Introduce Retry Mechanism for eth_call to handle Incomplete or delayed smart contract state updates

[laruh]

When using eth_call, the returned state may be outdated or incomplete in scenarios where:

1. A transaction intended to modify the state has not reached the queried node's mempool (with pending block parameter).
2. A transaction has not been confirmed and included in the blockchain (with latest block parameter).

This issue can result in unexpected or failed responses, especially in use cases like querying smart contract storage (eg TakerPaymentState of TakerPayment)
https://github.com/KomodoPlatform/etomic-swap/blob/bbdeb369eab0753454dae29640adaa4fdba221ba/contracts/EtomicSwapTakerV2.sol#L10

    enum TakerPaymentState {
        Uninitialized,
        PaymentSent,
        TakerApproved,
        MakerSpent,
        TakerRefunded
    }

    struct TakerPayment {
        bytes20 paymentHash;
        uint32 preApproveLockTime;
        uint32 paymentLockTime;
        TakerPaymentState state;
    }

    event TakerPaymentSent(bytes32 id);
    event TakerPaymentApproved(bytes32 id);
    event TakerPaymentSpent(bytes32 id, bytes32 secret);
    event TakerPaymentRefundedSecret(bytes32 id, bytes32 secret);
    event TakerPaymentRefundedTimelock(bytes32 id);

    mapping(bytes32 => TakerPayment) public takerPayments;

The inconsistency arises due to delayed transaction propagation or confirmation.

Proposed Solution

To address this, here is a proposal of a retry mechanism with the following features:

1. Retry on Unexpected or Failed Results:
   • If an eth_call returns an unexpected result or fails, retry the call until a valid result is returned or a timeout is reached.
2. Configurable Timeout and Delay:
   • Introduce a configurable timeout to limit the total retry duration.
   • Add an optional delay between retries (sleep duration) to allow time for state updates or transaction propagation.

This solution applies to scenario where accurate state retrieval depends on pending or recently confirmed transactions: eg Smart contract storage queries (e.g., mappings, enums) during atomic swap.

[laruh]

Retry on Unexpected or Failed Results:
If an eth_call returns an unexpected result or fails, retry the call until a valid result is returned or a timeout is reached.

This could be under discussion, we can return err immediately and retry only on Unexpected state results.