Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(forge test): add an option to continue fuzzing run on assertion failure #9727

Open
forkforkdog opened this issue Jan 21, 2025 · 0 comments
Labels
Cmd-forge-test Command: forge test T-feature Type: feature

Comments

@forkforkdog
Copy link

Component

Forge

Describe the feature you would like

Description:
Currently, Foundry’s invariant tests stop execution upon encountering the first assertion failure, ending the entire fuzzing campaign.

Feature Request:
Add an option to allow the fuzzing campaign to continue running until all configured runs are completed, even if assertion failures are encountered.

Rationale:
This feature would enable continuous fuzzing in an "Echidna-style" manner, where failing assertions are collected throughout the run without interrupting the campaign. Certain assertion failures require long-running fuzzing campaigns (e.g., 4+ hours), and stopping on the first failure can hinder the discovery of additional issues.

Expected Behavior:
When the continuous_run option is enabled, the fuzzer should:

  1. Continue running despite encountering assertion failures.

  2. Log all failures encountered during the campaign.

  3. Upon manual cancellation (e.g., CTRL+C/CMD+C) or completion of the configured runs/timeout, present all call traces for reproducing the assertion failures.

Example Configuration:

[invariant]
continuous_run = true

Example Output:

test/fuzzing/FuzzPositions.sol:FuzzPositions                                     
    ↪ invariant_GLOB_01: [9/4294967295] Runs  
    ❌ Failures: 2

Additional context

No response

@forkforkdog forkforkdog added T-feature Type: feature T-needs-triage Type: this issue needs to be labelled labels Jan 21, 2025
@github-project-automation github-project-automation bot moved this to Todo in Foundry Jan 21, 2025
@forkforkdog forkforkdog changed the title Add an Option to Continue Fuzzing Run on Assertion Failure FUZZING: Add an Option to Continue Fuzzing Run on Assertion Failure Jan 21, 2025
@grandizzy grandizzy added Cmd-forge-test Command: forge test and removed T-needs-triage Type: this issue needs to be labelled labels Jan 21, 2025
@zerosnacks zerosnacks changed the title FUZZING: Add an Option to Continue Fuzzing Run on Assertion Failure feat(forge test): add an option to continue fuzzing run on assertion failure Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Cmd-forge-test Command: forge test T-feature Type: feature
Projects
Status: Todo
Development

No branches or pull requests

2 participants