Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update fuzzer documentation and changelog #2899

Open
wants to merge 3 commits into
base: 2051-new-fuzzer-architecture-6
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- `snforge clean` command - used to manage and remove files generated by snforge. It supports cleaning the following components: coverage, profile, cache, trace, all
- `snforge new` now adds the `snfoundry_trace`, `coverage`, and `profile` directories to `.gitignore`.
- Custom types can be used in fuzz testing by implementing the `Fuzzable` trait

#### Changed

- It is now required to include the `#[fuzzer]` attribute for fuzz tests to work

#### Deprecated

Expand Down
2 changes: 2 additions & 0 deletions docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@
* [store](appendix/cheatcodes/store.md)
* [load](appendix/cheatcodes/load.md)
* [generate_random_felt](appendix/cheatcodes/generate_random_felt.md)
* [generate_arg](appendix/cheatcodes/generate_arg.md)
* [`snforge` Library Reference](appendix/snforge-library.md)
* [byte_array](appendix/snforge-library/byte_array.md)
* [declare](appendix/snforge-library/declare.md)
Expand All @@ -110,6 +111,7 @@
* [fs](appendix/snforge-library/fs.md)
* [env](appendix/snforge-library/env.md)
* [signature](appendix/snforge-library/signature.md)
* [fuzzable](appendix/snforge-library/fuzzable.md)
* [`sncast` Commands](appendix/sncast.md)
* [common flags](appendix/sncast/common.md)
* [account](appendix/sncast/account/account.md)
Expand Down
6 changes: 6 additions & 0 deletions docs/src/appendix/cheatcodes/generate_arg.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# `generate_arg`

> `fn generate_arg<T, +Serde<T>, +Drop<T>, +Into<T, felt252>>(min_value: T, max_value: T) -> T`

Returns a random number from a range `[min_value, max_value]`.
It is used mostly in the context of fuzz testing to implement [Fuzzable](../snforge-library/fuzzable.md) trait.
55 changes: 55 additions & 0 deletions docs/src/appendix/snforge-library/fuzzable.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# `fuzzable` Module

Module containing `Fuzzable` trait needed for fuzz testing and its implementations for basic types.

## `Fuzzable`

```rust
pub trait Fuzzable<T, +Debug<T>> {
fn blank() -> T;
fn generate() -> T;
}
```

This trait is used by `snforge` to generate random data for fuzz testing.
Any type that is used as a parameter in a test function with the [`#[fuzzer]`](../../testing/test-attributes.md#fuzzer) attribute must implement this trait.

- `blank()` returns an empty or default value. The specific value used does not matter much, as it is only used for configuration runs. For types that implement the `Default` trait, it is recommended to return `Default::default()`.
- `generate()` function is used to return a random value of the given type. To implement this function, it is necessary to either use the [generate_arg](../cheatcodes/generate_arg.md) cheatcode,
which can uniformly generate a random number within a specified range, or use a `Fuzzable` implementation from a different type.

## Example

Implementation for a custom type `Message`:

```rust
use core::num::traits::Bounded;
use snforge_std::fuzzable::{Fuzzable, generate_arg};

#[derive(Debug, Drop)]
struct Message {
id: u64,
text: ByteArray
}

impl FuzzableMessage of Fuzzable<Message> {
fn blank() -> Message {
Message {
// Implementation may consist of:
// Specifying a concrete value for the field
id: 0,
// Or using default value from `Default` trait
text: Default::default()
}
}

fn generate() -> Message {
Message {
// Using `generate_arg` cheatcode
id: generate_arg(0, Bounded::<u64>::MAX),
// Or calling `generate` function on a type that already implements `Fuzzable`
text: Fuzzable::<ByteArray>::generate()
}
}
}
```
17 changes: 8 additions & 9 deletions docs/src/snforge-advanced-features/fuzz-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ against a large number of possible arguments.

## Random Fuzzing

To convert a test to a random fuzz test, simply add arguments to the test function.
To convert a standard test into a random fuzz test, you need to add parameters to the test function
and include the [`#[fuzzer]`](../testing/test-attributes.md#fuzzer) attribute.
These arguments can then be used in the test body.
The test will be run many times against different randomly generated values.

Expand Down Expand Up @@ -43,17 +44,15 @@ Fuzzer seed: [..]

## Types Supported by the Fuzzer

Fuzzer currently supports generating values of these types
Fuzzer currently supports generating values for these types out of the box:

- `u8`
- `u16`
- `u32`
- `u64`
- `u128`
- `u256`
- `felt252`
- `u8`, `u16`, `u32`, `u64`, `u128`, `u256`
- `i8`, `i16`, `i32`, `i64`, `i128`
- `ByteArray`

Trying to use arguments of different type in test definition will result in an error.
To use other types, it is required to implement the [`Fuzzable`](../appendix/snforge-library/fuzzable.md) trait for them.
Providing non-fuzzable types will result in a compilation error.

## Fuzzer Configuration

Expand Down
5 changes: 5 additions & 0 deletions docs/src/testing/test-attributes.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ Currently, those attributes are supported:
- `#[fork]`
- `#[fuzzer]`

> 📝 **Note**
>
> If you want to use other attributes (for example custom ones or from different packages) on the test function, they should be placed above the `snforge` attributes.
> This ensures that they execute first on the raw code, rather than on the code generated by `snforge`. Otherwise, it may result in unexpected behavior.

### `#[test]`

Marks the function as a test case, to be visible for test collector.
Expand Down
Loading