Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add fuzz testing #14

Open
BurntSushi opened this issue Jul 10, 2024 · 7 comments
Open

add fuzz testing #14

BurntSushi opened this issue Jul 10, 2024 · 7 comments
Labels
testing An issue about testing.

Comments

@BurntSushi
Copy link
Owner

We should be able to do some fuzz testing. The minimal thing we can do is to throw data at the datetime and span parsers to ensure they don't panic.

@BurntSushi BurntSushi added the testing An issue about testing. label Jul 10, 2024
@sigaloid
Copy link

In my experience, a mutation-based fuzzer like AFL++/afl.rs converges on panics much more quickly when given a corpus to start out from, since they start out with examples that provide way better coverage. Something like this, except with some additional handling to generate test case files from src/tz/mod.rs: MaxOhn/rosu-map#3

I'd be happy to submit a PR if this is roughly what you're looking for!

@BurntSushi
Copy link
Owner Author

I don't have a ton of domain expertise with fuzzing. I just know some of the basics. Is that setup compatible with OSS-fuzz? I'd like to make sure that whatever I do, it can run inside of OSS-fuzz.

@BurntSushi
Copy link
Owner Author

cc @addisoncrump

@sigaloid
Copy link

According to OSS-fuzz, no - while they say they support AFL++ as an engine on the home page, they mention on the actual Rust integration page that they only support cargo fuzz, which is built around libFuzzer. However, similar methods can be used to seed libFuzzer with a corpus, so some code to extract the test cases out would help improve fuzzing coverage.

@addisoncrump
Copy link

Given that most of the issues that we would want to look for in this kind of project are correctness issues, any fuzzer capable of finding bugs beyond low hanging fruit would need a very strong grammar and a good oracle for correctness.

That's not to say that I don't think this would be valuable. For initial testing for crashes, this would certainly be valuable 👍 Prefer cargo-fuzz, though, as we can integrate this with other fuzzers (e.g., libafl) more easily.

@BurntSushi
Copy link
Owner Author

Yeah I think a good start here would be fuzzing the parsers. There's a fair bit of surface area there.

@addisoncrump
Copy link

I'll do a quick review of parse functions and make some simple harnesses.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing An issue about testing.
Projects
None yet
Development

No branches or pull requests

3 participants