-
Notifications
You must be signed in to change notification settings - Fork 400
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce CI fuzz iterations as we're now timing out #3691
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,10 +13,19 @@ rm *_target.rs | |
[ "$(git diff)" != "" ] && exit 1 | ||
popd | ||
|
||
export RUSTFLAGS="--cfg=secp256k1_fuzz --cfg=hashes_fuzz" | ||
|
||
mkdir -p hfuzz_workspace/full_stack_target/input | ||
pushd write-seeds | ||
RUSTFLAGS="$RUSTFLAGS --cfg=fuzzing" cargo run ../hfuzz_workspace/full_stack_target/input | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wanted to verify that using the hardcoded seed indeed increases coverage, but it seems that hongfuzz reports in the log always
Not sure why that is? Otherwise it would be easy to compare with and without this hardcoded seed. Or maybe there is another way to verify that it works indeed? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The fact that the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Okay, indirect evidence. But why is hongfuzz reporting 0? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it not using coverage to direct fuzzing? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yea, that I don't know, it certainly does locally and there's nothing different locally vs in CI. Also CI does turn up (fairly shallow) fuzz bugs as well, so I'd be surprised to learn it was able to do that with just general patterns and no instrumentation... There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does it show you |
||
popd | ||
|
||
cargo install --color always --force honggfuzz --no-default-features | ||
# Because we're fuzzing relatively few iterations, the maximum possible | ||
# compiler optimizations aren't necessary, so switch to defaults. | ||
sed -i 's/lto = true//' Cargo.toml | ||
sed -i 's/codegen-units = 1//' Cargo.toml | ||
TheBlueMatt marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
export RUSTFLAGS="--cfg=secp256k1_fuzz --cfg=hashes_fuzz" | ||
export HFUZZ_BUILD_ARGS="--features honggfuzz_fuzz" | ||
|
||
cargo --color always hfuzz build | ||
|
@@ -25,11 +34,13 @@ for TARGET in src/bin/*.rs; do | |
FILE="${FILENAME%.*}" | ||
HFUZZ_RUN_ARGS="--exit_upon_crash -v -n2" | ||
if [ "$FILE" = "chanmon_consistency_target" ]; then | ||
HFUZZ_RUN_ARGS="$HFUZZ_RUN_ARGS -F 64 -N100000" | ||
elif [ "$FILE" = "full_stack_target" ]; then | ||
HFUZZ_RUN_ARGS="$HFUZZ_RUN_ARGS -t0 -N1000000" | ||
HFUZZ_RUN_ARGS="$HFUZZ_RUN_ARGS -F 64 -N5000" | ||
elif [ "$FILE" = "process_network_graph_target" -o "$FILE" = "full_stack_target" -o "$FILE" = "router_target" ]; then | ||
HFUZZ_RUN_ARGS="$HFUZZ_RUN_ARGS -N50000" | ||
elif [ "$FILE" = "indexedmap_target" ]; then | ||
HFUZZ_RUN_ARGS="$HFUZZ_RUN_ARGS -N500000" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Any specific reasoning behind the 10x, 10x and 2x reductions? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No. I probably could have benchmarked, but in general there's not a lot of value in |
||
else | ||
HFUZZ_RUN_ARGS="$HFUZZ_RUN_ARGS -N1000000" | ||
HFUZZ_RUN_ARGS="$HFUZZ_RUN_ARGS -N2500000" | ||
fi | ||
export HFUZZ_RUN_ARGS | ||
cargo --color always hfuzz run $FILE | ||
|
Large diffs are not rendered by default.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
[package] | ||
name = "lightning-fuzz-write-seeds" | ||
version = "0.0.1" | ||
authors = ["Automatically generated"] | ||
publish = false | ||
edition = "2021" | ||
|
||
[features] | ||
|
||
[dependencies] | ||
lightning-fuzz = { path = "../" } | ||
|
||
# Prevent this from interfering with workspaces | ||
[workspace] | ||
members = ["."] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
fn main() { | ||
let mut iter = std::env::args(); | ||
iter.next().unwrap(); // program name | ||
let path = iter.next().expect("Requires a path as the first argument"); | ||
lightning_fuzz::full_stack::write_fst_seeds(&path); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why these two additions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because
RUSTFLAGS
aren't applied to doctests, so we have to restrict tests to only run library and binary tests.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could add comments when making changes like this. Make life easier for future devs.