Throwing Random Arguments at System Binaries: Real Segfaults vs. Ticket Noise

Why Would Anyone Do This?

Two and half reasons:

Smoke testing – You want to know if your system commands actually work, not just when you run them the way the docs say, but when users (or their scripts) feed them garbage.
AI is excellent at generating potential edge cases, and tracking systems are already all too eager to collect new tickets. I’m being careful not to dump every AI finding into Bugzilla; I don’t want to clutter the backlog and mainly waste developer time on theoretical bugs. Or Should I?

Plus, segfaults don’t lie – either the system crashed or it didn’t, and those are the issues that actually deserve the ticket.

Throwing Random Arguments at System Binaries Until They Crash

Script to do the work:

A pretty straightforward bash script, vibing with AI-generated chaos.

Grab all binaries from /usr/bin and /usr/sbin
Parse --help for flags (--whatever, -x, you know the drill)
Pick random combos of those flags (1-4 per run)
Feed them garbage: broken JSON/XML, binary junk, path traversal attempts, format strings, absurdly long lines

Only logs actual crashes – SIGSEGV, SIGABRT, SIGILL, SIGBUS. Exit code 1 from bad args gets ignored.

Core logic looks like this:

# Extract flags from --help                                            
  flags=$(timeout 3s "$bin" --help 2>&1 |                   
      grep -aoE -e '--[a-zA-Z0-9_-]+' -e '-[a-zA-Z]' |                                                    
      grep -avE 'help|version|usage')                                                                     
                                                                                                          
  # Pick random flags (1-4 of them)                                                                       
  chosen=$(echo "$flags" | shuf -n $((1 + RANDOM % 4)))                                                   
                                                                                                          
  # Add a random test file                                                                                
  fuzz_file="$WORKSPACE/$(random_pick: bad.json, random.bin, longline.txt, ...)"                          
                                                                                                          
  # Run it                                                  
  timeout 5s "$bin" $chosen $fuzz_file

Script skips the obvious no go zones – package managers, rm, network tools, editors. I’m glad to see the script finish with the machine still answering.

Try It Yourself!

Source: run-them-all

Run via Testing Farm (how-to):

testing-farm-public request --test-type fmf \                                                
    --git-url https://forge.fedoraproject.org/quality/fmf-tests.git \                                     
    --git-ref main \
    --compose Fedora-Cloud-Base-AmazonEC2.x86_64-44-1.3 \                                                 
    --arch x86_64 \                                                                                       
    --test run-them-all \                                                                                 
    --context "force=yes" \                                                                               
    --plan /plans/all

Run locally: Just use the try-all.sh binary from the test, no requirement needed, run:

curl -sSLO https://forge.fedoraproject.org/quality/fmf-tests/raw/branch/main/system-in-use/run-them-all/try-all-binaries-help-options.sh && bash try-all-binaries-help-options.sh

My results: Testing Farm Artifacts (Fedora 44 RC compose 1.3, ~950 binaries tested)

What I Found

grub2-mkrescue (bootloader utilities), perl (half the system depends on it), eqn from groff (man pages break without it).

What coredumpctl caught:

TIME                           PID UID GID SIG     COREFILE EXE                      SIZE
  Mon 2026-04-20 12:08:09 UTC  52378   0   0 SIGABRT present  /usr/bin/edgepaint      80.7K               
  Mon 2026-04-20 12:08:22 UTC  58926   0   0 SIGSEGV present  /usr/bin/eqn            57.8K
  Mon 2026-04-20 12:08:41 UTC  77687   0   0 SIGSEGV present  /usr/bin/gdbm_dump      23.9K               
  Mon 2026-04-20 12:09:01 UTC  97901   0   0 SIGSEGV present  /usr/bin/grub2-mkrescue 64.7K
  Mon 2026-04-20 12:09:08 UTC 106904   0   0 SIGSEGV present  /usr/bin/gtshapprox     48.3K               
  Mon 2026-04-20 12:10:15 UTC 161697   0   0 SIGABRT present  /usr/bin/perl5.42.1       94K

Breakdown:

edgepaint (graphviz) – 4× SIGABRT
gtshapprox (graphviz) – 3× SIGSEGV
perl – 4× SIGABRT
eqn (groff) – 1× SIGSEGV
gdbm_dump (gdbm) – 1× SIGSEGV
grub2-mkrescue (grub2) – 1× SIGSEGV

Reproducers

Run it on your side to see the reproducers in action. I’m curious to hear your thoughts on these major findings.

MY CRASHES:
COMMAND: /usr/bin/edgepaint -s -o --random_seed --angle --lightness -v --accuracy --share_endpoint 
COMMAND: /usr/bin/edgepaint -v -s --accuracy --color_scheme -o  /root/fuzz_lab/empty.dat
COMMAND: /usr/bin/edgepaint --share_endpoint -o --angle -v -s --random_seed --accuracy --color_scheme  -
COMMAND: /usr/bin/edgepaint -s --lightness --share_endpoint -o -v --angle --random_seed 
COMMAND: /usr/bin/efibootdump --guid -g -f  /root/fuzz_lab/large.dat
COMMAND: /usr/bin/eqn -C -f -M -d -v -m -T -s  /root/fuzz_lab/fake.png
COMMAND: /usr/bin/gdbm_dump --format  -
COMMAND: /usr/bin/grub2-file --is-x86-knetbsd  /root/fuzz_lab/longline.txt
COMMAND: /usr/bin/grub2-file --is-x86-knetbsd  /root/fuzz_lab/bad.json
COMMAND: /usr/bin/gtshapprox --flat -c  /root/fuzz_lab/fake.jpg
COMMAND: /usr/bin/gtshapprox -v -f --verbose -c  /root/fuzz_lab/cmd.txt
COMMAND: /usr/bin/gtshapprox -n -h --flat -l --closed -v  /root/fuzz_lab/gzip.dat
COMMAND: /usr/bin/gtshapprox --flat --number --log --cost --closed  -
COMMAND: /usr/bin/gtshapprox --log -l -f --keep  -
COMMAND: /usr/bin/mkfs.xfs -q  /root/fuzz_lab/bad_utf8.txt
COMMAND: /usr/bin/mkfs.xfs -K -L -m  /root/fuzz_lab/paths.txt
COMMAND: /usr/bin/perl -V -F -T -S -u -p -f -E -W -I  /root/fuzz_lab/bad.json
COMMAND: /usr/bin/perl -F -e -n -a -l -u -c  -
COMMAND: /usr/bin/perl -S -u -s -E -I -l 
COMMAND: /usr/bin/perl -s -p -n -u -D -E -d 
COMMAND: /usr/bin/tree -P --filelimit --matchdirs --hyperlink -i  /root/fuzz_lab/paths.txt

Look, these are edge cases. Nobody’s actually running edgepaint --wtf malformed.json in prod. But segfaults are segfaults – the binary should bail with “invalid option” or “bad input”, not dump core.

Now What?

So I’ve got a pile of crashes. Some in critical components. All reproducible.

File bugs for all of them? That’s a lot of BZ tickets for “yes hm this crashes if you feed it random garbage with weird flags”. Developers have better things to do.

Ignore them? They’re real bugs. And some of these are in grub2 and perl – not exactly throwaway packages.

Still figuring that out.

My Dilemma During Testing on Fedora 44 RC compose 1.X

Comments

crazylyle says:

A long long time ago, when I was first getting started on Unix, probably on a PDP-11, I tried something similar. I took troff and fed it every file in the file system to see if it would “work”. A simple find command:

find / -type f -exec troff ‘{}’ \;

It ran for days, and produced a fair number of core dumps.
solanum says:

File them with the lowest priority, and an absolute way to reproduce them. (if you can include compiler flags, libraries used, etc. don’t assume they are using fedora). They may or may not get fixed. IE not worth developers time or they are aware of the issue and it isn’t worth fixing. However, sometimes it can be indicative of a larger bug say like a change to a library that may actually be of concern or they are not aware there is an issue at all.

More importantly, it gives very low hanging fruit for someone who wants to help the project without stomping on the main devs which can lead to more people getting involved in the project and open source in general.

If you can reproduce them, you can also triage them which saves them time.