Add Atheris fuzzing coverage for convert_capec.py

[prasunsrivastav123-lang]

fixes #2166

Changes

• Added tests/scripts/convert_capec_fuzzer.py
• Fuzzes the CLI interface only, using temporary files and paths
• Covers malformed CAPEC JSON, invalid ASVS mappings, and unexpected arguments
• Mirrors the structure of convert_fuzzer.py

[prasunsrivastav123-lang]

Hi @sydseter
I’ve fixed the ClusterFuzzLite import issue, applied Black formatting, and kept the fuzzer interface-only as discussed. The remaining failures look CI-related (including the website build). if you’d like me to adjust anything further or if a rerun is preferred.

[sydseter]

Ï am getting the following error locally:

File "convert_capec_fuzzer.py", line 16, in
ModuleNotFoundError: No module named 'convert_capec'
[PYI-107:ERROR] Failed to execute script 'convert_capec_fuzzer' due to unhandled exception!

ERROR: 50.0% of fuzz targets seem to be broken. See the list above for a detailed information.
ERROR:main:Check build failed.

[sydseter]

Happens when I run:

python infra/helper.py check_build --external $env:PATH_TO_PROJECT --sanitizer address

[sydseter]

Whole trace:

INFO:__main__:Running: docker run --privileged --shm-size=2g --platform linux/amd64 --rm -i -e FUZZING_ENGINE=libfuzzer -e SANITIZER=address -e ARCHITECTURE=x86_64 -e FUZZING_LANGUAGE=python -e HELPER=True -v 'C:\Users\JSydseter\src\oss-fuzz\build\out\cornucopia-prasunsrivastav123-lang:/out' -t gcr.io/oss-fuzz-base/base-runner:latest test_all.py.
INFO: performing bad build checks for /tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer
INFO: performing bad build checks for /tmp/not-out/tmpmr6pu2_v/convert_fuzzer
Retrying failed fuzz targets sequentially 1
INFO: performing bad build checks for /tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer
Broken fuzz targets 1
('/tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer', CompletedProcess(args=['bad_build_check', '/tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer'], returncode=1, stdout=b'BAD BUILD: /tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer seems to have either startup crash or exit:\nvm.mmap_rnd_bits = 28\n/tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer -- -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null\nfuzzing now, this is what is here\nTraceback (most recent call last):\n  File "convert_capec_fuzzer.py", line 16, in <module>\nModuleNotFoundError: No module named \'convert_capec\'\n[PYI-54:ERROR] Failed to execute script \'convert_capec_fuzzer\' due to unhandled exception!\n', stderr=b''))
BAD BUILD: /tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer seems to have either startup crash or exit:
vm.mmap_rnd_bits = 28
/tmp/not-out/tmpmr6pu2_v/convert_capec_fuzzer -- -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null
fuzzing now, this is what is here
Traceback (most recent call last):
  File "convert_capec_fuzzer.py", line 16, in <module>
ModuleNotFoundError: No module named 'convert_capec'
[PYI-107:ERROR] Failed to execute script 'convert_capec_fuzzer' due to unhandled exception!

ERROR: 50.0% of fuzz targets seem to be broken. See the list above for a detailed information.
ERROR:__main__:Check build failed.

[prasunsrivastav123-lang]

@sydseter I’ve updated the fuzzer to prepend $PATH_TO_PROJECT/tests/scripts to sys.path (with a local fallback), which resolves the ModuleNotFoundError under infra/helper.py check_build and ClusterFuzzLite. I’ve pushed the fix in the latest commit.
I think the remaining failures are due to cached ClusterFuzzLite runs and the unrelated website CI.
Please let me know if you’d like me to adjust anything further or if a rerun looks good.

[sydseter]

Still the same issue:

INFO: performing bad build checks for /tmp/not-out/tmp_t3n6kdo/convert_capec_fuzzer
Broken fuzz targets 1
('/tmp/not-out/tmp_t3n6kdo/convert_capec_fuzzer', CompletedProcess(args=['bad_build_check', '/tmp/not-out/tmp_t3n6kdo/convert_capec_fuzzer'], returncode=1, stdout=b'BAD BUILD: /tmp/not-out/tmp_t3n6kdo/convert_capec_fuzzer seems to have either startup crash or exit:\nvm.mmap_rnd_bits = 28\n/tmp/not-out/tmp_t3n6kdo/convert_capec_fuzzer -- -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null\nfuzzing now, this is what is here\nTraceback (most recent call last):\n File "convert_capec_fuzzer.py", line 23, in \nModuleNotFoundError: No module named 'convert_capec'\n[PYI-54:ERROR] Failed to execute script 'convert_capec_fuzzer' due to unhandled exception!\n', stderr=b''))
BAD BUILD: /tmp/not-out/tmp_t3n6kdo/convert_capec_fuzzer seems to have either startup crash or exit:
vm.mmap_rnd_bits = 28
/tmp/not-out/tmp_t3n6kdo/convert_capec_fuzzer -- -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null
fuzzing now, this is what is here
Traceback (most recent call last):
File "convert_capec_fuzzer.py", line 23, in
ModuleNotFoundError: No module named 'convert_capec'

[prasunsrivastav123-lang]

@sydseter
The fuzzer was still adding tests/scripts to sys.path, but under infra/helper.py check_build it runs as a standalone binary and needs the project root on PYTHONPATH instead. That’s why convert_capec is still not being resolved in your environment.
I’m actively working on adjusting the import path to prepend the project root and align the import with the package layout. I’ll push the fix and notify you as soon as it’s ready.

[prasunsrivastav123-lang]

@sydseter Added an Atheris fuzzer for convert_capec.
Everything imports cleanly now; happy to adjust if you want changes.

[prasunsrivastav123-lang]

@sydseter
The ClusterFuzzLite failure appears to be due to a missing baseline coverage artifact (cifuzz-coverage-latest).
This is expected since this PR introduces a new fuzzer and no prior coverage exists yet.
The fuzzer builds and runs correctly; happy to adjust if a different coverage workflow is preferred.

[prasunsrivastav123-lang]

@sydseter Thanks for confirming!
Happy to follow the project’s preferred approach for initial coverage — please let me know if you’d like me to add a baseline coverage job or adjust the workflow for first-time fuzzers.

[sydseter]

It's not about baseline coverage. The reason this fails has to do with how the fuzzer is including and finding modules.
When I run your test on my cornucopia fork I get the error reported earlier. see: https://github.com/sydseter/cornucopia/actions/runs/21857981056/job/63079395998

INFO: Using built-in libfuzzer
WARNING: Failed to find function "__sanitizer_acquire_crash_state".
WARNING: Failed to find function "__sanitizer_print_stack_trace".
WARNING: Failed to find function "__sanitizer_set_death_callback".
INFO: libFuzzer ignores flags that start with '--'
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1337
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
 === Uncaught Python exception: ===
ImportError: cannot import name 'convert_capec' from 'scripts' (/tmp/_MEIUpebJP/scripts/__init__.py)
Traceback (most recent call last):
  File "convert_capec_fuzzer.py", line 20, in TestOneInput
ImportError: cannot import name 'convert_capec' from 'scripts' (/tmp/_MEIUpebJP/scripts/__init__.py)
INFO: Instrumentation complete.
Exception ignored in: <function _removeHandlerRef at 0x7f09ffef7880>
Traceback (most recent call last):
  File "logging/__init__.py", line 853, in _removeHandlerRef
TypeError: 'NoneType' object is not callable
==181== ERROR: libFuzzer: fuzz target exited
SUMMARY: libFuzzer: fuzz target exited
MS: 0 ; base unit: 0000000000000000000000000000000000000000
artifact_prefix='./'; Test unit written to ./crash-da39a3ee5e6b4b0d3255bfef95601890afd80709
Base64: 
ERROR: 50.0% of fuzz targets seem to be broken. See the list above for a detailed information.
stderr: 
2026-02-10 08:56:58,995 - root - ERROR - Error building fuzzers for (commit: c535845c186893c8f728e6b58a903b6a42eeb186, pr_ref: None).

So in order to fix this there need to be a way of including the modules and using the right path to the module in the test:

[sydseter]

You need to either develop the fuzzers locally or use your repository to build. Tweak around the cflite_pr.yml pipeline to make it work. Please tell me if you want to throw in the towel. It's no shame in admitting defeat. It took me months to understand how to run the test we have.

[prasunsrivastav123-lang]

@sydseter Thanks for the context — I’d like to keep working on this and learn the setup rather than give up.
I’ll try building and running the fuzzer locally and experiment with cflite_pr.yml, and I’ll report back if I get stuck or make progress👍

[prasunsrivastav123-lang]

Hi @sydseter
Thank you for your patience and for the detailed feedback earlier.
I've been digging into the fuzzer issue, but I’ve realized I still need to better understand Atheris/libFuzzer behavior — especially around coverage collection in CI.
I’ll come back with more information and a stronger solution .
In the meantime, if you’d prefer to close this PR for now to keep things tidy, please feel free — no problem at all.