Making AI-Guided Fuzzing More Reliable with Adversarial Probing

Cybersecurity researchers have used fuzzing to discover vulnerabilities for years, but it’s a painstaking process that requires significant human effort and expertise. This makes fuzzing a promising use case for AI.

However, current attempts to use AI-guided fuzzing have failed to produce the sort of reliable, informative results that security researchers can trust.

At AISLE, we’ve developed an AI-guided fuzzing workflow that avoids the four most common failure modes in the industry right now. In this article, we’ll explain why most LLM-generated fuzzing harnesses fall short, and what we’re doing differently to help defenders focus on genuine risks.

Fuzzing: More Than You Wanted to Know

If you’re already familiar with fuzzing, you can skip to the next section. If you’re not, here’s a short introduction to the field.

Key terms

A fuzzer repeatedly executes a program with mutated inputs to expose defects by getting the program to crash. In coverage-guided fuzzing, execution feedback helps it retain inputs that explore new code paths and use them for further mutation.

A fuzz harness is the test code the fuzzer uses to invoke the target library. With libFuzzer, a widely adopted fuzzing engine, the harness implements a well-defined LLVMFuzzerTestOneInput function. Note that while the fuzzer supplies inputs, the harness determines how those inputs exercise the library. The library is compiled with coverage instrumentation, which reports which code each input reaches so the fuzzer can favor inputs that explore new paths, and typically with sanitizers that turn memory errors into detectable crashes.

A seed is an input used as a starting point for mutation.

A corpus is the collection of inputs maintained by the fuzzer. An initial corpus may contain representative files or messages for the functionality under test, such as small PNG images for an image decoder.

The process

The harness and library are typically compiled with runtime instrumentation such as AddressSanitizer, which reports memory errors observed during execution. A sanitizer report establishes that the runtime crashed. Because crashes are hard to miss, they provide useful evidence for vulnerability discovery.

However, a crash doesn’t necessarily mean that there was an issue in the library you want to test. To make that link, you have to check whether the harness used its APIs correctly.

Figure 1. High-level conceptual flow of heuristics in coverage-guided fuzzing engines, such as libFuzzer or AFL.

As Figure 1 illustrates, the fuzzer generates lots of mutated (and therefore unexpected) corpus inputs and passes them through the harness to run the target library. Sanitizers detect errors, and coverage feedback is incorporated to improve further exploration.

AI-Guided Fuzzing: The State of Play

Like any other program, a fuzz harness can contain bugs. These bugs can affect a fuzzing campaign in two ways:

1.     The harness, not your code, might cause a crash. If the fuzzing harness has invalid arguments, incorrect object lifetimes, or missing cleanup, that can trigger a crash. You can’t simply conclude that each crash translates into a genuine vulnerability.

2.           Your harness may never reach its target. An incorrect guard or missing prerequisite can prevent the harness from reaching its target, leaving your campaign to perform ineffective tests.

As defenders well know, large volumes of misleading reports consume the attention needed to investigate genuine risks. And as LLMs are being used for vulnerability discovery, cybersecurity organizations are swamped with reports of varying quality.

So while AI can generate many fuzzing harnesses in a short amount of time, defenders are right to be skeptical. Our research shows that LLMs often build harnesses that generate crashes, only for further investigation to reveal that the harness fell prey to one or both of the failure modes described above.

Indeed, LLMs are known to exploit procedural loopholes in order to complete tasks when they calculate that doing things the “right way” may be less likely to achieve a satisfactory result.

That’s why we were unsatisfied with existing research into AI-guided fuzzing, which has largely focused on generating harnesses that build successfully, evaluating their coverage, or producing seeds for them. If AI-generated fuzzing harnesses cannot be trusted to call APIs correctly or contain sound logic, there is no reason to trust the coverage they claim to provide.

A libpng harness with a missing cleanup path

To see why a crash isn’t enough to establish that you’ve found a vulnerability on its own, consider the following example. This is an adaptation of a libpng harness defect we reported.

libFuzzer  works by calling a single function, LLVMFuzzerTestOneInput, many times, passing in a new block of semi-random bytes each time. Our harness is the small piece of glue code between the fuzzer and the library under test. It hands those bytes to libpng as if they were a PNG file, then asks libpng to decode the image one row at a time.

Before reading the code, you need to know one unusual detail: how libpng reports errors. Most C libraries return an error code. libpng instead uses two standard C functions, setjmp and longjmp. Think of setjmp as saving a checkpoint and longjmp as an emergency exit back to it. The harness saves a checkpoint before decoding begins. If libpng hits a fatal error deep inside the decoder, it jumps straight back to that checkpoint. Execution then resumes in the error-handling block, and every line in between that hadn’t run yet is skipped.

If you’ve used exceptions in Java, Python, or C++, this works much like throwing one, with one important difference: nothing is cleaned up on the way out. There is no finally block and no automatic destructor. Any cleanup the skipped code would have done simply never happens.

Before

Red outlines mark the faulty cleanup path

JavaScript
1static void test_png_transformations(const uint8_t *data, size_t size) {
2    // Create png_ptr and info_ptr; check failures (omitted).
3    if (setjmp(png_jmpbuf(png_ptr))) {  ◀ A
4        // The harness-owned row buffer is not freed.
5        png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
6        return;
7    }
8 
9    png_mem_buffer buffer = {data, size, 0};
10    png_set_read_fn(png_ptr, &buffer, png_read_from_buffer);
11    // Read metadata and set transformations (omitted).
12 
13    png_bytep row = (png_bytep)malloc(png_get_rowbytes(png_ptr, info_ptr));
14    if (row) {
15        for (int y = 0; y < height && y < 100; y++) {
16            png_read_row(png_ptr, row, NULL);
17        }
18        free(row); // A longjmp bypasses this call.  ◀ B
19    }
20    png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
21}
22 
23extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
24    test_png_transformations(data, size);
25    return 0;
26}
Copy

A · The error handler (lines 3–7). This is where execution lands after libpng’s emergency exit. The handler frees libpng’s own data structures, but not the row buffer that the harness allocated itself.

B · The only cleanup (line 18). The harness frees the row buffer here, but this line runs only if every row decodes successfully. If libpng fails partway through the image, the jump skips this line and the buffer is never freed.

After

Repair diff:  − removed · + added · line numbers: old / new

JavaScript
1  static void test_png_transformations(const uint8_t *data, size_t size) {
2      // Create png_ptr and info_ptr; check failures (omitted).
3+     png_bytep volatile row = NULL;
4      if (setjmp(png_jmpbuf(png_ptr))) {
5-         // The harness-owned row buffer is not freed.
6+         free(row); // Release the buffer on error, too.
7          png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
8          return;
9      }
10   
11      png_mem_buffer buffer = {data, size, 0};
12      png_set_read_fn(png_ptr, &buffer, png_read_from_buffer);
13      // Read metadata and set transformations (omitted).
14   
15-     png_bytep row = (png_bytep)malloc(png_get_rowbytes(png_ptr, info_ptr));
16+     row = (png_bytep)malloc(png_get_rowbytes(png_ptr, info_ptr));
17      if (row) {
18          for (int y = 0; y < height && y < 100; y++) {
19              png_read_row(png_ptr, row, NULL);
20          }
21-         free(row); // A longjmp bypasses this call.
22      }
23+     free(row); // Normal cleanup; free(NULL) is valid.
24      png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
25  }
26   
27  extern "C" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) {
28      test_png_transformations(data, size);
29      return 0;
30  }
Copy

What goes wrong. The harness allocates a buffer to hold one decoded row, decodes rows into it, and frees it at the end. That works for valid images. But a fuzzer mostly produces broken input, so libpng often fails partway through decoding. When it does, it jumps back to the checkpoint and bypasses free(row). The error handler cleans up libpng’s memory, but it doesn’t know about the harness’s buffer. Every such input leaks one buffer.

The fix. The repair makes the error handler aware of the buffer and ensures both exit paths free it:

1. Declare row at the top of the function, before the checkpoint, and set it to NULL. This makes the variable visible to the error handler. If an error occurs before the buffer is allocated, there is nothing to free, and that’s fine, because calling free(NULL) in C is defined to do nothing.
2. Free row in the error handler.
3. Move the normal-path free outside the if (row) block, so one call covers both the “allocated” and “allocation failed” cases.

Why volatile? There’s one subtle detail: row is declared volatile. Compilers are allowed to keep local variables in CPU registers for speed. The C standard warns that after a longjmp, a local variable changed since the setjmp call may not hold its latest value, because the jump can restore older register contents. Marking row as volatile forces the compiler to keep it in memory, so the error handler sees the real buffer address instead of a stale NULL. Without it, the fix could simply free nothing.

Whose bug is it? The bug is in the harness, not in libpng. LeakSanitizer, the memory-leak detector built into the Clang toolchain, correctly reports this memory leak. Libpng behaved exactly as designed, but the harness failed to properly handle its error-handling mechanism.

This is a case in point: fuzzing findings can come from the harness as easily as from the target.

Four Recurring Failure Modes

To investigate these problems systematically, we examined 586 production harnesses across 70 OSS-Fuzz projects. We identified four failure modes across harness implementation, API protocols, security boundaries, and target selection.

The production-harness audit evaluated P1 and P2; P3 and P4 additionally guide the selection of targets for new harnesses.

Table 1. Recurring failure modes and the corresponding harness quality principles.

P1 and P2 address distinct sources of error within the harnesses themselves. For instance, the libpng row-buffer leak is a harness cleanup defect, while an API protocol violation, such as using an output object after the operation that should initialize it has failed, can also trigger a crash.

P3 and P4, on the other hand, concern what is being tested and under which assumptions. A public API is not necessarily intended to accept attacker-controlled data. Conversely, an internal interface may be useful for testing when there is evidence that the relevant behavior is also reachable through legitimate public calls.

These principles define what we check when evaluating a harness. They guide the generator in checking API usage and harness logic, identifying problems, and repairing its output. Our fuzzing workflow builds these checks into harness generation before longer fuzzing runs begin.

Harness Generation and Validation

Based on these principles, AISLE’s AI-guided fuzzing system uses a multi-agent workflow with four stages that covers target selection, API protocol research, harness generation and build, and adversarial validation. The four principles guide these stages, while Adversarial Probing (AP) tests the generated harness. The agent examines its own output, identifies likely violations, and constructs inputs intended to expose them. Execution results then guide repair.

Figure 2. AISLE’s workflow for generating, testing, and repairing fuzz harnesses.

Fuzz target selection

We organize fuzz targets into Logic Groups (LGs), each representing a feature of the library. Each LG is defined as

where name identifies the feature, E is the set of API entry points that receive fuzzer input, C is a selected set of core functions reachable from those entries that implement the feature, and desc is an LLM-generated description of the feature and its security relevance.

The agent first examines existing harnesses to understand which features they should test, then explores the project source to identify additional features for new LGs. For each candidate, it traces callers from the core functions to find suitable API entry points, preferring public APIs in line with P3 and checking that they reach the intended implementation in line with P4. It then removes redundant candidates and ranks the remaining LGs by a score based on reachable memory operations and pointer dereferences. The selected LGs proceed to API protocol research and harness generation.

API research

For each LG, an API Research agent studies how to use the selected APIs before the harness is written. For each entry point, it reads the public headers, examines existing callers and internal dependencies, and searches tests and examples. It collects requirements for initialization, call order, parameters, object lifetimes, and error handling.

The agent produces a structured API protocol report for the Harness Generator agent. Each finding includes references to source files and line numbers, so the generator can check the evidence when writing the harness.

Harness generation and adversarial validation

The Harness Generator agent reads the project’s build configuration, the selected LG, and the API protocol report, then writes the harness. It reviews the draft against P1 and P2 through source analysis and attempts to build it. If compilation fails, the agent uses compiler feedback to revise the code and rebuild within a bounded number of attempts.

A successful build proceeds to Adversarial Probing (AP). The same agent reads its harness alongside the relevant library code and identifies the P1 and P2 checks that the harness appears most likely to violate. It then writes code to generate inputs targeting those suspected weaknesses. For example, it may trigger a library error after memory allocation to test cleanup, supply a buffer below the required minimum size, or run two inputs in sequence to expose state retained between executions.

AP provides two operations. A reach check uses GDB to observe whether an input reaches the target API. A run check executes the input under ASan and LSan to detect runtime errors. These observations are interpreted alongside the source code.

Note that reaching the API does not establish that the harness uses it correctly, and a sanitizer report does not by itself identify whether the defect belongs to the harness or the library.

The agent classifies each probe result into one of four cases.

1. Crash caused by the harness. A sanitizer reports an error, and the agent identifies a P1 or P2 violation in the harness. The harness returns to the repair loop.
2. No crash, but the target is not reached. The input does not drive execution into the target API. The agent investigates the input and harness logic, then revises and tests again.
3. Crash caused by the library. A sanitizer reports an error, and the agent attributes it to the library after checking the harness against the four principles. The finding proceeds to separate verification.
4. No crash, and the target is reached. The probe reaches the target API without exposing a harness defect. The agent continues validation.

After any harness repair, the agent rebuilds the binary and probes it again. Once the selected probes pass, it runs the harness under ASan and LSan for 600 seconds and collects line and branch coverage. Acceptance requires dynamic evidence that the target entry executes, nonzero line and branch coverage, and the agent’s assessment of the remaining quality checks. These results can also prompt further investigation and repair.

Some checks require source review rather than runtime probes, including API existence, local reimplementation of library code, and paired or mutually exclusive calls. P3 and P4 guide boundary and entry selection through source and call graph analysis, with runtime reach checks providing additional evidence that the selected entry executes.

The Results

We evaluate our approach against two state-of-the-art approaches, OSS-Fuzz-Gen and PromeFuzz, on 100 harness tasks from 39 C/C++ OSS-Fuzz projects. Human-written OSS-Fuzz harnesses serve as the reference and are withheld during generation. All generators receive the same project and target entry, use Claude Sonnet 4.6, and have a build budget of five attempts. The baselines are adapted to use a common input/output interface and the same pinned project versions.

Each harness is evaluated in 10 independent libFuzzer runs of 600 seconds each, starting with an empty corpus and using ASan and LSan. We take the median line and branch coverage for each harness, then average across the 100 tasks. Failed builds and harnesses with zero coverage contribute zero to these averages.

We count a harness as productive if it builds successfully and achieves nonzero line or branch coverage in the target library. Build failures and harnesses with zero coverage on both metrics are counted as unproductive.

Table 2. Baseline comparison on 100 tasks. Additionally reported as QuartetFuzz in the paper. Source: §5.4, Table 3.

Our workflow produces 96 productive harnesses, compared with 64 for OSS-Fuzz-Gen and 74 for PromeFuzz. Mean line coverage exceeds the baselines by 6.9 and 5.2 percentage points; mean branch coverage improves by 8.3 and 4.1 points. Coverage is statistically equivalent to the human-written reference within a ±2 percentage-point margin under paired TOST tests.

The contribution of adversarial probing

Disabling AP under the same evaluation conditions reduces productive harnesses from 96 to 84. Mean line coverage falls from 17.7% to 16.9%, and branch coverage from 17.5% to 16.6%. The twelve lost cases compile and run but fail to reach the target. Without the probe gate, those failures pass through the build-and-repair loop.

These results show that compilation and execution alone can miss failures to exercise the intended functionality. AP increases the productive rate by 12 percentage points, helping more harnesses perform effective testing. This improvement is less apparent in the aggregate coverage metrics, which rise by 0.8 percentage points for line coverage and 0.9 for branch coverage.

Source: §5.5, Table 4

All 96 productive harnesses invoke AP at least once, with 362 probe calls in total: a median of three and a maximum of twelve per harness. Twenty-two require at least six probes.

Real-World Impact

During the month of April 2026, we deployed our workflow on 23 open source projects spanning C/C++, Java, and JavaScript, and submitted 42 bug reports. Of these, 40 have been confirmed or fixed upstream, including three findings assigned CVEs. The remaining two were rejected as false positives, accounting for 4.8% of all submissions.

Our workflow achieved a false positive rate of 4.8%, lower than the rates reported by the systems listed below.

Table 3. False positive rates of popular fuzz harnesses

During generation and validation, the workflow identified and repaired 44 harness issues that triggered sanitizer errors, keeping those errors from becoming library bug reports. Repairing existing production harnesses also exposed two previously hidden library bugs.

Conclusion

LLMs can generate fuzz harnesses quickly, but those harnesses need testing too. Our workflow uses four quality principles to guide generation and adversarial probing to check whether a harness calls APIs correctly, has errors in its own logic, and reaches the intended code. The results guide repairs before longer fuzzing runs begin.

On 100 tasks, our workflow produced 96 productive harnesses, compared with 84 without adversarial probing, and achieved coverage comparable to human-written harnesses. These results support making harness validation part of generation, so that errors in the test code can be caught before they waste fuzzing time or become misleading bug reports.

Data and Version Notes

Evaluation numbers follow the controlled baseline and ablation experiments in Quality-Assured Fuzz Harness Generation via the Four Principles Framework (arXiv v1, May 2026). Costs reflect the reported experimental configuration.