GitHub Security Lab发布AI自动化Fuzzing工具Task
AI-powered fuzzing with the GitHub Security Lab Taskflow Agent
安全工程同学必看,GitHub把LLM Agent引入Fuzzing全流程,给出了完整的架构设计与参数取舍,直接落地能省大量人工调参时间。
If you’re new to fuzzing and want to learn the fundamentals first, check out our Fuzzing 101 course at gh.io/fuzzing101.
如果你是模糊测试(fuzzing)的新手,想先学习基础知识,请查看我们在 gh.io/fuzzing101 提供的 Fuzzing 101 课程。
Continuous fuzzing is not a magic solution that solves all your problems . Even projects that have been enrolled in OSS-Fuzz for years can still hide critical bugs, and the reason is almost always the same: someone needs to keep an eye on coverage, write new harnesses for the code that nobody is reaching, and triage the crashes that come out the other end. In other words, fuzzing still needs a human in the loop.
持续模糊测试并不是能解决所有问题的魔法方案。即使是在 OSS-Fuzz 中注册多年的项目,仍可能隐藏关键漏洞,原因几乎总是相同的:需要有人监控覆盖率,为无人触及的代码编写新的驱动(harnesses),并对产生的崩溃进行初步分类(triage)。换句话说,模糊测试仍然需要人工参与循环。
So the natural question I kept asking myself was: how much of that human work can we actually hand over to an LLM agent?
因此,我不断问自己的自然问题是:我们实际上能将多少这样的人工工作交给 LLM 智能体?
That is what led me to build the Fuzzing Taskflow, an autonomous fuzzing pipeline for C/C++ projects. You only need to point it at a GitHub repository, and it does the rest: it identifies the suitable entrypoints, analyzes the build system, writes the harnesses, runs AFL++, reads the coverage reports, improves the harnesses, triages every crash, and writes a vulnerability report for each unique bug, all without a human babysitting it.
这就是促使我构建 Fuzzing Taskflow 的原因,这是一个用于 C/C++ 项目的自主模糊测试流水线。你只需将其指向一个 GitHub 仓库,它便会完成其余工作:识别合适的入口点、分析构建系统、编写驱动、运行 AFL++、读取覆盖率报告、改进驱动、对每次崩溃进行分类,并为每个唯一漏洞撰写漏洞报告,全程无需人工 babysitting。
The Fuzzing Taskflow is built on top of the GitHub Security Lab Taskflow Agent, our framework for writing LLM-driven security automation, so the pipeline is expressed as a set of taskflows that an agent runs end to end.
Fuzzing Taskflow 建立在 GitHub Security Lab Taskflow Agent 之上,这是我们用于编写由 LLM 驱动的安全自动化框架,因此该流水线被表达为一组由智能体端到端运行的 taskflows。
In this post, I’ll walk you through how it works and the design decisions behind it. Let’s get going!
在本文中,我将带你了解其工作原理及背后的设计决策。让我们开始吧!
How to run it
如何运行它
The simplest way to run it’s just to go to https://github.com/GitHubSecurityLab/seclab-taskflows-fuzzing and start a codespace.
运行它的最简单方法是前往 https://github.com/GitHubSecurityLab/seclab-taskflows-fuzzing 并启动一个 codespace。
Then, run the script like this:
然后,按如下方式运行脚本:
./scripts/fuzzing/run_fuzzing.sh PROJECTSo, for example:
例如:
./scripts/fuzzing/run_fuzzing.sh tukaani-project/xzThat’s it. The argument is just a GitHub owner/repo slug. Then, the agent, takes care of all the preliminary steps on its own:
就是这样。参数只是一个 GitHub 所有者/仓库简称(owner/repo slug)。然后,智能体会自行处理所有预备步骤:
- Installing software such as AFL
- Cloning the repository
- Identifying the most relevant functions in the code
- Creating fuzz targets for those functions
- 安装 AFL 等软件
- 克隆仓库
- 识别代码中最相关的函数
- 为这些函数创建模糊测试目标
If you just want a quick smoke test before committing to a long campaign, point it at something small:
如果你只想在投入长期测试活动前进行快速冒烟测试,可以将其指向某个小型项目:
./scripts/fuzzing/run_fuzzing.sh DaveGamble/cJSONA word of warning before you run it: this taskflow runs afl-fuzz, clang, and arbitrary build commands chosen by the LLM directly on the host, with no container in between. A prompt-injected agent could, in principle, do anything your user can. So please run it only inside a disposable environment (e.g., a Codespace or a throwaway VM), without elevated privileges.
在运行之前提醒一句:此 taskflow 直接在宿主机上运行 afl-fuzz、clang 以及由 LLM 选择的任意构建命令,中间没有容器隔离。理论上,经过提示注入的智能体可以做任何你的用户所能做的事。因此,请务必仅在可丢弃的环境(如 Codespace 或一次性虚拟机)中运行,且不要使用提升的权限。
Model selection
模型选择
Some frontier models impose security guardrails on their outputs. For the fuzzing task flow, we use Claude Sonnet 5 by default because it passed all of our internal tests without issues. You can choose a different model by modifying the following file: src/seclab_taskflows_fuzzing/configs/model_config.yaml.
部分前沿模型会对输出施加安全护栏。对于模糊测试任务流,我们默认使用 Claude Sonnet 5,因为它在所有内部测试中均顺利通过。你可以通过修改以下文件来选择不同的模型:src/seclab_taskflows_fuzzing/configs/model_config.yaml。
The architecture in one minute
一分钟了解架构
Before getting into the interesting parts, it helps to know how the pieces fit together. There are three layers:
在深入有趣的部分之前,了解各组件如何协同工作会有所帮助。系统包含三个层次:
- A shell driver (run_fuzzing.sh) that chains the pipeline stages together.
- A set of taskflow YAMLs, one per stage, which are essentially the prompts that tell the LLM agent what to do at each step.
- A set of MCP tools that the agent calls to actually do the work: run AFL, compile a harness, store a crash, read a coverage report, and so on.
- 一个 Shell 驱动程序(run_fuzzing.sh),用于将流水线阶段串联起来。
- 一组任务流 YAML 文件,每个阶段一个,这些文件本质上是提示词,用于告知 LLM 智能体在每个步骤中该做什么。
- 一组 MCP 工具,智能体调用这些工具来实际执行工作:运行 AFL、编译桩代码、存储崩溃样本、读取覆盖率报告等。
The design rule I cared about most is a clean separation of responsibility: the LLM agent owns the decisions, and the MCP tools own the execution. The agent decides what to fuzz, what harness to write, and what coverage gap to chase next. The tools just expose primitives like run_afl_for or compile_harness. The agent never calls AFL or clang directly; it composes the pipeline out of these building blocks. All the state lives in a SQLite database (fuzz_context.db), so the stages never hand data to each other in memory, only through the database.
我最关心的设计原则是职责的清晰分离:LLM 智能体负责决策,MCP 工具负责执行。智能体决定要模糊测试什么、编写何种桩代码以及接下来要追踪哪个覆盖率缺口。工具仅暴露诸如 run_afl_for 或 compile_harness 等原语操作。智能体从不直接调用 AFL 或 clang;它通过组合这些构建块来搭建流水线。所有状态都保存在 SQLite 数据库(fuzz_context.db)中,因此各个阶段不会在内存中互相传递数据,而是仅通过数据库进行交互。
One small but important detail: each harness is built twice. AFL’s edge instrumentation is great for guiding the fuzzer but useless for human-readable coverage reports. So every harness becomes both a .afl binary (built with afl-clang-lto -fsanitize=address,undefined) and a .cov binary (built with clang -fprofile-instr-generate -fcoverage-mapping). The .afl binary does the fuzzing; the .cov binary replays AFL’s queue afterwards to produce real source-line and branch coverage.
一个虽小但重要的细节:每个桩代码都会构建两次。AFL 的边缘插桩技术非常适合引导模糊测试器,但对于人类可读的覆盖率报告则毫无用处。因此,每个桩代码都会生成两个版本:一个是 .afl 二进制文件(使用 afl-clang-lto -fsanitize=address,undefined 构建),另一个是 .cov 二进制文件(使用 clang -fprofile-instr-generate -fcoverage-mapping 构建)。.afl 二进制文件用于执行模糊测试;.cov 二进制文件随后会重放 AFL 的队列,以生成真实的源码行和分支覆盖率。
The coverage-feedback loop
覆盖率反馈循环
This is the heart of the whole pipeline, and it’s the part that most directly automates the manual workflow I described at the start.
这是整个流水线的核心,也是最直接自动化我在开头所述手动工作流程的部分。
If you have ever tried to improve fuzzing coverage by hand, you’ll know that it’s an iterative process that looks like this:
如果你曾经尝试过手动提升模糊测试的覆盖率,你就会知道这是一个迭代过程,大致如下:
The “check the coverage” step used to be completed by me, manually reading an LCOV report looking for uncovered branches. The “improve the coverage” step was also completed by me, this time, writing a new harness or crafting a new input. The Fuzzing Taskflow hands both of those steps to the agent.
“检查覆盖率”这一步过去由我手动完成,即阅读 LCOV 报告并寻找未覆盖的分支。“提升覆盖率”这一步也由我完成,这次是编写新的桩代码或构造新的输入。Fuzzing Taskflow 将这两个步骤都交给了智能体。
Each iteration, for each harness, the agent runs AFL for a time budget, replays the queue against the .cov binary to get a real coverage report, and then reads the list of uncovered branches. Based on what it finds, it picks one of a handful of actions:
每次迭代中,针对每个 harness,代理会运行 AFL 一个时间预算,将队列重放至 .cov 二进制文件以获取真实的覆盖率报告,然后读取未覆盖分支的列表。根据发现的内容,它会从少数几个动作中选择一个:
- Add a new seed crafted to reach an uncovered branch
- Edit the harness source to call an additional API
- Auto-enrich the AFL dictionary with the magic constants a guard is comparing against
- Simply skip the gap if it’s a cold error path or vendor code that isn’t worth chasing
- 添加一个专门构造用于到达未覆盖分支的新种子
- 编辑 harness 源代码以调用额外的 API
- 使用守卫(guard)正在比较的魔法常量自动扩充 AFL 字典
- 如果它是冷错误路径或不值得追踪的厂商代码,则直接跳过该间隙
The time budgets double every iteration:
时间预算在每次迭代中翻倍:
30s → 60s → 120s → 240s → 480s → 960s (≈ 32 min/target)
30s → 60s → 120s → 240s → 480s → 960s(≈ 32 分钟/目标)
The idea is to spend cheap, short rounds early (when there’s lots of low-hanging coverage to grab) and longer rounds later (when the fuzzer needs more time to break through a hard guard).
其理念是在早期花费廉价、短暂的轮次(此时有大量唾手可得的覆盖率可获取),而在后期进行更长的轮次(此时模糊测试器需要更多时间来突破困难的守卫条件)。
And just like in my manual workflow, I need an answer to the question: when do we stop? Here, the loop uses plateau detection: once two consecutive iterations each gain less than a configurable threshold (1% absolute line coverage by default), the loop decides it has hit diminishing returns and moves on. This keeps the agent from burning hours of compute squeezing out the last fraction of a percent.
就像在我的手动工作流中一样,我需要回答一个问题:我们何时停止?在这里,循环使用平台期检测:一旦连续两次迭代获得的增益均低于可配置的阈值(默认为绝对行覆盖率 1%),循环便判定已触及收益递减点并继续执行。这可以防止代理浪费大量计算资源去挤出最后那一点点百分比。
Structure-aware fuzzing
结构感知模糊测试
AFL’s default byte-level mutators (bit flips, arithmetic, block splicing) do a great job on binary formats but struggle with structured, text-based inputs. The classic solution is to hand-write custom mutators for each format, which is tedious work. This time I want the pipeline to do that work for me, so it ships four complementary mechanisms for producing structure-aware inputs.
AFL 默认的字节级变异器(位翻转、算术运算、块拼接)在二进制格式上表现优异,但在处理结构化、基于文本的输入时却力不从心。经典的解决方案是为每种格式手写自定义变异器,这是一项繁琐的工作。这次我希望流水线能替我完成这项工作,因此它提供了四种互补的机制来生成结构感知的输入。
1. Per-format dictionaries and custom mutators. For targets whose input format is recognized (JSON, XML, regex, PNG, length-prefixed binary TLV), the taskflow ships pre-built AFL dictionaries and LLVMFuzzerCustomMutator C files. The JSON mutator does token splicing and balanced-bracket duplication; the XML one knows about tags, entities, and billion-laughs tokens; the regex one carries real ReDoS patterns. Each mutator delegates half of its mutations back to AFL’s default byte mutator, so we keep the engine’s randomization instead of fighting it.
1. 每格式字典和自定义变异器。对于输入格式被识别的目标(JSON、XML、正则表达式、PNG、长度前缀二进制 TLV),taskflow 提供了预构建的 AFL 字典和 LLVMFuzzerCustomMutator C 文件。JSON 变异器执行令牌拼接和平衡括号复制;XML 变异器了解标签、实体和十亿笑话(billion-laughs)令牌;正则表达式变异器包含真实的 ReDoS 模式。每个变异器将其一半的变异委托给 AFL 的默认字节变异器,从而保留引擎的随机性,而不是与之对抗。
2. A source level dictionary. For formats the pipeline doesn’t recognize, it generates a custom mutator on the fly by scanning the target’s own .c/.h files. It extracts string literals and 32-bit numeric constants (from #define, case, and enum), filters out the noise, and uses them as splice tokens. The intuition is simple: the most interesting magic values that a parser checks for are usually written down somewhere in its own source.
2. 源码级字典。对于流水线无法识别的格式,它通过扫描目标自身的 .c/.h 文件来动态生成自定义变异器。它会提取字符串字面量和 32 位数值常量(来自 #define、case 和 enum),过滤掉噪声,并将它们用作拼接令牌。其直觉很简单:解析器检查的最有趣的魔法值通常都写在它自己的源码中的某个地方。
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力