构建模拟凭证泄露传播的Agent并验证记忆机制有效性
I built an agent that simulates credential-leak “outbreaks” and plans containment, then tested whether giving it memory actually helps (+7 points on held-out coverage)
提供了一套完整的AI安全Agent工程实践,从模拟数据生成到图神经网络训练及记忆机制的效果量化,参数与流程清晰可复现,对从事Agent安全与自动化响应的开发者极具参考价值。
Built this for the Hindsight “AI Agents That Learn” hackathon.
为 Hindsight “AI Agents That Learn” 黑客松构建。
The problem: when an API key leaks, scanners tell you it leaked. They don’t tell you which sessions to revoke, which OAuth grants to remove, or how much of production you’ll break doing it. The usual response is “rotate the most connected credentials”, which is slow, disruptive, and often leaves live sessions alive.
问题所在:当 API 密钥泄露时,扫描器只会告诉你它已泄露。它们不会告诉你应撤销哪些会话、移除哪些 OAuth 授权,或这样做会对生产环境造成多大破坏。通常的应对措施是“轮换连接度最高的凭据”,但这既缓慢又具有破坏性,且往往会导致活动会话仍然存活。
What I built (AEGIS//Contagion):
我构建的内容(AEGIS//Contagion):
Treats a leaked credential as an outbreak. It runs 2,600 Monte Carlo simulations over a credential/workload/resource graph. For the leaked AI key in my demo: R of 1.39, 58 min median time to the crown-jewel asset, 13% reach probability within a 90 min horizon.
将泄露的凭据视为疫情爆发。它在凭据/工作负载/资源图上运行 2,600 次蒙特卡洛模拟。在我演示中泄露的 AI 密钥案例中:R 值为 1.39,到达核心资产的中位时间为 58 分钟,在 90 分钟时间窗口内的触及概率为 13%。
Plans a portfolio of cuts under a disruption budget instead of a ranked list. For that incident: revoke sessions on 3 credentials, remove one OAuth grant, enforce step-up MFA. This gave 95% coverage at 7.2% disruption and $850 cost, and cut 15/15 dominant paths.
在干扰预算下规划一系列切断措施,而非仅生成排名列表。针对该事件:撤销 3 个凭据上的会话,移除一个 OAuth 授权,并强制实施逐步升级的多因素认证(MFA)。这实现了 95% 的覆盖率,同时干扰仅为 7.2%,成本为 850 美元,并切断了全部 15 条主导路径。
A 2-layer relation-aware GraphSAGE (26.7K params, NumPy, trained from scratch, no pretrained weights, no inference API) learns infectivity from the simulations. On a held-out estate: node AP 0.828 (base rate 0.425), edge AP 0.88, ECE 0.13 after temperature scaling.
一个两层关系感知的 GraphSAGE 模型(26.7K 参数,NumPy 实现,从头训练,无预训练权重,无推理 API)从模拟中学习传染性。在保留测试集上:节点 AP 为 0.828(基准率为 0.425),边 AP 为 0.88,温度缩放后 ECE 为 0.13。
Memory runs on Hindsight (local). It stores episodes, facts and analyst directives. Recalled memory changes the plan: observed transmissions override edge priors, and directives like “prefer revoking sessions before rotation” become constraints.
记忆系统基于 Hindsight(本地部署)。它存储经验片段、事实和分析师指令。被召回的记忆会改变计划:观察到的传播行为会覆盖边的先验概率,而诸如“优先撤销会话而非轮换”之类的指令则成为约束条件。
Does memory actually help? I replayed 6 scripted incidents. For each one, a cold plan and a warm plan use the same optimiser, same budget, same held-out outbreaks. Only memory differs.
记忆真的有帮助吗?我重放了 6 个脚本化事件。对于每个事件,冷启动计划和热启动计划使用相同的优化器、相同的预算和相同的保留测试集。唯一的区别在于是否有记忆。
Warm: 97% coverage. Cold: 90%. That’s +7.0 points. Naive top-k rotation: 16%.
热启动:覆盖率 97%。冷启动:覆盖率 90%。提升了 7.0 个百分点。简单的 Top-k 轮换策略:覆盖率仅 16%。
Per episode: +23, +15, +8, +0, +5. Memory helped most where the cold planner struggled, and did nothing on the incident it already solved perfectly. I’m reporting the zero on purpose.
各事件提升幅度分别为:+23、+15、+8、+0、+5。记忆在冷启动规划器表现不佳的场景中帮助最大,而在它已完美解决的事件中毫无作用。我特意报告了那个零值。
Caveats, because they matter:
注意事项,因为它们很重要:
The estate topology is synthetic, and the model’s labels come from my own simulator. So the AP numbers show it learned the simulator’s dynamics, not that it predicts real breaches.
资产拓扑是合成的,模型的标签来自我自己的模拟器。因此,AP 数值表明它学会了模拟器的动态特性,而非预测真实的入侵事件。
Six scripted incidents is a small sample.
六个脚本化事件的样本量较小。
It’s read-only and needs human approval. It doesn’t replace scanners or policy checks.
该系统为只读模式,需要人工批准。它不能替代扫描器或策略检查。
Repo: [https://github.com/shabrishraj0-lab/aegis-contagion\]
代码仓库:[https://github.com/shabrishraj0-lab/aegis-contagion\]
Demo video: [https://drive.google.com/file/d/1yO0KsM47Ng3Eltza3JvdlFJf7lx2qmXB/view\]
演示视频:[https://drive.google.com/file/d/1yO0KsM47Ng3Eltza3JvdlFJf7lx2qmXB/view\]
Feedback I’d love: does the “portfolio under a disruption budget” framing match how your IR team actually thinks about containment? And is a simulator-derived label a fair way to evaluate this, or is there a better option without real incident data?
我希望得到的反馈是:"在破坏性预算下的投资组合"这一框架是否与您所在IR团队实际思考风险管控的方式相符?此外,基于模拟器生成的标签是否是评估此方法的合理方式,或者在没有真实事故数据的情况下是否存在更好的替代方案?
更进一步:量化金融体系
看懂新闻只是起点——沿量化金融路径,把它变成能交付的工程能力