AI agents just automated the pentesting pipeline, and the open source version comes with a kill switch

Penetration testing has always been a labor-intensive craft, requiring hours of manual reconnaissance, careful tool orchestration, and methodical note-taking. A new open source project called VulnClaw aims to compress that workflow into a single natural language interaction, using an LLM agent that decides the next step, calls the appropriate tools, and records every finding as structured evidence. The approach fits into a broader shift: as more security teams explore AI-driven automation, incidents like the Hugging Face breach show what happens when an autonomous agent goes rogue, according to the first AI-on-AI attack blueprint.

What VulnClaw does

At its core, VulnClaw is a CLI tool that accepts plain language instructions, "pentest http://target.example.com", and then runs through a four-stage cycle: information gathering (fingerprinting, port scanning, directory enumeration), vulnerability discovery (injection points, known CVEs, misconfigurations), exploitation (PoC validation, privilege escalation), and report generation (structured Markdown plus a Python PoC script). The default engine is what the authors call a "model-led solve loop": the LLM decides which tool to invoke next, when to ask a clarifying question, and when to declare the task complete. This kind of autonomous decision-making is exactly what made the first fully AI-run ransomware attack possible, as Sysdig researchers documented.

This is not a fixed playbook. The agent supports 13 LLM providers, including OpenAI, Anthropic, DeepSeek, and several Chinese models like MiniMax, Zhipu, and Qwen, and switches between them via a single config command. It also integrates four MCP services: a built-in fetch tool for HTTP requests, a memory service for state persistence, and optional connections to Chrome DevTools (for browser automation) and Burp Suite (for HTTP interception and replay).

Evidence memory and the anti-hallucination gate

One design choice that sets VulnClaw apart from simpler scripted agents is its evidence layer. Every tool output is written to an AgentState.evidence store, with raw content retained alongside a hash and size. The active context that the model sees only contains a high-signal preview, response headers, form parameters, endpoint URLs, sink variables, flag-like tokens, while the full body stays available for on-demand retrieval via dedicated evidence_search and evidence_view tools. This gating mechanism addresses a core weakness: AI's inability to self-correct when it gets confident but wrong. A new benchmark on hierarchical reasoning recently showed that even top agents miss the mark badly, as the HSCodeComp benchmark revealed.

The anti-hallucination gate is the most explicit safeguard. A claimed flag or conclusion is only accepted if it appears character-for-character in a real tool output, or if it cites an evidence entry by number. If the model tries to declare victory based on a guess, the gate rejects the FINAL signal and sends the reason back to the model for continued exploration. A "near-success guard" also prevents the model from giving up too early when high-signal anchors (source code sinks, form inputs, response differences) remain unexhausted.

Built-in skill library and tool ecosystem

The tool ships with 50 specialty skills organized into 7 core and 16 domain-specific categories, covering web pentesting, Android app testing, client-side reverse engineering, CTF challenges, OSINT, and more. These skills are not injected as mandatory scripts; they serve as optional reference indices. The model must explicitly call load_skill_reference to read them, keeping the active prompt clean and avoiding the context bloat that plagues many LLM agents.

On the tooling side, VulnClaw bundles 29 encode/decode/cryptography operations (Base64, AES, JWT, Morse, and others), a batch HTTP probe tool for testing multiple URL/parameter variants in one call, a runtime diff probe for filter-bypass scenarios, and both a shell_command and a python_execute tool for local validation and payload construction. All tools write their outputs into the same evidence store, and a lightweight correction layer tracks repeated calls, failures, timeouts, and new findings to inject modest feedback signals into subsequent rounds.

Deployment and configuration

Installation is straightforward: pip install vulnclaw, set an API key, pick a provider, and run. A Docker image with the Web UI and MCP runtimes pre-installed is available. The project also includes a TUI workspace (vulnclaw tui) that displays authorized targets, scope boundaries, and runtime overview before launching a task. For persistent deep testing, a periodic mode runs 100 rounds per cycle for up to 10 cycles, generating an automatic report after each cycle.

Security warnings are conspicuous in the documentation. The project is labeled as a public Alpha intended only for authorized testing, CTF environments, and controlled research scenarios. The README warns explicitly that unauthorized penetration testing is illegal and that the authors assume no liability for misuse. With AI-powered attacks on the rise, from autonomous phishing to OAuth token theft, the stakes for responsible deployment have never been higher, as the rising tide of AI-driven phishing illustrates.

Why this matters for the security community

VulnClaw occupies a specific niche between fully manual pentesting and rigid vulnerability scanners. By letting a language model orchestrate the workflow, it can adapt to unexpected server responses, chain multiple vulnerabilities, and write custom PoCs, tasks that traditional scanners handle poorly. At the same time, the evidence gating and structured memory address the primary criticism of AI agents in security: that they confidently produce fabricated findings. This lines up with broader evidence that AI agents are rewriting the rules of cybersecurity, as our coverage of autonomous defense systems shows.

The tool is not mature enough for production use. The authors are upfront about its Alpha status, but it demonstrates a promising direction. Security teams already drowning in alert fatigue and manual triage may find that an AI agent, if properly constrained, can handle the repetitive parts of a pentest while leaving the judgment calls to a human operator. The next challenge will be proving that the anti-hallucination gates actually hold under real adversarial conditions.