AI pentesting 2025 / 2026 with open source tools

I am sure you also received a cold sales call or two. AI pentesting as a Service. Is this real or hype?

In the following, I test 5 different open source AI pentest tools and how they perform against a purposely vulnerable Java Web App.

While 2025 has been a boring year in InfoSec, it has been exciting for AI. Financially, at least. Currently, most AI services (Anthropic, Google) struggle with availability, basic software features, or UX. Or, in the case of OpenAI, their models are just not good enough anymore. The AI industry has problems.

2026 many AI ventures will go bankrupt. For a sustainable AI strategy, open-source and open-weight models are key. Because you cannot rely on a single vendor's performance. It's a very dynamic environment. Not only because of the investment activity.

When it comes to AI pentesting, it's best to do this as part of an Application Security (AppSec) program and to build up (AI) expertise in-house. This allows for dedicated and customized solutions to build up resilience and to support security processes.

In the following, I only use Gemini, but that's due to the holiday season 🙃. There are trained pentesting models, which will perform even better.

Summary and results

As a decision maker, there are three key insights to consider:

1. You (and everyone else) can get open-source AI security tools, which will exploit known and unknown vulnerabilities.
2. If the vulnerability is known, your time to patch must be in minutes if the systems are important. You do not have hours for customer / web-facing systems.
3. These are not toys. There is a billion-dollar investment in AI, which accelerates the overall playing field beyond technology. In security, finance, or even logistics. Cars, machines, or power grids are not exempt from macro-trends like security or AI. It's a changing force with a massive push behind it.

In this AI pentest tools test, the winner is LuaN1ao (鸾鸟), because it's highly sophisticated and intelligently designed to pentest (real-world) systems.

The code-review capabilities of Raptor can help to prevent security problems, like intrusions with ransomware, data theft, intellectual property exfiltration, etc. Security architecture investments like Zero Trust are key to moving forward now. Without security, AI can become everyone's Nokia moment.

AI pentesting with open source

I tested 5 different tools:

1. Hexstrike AI with Gemini CLI (MCP server) on Gemini 3 and 2.5 Pro (mixed)
2. LuaN1ao on Gemini 2.5 Pro (CLIProxyAPI)
3. EVA on Gemini 2.5 Pro (CLIProxyAPI)
4. Pentestagent on Gemini 2.5 Pro (CLIProxyAPI)
5. Raptor on Gemini 2.5 Pro (CLIProxyAPI)

The results are exciting and remind me of the era of Core Impact or Metasploit db_autopwn. The AI pentest tools would likely pass a CEH (Certified Ethical Hacker) exam. This also means that this goes beyond Tenable Nessus or Rapid7 InsightVM vulnerability scans.

In a pentest with live production systems, I would always make sure that there is a human in the loop. Some AI pentest tools are not designed for this.

The following tests focus on Marathon, a vulnerable Java web application for educational purposes. It is hosted in a dedicated security lab and not exposed to the public internet. Of course, it is much easier to pentest than a production-grade web app. The results, however, are indicative of what happens in the real world.

Which means: be responsible. Emphasize and communicate the facts, impacts, and risks. The time to communicate is now.

Hexstrike AI (v6.0)

Advanced AI-powered penetration testing MCP framework with 150+ security tools and 12+ autonomous AI agents

Hexstrike automates pentests with AI tools, based on existing and time-proven tools. It uses 100s of powerful tools, depending on the target. In a web app pentest against the vulnerable Java application Marathon, it used

• nuclei, nikto, nmap
• sqlmap
• and a couple of other tools

I used a Kali Linux (formerly known as Backtrack) host, given the amount of pentesting tools that are supported.

The results are valid, and the reporting makes sense. The tool is fast, and its output is reproducible. But you need to read the details (takes time), and you need to pay attention to your system.

LuaN1ao

LuaN1ao (鸾鸟) is an autonomous penetration testing agent based on state awareness and causal reasoning.

LuaN1ao is a sophisticated AI pentest tool written in Chinese. I translated most of the prompts to English with Gemini CLI (Gemini 3) and configured it to use CLIProxyAPI (Gemini 2.5 Pro). The results are excellent, but you have to be skilled to use this (MCP client-server architecture, graph-based documentation, maybe a little Chinese).

The translated version can be used by Western testers.

LuaN1ao will attempt to use Ghidra (or other tools) for decompilation and zero-day development. Given enough time, you will get a tested exploit. If you use it with CLIProxyAPI you can run it overnight on your subscription quota (20 bucks per month).

I have read that many people cycle free-tier Google accounts as well. Costs will not be a concern, because you can also use DeepSeek, MiniMax or Kimi. And these can be self-hosted. I'd also look at HuggingFace for special models (Dolphin maybe).

EVA

EVA is an AI-assisted penetration testing agent that enhances offensive security workflows by providing structured attack guidance, contextual analysis, and multi-backend AI integration.

EVA (Exploit Vector Agent) is much simpler but has good prompts to work with security tools like nikto, nmap or msfconsole .

The UI is rather minimal... and sadly, so are the results. EVA has an annoying disclaimer. In this test it failed completely and reverted to using security techniques from the early 2000s. Tried to hack the web server, looked for weird Perl scripts from the dot-com era 😏

Pentestagent

PentestAgent is an AI agent framework for black-box security testing, supporting bug bounty, red-team, and penetration testing workflows.

For this tool I used the docker compose installation on Kali. It's similar to Hexastrike, and in the test it used

• ZAP
• nuclei
• nikto
• gobuster

I needed to switch to the Kali-based container because the other Docker environment doesn't have the tools.

Pentestagent can generate PoC code for XSS and SQLi or guess weak passwords. Sadly, it selects time-consuming strategies (password brute-forcing) without asking the user. You also cannot stop such agent tasks or intervene to focus them on actual vulnerabilities.

The tool is well designed, and with slight modifications, it's probable that you can get excellent results.

Raptor

Raptor turns Claude Code into a general-purpose AI offensive/defensive security agent. By using Claude.md and creating rules, sub-agents, and skills, and orchestrating security tool usage, we configure the agent for adversarial thinking, and perform research or attack/defense operations.

Raptor is not a black box pentest tool but an offensive source code analyzer that will create PoC code. It has a couple of claude features (which I tested with CLIProxyAPI) on Gemini 2.5 Pro. Raptor delivers results in minutes (Java code assessment).

Raptor makes a point: vulnerable code needs to be patched, and Raptor delivers a patch as well. This is useful for developer training as well as to reduce the time to patch. Before AI, tools like GitHub Enterprise Security or Veracode sold this feature for very high prices.

Final words

Most people will not have the skills to compile and run these tools. Of course, again, AI tools can help here, but it still requires patience and knowledge. AI is a self-fulfilling prophecy of disruption.

I am not overly concerned that script kiddies will weaponize AI. Firstly, these AI tools rely on pentesting tools that have existed for decades. You can achieve similar results if you use Nessus well enough, which has also existed since the 2000s. Secondly, the tools require a human in the loop to be effective enough to cause damage. You need to steer the tools actively to exploit vulnerable systems. Thirdly, it still takes a lot of time and effort to get these tools to create zero-days. In 2016 we saw DARPA's CGC (Cyber Grand Challenge), where computers generated exploits. Now, roughly 10 years later, this type of tech becomes generally available without the need for a doctoral degree.

I am concerned that the darknet has much more sophisticated tools and that a large part of the industry is still in deep slumber. If everyone all of a sudden got their hands on high-end pentesting skills, that's a game changer. And that time will come when the tools get easier and AI skills become more widespread.

Technology is always a disruptive element. Without the right strategy, being left behind isn't the worst that can happen. 2025 was the year to define a strategy; 2026 is the year to execute it.