AI Hackbots: My Tips, Tools, and Hard-Earned Lessons, Pt 1

What I’ve learned building AI-powered attack workflows (so far)

Hey everyone!

I’ve been spending real hours building and breaking AI hackbots, and I want to get some notes down while the pain and the wins are still fresh. If you’re experimenting with LLM-driven agents, this could save you some dead ends. I’ll walk through how I think about context engineering, how to scope agents so they don’t spiral, when to lean on automation versus AI-led analysis, the stack of tools and data sources I actually use, and some videos that shaped my approach. I want to give a heavy shout-out here to my brother Daniel Miessler as many of these we have discussed over hundreds of calls and coding sessions.

The TL;DR

• Context engineering is key. Research terms, clean RAG, structured formats matter.

• Agents need a tight, single-mission scope or they burn tokens and hallucinate.

• Automate the repeatable stuff. AI is where analysis or synthesis matters.

• My stack is open tools plus curated data feeds (reports, fuzz strings, RAG).

• I’ve got 3–5 essential videos for you from awesome people.

/ Context Engineering in the Wild

Context engineering is what makes or breaks a hackbot. The way you design the setup, not just the ask, determines whether the AI stays useful or drifts. Think of it like building a briefing packet for an intern: give them tight context, examples, and format, and you’re golden. Be vague, and they’ll return slop.

// Related Research Terms Trick

When targeting a domain, say Kerberos attacks, I seed agent system prompts with a section called “Related Research Terms” like "TGT, AS-REQ, golden ticket, NTLM relay, pass-the-hash." That narrows the model’s attention mechanism. These aren’t things to act on but rather guide the model to the right mental neighborhood. In fact I do this for almost every system prompt I write for any AI, here's an example of a general one for SSRF:

// Clean RAG

Retrieval-augmented generation is powerful, but raw internet hits add noise. Instead, I build clean, curated RAG packs customized per agent:

• Formatted reports: Bug bounty write-ups, blog dissections, pentest findings, all normalized into a consistent schema, sometimes even using a standalone AI to clean them up.

• Targeted fuzz strings: Suppose the agent hunts SQLi or buffer overflow flaws. I feed it known fuzz patterns as context so it recognizes what to test for.

Each hackbot agent gets its own knowledge pack instead of grazing the whole web. This tightens relevance and improves consistency. More on sources for this later…

// DSPy—Context Engineering, Programmatic-Style

There’s a new library emerging for programmatic context engineering: DSPy (Declarative Self-improving Python). It’s a modular framework where you define Signatures, Modules, and Optimizers, and DSPy generates and refines the prompt logic for you. Think of it as moving from hand-rolled strings to structured code. It supports RAG pipelines, ReAct-style reasoning, and has built-in optimizers that self-tune weights.

• DSPy site: https://dspy.ai

• Intro article: https://ritikjain51.medium.com/dspy-a-framework-for-programming-and-foundation-models-fe9d78d0af03

Ads Dawson wrote a killer Bugcrowd article on using DSPy for AI hackbots. He shows how DSPy can make hackbot agents more structured, less brittle, and easier to evolve.

• https://www.bugcrowd.com/blog/hacking-llm-applications-in-the-trenches-with-dspy/

// Best Practices That Stick

• Examples beat long instructions. Input to output pairs anchor behavior really well. Some context engineering best practices say include up to 10 examples of input and output pairs that show how you would ideally like the AI to respond to common requests.

• Define your output first. JSON schema, section headers, bullet templates, clean structure makes chaining and consumption predictable.

• Guardrails prompting help. “Do not summarize, only extract IPs” works better than hoping for discipline. Same as a winner like “analyze and cite all your sources when suggesting a finding” and “for each finding explain your reasoning and chain of thought for suggesting the finding is valid”.

(Sponsor)

/ Scoping Your Agents

Agents are like interns. If you don’t give them a one-sentence job, they’ll try to boil the ocean. Stick to the single-mission rule: one agent, one job. ENUM agent, TRIAGE agent, REPORT agent. Don’t build a mega agent that tries to own the whole chain.

// Automation vs AI-Led Analysis

Automate the predictable: general and easy directory enumeration, content discovery, port scanning, parsing JSON, API calls, anything repeatable. It’s cheaper and faster. AI shines when you need analysis, interpretation, or synthesis. The only caveats here are very specific AI agents like ones that parse JavaScript which I found to be a particular strength of AI agents, so I created a standalone one for that.

TLDR; Don’t replace automation with AI. Bolt AI on top: run the script, send clean structured output over, and let the model explain or prioritize, but don’t use it for raw grunt work.

// Micro-Scoping Vulnerability Agents

The biggest wins come from micro-scoping. Don’t build a “SQL injection bot.” Build an error-based SQLi agent, a blind SQLi agent, and a union-based SQLi agent. Each one has its own curated context pack: fuzzing strings, bug bounty write-ups, and formatted reports specific to that injection style.

By narrowing scope and tuning context, accuracy goes way up. The agent isn’t guessing across too much ground, it’s hunting in a single lane. The same principle works for XSS: split into DOM-based, stored, and reflected agents, each carrying its own focused dataset. In practice, this is the only way I’ve seen AI hackbots evolve past demos and into tools people actually trust.

/ Tools and Data Sources

When building your AI hackbot architecture, there are several ways you can go. I started off with LangChain but then moved over to Claude Desktop and eventually Claude Code with MCPs and Agents. I've seen people build successful systems in n8n, and have done a few agents in there as well. Two foundational Tools in my opinion are the MCPs for Puppeteer and playwright, to execute browsing and Analysis of web code

// Data Sources

• Exploit-DB / Vulners feeds: Raw fuel.

• HackerOne & Bugcrowd disclosures: Clean bounty write-ups for context.

• Wordlists (SecLists, PayloadsAllTheThings ): Still gold for enumeration.

• Research blogs and research presentations.

• Synthetic Data (note): in some cases for vulnerabilities that are very esoteric I used AI to create synthetic data (reports) that didn't exist but looked like the vulnerability the agent was searching for, in order to confidently tune the ai's attention mechanism to find that vuln. This is not a fully automated process and requires a lot of manual tuning to build good synthetic data.

Pro tip: always sanitize. I run all data through scripts that clean, dedupe, and tag before the model sees anything.

/ A Few Must-Watch Videos + Resources

Here few of the bookmark videos that I definitely learned a lot from. That being said I didn't implement in my systems all of the recommendations inside of these talks… take the ones that work for you and leave the rest.

I also highly suggest researching all of the code and presentations done by any of the teams in the AIxCC DARPA challenge …

/ Caveats and Pitfalls

• Agents running too long equals hallucinations and token burn. Use kill switches.

• Never trust output blindly, always verify.

• Models without Context Engineering are not even 50% of the way to being an effective hackbot. Prompt engineering and Context Engineering are paramount skills in building these AIs.

• Over-engineered stacks collapse under their own weight. Keep things lean and testable.
  

/ On Human-in-the-Loop

Even my own success with my hackbot finding bounties and implementing all of the techniques above has in some cases required me stopping the workflow of the bot and guiding it using human in the loop prompting. Instances where I see a juicy named function, and have to tell the bot to zero in on it, or something like that happen all the time. Usually when I do that the bot is able to execute its vulnerability workflow but sometimes it takes a human's corpus of experience to get the best out of the system. Additionally some of my best micro agents, are not crafted from write-ups that other researchers have produced but from my long-standing experience that's never been documented anywhere. An example of this is a list of vulnerabilities in Enterprise hosted ServiceNow that I built into an AI agent that none of those tricks are available online so I had to build that agent prompting by hand.

/ Outro

To be honest there's so many more tips and tricks I want to share but it would just make this newsletter gigantic. Be on the lookout for part two sometime soon, as AI hackbots are evolving fast.

The fundamentals are clear. Scope them tight, automate the bones, use AI for judgment + deep analysis, and design context intentionally, whether by hand or with frameworks like DSPy. This stack isn’t perfect, but it’s iterative.

I’ll keep sharing my mistakes and discoveries, and I want to see what you’re building too.

Happy hacking 😎
󠅘󠅙󠅔󠅔󠅕󠅞󠅔󠅑󠅤󠅑󠄢󠄠󠄢󠄥󠅘󠅙󠅔󠅔󠅕󠅞󠅔󠅑󠅤󠅑󠄢󠄠󠄢󠄥