{"id":47004292,"title":"LangChain Agent Testing Guide Tool (Free)","points":1,"user":"exordex","time":1770998845,"time_ago":"6 hours ago","type":"link","content":"

Hey HN,\n If you're building LangChain agents, you've probably seen them break in creative ways - prompt injection bypassing your chain logic, tools getting called with bad parameters, or cascading failures when an API times out mid-chain.

I built Khaos to test these failure modes before production.

Example LangChain agent:\n ```python\n from langchain.agents import AgentExecutor, create_openai_functions_agent\n from khaos import khaosagent

  @khaosagent(name="research-agent", framework="langgraph")\n  def agent(query: str) -> dict:\n      executor = AgentExecutor(agent=agent, tools=tools)\n      result = executor.invoke({"input": query})\n      return {"response": result["output"]}\n

\nTest it:\n pip install khaos-agent\n khaos discover\n khaos run research-agent --pack security

Khaos injects:\n - 242+ security attacks - Prompt injection variations that bypass LangChain's prompt templates\n - Tool misuse - Malicious parameters in tool calls (e.g., os.system injection in code execution tools)\n - Chain failures - What happens when your 3rd step in a 5-step chain times out?\n - LLM faults - Rate limits, token overflows, model unavailability

  Why this matters for LangChain specifically:\n\n  LangChain's abstraction layers can hide vulnerabilities:\n  - Prompt templates can still be injected via tool outputs\n  - AgentExecutor doesn't validate tool parameters\n  - Chains fail silently or propagate corrupted state\n  - ReAct/Plan-and-Execute patterns have unique attack surfaces\n\n  Works with LangGraph, LCEL chains, and classic LangChain agents. Auto-instruments your chains to inject faults at each step.\n\n  Repo: https://github.com/ExordexLabs/khaos-sdk\n  Examples: https://github.com/ExordexLabs/khaos-examples/tree/master/code-execution-agent

","url":"item?id=47004292","comments":[],"comments_count":0}