End-to-End Multi-Tool Agent Tutorial
This tutorial walks through a complete testing workflow for a LangGraph agent that uses multiple tools. We will:
Build a small billing support agent with three tools
Mock each tool with StuntDouble using input matchers and dynamic resolvers
Run the agent end-to-end with
scenario_metadataAssert tool call order and arguments with
CallRecorderin pytestCover a few common edge cases
If you already know the individual pieces, this guide shows how they fit together in one realistic test.
Prerequisites: StuntDouble requires Python 3.12+. This tutorial also assumes you already have a working LangGraph agent node and tool setup.
What We Are Testing
Assume the agent handles a request like:
“Check account CUST-100. If they have overdue bills, create a follow-up invoice.”
At runtime the agent may call:
get_customer(customer_id)list_bills(customer_id, status)create_invoice(customer_id, amount, reason)
In production these tools might hit CRMs, billing systems, or payment APIs. In tests we want:
deterministic responses
no network calls
assertions about which tools were called
confidence that the agent took the expected path
Step 1: Define the Tools
The tools below are intentionally simple. The important part is that they are real LangChain tools wired into a real ToolNode.
from langchain_core.tools import tool
@tool
def get_customer(customer_id: str) -> dict:
"""Fetch a customer record from the CRM."""
raise NotImplementedError("Production CRM call")
@tool
def list_bills(customer_id: str, status: str = "open") -> dict:
"""List bills for a customer."""
raise NotImplementedError("Production billing call")
@tool
def create_invoice(customer_id: str, amount: int, reason: str) -> dict:
"""Create a follow-up invoice."""
raise NotImplementedError("Production invoice call")
Step 2: Build a LangGraph Agent
StuntDouble plugs into LangGraph through ToolNode(..., awrap_tool_call=wrapper). That means your graph stays production-shaped.
from langgraph.graph import START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode, tools_condition
from stuntdouble import (
CallRecorder,
MockToolsRegistry,
create_mockable_tool_wrapper,
validate_registry_mocks,
)
tools = [get_customer, list_bills, create_invoice]
registry = MockToolsRegistry()
recorder = CallRecorder()
# Data-driven mocks read cases from scenario_metadata["mocks"][tool_name]
registry.register_data_driven("get_customer")
registry.register_data_driven("list_bills")
registry.register_data_driven("create_invoice")
wrapper = create_mockable_tool_wrapper(
registry,
recorder=recorder,
)
builder = StateGraph(MessagesState)
builder.add_node("agent", agent_node) # Your LLM/planner node; see langgraph-integration.md for a full example.
builder.add_node("tools", ToolNode(tools, awrap_tool_call=wrapper))
builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", tools_condition)
builder.add_edge("tools", "agent")
graph = builder.compile()
Why register_data_driven() here?
It keeps the tutorial focused on test scenarios instead of custom factory code
It supports matcher-based conditional responses
It resolves placeholders like
{{input.customer_id}},{{now}}, and{{sequence('INV')}}
Step 3: Define a Multi-Tool Scenario
Now we provide one scenario that mocks all three tools. Each tool can have multiple cases.
from stuntdouble import inject_scenario_metadata, validate_registry_mocks
scenario_metadata = {
"scenario_id": "billing-agent-overdue-flow",
"mocks": {
"get_customer": [
{
"input": {"customer_id": {"$regex": "^CUST-"}},
"output": {
"id": "{{input.customer_id}}",
"name": "Acme Manufacturing",
"tier": "gold",
"status": "active",
},
}
],
"list_bills": [
{
"input": {
"customer_id": "CUST-100",
"status": "overdue",
},
"output": {
"bills": [
{"id": "BILL-900", "amount": 1200, "status": "overdue"},
{"id": "BILL-901", "amount": 300, "status": "overdue"},
],
"total_overdue": 1500,
},
},
{
"output": {
"bills": [],
"total_overdue": 0,
}
},
],
"create_invoice": [
{
"input": {"amount": {"$gt": 1000}},
"output": {
"invoice_id": "{{sequence('INV')}}",
"customer_id": "{{input.customer_id}}",
"amount": "{{input.amount}}",
"reason": "{{input.reason}}",
"created_at": "{{now}}",
"status": "pending_review",
},
},
{
"output": {
"invoice_id": "{{sequence('INV')}}",
"customer_id": "{{input.customer_id}}",
"amount": "{{input.amount}}",
"reason": "{{input.reason}}",
"created_at": "{{now}}",
"status": "created",
}
},
],
},
}
errors = validate_registry_mocks(tools, scenario_metadata)
if errors:
raise ValueError(f"Invalid scenario metadata: {errors}")
config = inject_scenario_metadata({}, scenario_metadata)
This single scenario demonstrates a few useful patterns:
get_customeruses a$regexmatcherlist_billshas a specific case plus a catch-all fallbackcreate_invoicereturns different outputs based on the amountplaceholder resolution echoes tool inputs into the response
Step 4: Run the Agent End-to-End
Invoke the compiled graph exactly as you would in production, but pass the scenario through config.
from langchain_core.messages import HumanMessage
result = await graph.ainvoke(
{
"messages": [
HumanMessage(
"Check account CUST-100. If they have overdue bills, create a follow-up invoice."
)
]
},
config=config,
)
If the agent decides to:
fetch the customer
fetch overdue bills
create an invoice for the overdue amount
then all three calls will be intercepted by StuntDouble and recorded by CallRecorder.
Step 5: Assert Tool Behavior in pytest
This is the part many teams want in practice: not just “did the agent return something reasonable?” but “did it use the right tools, in the right order, with the right arguments?”
import pytest
from langchain_core.messages import HumanMessage
from langgraph.graph import START, MessagesState, StateGraph
from langgraph.prebuilt import ToolNode, tools_condition
from stuntdouble import (
CallRecorder,
MockToolsRegistry,
create_mockable_tool_wrapper,
inject_scenario_metadata,
validate_registry_mocks,
)
@pytest.fixture
def recorder():
return CallRecorder()
@pytest.fixture
def graph_with_mocks(recorder):
registry = MockToolsRegistry()
registry.register_data_driven("get_customer")
registry.register_data_driven("list_bills")
registry.register_data_driven("create_invoice")
tools = [get_customer, list_bills, create_invoice]
wrapper = create_mockable_tool_wrapper(
registry,
recorder=recorder,
)
builder = StateGraph(MessagesState)
builder.add_node("agent", agent_node) # Your LLM/planner node; see langgraph-integration.md for a full example.
builder.add_node("tools", ToolNode(tools, awrap_tool_call=wrapper))
builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", tools_condition)
builder.add_edge("tools", "agent")
return builder.compile()
@pytest.mark.asyncio
async def test_overdue_billing_flow(graph_with_mocks, recorder):
scenario_metadata = {
"scenario_id": "pytest-overdue-billing-flow",
"mocks": {
"get_customer": [
{
"output": {
"id": "{{input.customer_id}}",
"name": "Acme Manufacturing",
"status": "active",
}
}
],
"list_bills": [
{
"input": {"status": "overdue"},
"output": {
"bills": [{"id": "BILL-900", "amount": 1200}],
"total_overdue": 1200,
},
}
],
"create_invoice": [
{
"output": {
"invoice_id": "{{sequence('INV')}}",
"customer_id": "{{input.customer_id}}",
"amount": "{{input.amount}}",
"reason": "{{input.reason}}",
"status": "created",
}
}
],
},
}
errors = validate_registry_mocks([get_customer, list_bills, create_invoice], scenario_metadata)
assert errors == {}
config = inject_scenario_metadata({}, scenario_metadata)
await graph_with_mocks.ainvoke(
{
"messages": [
HumanMessage(
"Check account CUST-100. If they have overdue bills, create a follow-up invoice."
)
]
},
config=config,
)
recorder.assert_call_order("get_customer", "list_bills", "create_invoice")
recorder.assert_called_with("get_customer", customer_id="CUST-100")
recorder.assert_called_with("list_bills", customer_id="CUST-100", status="overdue")
recorder.assert_called_once("create_invoice")
invoice_call = recorder.get_last_call("create_invoice")
assert invoice_call is not None
assert invoice_call.was_mocked is True
assert invoice_call.args["customer_id"] == "CUST-100"
assert invoice_call.args["amount"] == 1200
invoice_result = recorder.get_result("create_invoice")
assert invoice_result["status"] == "created"
assert invoice_result["invoice_id"].startswith("INV-")
This test checks:
the workflow path
the arguments used for each tool call
that the invoice tool was mocked rather than executed for real
the actual mock result returned to the agent
Common Patterns
Conditional Responses Per Tool
Use multiple cases ordered from specific to general:
"list_bills": [
{
"input": {"status": "overdue", "customer_id": "CUST-100"},
"output": {"bills": [{"id": "BILL-1"}], "total_overdue": 800},
},
{
"input": {"status": "overdue"},
"output": {"bills": [{"id": "BILL-2"}], "total_overdue": 200},
},
{
"output": {"bills": [], "total_overdue": 0},
},
]
Put your most specific matcher first and leave a catch-all case last.
Dynamic Outputs That Echo Inputs
Resolvers are useful when your assertions should depend on agent-generated inputs:
"create_invoice": [
{
"output": {
"invoice_id": "{{sequence('INV')}}",
"customer_id": "{{input.customer_id}}",
"amount": "{{input.amount}}",
"created_at": "{{now}}",
}
}
]
This makes the mock feel realistic without hard-coding every possible request.
Simulating a Tool Error
For error cases, use a custom mock that raises:
def failing_invoice_mock(scenario_metadata):
def mock_fn(customer_id: str, amount: int, reason: str) -> dict:
raise ValueError("Billing API timeout")
return mock_fn
registry = MockToolsRegistry()
registry.register_data_driven("get_customer")
registry.register_data_driven("list_bills")
registry.register("create_invoice", failing_invoice_mock)
wrapper = create_mockable_tool_wrapper(
registry,
strict_mock_errors=True,
)
With strict_mock_errors=True, the wrapper re-raises mock failures so your test can use:
with pytest.raises(ValueError, match="Billing API timeout"):
await graph.ainvoke(state, config=config)
Allowing Some Tools to Stay Real
If you want a hybrid test where only some tools are mocked:
wrapper = create_mockable_tool_wrapper(
registry,
require_mock_when_scenario=False,
)
That tells StuntDouble to fall back to the real tool when no mock is registered for a tool in the current scenario.
Troubleshooting
“Mock for tool X is registered but its input conditions were not met”
This usually means the tool was mocked, but none of its input cases matched the actual arguments.
Fix it by either:
adding a catch-all case with no
inputrelaxing the matcher
asserting the recorded args to see what the agent actually sent
print(recorder.get_calls("list_bills"))
“No mock registered for tool X”
This happens when scenario_metadata is present and the wrapper is in strict mode, but the registry has no mock for that tool.
Either:
register the missing tool with
register_data_driven()orregister()or set
require_mock_when_scenario=Falsefor hybrid runs
Signature Mismatch Errors
register_data_driven() scenarios are best validated with validate_registry_mocks(tools, scenario_metadata) before invocation.
If you register a custom mock factory with register() instead, then tools=tools and validate_signatures=True are useful for checking that the returned callable matches the real tool signature.
When to Use This Pattern
This end-to-end setup works especially well when:
your agent can take multiple tool paths depending on the prompt
you want one test to verify both final behavior and internal tool usage
you need scenario-specific mock data per invocation
you want fast tests without modifying production graph wiring
If you only need a single static mock, the Quickstart is simpler. If you need deeper detail on matcher syntax or placeholder support, continue with the dedicated guides below.