Service test planning#478
Conversation
| mc.last_request.headers["x-api-key"] | ||
| mock = MockAnthropic() | ||
| mock.set_response(r"haiku", "sure, here's a haiku") | ||
| mock.set_response(r"create workflow", tool_use("call_workflow_agent", {...})) |
There was a problem hiding this comment.
Not sure I understand this bit
There was a problem hiding this comment.
Oh, is it changing the response structure to be a like a strucuted tool output, rather than free text?
So the regular response is like { content: "here's a haiku" }, but the tool response is like { type: "tool_use", content: { rag_results: [] } } or whatever
It might be nicer to have functions like mock.set_tool_response(type, input, output) , which give us more control when simulating these particular requests.
But we can resolve that in code, maybe we don't need to interfere with the spec yet
There was a problem hiding this comment.
By set_tool_response(type, input, output), do you mean one call that handles the whole round trip like "when the model decides to call tool x with input y, the tool returns z"? Now they're separate as set_response + tool_use() for the model request and tool_stubs for what the tool returns
There was a problem hiding this comment.
Yeah one call for the round trip is what I was thinking of
This approach with the tool_use call just feels weird to me.
But maybe we should just generate with this approach and take a close look at it, then maybe experiment with other approaches. Sometimes it's easier to make these decisions against a diff.
2 participants
Short Description
Updated section 2) service tests in agent-team-architecture-plan/2-service-tests.md
AI Usage
Please disclose how you've used AI in this work (it's cool, we just want to know!):
You can read more details in our Responsible AI Policy