Mock Prompt Executor - Structured Responses

[samDobsonDev]

Hello everyone,
I'm using a mock Prompt Executor in my tests, and I'm curious if it's possible to mock structured responses to LLM calls, not just tool calls and string messages?

Take this example:

val mockPromptExecutor: PromptExecutor = getMockExecutor(toolRegistry = toolRegistry) {
    mockTool(getUndeliveredOrdersForUserTool) alwaysReturns GetUndeliveredOrdersForUserTool.Result("Mocked Tool Call Response")
    mockLLMToolCall(getUndeliveredOrdersForUserTool, ToolArgs.Empty()) onRequestEquals "Hi there, where is my order?"
    mockLLMAnswer("Mocked LLM Response") onRequestContains "Mocked Tool Call Response"
    // TODO - How do we mock structured responses?
}

I want to mock an LLM structured response so that when the input contains "Mocked LLM Response", it returns an instance of my structured response class. How can I achieve this?

[Ololoshechkin]

Hi! Thanks for your question!
Currently there's no DSL for that but you can do that using the ordinary mockLLMAnswer(String). Just provide the JSON structure of your response as String -- the way you would expect the LLM to generate it, and it should work.

If you check how PromptExecutor.executeStructured is implemented, you'll see the following:

repeat(retries) { attempt ->
        ...
        val response = execute(prompt = prompt, model = mainModel) // normal request, response is Message.Response
        ...
        val structure = structureParser.parse(structure, response.content) // trying to parse `response.content` from String
        ...
}

So providing a JSON string in tests should work.