Fix: Vertex streaming + native tool calling leading to blank responses

[ctroche99]

Root Cause & Fix: Native Tool Calling Producing Blank Output

After adding INFO-level logging at every decision point in a live OpenWebUI instance, the blank-output bug with native tool calling + streaming traces to three distinct issues. The first is the dominant one and the reason every prior attempt at a fix looked correct on paper but failed at runtime.

1. The Google Genai SDK's Automatic Function Calling (AFC) was silently competing with the pipe's tool loop

The google-genai SDK enables Automatic Function Calling (AFC) by default whenever you pass Python callables (rather than types.FunctionDeclaration objects) as tools in GenerateContentConfig. AFC:

1. Intercepts the model's function_call parts before they reach the caller's iterator
2. Executes the Python callable itself
3. Sends the result back to Gemini as a FunctionResponse
4. Only yields the final text response to the pipe's stream loop

This means the pipe's custom tool-execution loop never fired. The diagnostic logs proved it: function_calls=0 on every stream round, yet server logs showed the underlying tool APIs being hit. AFC was doing the work invisibly. You can spot it in the SDK's own log line:

google.genai.models:async_generator - AFC is enabled with max remote calls: 10.

AFC and the pipe's tool loop are doing exactly the same job — they're two competing implementations of the same round-trip. AFC wins the race because it runs inside the SDK. The fix is to explicitly disable AFC so the pipe's loop owns the execution:

if tools:
    gen_config_params["tools"] = tools
    gen_config_params["automatic_function_calling"] = (
        types.AutomaticFunctionCallingConfig(disable=True)
    )

This isn't a loss of functionality. AFC is a convenience feature for simple scripts that don't need custom serialization, error handling, or UI rendering. The pipe's loop replaces it with a more capable implementation that handles OWUI-specific requirements.

2. OpenWebUI tools can return (Response, str) tuples that AFC cannot serialize for Gemini

OWUI's tool convention allows callables to return a tuple where element [0] is a Response object for the UI renderer and element [1] is the plain-text string for the model's context. When AFC executed these tools, it tried to pass the raw Python tuple — including the response object — into the FunctionResponse it sends back to Gemini. Gemini cannot deserialize that, so the model produced empty output.

This is why the symptom was "the tool runs (visible in server logs) but the chat shows nothing." The tool executed successfully; the serialization of its result to Gemini was what broke.

Fix in the manual tool loop — extract the str component when a tool returns a tuple:

_raw = tool_callable(**tool_args)
tool_result = (await _raw) if inspect.isawaitable(_raw) else _raw
if isinstance(tool_result, tuple):
    tool_result = next(
        (v for v in tool_result if isinstance(v, str)), None
    )
if tool_result is None:
    tool_result = ""
tool_result = str(tool_result)

3. Gemini sometimes namespaces tool names as default_api:foo

Once AFC was disabled and the raw function_call parts were visible, a secondary issue appeared. Gemini occasionally emits function_call.name as "default_api:tool_name" or "default_api.tool_name" rather than the plain registered name. The if tool_name in __tools__ lookup failed silently and we sent back an error FunctionResponse, after which the model gave up producing output.

Fix: strip the prefix for the lookup, but preserve the original name in the FunctionResponse so Gemini can pair it with the original FunctionCall:

raw_tool_name = fc_part.function_call.name
tool_name = raw_tool_name
for prefix in ("default_api:", "default_api."):
    if tool_name.startswith(prefix):
        tool_name = tool_name[len(prefix):]
        break

# ... execute via __tools__[tool_name] ...

types.FunctionResponse(
    name=raw_tool_name,  # ← original name, not normalized
    response={"result": tool_result},
)

Why earlier fixes didn't work

Several prior PRs correctly built a multi-round tool-execution loop over generate_content_stream with function_call detection, tool execution, follow-up streams, and <details type="tool_calls"> rendering. The code was structurally sound. But because AFC consumed the function_call parts before they reached the loop, the loop body was unreachable at runtime. And because AFC passed unserializable return values to Gemini, the final output was empty rather than an error. The bug was invisible without per-step logging.

TL;DR

The pipe's tool loop was correct in structure. AFC was just running first and producing unserializable payloads. Disabling AFC, extracting the str from OWUI tuple returns, and normalizing the default_api: prefix produces the expected end-to-end behavior in both streaming and non-streaming paths.

[ctroche99]

Hello,

Let me first start off by saying thank you for building this project! I am restricted to a VertexAI account in my environment and this PIPE function allowed me to set it up in OpenWebUI. I quickly ran into the issue reported in #155 and then the issue also seen in #135 . Considering OpenWebUI now considers default tool calling deprecated and "native" is required, I felt this needed to be addressed. I setup server side logging for your tool calling functions and then, based on those, learned about AFC. From there it was easy to have Claude re-write the functions with AFC disabled and handle the process via the Pipe. In my testing, it appears to have worked. Of course, I welcome your own testing and verification. I wanted to leave this comment to let you know I am not just pushing bling Claude PRs to your project. If you prefer non-AI coded PRs, that is fine, go ahead and close this out, but I did want to pass the fix along in case you found it sufficient.

[olivier-lacroix]

@ctroche99 thanks for this! Have been having this blank response issue, which is quite annoying.

Had a quick look: how do you handle the thought signature that Gemini 3 models need to? Without sending it back, things will crash.

Does your solution handle parallel tool calls?

[ctroche99]

@ctroche99 thanks for this! Have been having this blank response issue, which is quite annoying.

Had a quick look: how do you handle the thought signature that Gemini 3 models need to? Without sending it back, things will crash.

Does your solution handle parallel tool calls?

Very welcome!

Thought signatures: good catch, I forgot about that requirement; now fixed in 16e3641. I was only including function_call parts in the model turn when building tool-call history, dropping the thought Parts entirely. To fix it I collect thought Parts into thought_parts_this_round alongside the existing function_call_parts_this_round and prepend them when constructing the model Content, so the entire turn — thoughts (with signatures) + function calls — gets echoed back. Applied to both streaming and non-streaming paths.

Parallel tool calls — yes, fully handled. function_call_parts_this_round accumulates every function_call Part the model emits across a single stream round; we iterate them in one loop, execute each, build one FunctionResponse per call, and batch all responses into a single user Content before the follow-up request.

Heads up — two meaningful changes since the PR was opened that you may want to pull:

51f9fcd — emit OWUI embeds/files events for rich tool returns.

Because the pipe drives its own tool-calling loop, it bypasses OWUI's middleware — so tools that return (HTMLResponse, str) tuples or data:image/... base64 strings never get their persisted events fired. Cards would render live then vanish on reload. Added a _process_tool_result_for_owui helper that mirrors what OWUI's middleware process_tool_result does locally: detects HTMLResponse (tuple or bare), image data URLs, dicts/lists, etc., emits {"type": "embeds", "data": {"embeds": [...]}} or {"type": "files", ...}, and returns the LLM-visible string. Rich-UI tools now persist correctly across reloads. This was a bit of a rabbit hole but ultimately worked out very well in the end. I had a tool with a HTML injection that kept wiping the HTML but leaving the context injection. Turns out OWUI uses this 'embed' render tag for non text displays. Updated to catch common ones. Although, I did skip the MCP and external-tool-tuple branches because I had already gone overboard lol.

16e3641 — Gemini 3 thought-signature round-tripping (described above).

The other 6 commits in between were widget-positioning experiments that got reverted — net delta since the PR is just those two fixes.