fix(llm): improve rag demo error handling by zxuexingzhijie · Pull Request #335 · apache/hugegraph-ai

zxuexingzhijie · 2026-05-18T16:26:08Z

Summary

Make RAG demo auto-reload opt-in through HG_DEV_RELOAD instead of enabling reload by default.
Improve LLM error handling so LiteLLM/OpenAI runtime failures surface consistently instead of being silently converted to ordinary result strings.
Update Ollama async embeddings to use the batch embed(input=...) API and validate malformed embedding responses.
Make graph summary fetching handle HugeGraph Gremlin response shapes more defensively while preserving real execution failures.
Clarify HugeGraph schema and HTTP error messages so callers receive more actionable failures.

Change-Id: I1658c1a7aefe353f48be4d48c7d4425aded13a3d

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Change-Id: I275a618f6b6a43448c40d91fc6e4667084d26f4f

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

imbajin

这轮主要有几处错误处理会导致运行时结果被静默改写或解析错误

Change-Id: I65b37b72baea21e2cf2cf2c471ff14c56869520e

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

+def test_response_validation_raises_http_error_with_numeric_status_body():
+    response = Mock(spec=requests.Response)
+    response.status_code = 400
+    response.text = '{"status":400,"message":"bad gremlin"}'
+    response.content = response.text.encode("utf-8")
+    response.json.return_value = {"status": 400, "message": "bad gremlin"}
+    response.request = Mock()
+    response.request.body = "g.V2()"
+    response.raise_for_status.side_effect = requests.exceptions.HTTPError("400 Client Error")
+
+    validator = ResponseValidation()
+
+    with pytest.raises(Exception, match="bad gremlin"):
+        validator(response, "POST", "/gremlin")


+        except RequestException as e:
+            raise ValueError(f"Failed to connect HugeGraph to get schema '{self.graph_name}': {e}") from e
        if not schema["vertexlabels"] and not schema["edgelabels"]:
-            raise Exception(f"Can not get {self.graph_name}'s schema from HugeGraph!")
+            raise ValueError(f"Can not get {self.graph_name}'s schema from HugeGraph!")


            log.info("Token usage: %s", response.usage)
            return response.choices[0].message.content
        except (RateLimitError, BudgetExceededError, APIError) as e:
            log.error("Error in LiteLLM call: %s", e)
-            return f"Error: {str(e)}"
+            raise

    @retry(
        stop=stop_after_attempt(2),
        wait=wait_exponential(multiplier=1, min=2, max=5),
        retry=retry_if_exception_type((RateLimitError, BudgetExceededError, APIError)),
    )
    async def agenerate(
        self,
        messages: Optional[List[Dict[str, Any]]] = None,
        prompt: Optional[str] = None,
    ) -> str:
        """Generate a response to the query messages/prompt asynchronously."""
        if messages is None:
            assert prompt is not None, "Messages or prompt must be provided."
            messages = [{"role": "user", "content": prompt}]
        try:
            response = await acompletion(
                model=self.model,
                messages=messages,
                temperature=self.temperature,
                max_tokens=self.max_tokens,
                api_key=self.api_key,
                base_url=self.api_base,
            )
            log.info("Token usage: %s", response.usage)
            return response.choices[0].message.content
        except (RateLimitError, BudgetExceededError, APIError) as e:
            log.error("Error in async LiteLLM call: %s", e)
-            return f"Error: {str(e)}"
+            raise



imbajin

本轮整体方向正确，但 base_node 的 catch-all 与 schema_build 的 raise 相互抵消、ollama/litellm/openai 的契约改动缺少测试覆盖，建议先收敛错误处理边界并补齐回归测试再合入。

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Change-Id: I939ca3a2df3ecc55471f9b98155b5ab03fa43d03

Change-Id: Ib8bb730fb9c28e4b3e0a05e72d531544d0f623a9

imbajin

CI 已恢复，主要阻塞问题已解决。下面还剩两处非阻塞的测试/维护性建议

Change-Id: I815753ca0350e6b5f734937973eca956d9ef422a

zxuexingzhijie · 2026-05-20T07:08:13Z

done

fix(llm): improve rag demo error handling

278bf94

Change-Id: I1658c1a7aefe353f48be4d48c7d4425aded13a3d

dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label May 18, 2026

github-actions Bot added llm python-client labels May 18, 2026

imbajin requested a review from Copilot May 19, 2026 05:00

Copilot started reviewing on behalf of imbajin May 19, 2026 05:00 View session

Copilot AI reviewed May 19, 2026

View reviewed changes

fix(llm): restore ci error handling contracts

334711c

Change-Id: I275a618f6b6a43448c40d91fc6e4667084d26f4f

imbajin requested a review from Copilot May 19, 2026 07:41

Copilot started reviewing on behalf of imbajin May 19, 2026 07:42 View session

Copilot AI reviewed May 19, 2026

View reviewed changes

imbajin reviewed May 19, 2026

View reviewed changes

Comment thread hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/fetch_graph_data.py Outdated

Comment thread hugegraph-python-client/src/pyhugegraph/utils/util.py Outdated

Comment thread hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/fetch_graph_data.py Outdated

fix(llm): handle graph summary response shapes

f558261

Change-Id: I65b37b72baea21e2cf2cf2c471ff14c56869520e

imbajin requested a review from Copilot May 19, 2026 14:19

Copilot started reviewing on behalf of imbajin May 19, 2026 14:38 View session

Copilot AI reviewed May 19, 2026

View reviewed changes

imbajin requested a review from Copilot May 20, 2026 03:18

Copilot started reviewing on behalf of imbajin May 20, 2026 03:18 View session

Copilot AI reviewed May 20, 2026

View reviewed changes

imbajin reviewed May 20, 2026

View reviewed changes

zxuexingzhijie and others added 2 commits May 20, 2026 13:02

Potential fix for pull request finding

ddcc96f

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

fix(llm): address review error contracts

22ec7e4

Change-Id: I939ca3a2df3ecc55471f9b98155b5ab03fa43d03

dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels May 20, 2026

test(llm): update schema manager error expectation

76248e6

Change-Id: Ib8bb730fb9c28e4b3e0a05e72d531544d0f623a9

imbajin reviewed May 20, 2026

View reviewed changes

Comment thread hugegraph-llm/src/hugegraph_llm/models/embeddings/ollama.py

Comment thread hugegraph-llm/src/tests/models/llms/test_litellm_client.py Outdated

fix(llm): align ollama embedding response validation

35f4604

Change-Id: I815753ca0350e6b5f734937973eca956d9ef422a

imbajin approved these changes May 20, 2026

View reviewed changes

dosubot Bot added the lgtm This PR has been approved by a maintainer label May 20, 2026

Merge branch 'main' into fix/hugegraph-ai-rag-runtime-errors

7b5b8d7

imbajin merged commit 2df0b6e into apache:main May 20, 2026
13 checks passed

Conversation

zxuexingzhijie commented May 18, 2026 • edited by imbajin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

imbajin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

imbajin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

imbajin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zxuexingzhijie commented May 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zxuexingzhijie commented May 18, 2026 •

edited by imbajin

Loading

imbajin left a comment •

edited

Loading

imbajin left a comment •

edited

Loading