Skip to content

fix: local AI chat backend wiring and fallback handling#1119

Open
BenjaminMichaelis wants to merge 10 commits into
mainfrom
copilot/local-ai-chat-wiring-fix
Open

fix: local AI chat backend wiring and fallback handling#1119
BenjaminMichaelis wants to merge 10 commits into
mainfrom
copilot/local-ai-chat-wiring-fix

Conversation

@BenjaminMichaelis
Copy link
Copy Markdown
Member

Summary

  • replace environment-gated chat registration with config-driven backend selection
  • add IChatCompletionService abstraction with Azure, Local, and Unavailable implementations
  • wire local Ollama-compatible chat path with endpoint/model config and safe fallback behavior
  • return explicit 503 + chat_unavailable contracts when chat backend is unavailable
  • add tests covering unavailable chat contracts for /api/chat/message and /api/chat/stream

Validation

  • pwsh .github/hooks/scripts/validate-repos.ps1 -Projects web (pass)

Copilot AI review requested due to automatic review settings May 16, 2026 17:55
Comment thread EssentialCSharp.Chat.Shared/Services/LocalChatService.cs Fixed
Comment thread EssentialCSharp.Web/Controllers/ChatController.cs Fixed
Comment thread EssentialCSharp.Web/Controllers/ChatController.cs Fixed
Comment thread EssentialCSharp.Web/Program.cs Fixed
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR moves chat backend registration to configuration-driven selection and introduces local/unavailable chat service abstractions so the web app can continue running when chat is not configured.

Changes:

  • Replaces direct Azure chat registration with AddConfiguredChatServices.
  • Adds IChatCompletionService implementations for Azure, local OpenAI-compatible backends, and unavailable fallback.
  • Adds controller/test coverage for 503 chat_unavailable responses, plus unrelated development CAPTCHA/login bypass changes.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
EssentialCSharp.Web/Program.cs Wires chat registration through the new configuration-driven extension.
EssentialCSharp.Web/Controllers/ChatController.cs Uses IChatCompletionService and returns unavailable/error contracts.
EssentialCSharp.Web/Areas/Identity/Pages/Account/Register.cshtml.cs Adds development-only CAPTCHA test-token bypass.
EssentialCSharp.Web/Areas/Identity/Pages/Account/Login.cshtml.cs Adds development-only CAPTCHA and email-confirmation bypass behavior.
EssentialCSharp.Web.Tests/ChatAvailabilityTests.cs Adds tests for unavailable chat responses.
EssentialCSharp.Chat.Shared/Services/UnavailableChatService.cs Adds fallback unavailable chat implementation.
EssentialCSharp.Chat.Shared/Services/LocalChatService.cs Adds local OpenAI-compatible chat backend implementation.
EssentialCSharp.Chat.Shared/Services/IChatCompletionService.cs Defines the chat completion abstraction.
EssentialCSharp.Chat.Shared/Services/ChatBackendUnavailableException.cs Adds domain exception for unavailable backends.
EssentialCSharp.Chat.Shared/Services/AIChatService.cs Implements the new abstraction for Azure chat.
EssentialCSharp.Chat.Shared/Models/AIOptions.cs Adds local AI configuration options.
EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs Adds backend selection and local/unavailable service registration.
Comments suppressed due to low confidence (3)

EssentialCSharp.Web.Tests/ChatAvailabilityTests.cs:39

  • This test also depends on ambient app configuration leaving chat unavailable. Because the factory does not clear AIOptions/user secrets/environment settings, machines with a configured Azure or local backend will not return 503 here. Use a test-specific factory/service override so the unavailable path is guaranteed.
        HttpClient client = factory.CreateClient(new WebApplicationFactoryClientOptions { AllowAutoRedirect = false });

        string userId = await McpTestHelper.CreateUserAsync(factory, "chat-stream-unavailable");
        (string cookieName, string cookieValue) = await McpTestHelper.CreateIdentityApplicationCookieAsync(factory, userId);

EssentialCSharp.Web/Areas/Identity/Pages/Account/Login.cshtml.cs:110

  • This change adds an authentication bypass for unconfirmed accounts when a hard-coded test CAPTCHA token is submitted in Development. That is a significant auth behavior change unrelated to the chat-backend PR scope and can grant access to any unconfirmed local/development account that knows the token; keep this in the test host or a dedicated test-only service override instead of production code paths.
                // For test tokens, bypass email confirmation requirement
                if (isTestToken && !foundUser.EmailConfirmed)
                {
                    // Temporarily set email as confirmed for this sign-in when using test token
                    var tempConfirmed = foundUser.EmailConfirmed;
                    foundUser.EmailConfirmed = true;
                    result = await signInManager.PasswordSignInAsync(foundUser, Input.Password, Input.RememberMe, lockoutOnFailure: true);
                    foundUser.EmailConfirmed = tempConfirmed;

EssentialCSharp.Web/Controllers/ChatController.cs:208

  • The streaming path has the same gap as SendMessage: backend failures from the Azure/OpenAI client that are not already wrapped as ChatBackendUnavailableException fall into the generic 500/error event path, so the new chat_unavailable/503 contract is only used for the unavailable stub/local wrapper. Convert known backend/network exceptions to ChatBackendUnavailableException before this generic handler.
        catch (Exception ex) when (!Response.HasStarted)
        {
            LogChatStreamErrorBeforeResponseStarted(_Logger, ex, User.Identity?.Name);
            Response.StatusCode = 500;
            Response.ContentType = "application/json";
            await Response.WriteAsJsonAsync(new { error = "Chat service unavailable" }, CancellationToken.None);
        }

Comment thread EssentialCSharp.Web.Tests/ChatAvailabilityTests.cs
Comment thread EssentialCSharp.Chat.Shared/Services/LocalChatService.cs Outdated
Comment thread EssentialCSharp.Chat.Shared/Services/LocalChatService.cs Outdated
Comment thread EssentialCSharp.Web/Controllers/ChatController.cs Outdated
Comment thread EssentialCSharp.Chat.Shared/Services/LocalChatService.cs Outdated
Comment thread EssentialCSharp.Web/Areas/Identity/Pages/Account/Login.cshtml.cs Outdated
Comment thread EssentialCSharp.Web/Controllers/ChatController.cs Outdated
Comment thread EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs Outdated
Comment thread EssentialCSharp.Web/Controllers/ChatController.cs
Comment thread EssentialCSharp.Web/Controllers/ChatController.cs
@BenjaminMichaelis BenjaminMichaelis force-pushed the copilot/local-ai-chat-wiring-fix branch from d3272ea to c6f7ea6 Compare May 16, 2026 18:30
@BenjaminMichaelis BenjaminMichaelis changed the title Fix local AI chat backend wiring and fallback handling fix: local AI chat backend wiring and fallback handling May 16, 2026
@BenjaminMichaelis
Copy link
Copy Markdown
Member Author

Memory leak fix pushed in 1b89b5b: replaced unbounded \ConcurrentDictionary\ with a private \MemoryCache\ (SizeLimit=500, 30-min sliding TTL). Also implemented \IDisposable\ with \GC.SuppressFinalize\ for correct singleton cleanup.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Comment thread EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs Outdated
Comment thread EssentialCSharp.Web/Areas/Identity/Pages/Account/Login.cshtml.cs Outdated
@BenjaminMichaelis
Copy link
Copy Markdown
Member Author

Security cleanup pushed in 413e352: removed the hard-coded development hCaptcha bypass and the temporary email-confirmation mutation from the Identity login/register flows. Local/E2E testing should rely on the configured official hCaptcha test keys or test-host service overrides instead of bypass logic in production page handlers.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs:122

  • The Azure path now calls the overload that only validates EmbeddingRetryOptions defaults, so AIOptions:EmbeddingRetry values from appsettings or environment variables are no longer bound. This regresses the previous AddAzureOpenAIServices(configuration) behavior and makes customized embedding retry/backoff settings silently ignored whenever Azure is selected.
                services.AddAzureOpenAIServices(aiOptions, postgresConnectionString!);

EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs:142

  • This 120-second timeout is still subject to the app-wide ConfigureHttpClientDefaults(http => http.AddStandardResilienceHandler()) in Program.cs, whose standard timeout policy can cancel requests earlier than HttpClient.Timeout. Local models often take longer on first token/model load, so the new local backend may report chat_unavailable before the configured 120 seconds unless this named client overrides the resilience timeouts or opts out.
                client.Timeout = TimeSpan.FromSeconds(120);

Comment thread EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs Outdated
Comment thread EssentialCSharp.Web/Areas/Identity/Pages/Account/Register.cshtml.cs Outdated
Comment thread EssentialCSharp.Web/Areas/Identity/Pages/Account/Login.cshtml.cs Outdated
Copilot AI review requested due to automatic review settings May 16, 2026 22:56
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Comment thread EssentialCSharp.Chat.Shared/Services/LocalChatService.cs
Comment thread EssentialCSharp.Chat.Tests/LocalChatServiceTests.cs Fixed
Comment thread EssentialCSharp.Chat.Tests/LocalChatServiceTests.cs Fixed
Copilot AI review requested due to automatic review settings May 17, 2026 04:05
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

EssentialCSharp.Chat.Shared/Services/UnavailableChatService.cs:41

  • The streaming unavailable path also yields normal SSE content plus a response ID. If a caller forgets the IsAvailable pre-check, the fallback backend will appear to produce a valid stream rather than raising ChatBackendUnavailableException and letting the API return the unavailable contract.
        yield return ("Chat service is unavailable in this environment.", responseId: null);
        yield return (string.Empty, Guid.NewGuid().ToString("N"));

Comment thread EssentialCSharp.Chat.Shared/Services/UnavailableChatService.cs Outdated
Comment thread EssentialCSharp.Chat.Shared/EssentialCSharp.Chat.Common.csproj Outdated
Comment thread EssentialCSharp.Chat.Shared/Services/LocalChatService.cs Outdated
Comment thread EssentialCSharp.Chat.Shared/Services/LocalChatService.cs Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comment thread EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

… 30min TTL, 500 entry limit)

- Add private MemoryCache with SizeLimit=500 and 30-minute sliding expiration
- Implement IDisposable with GC.SuppressFinalize for proper DI singleton cleanup
- Update Set call to use MemoryCacheEntryOptions
- Update TryGetValue to use generic overload with null guard
Remove the hard-coded development hCaptcha token bypass and temporary email-confirmation mutation from Identity login/register page handlers. Local testing should use the configured hCaptcha test keys or test-host service overrides instead of changing production auth flow code.
…nfig, remove resilience handler from local AI client

- IsValidNpgsqlConnectionString() helper validates that a connection string
  has a non-empty Host instead of just checking for non-whitespace text,
  preventing the 'your-postgres-connection-string-here' placeholder from
  passing validation.
- AddOptions<EmbeddingRetryOptions>().Bind() wired in AddConfiguredChatServices
  so retry config from appsettings is not silently ignored when using the
  AIOptions overload of AddAzureOpenAIServices.
- .RemoveAllResilienceHandlers() added to LocalAIChat HttpClient to prevent
  the global ConfigureHttpClientDefaults resilience handler (30s attempt /
  90s total timeouts) from cutting off long LLM completions. EXTEXP0001
  suppressed in the project file.
- Narrow IsValidNpgsqlConnectionString exception handling to ArgumentException.
- Add LocalChatService unit tests for request construction, non-success/error mapping,
  invalid payload handling, and previousResponseId conversation reuse.
- Add AddConfiguredChatServices selection tests for Azure, local, fallback, and missing
  configuration, including EmbeddingRetry options binding coverage.
- Ensure LocalChatService tests own and dispose created HttpClient instances.
- Ensure non-success response object is created in a local using variable before enqueue.
- Make UnavailableChatService throw ChatBackendUnavailableException for both completion APIs.
- Use per-entry MemoryCacheEntryOptions in LocalChatService history cache writes.
- Wrap response body-read transport/timeouts as ChatBackendUnavailableException.
- Replace project-wide EXTEXP0001 suppression with targeted pragma around
  RemoveAllResilienceHandlers usage.
Reject non-HTTPS Azure endpoint values during backend selection to avoid sending
DefaultAzureCredential bearer tokens over cleartext transport when misconfigured.
@BenjaminMichaelis BenjaminMichaelis force-pushed the copilot/local-ai-chat-wiring-fix branch from 22c7605 to e1abc0b Compare May 17, 2026 07:13
Pass the official hCaptcha test token in chat unavailable endpoint tests so
captcha enforcement does not short-circuit with 403 before backend-availability
assertions.
Copilot AI review requested due to automatic review settings May 17, 2026 08:23
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

EssentialCSharp.Web.Tests/ChatAvailabilityTests.cs:45

  • This test can be affected by real hCaptcha configuration from user secrets/environment variables. With HCaptcha:SecretKey configured, the stream action validates this hard-coded token before checking chat availability, so the test may fail before exercising the intended chat_unavailable contract. Override ICaptchaService or clear HCaptcha options in the test factory.
            Content = JsonContent.Create(new { message = "Hello", enableContextualSearch = false, captchaResponse = HCaptchaTestToken })


using var request = new HttpRequestMessage(HttpMethod.Post, "/api/chat/message")
{
Content = JsonContent.Create(new { message = "Hello", enableContextualSearch = false, captchaResponse = HCaptchaTestToken })
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants