fix: local AI chat backend wiring and fallback handling#1119
fix: local AI chat backend wiring and fallback handling#1119BenjaminMichaelis wants to merge 10 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR moves chat backend registration to configuration-driven selection and introduces local/unavailable chat service abstractions so the web app can continue running when chat is not configured.
Changes:
- Replaces direct Azure chat registration with
AddConfiguredChatServices. - Adds
IChatCompletionServiceimplementations for Azure, local OpenAI-compatible backends, and unavailable fallback. - Adds controller/test coverage for 503
chat_unavailableresponses, plus unrelated development CAPTCHA/login bypass changes.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
EssentialCSharp.Web/Program.cs |
Wires chat registration through the new configuration-driven extension. |
EssentialCSharp.Web/Controllers/ChatController.cs |
Uses IChatCompletionService and returns unavailable/error contracts. |
EssentialCSharp.Web/Areas/Identity/Pages/Account/Register.cshtml.cs |
Adds development-only CAPTCHA test-token bypass. |
EssentialCSharp.Web/Areas/Identity/Pages/Account/Login.cshtml.cs |
Adds development-only CAPTCHA and email-confirmation bypass behavior. |
EssentialCSharp.Web.Tests/ChatAvailabilityTests.cs |
Adds tests for unavailable chat responses. |
EssentialCSharp.Chat.Shared/Services/UnavailableChatService.cs |
Adds fallback unavailable chat implementation. |
EssentialCSharp.Chat.Shared/Services/LocalChatService.cs |
Adds local OpenAI-compatible chat backend implementation. |
EssentialCSharp.Chat.Shared/Services/IChatCompletionService.cs |
Defines the chat completion abstraction. |
EssentialCSharp.Chat.Shared/Services/ChatBackendUnavailableException.cs |
Adds domain exception for unavailable backends. |
EssentialCSharp.Chat.Shared/Services/AIChatService.cs |
Implements the new abstraction for Azure chat. |
EssentialCSharp.Chat.Shared/Models/AIOptions.cs |
Adds local AI configuration options. |
EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs |
Adds backend selection and local/unavailable service registration. |
Comments suppressed due to low confidence (3)
EssentialCSharp.Web.Tests/ChatAvailabilityTests.cs:39
- This test also depends on ambient app configuration leaving chat unavailable. Because the factory does not clear AIOptions/user secrets/environment settings, machines with a configured Azure or local backend will not return 503 here. Use a test-specific factory/service override so the unavailable path is guaranteed.
HttpClient client = factory.CreateClient(new WebApplicationFactoryClientOptions { AllowAutoRedirect = false });
string userId = await McpTestHelper.CreateUserAsync(factory, "chat-stream-unavailable");
(string cookieName, string cookieValue) = await McpTestHelper.CreateIdentityApplicationCookieAsync(factory, userId);
EssentialCSharp.Web/Areas/Identity/Pages/Account/Login.cshtml.cs:110
- This change adds an authentication bypass for unconfirmed accounts when a hard-coded test CAPTCHA token is submitted in Development. That is a significant auth behavior change unrelated to the chat-backend PR scope and can grant access to any unconfirmed local/development account that knows the token; keep this in the test host or a dedicated test-only service override instead of production code paths.
// For test tokens, bypass email confirmation requirement
if (isTestToken && !foundUser.EmailConfirmed)
{
// Temporarily set email as confirmed for this sign-in when using test token
var tempConfirmed = foundUser.EmailConfirmed;
foundUser.EmailConfirmed = true;
result = await signInManager.PasswordSignInAsync(foundUser, Input.Password, Input.RememberMe, lockoutOnFailure: true);
foundUser.EmailConfirmed = tempConfirmed;
EssentialCSharp.Web/Controllers/ChatController.cs:208
- The streaming path has the same gap as SendMessage: backend failures from the Azure/OpenAI client that are not already wrapped as ChatBackendUnavailableException fall into the generic 500/error event path, so the new chat_unavailable/503 contract is only used for the unavailable stub/local wrapper. Convert known backend/network exceptions to ChatBackendUnavailableException before this generic handler.
catch (Exception ex) when (!Response.HasStarted)
{
LogChatStreamErrorBeforeResponseStarted(_Logger, ex, User.Identity?.Name);
Response.StatusCode = 500;
Response.ContentType = "application/json";
await Response.WriteAsJsonAsync(new { error = "Chat service unavailable" }, CancellationToken.None);
}
d3272ea to
c6f7ea6
Compare
|
Memory leak fix pushed in 1b89b5b: replaced unbounded \ConcurrentDictionary\ with a private \MemoryCache\ (SizeLimit=500, 30-min sliding TTL). Also implemented \IDisposable\ with \GC.SuppressFinalize\ for correct singleton cleanup. |
|
Security cleanup pushed in 413e352: removed the hard-coded development hCaptcha bypass and the temporary email-confirmation mutation from the Identity login/register flows. Local/E2E testing should rely on the configured official hCaptcha test keys or test-host service overrides instead of bypass logic in production page handlers. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (2)
EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs:122
- The Azure path now calls the overload that only validates
EmbeddingRetryOptionsdefaults, soAIOptions:EmbeddingRetryvalues from appsettings or environment variables are no longer bound. This regresses the previousAddAzureOpenAIServices(configuration)behavior and makes customized embedding retry/backoff settings silently ignored whenever Azure is selected.
services.AddAzureOpenAIServices(aiOptions, postgresConnectionString!);
EssentialCSharp.Chat.Shared/Extensions/ServiceCollectionExtensions.cs:142
- This 120-second timeout is still subject to the app-wide
ConfigureHttpClientDefaults(http => http.AddStandardResilienceHandler())inProgram.cs, whose standard timeout policy can cancel requests earlier thanHttpClient.Timeout. Local models often take longer on first token/model load, so the new local backend may reportchat_unavailablebefore the configured 120 seconds unless this named client overrides the resilience timeouts or opts out.
client.Timeout = TimeSpan.FromSeconds(120);
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
EssentialCSharp.Chat.Shared/Services/UnavailableChatService.cs:41
- The streaming unavailable path also yields normal SSE content plus a response ID. If a caller forgets the IsAvailable pre-check, the fallback backend will appear to produce a valid stream rather than raising ChatBackendUnavailableException and letting the API return the unavailable contract.
yield return ("Chat service is unavailable in this environment.", responseId: null);
yield return (string.Empty, Guid.NewGuid().ToString("N"));
… 30min TTL, 500 entry limit) - Add private MemoryCache with SizeLimit=500 and 30-minute sliding expiration - Implement IDisposable with GC.SuppressFinalize for proper DI singleton cleanup - Update Set call to use MemoryCacheEntryOptions - Update TryGetValue to use generic overload with null guard
Remove the hard-coded development hCaptcha token bypass and temporary email-confirmation mutation from Identity login/register page handlers. Local testing should use the configured hCaptcha test keys or test-host service overrides instead of changing production auth flow code.
…nfig, remove resilience handler from local AI client - IsValidNpgsqlConnectionString() helper validates that a connection string has a non-empty Host instead of just checking for non-whitespace text, preventing the 'your-postgres-connection-string-here' placeholder from passing validation. - AddOptions<EmbeddingRetryOptions>().Bind() wired in AddConfiguredChatServices so retry config from appsettings is not silently ignored when using the AIOptions overload of AddAzureOpenAIServices. - .RemoveAllResilienceHandlers() added to LocalAIChat HttpClient to prevent the global ConfigureHttpClientDefaults resilience handler (30s attempt / 90s total timeouts) from cutting off long LLM completions. EXTEXP0001 suppressed in the project file.
- Narrow IsValidNpgsqlConnectionString exception handling to ArgumentException. - Add LocalChatService unit tests for request construction, non-success/error mapping, invalid payload handling, and previousResponseId conversation reuse. - Add AddConfiguredChatServices selection tests for Azure, local, fallback, and missing configuration, including EmbeddingRetry options binding coverage.
- Ensure LocalChatService tests own and dispose created HttpClient instances. - Ensure non-success response object is created in a local using variable before enqueue.
- Make UnavailableChatService throw ChatBackendUnavailableException for both completion APIs. - Use per-entry MemoryCacheEntryOptions in LocalChatService history cache writes. - Wrap response body-read transport/timeouts as ChatBackendUnavailableException. - Replace project-wide EXTEXP0001 suppression with targeted pragma around RemoveAllResilienceHandlers usage.
Reject non-HTTPS Azure endpoint values during backend selection to avoid sending DefaultAzureCredential bearer tokens over cleartext transport when misconfigured.
22c7605 to
e1abc0b
Compare
Pass the official hCaptcha test token in chat unavailable endpoint tests so captcha enforcement does not short-circuit with 403 before backend-availability assertions.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
EssentialCSharp.Web.Tests/ChatAvailabilityTests.cs:45
- This test can be affected by real hCaptcha configuration from user secrets/environment variables. With
HCaptcha:SecretKeyconfigured, the stream action validates this hard-coded token before checking chat availability, so the test may fail before exercising the intendedchat_unavailablecontract. Override ICaptchaService or clear HCaptcha options in the test factory.
Content = JsonContent.Create(new { message = "Hello", enableContextualSearch = false, captchaResponse = HCaptchaTestToken })
|
|
||
| using var request = new HttpRequestMessage(HttpMethod.Post, "/api/chat/message") | ||
| { | ||
| Content = JsonContent.Create(new { message = "Hello", enableContextualSearch = false, captchaResponse = HCaptchaTestToken }) |
Summary
Validation