Cap admin dashboard pending/scheduled details to avoid OOM on backlogs#497
Merged
Conversation
The admin index unconditionally materialised every pending and every scheduled queued_jobs row, then passed both arrays to the view. With a realistic backlog (thousands of unfetched pending jobs) the response balloons in two ways: - the rendered table HTML grows linearly with rows (the modern Bootstrap dashboard is heavier per row than the pre-overhaul one), - DebugKit's Variables panel serialises every view variable and can blow past memory_limit while building its snapshot. Fix: cap the visible pending/scheduled lists at Queue.adminDetailsLimit (default 200), detect truncation with a +1 row probe so we avoid a second count() on small queues, and compute the "new" / pendingJobs / scheduledJobs tile counts via DB count() queries against the same conditions so the tiles still reflect the true totals. Adds QueuedJobsTable::getPendingCount() / getScheduledCount() / getNewCount() to keep the count-query conditions next to the matching SelectQuery getters. Template renders a "Showing N most recent of M" notice when a list was truncated, with a link to the QueuedJobs admin for the full view. Documents Queue.adminDetailsLimit in config/app.example.php. Tests cover both the truncated and within-limit paths.
When the dashboard caps the pending/scheduled lists, the 'See QueuedJobs admin' link from the truncation banners now carries ?status=in_progress (pending) or ?status=scheduled, matching the existing search-collection filter names. Lands you exactly on the rows the truncated list represents instead of the full mixed list of completed + failed + running entries.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #497 +/- ##
============================================
+ Coverage 77.55% 78.38% +0.82%
- Complexity 972 1007 +35
============================================
Files 45 45
Lines 3262 3415 +153
============================================
+ Hits 2530 2677 +147
- Misses 732 738 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
QueueController::index()unconditionally materialises every pending and every scheduled queued_jobs row, then passes both arrays to the view. With a realistic backlog (thousands of pending or failed-and-retried jobs) two things go wrong:Fix
Queue.adminDetailsLimit(default 200). New config key documented inconfig/app.example.php.+1row probe so we avoid a secondcount()query on small queues.count()against the same conditions so they keep reflecting the true totals regardless of the visible-list cap.QueuedJobsTable:getPendingCount(),getScheduledCount(),getNewCount(). Pair with the existinggetPendingStats()/getScheduledStats()SelectQuerygetters.?status=in_progressfor pending,?status=scheduledfor scheduled) so the admin lands on exactly the rows the truncated list represents.Tests
testIndexTruncatesPendingDetailsAtDetailsLimit— sets the cap to 3, queues 5 jobs, asserts the truncation flag plus the true total.testIndexNotTruncatedWhenWithinLimit— single job, asserts truncation flags stay false (avoids the extra count() query in the common case).Both branches green: 16 tests / 40 assertions.