Skip to content

refactor: consolidate audit-log code into dojo/auditlog/ package#14763

Open
Maffooch wants to merge 1 commit intoDefectDojo:devfrom
Maffooch:auditlog-cleanup
Open

refactor: consolidate audit-log code into dojo/auditlog/ package#14763
Maffooch wants to merge 1 commit intoDefectDojo:devfrom
Maffooch:auditlog-cleanup

Conversation

@Maffooch
Copy link
Copy Markdown
Contributor

@Maffooch Maffooch commented Apr 27, 2026

Summary

  • Moves dojo/auditlog.py (1,083 lines), dojo/pghistory_models.py, and dojo/pghistory_utils.py into a self-contained dojo/auditlog/ package that mirrors the dojo/url/ canonical layout from CLAUDE.md.
  • Also pulls the audit-log view (action_history), its template, and LogEntryFilter/PgHistoryFilter into the package — they previously lived in dojo/views.py, dojo/templates/, and dojo/filters.py.
  • New dojo/auditlog/settings.py becomes the source of truth for the audit-log env-var schema and pghistory field defaults; settings.dist.py imports from it.
  • Backward-compatible re-exports stay at the original locations per the playbook, so the 21 existing call sites keep working without churn.

Layout

dojo/auditlog/
├── __init__.py        # PEP 562 lazy re-exports
├── models.py          # DojoEvents (app_label auto-inferred to "dojo")
├── services.py        # registration + cleanup + config
├── helpers.py         # display formatting + Celery context wrappers
├── backfill.py        # backfill subsystem
├── filters.py         # LogEntryFilter, PgHistoryFilter
├── settings.py        # env schema + pghistory field defaults
├── ui/
│   ├── __init__.py
│   ├── views.py       # action_history
│   └── urls.py        # action_history route
└── templates/dojo/action_history.html

Notable details

  • __init__.py uses PEP 562 lazy attribute resolution. settings.dist.py imports dojo.auditlog.settings at Django settings-load time, before django.conf.settings is built; eager re-exports would circularly pull in django-dependent submodules.
  • DojoEvents drops the explicit app_label = "dojo" per CLAUDE.md ("DO NOT set app_label in Meta — auto-inferred"). makemigrations --check reports zero new migrations.
  • Defensive try/except blocks in services and backfill are intentionally dropped — let pgtrigger / COPY failures surface loudly rather than silently swallowing them.
  • dojo/settings/settings.dist.py becomes a consumer of dojo.auditlog.settings: imports the env-schema dict (splatted into environ.Env via **AUDITLOG_ENV_SCHEMA), the three PGHISTORY_*_FIELD constants, and AUDITLOG_DISABLE_ON_RAW_SAVE; adds dojo/auditlog/templates to TEMPLATES[0][\"DIRS\"] since dojo.auditlog isn't a separate Django app.
  • unittests/test_auditlog.py: two @patch targets updated from dojo.auditlog.call_command to dojo.auditlog.services.call_command, reflecting where the import now lives.

Out of scope (deferred per playbook)

  • Updating the 21 stub-using call sites (apps.py, tasks.py, signals, management commands, migrations, two test files) to import from new paths — explicitly a separate cleanup pass.
  • Moving per-domain audit-log signal handlers (dojo/test/signals.py etc.) into dojo/auditlog/signals.py — they're domain-coupled by design.
  • Removing the legacy django-auditlog library; LogEntryFilter still references it for the page's "legacy entries" pane.

Test plan

  • python manage.py check — no issues
  • python manage.py makemigrations --checkNo changes detected (the critical migration gate)
  • unittests.test_auditlog — 7/7 pass
  • unittests.test_importers_performance — 10/10 pass
  • reverse('action_history', cid=1, oid=2) — resolves to /history/1/2 (route name preserved)
  • End-to-end: edit a Finding in the UI; visit /history/<cid>/<oid>; confirm template renders, filters work (event type, user, IP, diff search), and a pghistory event is recorded with user/url/remote_addr/source/scan_type context fields populated.

🤖 Generated with Claude Code

@Maffooch Maffooch requested a review from mtesauro as a code owner April 27, 2026 21:53
@github-actions github-actions Bot added settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR unittests labels Apr 27, 2026
Move dojo/auditlog.py (1,083 lines), dojo/pghistory_models.py, and
dojo/pghistory_utils.py into a self-contained dojo/auditlog/ package
that mirrors the dojo/url/ canonical layout from CLAUDE.md. Also pulls
in the action_history view, its template, and the LogEntryFilter /
PgHistoryFilter that previously lived in dojo/views.py, dojo/templates/,
and dojo/filters.py.

Layout:
- models.py        DojoEvents proxy model (app_label dropped, auto-inferred)
- services.py      pghistory registration, retention/cleanup, config
- helpers.py       display formatting + Celery context wrappers
- backfill.py      backfill subsystem (used by migrations + commands)
- filters.py       LogEntryFilter, PgHistoryFilter
- settings.py      env-var schema + pghistory field defaults (source of truth)
- ui/views.py      action_history view
- ui/urls.py       action_history route (name preserved)
- templates/dojo/action_history.html

Backward-compatible re-exports stay at dojo/pghistory_models.py,
dojo/pghistory_utils.py, and dojo/filters.py per the playbook, so the
21 existing call sites keep working without churn. dojo/auditlog.py and
the original template are deleted (the package replaces the file).

Notable details:
- __init__.py uses PEP 562 lazy attribute resolution. settings.dist.py
  imports dojo.auditlog.settings at Django settings-load time, before
  django.conf.settings is built; eager re-exports would pull in
  django-dependent submodules and circularly import settings.
- DojoEvents drops the explicit app_label = "dojo" per CLAUDE.md
  ("DO NOT set app_label in Meta — auto-inferred"). makemigrations
  --check reports zero new migrations.
- Defensive try/except blocks in services and backfill are dropped per
  the user's directive — let pgtrigger / COPY failures surface loudly
  rather than silently swallowing them.
- settings.dist.py becomes a consumer of dojo.auditlog.settings:
  imports the env-schema dict (splatted into environ.Env via
  **AUDITLOG_ENV_SCHEMA), the three PGHISTORY_*_FIELD constants, and
  AUDITLOG_DISABLE_ON_RAW_SAVE; adds dojo/auditlog/templates to
  TEMPLATES[0]["DIRS"] since auditlog isn't a separate Django app.
- unittests/test_auditlog.py: two @patch targets updated from
  dojo.auditlog.call_command to dojo.auditlog.services.call_command,
  reflecting where the import now lives.

Verified end-to-end against the running stack:
- python manage.py check                       -> no issues
- python manage.py makemigrations --check      -> No changes detected
- unittests.test_auditlog                      -> 7/7 pass
- unittests.test_importers_performance         -> 10/10 pass
- reverse('action_history', cid=1, oid=2)      -> /history/1/2

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Maffooch Maffooch added this to the 2.58.0 milestone Apr 27, 2026
Copy link
Copy Markdown
Member

@valentijnscholten valentijnscholten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defensive try/except blocks in services and backfill are intentionally dropped — let pgtrigger / COPY failures surface loudly rather than silently swallowing them.

Why are these being dropped? In case of some unexpected error this would block users from starting their instances after an upgrade.

@Maffooch
Copy link
Copy Markdown
Contributor Author

I went back and looked at each spot to make sure the upgrade path wouldn't blow up, and I think we are actually safe here. For configure_pghistory_triggers has the same try/catch that was just an error log then a raise. The exception already propagated, so removing the wrapper only loses the friendlier log line ahead of the traceback, but the startup blocking behavior is identical.

For the outer try/catch around process_model_backfill, the only callers on the upgrade path are the backfill migrations, and both already wrap the call in their own try/catch. A backfill failure during migration wont abort the migration or block startup after an upgrade.

The one real behavior change is in the manual pghistory_backfill_fast. it now exits on model failure instead of silently moving on. If you'd like the friendlier log message ahead of the traceback, happy to add the error log back and let it continue if there was an error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR unittests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants