Skip to content

Expand constraint coverage and document audit decisions for #14#28

Open
Abhishek-Kumar-Rai5 wants to merge 1 commit intoPecanProject:mainfrom
Abhishek-Kumar-Rai5:feat/migration-of-constraints-validation
Open

Expand constraint coverage and document audit decisions for #14#28
Abhishek-Kumar-Rai5 wants to merge 1 commit intoPecanProject:mainfrom
Abhishek-Kumar-Rai5:feat/migration-of-constraints-validation

Conversation

@Abhishek-Kumar-Rai5
Copy link
Copy Markdown

Description

Expands constraint coverage for Issue #14 by adding high-value data integrity constraints derived from the BETYdb constraint audit, extending validation logic and tests, and documenting explicit coverage decisions for implemented vs deferred constraints.

This PR includes:

  • Expanded Frictionless-native constraints in datapackage.json for:

    • species completeness and uniqueness
    • citation completeness
    • PFT completeness and uniqueness
    • priors distribution validation
  • Expanded custom constraints in inst/extdata/custom_constraints.yaml for:

    • citation natural-key uniqueness
    • methods uniqueness within citation
    • conditional units requirement when management level is present
  • Additional candidate constraints were reviewed and explicitly deferred after empirical validation (rather than silently omitted), including management event uniqueness and prior uniqueness keys.

  • Added/expanded validation logic and tests in:

    • R/validate.R
    • R/validate_custom.R
    • tests/testthat/test_constraints.R
  • Added documentation:

    • docs/constraint-validation.md
    • docs/constraint-decisions.md
  • Removed duplicate obsolete constraint source:

    • deleted data-raw/custom_constraints.yaml

Build-time filtering/quarantine in make-data.R is intentionally retained and documented, but not redesigned in this PR.

Note:
A pre-existing legacy test expecting traitsview > 40000 rows continues to fail locally because current packaged data contains ~18010 filtered/public records; this is unrelated to the constraint additions in this PR.

Related Issue(s)

Closes #14

Also fulfills and supersedes the machine-readable constraint definition work in #22, now extending that schema foundation with validation logic, tests, and coverage documentation.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Data update (changes to datasets)
  • Documentation update
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have run devtools::check() with no errors or warnings
    (devtools::test() passes except for one unrelated pre-existing row-count test)
  • I have updated NEWS.md if applicable
  • I have updated documentation if applicable
  • I have added/updated tests if applicable

Data Changes (if applicable)

  • Data rebuilt from source using data-raw/make-data.R
  • datapackage.json updated
  • Row counts verified

Additional Notes

Scope intentionally focuses on high-value constraints for shipped tables and documents deferred constraints with rationale, rather than attempting exhaustive migration of all historical BETYdb constraints.

Two candidate uniqueness constraints were intentionally moved from proposed implementation to deferred status after validation against shipped data showed current uniqueness assumptions were too broad.

This PR includes both implementation and coverage-decision documentation in response to review comments requesting explicit treatment of constraints not migrated.

@Abhishek-Kumar-Rai5
Copy link
Copy Markdown
Author

@dlebauer sir, please have a look into this whenever you have a second and provide your review and suggestions on this.

@Abhishek-Kumar-Rai5 Abhishek-Kumar-Rai5 force-pushed the feat/migration-of-constraints-validation branch from d74da8e to 97bc47a Compare April 26, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement (migrate) constraints and validation from postgres betydb

1 participant