Clustering correlated parameters for block-wise compression (for posteriorHCA brms fit object compression ) #2
Open
zhanchen wants to merge 19 commits into
Open
Conversation
When mclust selects an axis-aligned covariance model
(EII/VII/EEI/VEI/EVI/VVI), keep the per-component covariances as a
d x G matrix of diagonals instead of a mostly-zero d x d x G array,
so high-dimensional posteriors actually compress (d^2 G -> d G).
A `covariance_type` field ("full" | "diagonal" | "shared") drives
fast diagonal paths in `.sample_mclust()` and `.density_mclust()`
that skip building the d x d sigma entirely; legacy compressed
objects without `covariance_type` are handled by inferring from
shape, so existing files keep working.
Also stop `print()` on the result of `compress_brmsfit()` /
`compress_sccomp()` from dispatching to `brms::print.brmsfit` on the
draws-stripped structure (which fails inside brms's summary path).
The wrapper list is now classed `compressed_brmsfit` /
`compressed_sccomp` (both inheriting `compressed_fit`) with a small
print method that summarises both halves and points the user at the
right `reconstruct_*()`.
Introduce correlation-driven partitioning, blockwise compression with constrained mclust covariance families, and blockwise-aware sampling/density/reconstruction diagnostics while keeping dev artifacts out of the commit.
…d dev notes and tests.
… colnames Register print S3 methods for compressed_brmsfit/sccomp and brmsfit_stripped. Unwrap compressed_fit in density_posterior. evaluate_compression documents compressed_fit; ranger uses x/y interface with safe colnames. Tests and qmd. Restore NAMESPACE exports (evaluate_compression, print.compression_fidelity, stats::predict).
Refactor cluster partitioning around partition_parameters_clusters(): add simple_output (default TRUE) so the default return is a plain list of cluster vectors for compress_* partition, with simple_output = FALSE restoring the full poco_partition_clusters object. Remove the legacy rest field everywhere and update partition_remainder(), blockwise helpers, and tests. Teach plot_posterior_correlation() heatmaps to annotate simple partitions via .enrich_partition_for_heatmap_plot(). Allow .coerce_blockwise_partition() to coerce list() (no clusters) to an all-remainder partition; clarify errors and partition_blocks() docs; add a blockwise test. Refresh compress_posterior() partition docs, correlation roxygen/examples, and the posteriorHCA.
Use univariate E/V when d=1 but model_name lists multivariate diagonal families. Store univariate covariances as 1×G diagonal to avoid mvtnorm isSymmetric errors on scalars. Coerce full-cov slices to matrices; infer diagonal vs shared when covariance_type is missing; normalize mclust means if returned as G×d. Add tests for univariate fit, stripped covariance_type, and blockwise remainder sampling/density.
Only transpose means when shape is G×d but not already d×G. The previous rule nrow(means)==G && ncol(means)==d fired for correct d×G matrices whenever d==G (common for small cluster blocks), swapping parameter and component axes and tanking sample fidelity / evaluate_compression. Add regression test on two-blob d=G=2 fit.
Retry failed default ellipsoidal mclust cluster fits with diagonal/spherical models once, with user-visible messages and a leading newline on stderr so BiocParallel progress lines stay readable. Accept scalar numeric cluster_BPPARAM as a worker count (ceiling for non-integers, cap by cluster blocks and CPUs, values below two or non-finite coerced to sequential with messages) and build MulticoreParam or SnowParam accordingly. Document cluster_BPPARAM and list every compress_brmsfit argument in roxygen and compress_brmsfit.Rd. Add blockwise tests for numeric cluster_BPPARAM.
… compatibale with map_dfr, etc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.