Evaluate Comprehensive Simulation Accuracy (Unimodal Omics Layer)

Computes distribution distances, cellular and feature-level properties, zero-inflation curves, and manifold distances between reference and simulated datasets.

Usage

evaluate_simulation_accuracy(
  ref_data,
  sim_data,
  compute_bivariate = TRUE,
  threads = 1,
  memory_mb = NULL,
  elapsed_time = NULL,
  peak_memory_mb = NULL,
  verbose = TRUE
)

Arguments

ref_data

Reference count matrix (features x cells).

sim_data

Simulated count matrix (features x cells).

compute_bivariate

Logical, whether to compute 2D bivariate tests. Default is TRUE.

threads

Number of threads for parallel computation. Default is 1.

memory_mb

Optional numeric value of peak memory allocation in MB (or peak_memory_mb).

elapsed_time

Optional numeric value of wall-clock elapsed time in seconds.

peak_memory_mb

Optional alias for memory_mb.

verbose

Logical, whether to print execution messages. Default is TRUE.

Value

A list containing tidy summary tables and detailed metric lists.

Details

Metric Measurement Capacity by Data & Metadata Mode:

• Count Matrix Alone (No Metadata): Measures approximately 23 baseline metrics spanning Category I (Distributional Properties: KS, Wasserstein, MAD, MAE, RMSE, OV, Bhattacharyya, ECDF difference, Runs test, 2D Fasano-Franceschini, 2D Peacock, KDE test, MMD, Fréchet distance), Category II (Correlations & Zero-Inflation: BCV discrepancy, dropout curve midpoint/slope, mean-variance R², cell/gene Pearson correlations), and Category VI (Trajectory & Lineage Dynamics: pseudotime correlation, tree height RMSE).

• + Cell Types (Optional): Unlocks 12–15 additional metrics spanning Category III (Cellular Structure & Concordance: ARI, NMI, AMI, Silhouette width, Dunn, Davies-Bouldin, Calinski-Harabasz, V-measure, Neighborhood Purity, Generative Precision/Recall) and Category V (Biological Signal & Downstream Fidelity: SimBench DE/DP/DM/DB/DV signal fidelity, top DEG Jaccard overlap, Log2FC correlation, classifier accuracy and Macro-F1).

• + Batch Annotations (Optional): Unlocks 7 additional metrics spanning Category IV (Batch Effects & Confounder Mixing: Batch Silhouette width, local neighborhood Shannon Entropy, PCR R², CMS CellMixS, Inverse Simpson Index ISI, Seurat Mixing Metric, LDF difference).

• Full Metadata (Count Matrix + Cell Types + Batch): Unlocks the complete unimodal benchmarking suite of up to 55 metrics.

Examples

data(example_scrna, package = "scSimEval")
res <- evaluate_simulation_accuracy(example_scrna$ref, example_scrna$sim)
#> [1/5] Extracting cell-level properties...
#> [2/5] Extracting feature-level properties...
#> [3/5] Computing univariate accuracy metrics...
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> [4/5] Computing bivariate metrics (Fasano-Franceschini, Peacock, KDE zstat, 2D EMD)...
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> [5/5] Computing zero-probability and manifold distances...
#> Unimodal accuracy evaluation complete.