Package index • scSimEval

Master Benchmarking Workflows

Unified end-to-end evaluation pipelines executing the 62 canonical metrics across unimodal and multiomics simulations.

evaluate_simulation_accuracy(): Evaluate Comprehensive Simulation Accuracy (Unimodal Omics Layer)
evaluate_multiomics_accuracy(): Master Multiomics Simulation Benchmarking Pipeline
evaluate_multiple_datasets(): Evaluate Multiple Single-Cell Datasets Simultaneously
benchmark_simulation_suite(): Benchmark Simulation Methods Across Multiple Single-Cell Datasets

High-Resolution Visualization Suite (600 DPI)

Publication-grade visualization functions designed for multi-metric and cross-simulator benchmarking.

plot_benchmark_bubble_matrix() plot_bubble_matrix(): Plot Multi-Dimensional Benchmarking Bubble Matrix
plot_evaluation_summary() plot_benchmark_summary_bars() plot_summary_bars(): Plot Single-Cell Simulator Evaluation Summary
plot_distribution_qc(): Plot Single-Cell Summary Distribution Quality (Comparative Overlays)
plot_scalability_benchmark(): Plot Multi-Dimensional Computational Scalability Benchmark
plot_metric_boxplots(): Plot Comparison Boxplots Across Simulators Plot Benchmark Metric Score Distributions Across Simulators
plot_category_metric_bars(): Plot Benchmark Metric Performance Barplots by Category
plot_metric_heatmap(): Plot Multi-Simulator Comparative Metric Heatmap Across Canonical Categories
plot_metric_pca(): Principal Component Analysis (PCA) Dashboard of Benchmark Metrics and Methods
plot_metric_mds(): Multi-Dimensional Scaling (MDS) Ordination of Evaluation Metrics or Simulators
plot_individual_metric_bar(): Plot Individual Metric Performance Barplot
compute_dataset_embeddings(): Compute Low-Dimensional Embeddings for Reference and Simulated Single-Cell Datasets
plot_dataset_embeddings(): Plot High-Dimensional Dataset Embeddings for Reference and Simulated Single-Cell Data
compute_embedding_quality_metrics(): Compute Quantitative Quality Metrics for Low-Dimensional Cell Embeddings
compute_method_leaderboard(): Compute Simulator Method Performance Leaderboard

Interactive Graphical Interface (Shiny App Studio)

Interactive Shiny web application providing point-and-click simulation benchmarking, UMAP/t-SNE embedding explorations, and dynamic data packaging.

launch_scSimEval_app(): Launch Interactive scSimEval Benchmarking Studio

Category-Specific Evaluation Suites

Targeted evaluation functions executing specialized metric pipelines.

evaluate_batch_metrics(): Calculate Batch Integration Metrics
evaluate_clustering_metrics(): Full Clustering Performance Evaluation
evaluate_trajectory_metrics(): Full Trajectory Accuracy Evaluation
evaluate_cross_batch_prediction(): Evaluate Cross-Batch Cell Classification Transfer Accuracy
evaluate_cross_modal_prediction(): Cross-Modal Cell-Type Label Transfer Accuracy
evaluate_deg_fidelity(): Evaluate Differentially Expressed Gene (DEG) Fidelity
evaluate_predictive_de_model(): Predictive Cell Identity Classification Using DE / Top Variable Features
evaluate_simbench_signals(): Evaluate the 5 SimBench Biological Signal Proportions
evaluate_epigenomic_annotation(): Evaluate Epigenomic Supervised Cell-Type Annotation (EpiAnno / SCAN-ATAC-Sim)
evaluate_zero_probability_curve(): Evaluate Zero-Probability Dropout Curve Fidelity (Splatter & ZINB-WaVE)
evaluate_accessibility_sparsity_curve(): Evaluate Accessibility-Sparsity Curve Concordance (simATAC)
benchmark_resource_usage(): Benchmark Execution Runtime and Memory Usage

Data Extraction & Empirical Properties

Extraction of feature-level and cell-level empirical properties.

extract_cell_properties(): Extract Comprehensive Cell-Level Properties
extract_feature_properties(): Extract Comprehensive Feature-Level (Gene / Peak) Properties
extract_dataset_summary(): Extract Dataset Dimensional Properties and Summary Statistics

Example Multiomics Datasets & Metadata

Pre-packaged synthetic and empirical single-cell multiomics benchmark datasets and metadata annotation templates.

example_scrna: Synthetic Example scRNA-seq Benchmarking Dataset
example_scatac: Synthetic Example scATAC-seq Benchmarking Dataset
example_multiomics: Synthetic Example Multiomics Benchmarking Dataset
example_cell_types: Example Cell-Type Annotations Vector
example_batch_info: Example Batch Annotations Vector

Computational Utilities & Fast Kernels

High-performance matrix calculation helpers and internal dimension-reduction utilities.

fast_row_vars(): Fast Row-Wise Variances for Dense or Sparse Single-Cell Matrices
fast_col_vars(): Fast Column-Wise Variances for Dense or Sparse Single-Cell Matrices
fast_sparse_zero_fraction_cell(): Fast Zero Fraction per Cell (Column-wise)
fast_sparse_zero_fraction_feature(): Fast Zero Fraction per Feature (Row-wise)
fast_cross_distance_sq(): Fast BLAS-Accelerated Cross-Distance Squared Matrix
fast_pca(): Fast Truncated PCA on Top Highly Variable Features
subsample_dist_vector(): Subsample Large Vectors for Accelerated Distributional Distance Estimation

Category 1: Global & Marginal Distributions

Evaluation functions assessing univariate and bivariate distribution divergence.

calc_ks(): Calculate Kolmogorov-Smirnov Distance (1D KS)
calc_wasserstein_1d(): Calculate 1D Wasserstein Metric / Earth Mover's Distance (WS)
calc_mad(): Calculate Median Absolute Deviation (MAD)
calc_mae(): Calculate Mean Absolute Error (MAE)
calc_rmse(): Calculate Root Mean Squared Error (RMSE)
calc_overlap(): Calculate Distribution Overlapping Index (OV)
calc_bhattacharyya(): Calculate Bhattacharyya Distance (BH)
calc_ecdf_diffarea(): Calculate Area Between Empirical Cumulative Distribution Functions (eCDFs)
calc_runs_test(): Calculate Wald-Wolfowitz Runs Test on Pooled Distributions
calc_nn_mismatch(): Calculate Nearest-Neighbor Label Mismatch Proportion
calc_between_dataset_silhouette(): Calculate Between-Dataset Silhouette Width
calc_fasano_franceschini(): Calculate Fasano-Franceschini 2D Kolmogorov-Smirnov Test Statistic
calc_peacock_2d(): Calculate Peacock 2D Kolmogorov-Smirnov Test Statistic
calc_kde_test(): Calculate 2D Bivariate Kernel Density Estimation (KDE) Test Statistic
calc_emd_2d(): Calculate 2D Earth Mover's Distance (2D EMD)
calc_mmd(): Maximum Mean Discrepancy (MMD) with Gaussian RBF Kernel
calc_frechet_singlecell_distance(): Frechet Single-Cell Distance (FSD)
calc_model_aic_bic(): Compute Model Information Criteria (AIC and BIC)
calc_marginal_aic_bic(): Marginal Model Goodness of Fit and Information Criteria for Single-Cell Simulators
calc_likelihood_ratio_test(): Perform Likelihood Ratio Test for Comparing Nested Single-Cell Simulation Models
calc_outlier_proportion(): Calculate Proportion of Outliers in a Numeric Vector
calc_all_univariate_metrics(): Compute All Univariate Distance & Accuracy Metrics for a Feature
calc_all_bivariate_metrics(): Compute All Bivariate (2D) Distance & Accuracy Metrics for Joint Distributions

Category 2: Cellular & Feature-Level Properties

Assessment of zero-inflation dynamics, library size distributions, and kinetic noise decomposition.

calc_zero_probability_curve(): Fit Zero-Probability Dropout Curve (Splatter & ZINB-WaVE)
calc_excess_zero_weights(): Calculate Posterior Excess Zero Weights (zingeR & ZINB-WaVE)
calc_kinetic_noise_decomposition(): Kinetic Noise Decomposition (SymSim & Elowitz et al.)
calc_accessibility_sparsity_curve(): Fit Chromatin Accessibility-Sparsity Polynomial Curve (simATAC)

Category 3: Cell Identity & Clustering Concordance

Concordance of unsupervised cluster definitions, cluster separation, and latent manifold recovery.

calc_ari(): Calculate Adjusted Rand Index (ARI)
calc_nmi(): Calculate Normalized Mutual Information (NMI)
calc_ami(): Calculate Adjusted Mutual Information (AMI)
calc_clustering_accuracy(): Calculate Clustering Accuracy (ACC)
calc_fmi(): Calculate Fowlkes-Mallows Index (FMI)
calc_silhouette(): Calculate Average Silhouette Width (ASW)
calc_dunn(): Calculate Dunn Index
calc_connectivity(): Calculate Cluster Connectivity
calc_davies_bouldin(): Calculate Davies-Bouldin Index (DB)
calc_calinski_harabasz(): Calculate Calinski-Harabasz Index (CH)
calc_homogeneity(): Calculate Homogeneity Score
calc_completeness(): Calculate Completeness Score
calc_v_measure(): Calculate V-Measure
calc_homogeneity_completeness_v_measure(): Calculate Homogeneity, Completeness, and V-Measure
calc_neighborhood_purity(): Calculate Neighborhood Purity
calc_cdi(): Compute Clustering Deviation Index (CDI) for Single-Cell Clustering Evaluation
calc_generative_precision_recall(): Generative Precision and Recall for Single-Cell Manifolds

Category 4: Batch Effect & Confounder Dynamics

Quantification of technical variation, batch mixing, and local density factor conservation.

calc_cms(): Cell-Specific Mixing Score (CMS)
calc_isi(): Inverse Simpson Index (ISI) for Batch Mixing
calc_seurat_mixing_metric(): Seurat Mixing Metric
calc_ldf(): Local Density Factor (LDF)
calc_ldf_diff(): Local Density Differences (ldfDiff)
calc_local_structure_metric(): Local Structure Preservation Metric

Category 5: Differential Expression & Signal Preservation

Recovery of biological contrast, differential gene detection accuracy, and cell-type deconvolution fidelity.

calc_signal_de(): Detect Differential Expression (DE) Genes (Mean Shift via Limma or T-Test)
calc_signal_dv(): Detect Differential Variability (DV) Genes (Bartlett Test)
calc_signal_dd(): Detect Differential Distribution (DD) Genes (Kolmogorov-Smirnov Test)
calc_signal_dp(): Detect Differential Proportion / Zero-Inflation (DP) Genes (Chisq Test)
calc_signal_bd(): Detect Bimodal Distribution (BD) Genes (Bimodal Separation Index)
calc_delta_variance(): Calculate Delta Variance (Pseudoreplication Bias Metric)
calc_deconvolution_accuracy(): Evaluate Cell-Type Deconvolution & Mixture Proportion Accuracy
calc_variance_decomposition(): Variance Component Decomposition (BASiCS & muscat)
calc_intraclass_correlation(): Calculate Intraclass Correlation Coefficient (ICC) (hierarchicell & rescueSim)
calc_expected_mi(): Expected Mutual Information for Cluster Comparison

Category 6: Trajectory & Lineage Dynamics

Inference and benchmarking of differentiation pseudotime trajectories and lineage branching topologies.

infer_scrna_pseudotime(): Automatically Infer Pseudotime Trajectory from scRNA-seq Counts
infer_scrna_lineage_tree(): Automatically Infer Lineage Tree (hclust) from scRNA-seq Counts & Cell Types
calc_pseudotime_correlation(): Calculate Correlation of Geodesic Pseudotime Distances
calc_tree_height_discrepancy(): Calculate Lineage Tree Branch Height Discrepancy (RMSE)

Category 7: Multiomics Cross-Modality Coupling

Inter-modality regulatory coupling, peak-to-gene linkages, and joint chromatin-expression modularity.

calc_cross_modality_correlation(): Calculate Cross-Modality Correlation Fidelity
calc_foscttm(): Fraction of Samples Closer Than The True Match (FOSCTTM)
calc_cross_modal_generation(): Cross-Modality In Silico Generation & Translation Fidelity
calc_modality_alignment(): Modality Alignment & Omics Layer Mixing in Joint Latent Space
calc_network_jaccard(): Jaccard Similarity of Predicted Regulatory Networks / Links
calc_atac_rna_coupling(): Chromatin Accessibility to RNA Regulatory Effect Coupling
calc_coregulation_fidelity(): Multi-Omics Co-Regulation & Modularity Fidelity
calc_peak_coaccessibility_fidelity(): Evaluate Single-Cell ATAC Peak Co-Accessibility Fidelity (SCRIP)
calc_coexpression_module_fidelity(): Evaluate Gene Co-Expression Module Fidelity (ESCO)
calc_accessibility_profile_concordance(): Evaluate Chromatin Accessibility Profile Concordance (DiTSim)

Category 8: Computational Scalability & Resource Footprint

Empirical benchmarking of CPU runtime, peak RAM consumption, and execution efficiency.

benchmark_resource_usage(): Benchmark Execution Runtime and Memory Usage