Skip to contents

Compute Simulator Method Performance Leaderboard

Source: R/10_visualizations.R
compute_method_leaderboard.Rd

Computes the overall performance leaderboard ranking simulation methods across all benchmark evaluation metrics. Metrics are first direction-inverted and min-max standardized into fidelity scores in [0, 1] (where 1.0 represents best observed performance). Simulators are then rank-ordered by mean overall fidelity score.

Usage

compute_method_leaderboard(benchmark_data)

Arguments

benchmark_data

A benchmark data.frame (such as demo$benchmark_summary_table or output from evaluate_simulation_accuracy()) or a named list of benchmark result tables.

Value

A data.frame with columns:

  • Overall_Rank: Integer rank (1 = top-performing simulator).

  • Method: Simulator method name.

  • Average_Fidelity: Formatted percentage string (e.g., "58.5%").

  • Fidelity_Score: Numeric composite fidelity score in [0, 1] rounded to 4 decimals.

Examples

data(example_scrna, package = "scSimEval")
res <- evaluate_simulation_accuracy(example_scrna$ref, example_scrna$sim)
#> [1/5] Extracting cell-level properties...
#> [2/5] Extracting feature-level properties...
#> [3/5] Computing univariate accuracy metrics...
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> Metric: 'bhattacharyya' with unit: 'log'; comparing: 2 vectors
#> [4/5] Computing bivariate metrics (Fasano-Franceschini, Peacock, KDE zstat, 2D EMD)...
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> Warning: argument is not numeric or logical: returning NA
#> Warning: emd: Maximum number of iterations has been reached (500)
#> [5/5] Computing zero-probability and manifold distances...
#> Unimodal accuracy evaluation complete.
lb <- compute_method_leaderboard(res)