Meta-analysis of (single-cell method) benchmarks reveals the need for extensibility and interoperability.
Genome Biol
; 24(1): 119, 2023 05 17.
Article
in En
| MEDLINE
| ID: mdl-37198712
ABSTRACT
Computational methods represent the lifeblood of modern molecular biology. Benchmarking is important for all methods, but with a focus here on computational methods, benchmarking is critical to dissect important steps of analysis pipelines, formally assess performance across common situations as well as edge cases, and ultimately guide users on what tools to use. Benchmarking can also be important for community building and advancing methods in a principled way. We conducted a meta-analysis of recent single-cell benchmarks to summarize the scope, extensibility, and neutrality, as well as technical features and whether best practices in open data and reproducible research were followed. The results highlight that while benchmarks often make code available and are in principle reproducible, they remain difficult to extend, for example, as new methods and new ways to assess methods emerge. In addition, embracing containerization and workflow systems would enhance reusability of intermediate benchmarking results, thus also driving wider adoption.
Full text:
1
Database:
MEDLINE
Main subject:
Computational Biology
/
Benchmarking
Type of study:
Guideline
/
Systematic_reviews
Language:
En
Year:
2023
Type:
Article