CoIR (Code Information Retrieval Benchmark)
Introduced by Li et al. in CoIR: A Comprehensive Benchmark for Code Information Retrieval Models
CoIR (Code Information Retrieval) benchmark, is designed to evaluate code retrieval capabilities. CoIR includes 10 curated code datasets, covering 8 retrieval tasks across 7 domains. In total, it encompasses two million documents. It also provides a common and easy Python framework, installable via pip, and shares the same data schema as benchmarks like MTEB and BEIR for easy cross-benchmark evaluations.
Benchmarks
| Trend | Task | Dataset Variant | Best Model | Paper | Code |
|---|---|---|---|---|---|
|
|
|
|||
|
|
CovidCTNet
|
|||
|
|
Fine tuned Yolov5xu
|
|||
|
|
Voyage-code-002
|
|||
|
CoIR
|
Voyage-code-002
|
|||
|
|
|
Papers
| Paper | Code | Results | Date | Stars |
|---|
Dataset Loaders
No data loaders found. You can
submit your data loader here.
No data loaders found. You can
submit your data loader here.
