Name		Name	Last commit message	Last commit date
parent directory ..
.gitignore		.gitignore
README.md		README.md
chatbot_arena.csv		chatbot_arena.csv
chatbot_arena.py		chatbot_arena.py
requirements.txt		requirements.txt
rust_python.csv		rust_python.csv
rust_python.py		rust_python.py
scale.csv		scale.csv
scale_compute.py		scale_compute.py
scale_data.py		scale_data.py

README.md

Code to Reproduce Experiments from the COLING 2025 Paper

Ustalov, D. Reliable, Reproducible, and Really Fast Leaderboards with Evalica. 2024. arXiv: 2412.11314 [cs.CL].

Prerequisites

requirements.txt
Chatbot Arena's Dump (August 2024): https://storage.googleapis.com/arena_external_data/public/clean_battle_20240814_public.json
LLMFAO Dataset: https://raw.githubusercontent.com/dustalov/llmfao/refs/heads/master/crowd-comparisons.csv → llmfao.csv

Table 1: chatbot_arena.csv

python3 -m chatbot_arena

Table 2: rust_python.csv

python3 -m rust_python

Figure 3: scale.csv

python3 -m scale_data
python3 -m scale_compute