Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
108 Open 1,345 Closed
108 Open 1,345 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Capture gen_kwargs from CLI in squad_completion
#2727 opened Feb 23, 2025 by ksurya Loading…
1
Groundcocoa
#2724 opened Feb 22, 2025 by HarshKohli Loading…
1
Add cocoteros_es task in spanish_bench
#2721 opened Feb 21, 2025 by sgs97ua Loading…
1
Add support for sequence labeling
#2718 opened Feb 20, 2025 by jogonba2 Loading…
New healthcare benchmark: careqa
#2714 opened Feb 19, 2025 by PabloAgustin Loading…
4
Add AIBE task and utilities
#2712 opened Feb 18, 2025 by parimalthakre01 Loading…
1
feat: Add mmlu-redux and it's spanish transaltion as generative task definitions
#2705 opened Feb 16, 2025 by luiscosio Loading…
1
Support SGLang as Potential Backend for Evaluation
#2703 opened Feb 15, 2025 by Monstertail Loading…
38
Add Task (Financial mmlu ko)
#2699 opened Feb 14, 2025 by choics2623 Loading…
1
add o3-mini support
#2697 opened Feb 14, 2025 by HelloJocelynLu Loading…
1
1
add audio modality (qwen2 audio only)
#2689 opened Feb 12, 2025 by artemorloff Loading…
7
Add generation variants of some tasks
#2688 opened Feb 11, 2025 by baberabb Loading…
Convert gen tasks to multiple_choice
#2670 opened Feb 4, 2025 by baberabb • Draft
[hf-multimodal] pass kwargs to self.processor
#2667 opened Jan 31, 2025 by baberabb Loading…
1
3
Add from dataframe
#2655 opened Jan 25, 2025 by AMindToThink Loading…
1
humaneval instruct
#2650 opened Jan 22, 2025 by baberabb Loading…
1
Easily evaluate models steered by SAEs
#2641 opened Jan 21, 2025 by AMindToThink Loading…
1
Include all test files in sdist
#2634 opened Jan 19, 2025 by booxter Loading…
3
Add loncxt tasks
#2629 opened Jan 17, 2025 by baberabb • Draft
1
fix: nemo eval in containers with TransformerEngine > 1.10 won't error
#2621 opened Jan 10, 2025 by terrykong • Draft
1
Added EU20 task suite
#2620 opened Jan 10, 2025 by KlaudiaTH Loading…
2
Add support for generative answering of multiple_choice tasks
#2601 opened Dec 29, 2024 by pasky Loading…
9
change to single process for bootstrap_stderr
#2593 opened Dec 23, 2024 by zhuyuhua-v Loading…
4
Update dependencies in pyproject.toml: remove duplicated 'evaluate', add 'anthropic' and 'openai' to optional dependencies
#2582 opened Dec 19, 2024 by Technolog796 Loading…
5
Added caseHOLD task
#2570 opened Dec 16, 2024 by zolastro Loading…
2
ProTip! Filter pull requests by the default branch with base:main.