Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Geekbench crowdsources CPU performance matrix. Should SCER explore similar crowdsourcing approach to LLMs carbon footprint/efficiencies? #56

Open
Tracked by #57
chrisxie-fw opened this issue May 23, 2024 · 0 comments
Open
Tracked by #57

Geekbench crowdsources CPU performance matrix. Should SCER explore similar crowdsourcing approach to LLMs carbon footprint/efficiencies? #56

chrisxie-fw opened this issue May 23, 2024 · 0 comments
Assignees
@chrisxie-fw
Labels
question Further information is requested

Comments

@chrisxie-fw
Copy link
Contributor

Geekbench: https://browser.geekbench.com/v6/cpu/singlecore

Huggingface: It appears that huggingface is doing evaluations of LLMs in their backend in the form of leaderboard. Is it possible to democratize that? Are there ways for huggingface or the open source community to make the LLMs evaluation process more transparent and make the results more easily reproducible?
@chrisxie-fw chrisxie-fw added the question Further information is requested label May 23, 2024
@seanmcilroy29 seanmcilroy29 mentioned this issue May 24, 2024
Open
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chrisxie-fw chrisxie-fw

Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@chrisxie-fw