Skip to content

Issues: UKGovernmentBEIS/inspect_ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

45 Open 259 Closed
45 Open 259 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Installation raises: "The dependency name for inspect-ai does not match the actual package's name: inspect-evals"
#1306 opened Feb 12, 2025 by tadamcz
1
Include sample id in docker project name
#1304 opened Feb 12, 2025 by AlexLloyd0
2
web_browser: strip base64 encoded images from accessibility trees web browser
#1303 opened Feb 12, 2025 by jjallaire-aisi
Non-deterministic error in proto package when using Google models
#1299 opened Feb 12, 2025 by tadamcz
4
[feature request] progress of forked solvers / subtasks are not visible in terminal viewer
#1296 opened Feb 12, 2025 by skinnerjc
2
[feature request] set default cache policy
#1292 opened Feb 11, 2025 by kaifronsdal
1
MATH dataset used for mathematics tutorial no longer accessible
#1283 opened Feb 10, 2025 by ChiWilliams
1
improved retry behavior
#1277 opened Feb 10, 2025 by jjallaire
1
Custom Displays
#1267 opened Feb 7, 2025 by Baidicoot
1
Run cleanup and finalizers
#1265 opened Feb 7, 2025 by sjawhar
Unique ID for each run of each sample
#1262 opened Feb 7, 2025 by sjawhar
Log viewer: scroll bar resets when using the scorer drop-down
#1220 opened Jan 30, 2025 by tadamcz
@dragonstyle
[Feature request] Support for LiteLLM and LiteLLM proxy
#1213 opened Jan 30, 2025 by danrr
Execute --runslow tests in CI
#1212 opened Jan 30, 2025 by jjallaire-aisi
@art-dsit
1
Tutorial: Mathematics dataset not available in HF anymore
#1183 opened Jan 24, 2025 by guiem
Broader notion of retryable exception
#1174 opened Jan 22, 2025 by tadamcz
10
Feature Request: Grouped output metrics & exportable result files for benchmarks
#1172 opened Jan 22, 2025 by owenparsons
@dragonstyle
3
Enable sandboxes to connect to existing Docker or k8s instances
#1153 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Enforcing cost limits
#1152 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
2
Broader notions of "model" for evals
#1149 opened Jan 19, 2025 by jjallaire
@jjallaire
Bridge to convert from compose.yaml to HELM values
#1148 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
1
Improve task scheduler to not require model as an axis
#1146 opened Jan 19, 2025 by jjallaire
@jjallaire
Direct linking to samples and transcript entries in Inspect View
#1145 opened Jan 19, 2025 by jjallaire
@dragonstyle
Checkpointing and restarting from checkpoints
#1144 opened Jan 19, 2025 by jjallaire
@jjallaire
Log annotation UI
#1143 opened Jan 19, 2025 by jjallaire
@dragonstyle
ProTip! Follow long discussions with comments:>50.