Fix device handling and logits concatenation in OliveEvaluator #1615
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
- Add exception handling for unsupported devices in `device_string_to_torch_device` method. - Correct logits concatenation in `OnnxEvaluator` by using `logits_dict` instead of `logits`. - Initialize `logits_dict` in `PyTorchEvaluator` to handle different result types. - Update `_inference` method in `PyTorchEvaluator` to handle different result types and concatenate logits correctly.
tezheng
force-pushed
the
fix/eval_support_dict_as_output
branch
from
306da8e to
09138a8
Compare
jambayk
reviewed
Feb 14, 2025
| @@ -196,7 +196,11 @@ def compute_throughput(metric: Metric, latencies: Any) -> MetricResult: | |||
| class _OliveEvaluator(OliveEvaluator): | |||
| @staticmethod | |||
| def device_string_to_torch_device(device: Device): | |||
| return torch.device("cuda") if device == Device.GPU else torch.device(device) | |||
| try: | |||
There was a problem hiding this comment.
maybe we can just do torch.device("cuda") if device == Device.GPU else torch.device("cpu"). Otherwise, this would raise the warning for npu targets everytime. since this method is only used when evaluating pytorch models, I think it's save to just return cuda or cpu only.
There was a problem hiding this comment.
So we will only consider cuda and cpu, for now at least? Are we planning to support device like Apple mps?
There was a problem hiding this comment.
yes, there are no plans for other torch devices like mps for now.
xiaoyu-work
reviewed
Feb 18, 2025
Comment on lines
+756
to
+762
| if isinstance(result, torch.Tensor): | ||
| logits.append(result.cpu()) | ||
| elif isinstance(result, (list, tuple)): | ||
| logits.append([r.cpu() for r in result]) | ||
| elif isinstance(result, dict): | ||
| for k in result: | ||
| logits_dict[k].append(result[k].cpu()) |
There was a problem hiding this comment.
How can we handle the case when result has logits attribute?
There was a problem hiding this comment.
You mean result like { "logits": tensor }?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
device_string_to_torch_devicemethod.OnnxEvaluatorby usinglogits_dictinstead oflogits.logits_dictinPyTorchEvaluatorto handle different result types._inferencemethod inPyTorchEvaluatorto handle different result types and concatenate logits correctly.Describe your changes
Checklist before requesting a review
lintrunner -a(Optional) Issue link