Skip to content

[Tracker] All the issue related with ONNX model zoo models #886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pdhirajkumarprasad opened this issue Nov 20, 2024 · 2 comments
Open

Comments

@pdhirajkumarprasad
Copy link

pdhirajkumarprasad commented Nov 20, 2024

We have enabled total 2219 unique models from ONNX library. This issue is to track all the issues seen in the models.

Passing Summary

TOTAL TESTS = 2219

Stage # Passing % of Total % of Attempted
Setup 2217 99.9% 99.9%
IREE Compilation 2149 96.8% 96.9%
Gold Inference 2149 96.8% 100.0%
IREE Inference Invocation 2146 96.7% 99.9%
Inference Comparison (PASS) 2011 90.6% 93.7%

Fail Summary

TOTAL TESTS = 2219

Stage # Failed at Stage % of Total
Setup 2 0.1%
IREE Compilation 68 3.1%
Gold Inference 0 0.0%
IREE Inference Invocation 3 0.1%
Inference Comparison 135 6.1%

CPU

# issue type issue no #model impacted list of model assignee status
1 failed to legalize operation 'torch.operator' -> QLinearAdd / QLinearLeakyRelu / QLinearConcat / onnx.QLinearGlobalAveragePool 949 19 QLinearAdd Importer - Torch MLIR - (Assigned)
2 failed to legalize operation 'torch.constant.float 908 6 Roi Align Torch MLIR (In Progress)
3 failed to legalize operation 'onnx.Loop' 948 7 Loop Torch MLIR - (In Progress)
4 Assertion `result && ""Zero-sized vectors are not allowed""' failed.\n' 913 3 If Torch MLIR - (Unassigned)
5 expected offsets to be non-negative, but got -1 19935 2 Offsets IREE - (Unassigned)
6 Incorrect imported IR 905 8 Incorrect IR Torch MLIR - (Unassigned)
7 failed to legalize operation 'torch.operator' -> AvgPool 895 3 AvgPool Torch MLIR (Review) - 3923
8 'tensor.cast' op operand type 'tensor<80x15130xf32>' and result type 'tensor<15130xf32>' are cast incompatible 891 2 NMS Torch MLIR - Review 3999, IREE - Assigned19474
9 error: invalid size -9223372036854775808 for !torch.tensor type 3 Invalid
10 func.func' op exceeded stack allocation limit of 32768 bytes for function. Got 786432 bytes 2 Stack allocation limit
11 failed to legalize operation "onnx.CategoryMapper" 910 1 CategoryMapper
12 'hal.tensor.barrier' op failed to verify that all of {sources, results} have same type 1 Barrier Resolves with onnx runtime shape infer
13 failed to legalize operation 'torch.operator' that was explicitly marked illegal onnx.If 930 1 Onnx.If Importer issue
14 failed to legalize operation 'onnx.LSTM' 947 1 onnx.LSTM Torch MLIR- (In Progress)
15 'linalg.generic' op write affecting operations on global resources are restricted to workgroup distributed contexts 20156 1 Conv IREE - (Unassigned)
@pravg-amd
Copy link

Latest categorization of the errors is updated here

@pravg-amd
Copy link

Models failing at setup stage are disabled as part of the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants