Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracker] All the issue related with ONNX model zoo models #886

Open
pdhirajkumarprasad opened this issue Nov 20, 2024 · 0 comments
Open

Comments

@pdhirajkumarprasad
Copy link

pdhirajkumarprasad commented Nov 20, 2024

We have enabled total 2325 models from ONNX model zoo. This issue is to track all the issues seen in the models.

Passing Summary

TOTAL TESTS = 2325

Stage # Passing % of Total % of Attempted
Setup 2311 99.4% 99.4%
IREE Compilation 2024 87.1% 87.6%
Gold Inference 2024 87.1% 100.0%
IREE Inference Invocation 2015 86.7% 99.6%
Inference Comparison (PASS) 1959 84.3% 97.2%

Fail Summary

TOTAL TESTS = 2325

Stage # Failed at Stage % of Total
Setup 14 0.6%
IREE Compilation 287 12.3%
Gold Inference 0 0.0%
IREE Inference Invocation 9 0.4%
Inference Comparison 56 2.4%

Detail categorization is present here

CPU

# issue type issue no #model impacted list of model assignee status
1 failed to legalize operation 'torch.operator' that was explicitly marked illegal 44 NonMaxSuppression, MatMulInteger, Scan, Conv, Loop, QLinearConv, DequantizeLinear, AveragePool, QLinearMatMul, Resize
2 One or more operations with large vector sizes (16384 bytes) were found 139 Resolves with 443
3 expected offsets to be non-negative, but got -1 19935 6
4 'vector.transfer_write' op inferred mask type ('vector<1x1x8xi1>') and mask operand type ('vector<1x8x1xi1>') don't match 19383 9 Resolves with 443
5 failed to legalize operation 'torch.aten.nonzero' 3876 8 Resolved
6 'tensor.reshape' op source and destination tensor should have the same number of elements 3
7 Assertion `getHPadArgument(LEFT) < inputType.getShape()[hDim] && "Left padding too large"' failed 909 5
8 failed to legalize operation 'torch.constant.float 908 15
9 Assertion `this->isReferenceToStorage(I) && ""Insertion iterator is out of bounds.""' failed. 912 3
10 'tensor.cast' op operand type 'tensor<80x15130xf32>' and result type 'tensor<15130xf32>' are cast incompatible 891 3
11 Assertion `result && ""Zero-sized vectors are not allowed""' failed.\n' 913 5
12 error: operand #0 does not dominate this use\n' 19947 3
13 error: 'iree_linalg_ext.scan' op expected type of operand #1 ('tensor<1x128xi64>') to match type of corresponding result ('tensor<1x?xi64>') 17526 9
14 failed to legalize operation "onnx.CategoryMapper" 910 1
15 Invalid IR/Import failures 905 40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant