Inference Error on ONNX

#1
by RayyanAkhtar - opened

I am trying to use the onnx model from this repository with Python3.11 and onnxruntime-qnn=1.19.0 on Windows Platform on Snapdragon 8cx Gen 3 Processor (Windows Dev kit 2023). But during the inference I am getting this error:
FAIL : Node 'Conv' OpType:Conv with domain:com.ms.internal.nhwc was inserted using the NHWC format as requested by QNNExecutionProvider, but was not selected by that EP. This means the graph is now invalid as there will not be an EP able to run the node. This could be a bug in layout transformer, or in the GetCapability implementation of the EP.

Code:

options.add_session_config_entry("session.disable_cpu_ep_fallback", "0")

options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_ENABLE_EXTENDED
self.session = onnxruntime.InferenceSession(path,
sess_options=options,
providers=["QNNExecutionProvider"],
provider_options=[{"backend_path": "QnnHtp.dll")

outputs = self.session.run(self.output_names, {self.input_names[0]: input_tensor})

Environment:

QNN_SDK_ROOT=C:\Qualcomm\AIStack\QAIRT\2.22.0.240425

Error:

`
unknown ARM CPU part 0xd4b ignored

unknown ARM CPU part 0xd4b ignored

unknown ARM CPU part 0xd4b ignored

unknown ARM CPU part 0xd4b ignored

unknown ARM CPU part 0xd4c ignored

unknown ARM CPU part 0xd4c ignored

unknown ARM CPU part 0xd4c ignored

unknown ARM CPU part 0xd4c ignored

2024-11-29 13:12:11.8863606 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.8919005 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/3/net/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.8972361 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/4/net/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.9042892 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/5/net/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.9111319 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/tail/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.9166808 [W:onnxruntime:, qnn_execution_provider.cc:664 onnxruntime::QNNExecutionProvider::GetCapability] QNN EP does not support a single Quantize/Dequantize node in a partition.
2024-11-29 13:12:11.9239293 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.0.weight_dq
2024-11-29 13:12:11.9291546 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv of type Conv2d with error code 3110

2024-11-29 13:12:11.9339917 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.9387543 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.2.weight_dq
2024-11-29 13:12:11.9431608 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_6 of type Conv2d with error code 3110

2024-11-29 13:12:11.9490821 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.3.net.0.weight_dq
2024-11-29 13:12:11.9528103 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_12 of type Conv2d with error code 3110

2024-11-29 13:12:11.9584756 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/3/net/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.9640692 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.3.net.2.weight_dq
2024-11-29 13:12:11.9677988 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_18 of type Conv2d with error code 3110

2024-11-29 13:12:11.9724161 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.4.net.0.weight_dq
2024-11-29 13:12:11.9771572 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_24 of type Conv2d with error code 3110

2024-11-29 13:12:11.9818775 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/4/net/1/Relu of type Relu with error code 3110

2024-11-29 13:12:11.9871576 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.4.net.2.weight_dq
2024-11-29 13:12:11.9915960 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_30 of type Conv2d with error code 3110

2024-11-29 13:12:11.9965127 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.5.net.0.weight_dq
2024-11-29 13:12:11.9999194 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_36 of type Conv2d with error code 3110

2024-11-29 13:12:12.0044381 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/cnn/5/net/1/Relu of type Relu with error code 3110

2024-11-29 13:12:12.0107981 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.cnn.5.net.2.weight_dq
2024-11-29 13:12:12.0146768 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_42 of type Conv2d with error code 3110

2024-11-29 13:12:12.0198233 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.residual.weight_dq
2024-11-29 13:12:12.0243246 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_48 of type Conv2d with error code 3110

2024-11-29 13:12:12.0296923 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.tail.0.weight_dq
2024-11-29 13:12:12.0332154 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_54 of type Conv2d with error code 3110

2024-11-29 13:12:12.0376411 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node /model/tail/1/Relu of type Relu with error code 3110

2024-11-29 13:12:12.0447864 [E:onnxruntime:, qnn_model_wrapper.cc:138 onnxruntime::qnn::QnnModelWrapper::CreateQnnInputOutputTensors] Input name not exist: model.tail.2.weight_dq
2024-11-29 13:12:12.0497153 [W:onnxruntime:, qnn_model_wrapper.cc:244 onnxruntime::qnn::QnnModelWrapper::CreateQnnNode] QNN.backendValidateOpConfig() failed for node Conv_token_60 of type Conv2d with error code 3110

Traceback (most recent call last):
File "C:\volterra_research\refactored_code\python\super-resolution\super_resolution.py", line 54, in
sr = SuperResolution(
^^^^^^^^^^^^^^^^
File "C:\volterra_research\refactored_code\python\super-resolution\super_resolution\resolution.py", line 12, in init
self.initialize_model(path)
File "C:\volterra_research\refactored_code\python\super-resolution\super_resolution\resolution.py", line 24, in initialize_model
self.session = onnxruntime.InferenceSession(path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\volterra_research\volterravenv\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "C:\volterra_research\volterravenv\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 491, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node 'Conv' OpType:Conv with domain:com.ms.internal.nhwc was inserted using the NHWC format as requested by QNNExecutionProvider, but was not selected by that EP. This means the graph is now invalid as there will not be an EP able to run the node. This could be a bug in layout transformer, or in the GetCapability implementation of the EP.
`

Kindly help me to resolve this issue.

This is the system information.
Screenshot 2024-11-29 100719-copy.png

Sign up or log in to comment