NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/doc_template/UsingSDKAsLibrary.ipynb.
In this tutorial, we'll walk through an example of using the Chimera SDK as a library.
You may want to use the SDK as a library when the SDK's Command Line Interface does not have commands implemented for your use case. Using the SDK as a library can provide you with more control over how you use the underlying tools that comprise the SDK.
To learn more about the APIs used in this tutorial, refer to Quadric's API documentation.
Begin with a pre-trained FP32 ONNX Model from the ONNX Model Hub
onnx_model_path = "../common/float_onnx/resnet18_float32.onnx"
Quickstart: compile the float ONNX directly for performance estimation:
In order to obtain the performance estimation of the float ONNX model, you can use the following code snippet. This will quantize the graph with synthetic inputs. The quantized graph will be compiled and executed on Chimera ISS. The performance estimation will be printed out.
[!WARNING]
This is useful for performance estimate only. Forward pass inference is not recommend.
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
cgc_job = ChimeraJob(onnx_model_path)
cgc_job.compile()
cgc_job.run_inference_harness()
print(cgc_job)
cgc_job.plot_run_statistics()
2026-06-19 04:07 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 04:07 - WARNING - epu - chimera_job - The network appears to not be quantized. Attemping synthetic quantization on the model. A successful complition will produce valid performance results. However the forward pass of the model will NOT produce accurate outputs. If you're interested in accurate outputs, quantize the model prior to ingestion.
2026-06-19 04:07 - INFO - epu - quantize - Generating synthetic data
2026-06-19 04:07 - INFO - epu - quantize - Optimized model to opset
2026-06-19 04:07 - INFO - epu - quantize - Saved optimized model to resnet18_float32_float32_opt.onnx
2026-06-19 04:07 - INFO - epu - quantize - Input shapes: [1, 3, 224, 224]. Input names: input
2026-06-19 04:07 - INFO - epu - quantize - Output shapes: [[1, 1000]]. Output names: ['output']
2026-06-19 04:07 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-06-19 04:07 - INFO - epu - quantize - Quantization done succesfully!
2026-06-19 04:07 - INFO - epu - quantize - ONNX full precision model size: 44.59 MB
2026-06-19 04:07 - INFO - epu - quantize - ONNX quantized model size: 11.2 MB
2026-06-19 04:07 - INFO - epu - quantize - Saved quantized model to /quadric/sdk-cli/examples/doc_template/resnet18_opt_sym_int8_q.onnx
2026-06-19 04:07 - INFO - epu - quantize - Saved shape inferenced model to /quadric/sdk-cli/examples/doc_template/resnet18_opt_sym_int8_q.onnx
2026-06-19 04:07 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-06-19 04:07 - INFO - epu - quantize - Model still has FLOAT/FLOAT16 types. Creating ranges for floating point tensors using calibration data
2026-06-19 04:07 - INFO - epu - quantize - Saved tensor ranges to /quadric/sdk-cli/examples/doc_template/resnet18_opt_sym_int8_q.onnx.tranges
2026-06-19 04:07 - INFO - epu - chimera_job - Numerical ranges provided
2026-06-19 04:07 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 04:07 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 04:07 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
----------------------- ---------------------- --------------------- --------------------
output_DequantizeLinear contrib.epu.dequantize [-4.66049f, 5.69615f] 28
2026-06-19 04:07 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-06-19 04:07 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-06-19 04:07 - INFO - epu - codegen - START=================================relay_to_tir
2026-06-19 04:07 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-06-19 04:07 - INFO - epu - codegen - START==============================adapt_and_order
2026-06-19 04:07 - INFO - epu - mac_counter -
2026-06-19 04:07 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:07 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-06-19 04:07 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 236,027,904 ops (118,013,952 MACs) - /conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer2/layer2.0/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.0/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer2/layer2.0/downsample/downsample.0/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer3/layer3.0/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.0/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer3/layer3.0/downsample/downsample.0/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer4/layer4.0/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.0/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer4/layer4.0/downsample/downsample.0/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv1/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv2/Conv_quant
2026-06-19 04:07 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-06-19 04:07 - INFO - epu - mac_counter - Total: 3,627,122,688 ops (1,813,561,344 MACs)
2026-06-19 04:07 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:07 - INFO - epu - mac_counter -
2026-06-19 04:07 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:07 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-06-19 04:07 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:07 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-06-19 04:07 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:07 - INFO - epu - codegen - START================================lrm_splitting
2026-06-19 04:08 - INFO - epu - codegen - START==============================ext_split_relay
2026-06-19 04:08 - INFO - epu - codegen - START====================================build_tir
2026-06-19 04:08 - INFO - epu - chimera_job - Compilation of resnet18_float32_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful
2026-06-19 04:08 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x7661a93068f0>
FILM 12/12: 100%|███████████████████████████████████████████████████| 12/12 [00:08<00:00, 1.50it/s]
2026-06-19 04:08 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x7661a93068f0>
2026-06-19 04:08 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 04:08 - INFO - epu - iss_testing - Done 0:00:00.083581
╒═════════════════════╤═════════════════════════════════════════════════════════════════╕
│ Module Name │ resnet18_float32_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ ONNX File │ ../common/float_onnx/resnet18_float32.onnx │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ L2M Size │ 16MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Max L2M │ 9.749MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Max LRM │ 2.375kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 1.814 │
╘═════════════════════╧═════════════════════════════════════════════════════════════════╛
╒════╤════════╤════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input │ [1, 3, 224, 224] │ tensor[FixedPoint32<29>] │ n/a │
├────┼────────┼────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ output │ [1, 1000] │ tensor[FixedPoint32<28>] │ 0.033 │
╘════╧════════╧════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 0.28 │
├──────────────────────────────────┼─────────┤
│ FPS │ 3575.14 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 1517.44 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 2356.03 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 11.83 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 0.00 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 41.29 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 0.01 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 23.29% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 475,506
[SDK-CLI] : Executions/second: 3,575.14
compute : ▇▇▇▇▇▇▇▇▇ 51.055K
data_array : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 74.642K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 258.863K
data_external: ▇ 8.115K
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇ 61.363K
for more information check run directory: /quadric/sdk-cli/examples/doc_template/ccl_build/resnet18_float32_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040825_583156
2026-06-19 04:08 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/doc_template/ccl_build/resnet18_float32_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040825_583156/data/resnet18_float32_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/doc_template/ccl_build/resnet18_float32_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040825_583156/data'
Quantize an ONNX model using the sdk_cli.lib.quantize.quantize_onnx_model function
To quantize an ONNX model, we'll need the following:
- A PyTorch dataset containing images, i.e. any dataset that inherits from the
VisionDatasetbase class - Path to an ONNX model file (
*.onnx) - Input resolution of the ONNX model
- Optional: channelwise means and standard deviations used to normalize the data on which the model was trained
Lastly, we'll need to choose whether the model should be statically quantized with symmetric or asymmetric activations. To learn more about the different types of quantization, refer to the Accepted Model Formats - Quantized ONNX document in the Chimera Software User Guide.
from torchvision.transforms import CenterCrop, Compose, Normalize, Resize, ToTensor
from sdk_cli.lib.quantize import QuantizedONNXModel, quantize_onnx_model
from sdk_cli.utils.datasets import ImageNet_Mini_Quadric
from sdk_cli.utils.datasets.ImageNet import IMAGENET_1K_NORMALIZATION_PARAMETERS
onnx_model_input_resolution = (224, 224)
## NOTE: Change normalization parameters below if using dataset other than ImageNet
transforms = Compose(
[
Resize(onnx_model_input_resolution),
CenterCrop(onnx_model_input_resolution),
ToTensor(),
Normalize(
IMAGENET_1K_NORMALIZATION_PARAMETERS.channel_means,
IMAGENET_1K_NORMALIZATION_PARAMETERS.channel_standard_deviations,
),
]
)
## Path to directory containing data to use for for numerical range calibration during quantization
## Data is used also used to compare accuracy of fp32 and int8 models
dataset = ImageNet_Mini_Quadric.Dataset(transform=transforms)
quantized_onnx_model: QuantizedONNXModel = quantize_onnx_model(
onnx_model_path, dataset, max_calibration_samples=20, asymmetric_activation=False
)
2026-06-19 04:08 - INFO - sdk - quantize - Limiting calibration dataset to 20 samples
2026-06-19 04:08 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-06-19 04:08 - DEBUG - sdk - quantize - Forcing node types: []
2026-06-19 04:08 - DEBUG - sdk - quantize - ONNX Node types excluded from quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-06-19 04:08 - DEBUG - sdk - quantize - ONNX Node names excluded from quantization: []
2026-06-19 04:08 - INFO - sdk - quantize - Starting quantization...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-06-19 04:08 - INFO - sdk - quantize - Quantization completed! Quantized model saved to /quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q.onnx
2026-06-19 04:08 - INFO - sdk - quantize - ONNX full precision model size: 44.59MB
2026-06-19 04:08 - INFO - sdk - quantize - ONNX quantized model size: 11.2MB
2026-06-19 04:08 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-06-19 04:08 - INFO - sdk - quantize - ONNX Model with well-defined shapes has been saved at `/quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q_shaped.onnx`.
2026-06-19 04:08 - DEBUG - sdk - quantize - Checking for FLOAT/FLOAT16 types...
2026-06-19 04:08 - INFO - sdk - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-06-19 04:08 - INFO - sdk - quantize - Model still has FLOAT/FLOAT16 types after quantization. Creating ranges for floating point tensors using calibration data...
2026-06-19 04:08 - INFO - sdk - quantize - Saved computed tensor ranges to /quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q_shaped.tranges.
2026-06-19 04:08 - INFO - sdk - quantize -
╒══════════════════════════════════════════════════════════════════════════════════════════╤═════════════════════════════════════════════════════════════════════════════════════════════╕
│ Quantized ONNX Model │ Tensor Ranges File │
╞══════════════════════════════════════════════════════════════════════════════════════════╪═════════════════════════════════════════════════════════════════════════════════════════════╡
│ /quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q_shaped.onnx │ /quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q_shaped.tranges │
╘══════════════════════════════════════════════════════════════════════════════════════════╧═════════════════════════════════════════════════════════════════════════════════════════════╛
Compile the quantized ONNX model with CGC for a specific Chimera Hardware Configuration with the tvm.contrib.epu.chimera_job.chimera_job.ChimeraJob class
To compile the quantized ONNX model with the Chimera Graph Compiler (CGC), you'll need the:
- Path to the Quantized ONNX Model (accessible using
QuantizedONNXModel.model_path) - Chimera hardware configuration to target for compilation
- Chimera Processor Types: 'QB1', 'QB4', 'QB16'
- L2M Memory Size Options: '4MB', '8MB', '16MB', '32MB'
- External Read/Write Bandwidth Options: '4GBs', '8GBs', '16GBs', '32GBs'
The compiled ONNX model will be represented as a C++ kernel defined using Quadric's Chimera Compute Language (CCL) APIs. This kernel can subsequently be compiled into an executable binary using the Chimera LLVM C++ Compiler.
## Create a `ChimeraJob` object
from tvm.contrib.epu.chimera_job.hw_config import HWConfig
hw_config = HWConfig(
product="QC-U",
ocm_size="8MB",
ext_rd_bw="128GBps",
ext_wr_bw="128GBps",
)
cgc_job = ChimeraJob(
model_p=str(quantized_onnx_model.model_path),
hw_config=hw_config,
quiet_iss=False,
)
print(cgc_job)
## Compile the quantized ONNX model to the specified hardware configuration
cgc_job.compile()
## Optional: Printing the `ChimeraJob` object will print a summary of the compilation process
print(cgc_job)
╒═════════════════════╤════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q_shaped.onnx │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
╘═════════════════════╧════════════════════════════════════════════════════════════════════════════════════════════╛
2026-06-19 04:08 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 04:08 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 04:08 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 04:08 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
----------------------- ---------------------- ---------------------------------------- --------------------
output_DequantizeLinear contrib.epu.dequantize (-23.103130340576172, 22.92263713479042) 26
2026-06-19 04:08 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-06-19 04:08 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-06-19 04:08 - INFO - epu - codegen - START=================================relay_to_tir
2026-06-19 04:08 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-06-19 04:08 - INFO - epu - codegen - START==============================adapt_and_order
2026-06-19 04:08 - INFO - epu - mac_counter -
2026-06-19 04:08 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:08 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-06-19 04:08 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 236,027,904 ops (118,013,952 MACs) - /conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer2/layer2.0/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.0/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer2/layer2.0/downsample/downsample.0/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer3/layer3.0/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.0/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer3/layer3.0/downsample/downsample.0/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer4/layer4.0/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.0/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer4/layer4.0/downsample/downsample.0/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv1/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv2/Conv_quant
2026-06-19 04:08 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-06-19 04:08 - INFO - epu - mac_counter - Total: 3,627,122,688 ops (1,813,561,344 MACs)
2026-06-19 04:08 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:08 - INFO - epu - mac_counter -
2026-06-19 04:08 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:08 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-06-19 04:08 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:08 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-06-19 04:08 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:08 - INFO - epu - codegen - START================================lrm_splitting
2026-06-19 04:09 - INFO - epu - codegen - START==============================ext_split_relay
2026-06-19 04:09 - INFO - epu - codegen - START====================================build_tir
2026-06-19 04:09 - INFO - epu - chimera_job - Compilation of resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful
╒═════════════════════╤════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q_shaped.onnx │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 7.483MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 2.375kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 1.814 │
╘═════════════════════╧════════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input │ [1, 3, 224, 224] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ output │ [1, 1000] │ tensor[FixedPoint32<26>] │ n/a │
╘════╧════════╧════════╧══════════════════╧══════════════════════════╧═══════╛
Generate performance results for the compiled model on synthetic data
We can run inference on the compiled model using the ChimeraJob.run_inference_harness method. The method generates synthetic input data which is sufficient for obtaining performance benchmarks for the compiled model.
from typing import Dict
import numpy as np
## Run inference on ONNX model compiled for Chimera hardware configuration using synthetic input data
## NOTE: Inference output '{key: value}' pairs are '{ONNX output name: output tensor as NumPy array}'
inference_outputs: Dict[str, np.ndarray] = cgc_job.run_inference_harness()
## Printing the `ChimeraJob` again and it will show more run-time execution stats in addition to the compile-time information.
print(cgc_job)
2026-06-19 04:09 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x76630c1c73a0>
FILM 12/12: 100%|███████████████████████████████████████████████████| 12/12 [00:07<00:00, 1.61it/s]
2026-06-19 04:09 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7661a4135570>
2026-06-19 04:09 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 04:09 - INFO - epu - iss_testing - Done 0:00:00.073548
╒═════════════════════╤════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/doc_template/resnet18_OpSet16_optimized_sym_int8_q_shaped.onnx │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 7.483MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 2.375kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 1.814 │
╘═════════════════════╧════════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input │ [1, 3, 224, 224] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ output │ [1, 1000] │ tensor[FixedPoint32<26>] │ 0.082 │
╘════╧════════╧════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 0.28 │
├──────────────────────────────────┼─────────┤
│ FPS │ 3578.30 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 1518.54 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 2356.41 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 11.83 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 0.00 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 41.33 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 0.01 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 23.31% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 475,086
[SDK-CLI] : Executions/second: 3,578.30
compute : ▇▇▇▇▇▇▇▇▇ 51.058K
data_array : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 74.706K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 258.731K
data_external: ▇ 8.115K
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇ 61.008K
for more information check run directory: /quadric/sdk-cli/examples/doc_template/ccl_build/resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040922_5c3ab3
## Plot the run-time execution statistics
cgc_job.plot_run_statistics()
2026-06-19 04:09 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/doc_template/ccl_build/resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040922_5c3ab3/data/resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/doc_template/ccl_build/resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040922_5c3ab3/data'
Key information to note from the Compiled Model's Performance Profile:
- Total number of cycles
- Model input and output shapes, names, and types
- Total number of external bytes transferred to and from the Chimera Core
- Average real read/write bandwidth over the execution of the kernel
- Maximum L2M memory allocation
Verify the compiled model's performance on real data
In order to run the compiled network on a real image, the following information is required:
- Model compiled for Chimera hardware configuration (
ChimeraJobobject) - ONNX Model's input name(s)
- Path to an image
from PIL import Image
## Load the image using PIL
input_image_path = "../common/calibration/raw/daniel_dog.jpg"
input_image = Image.open(input_image_path)
## Pre-process the image
transformed_image = transforms(input_image)
## Run inference on ONNX model compiled for Chimera hardware configuration using real input data
## `ChimeraJob.run_inference_harness` expects `inputs` to be dictionary with '{key: value}' pairs of '{input names: input tensors}'.
inference_output = cgc_job.run_inference_harness(inputs={"input": transformed_image.numpy()})
2026-06-19 04:09 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7661a6fbf400>
FILM 12/12: 100%|███████████████████████████████████████████████████| 12/12 [00:07<00:00, 1.54it/s]
2026-06-19 04:09 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7661a6d5a1d0>
2026-06-19 04:09 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 04:09 - INFO - epu - iss_testing - Done 0:00:00.094054
Since the network is a classfier, we can use the SDK-provided ClassifyResult class to help us interpret the output of the network.
ClassifyResult requires the following at initialization:
name: a name for the resultlabels: a list of labels for the network to be applied to the outputsoftmax: whether or not to applysoftmaxto the output of the network, as some network implementations include softmax in the neural network itself and some do not.
from sdk_cli.utils.model_helpers import ClassifyResult
IMAGENET_LABELS_PATH = r"../common/validation/imagenet_classes.txt"
## Load ImageNet labels
imagenet_labels = {idx: x.replace("\n", "") for idx, x in enumerate(open(IMAGENET_LABELS_PATH))}
## Create `ClassifyResult` object to calculate and render classifier stats
result = ClassifyResult("classification output", imagenet_labels, softmax=True)
result.output = inference_output["output"]
result.calculate_stats()
## Display input image and print stats
from IPython.display import Image, display
display(Image(filename=input_image_path, width=400))
print(result)
--------------classification output---------------
golden retriever.............................0.406
Labrador retriever...........................0.406
kuvasz.......................................0.047
cocker spaniel...............................0.039
Great Pyrenees...............................0.027
Top1: golden retriever 0.406
Enabling onnxruntime versus Instruction Set Simulator Fusion Region Validation
In order to validate the ISS outputs against the appropriate ORT vectors, we need to figure initlizate the ChimeraJob with the validate_iss keyword argument set, and recompile. This will reconfigure ISS to output intermediary Fusion Region tensors from ISS in order to compare them with onnxruntime.
hw_config = HWConfig(ocm_size="2MB")
cgc_job = ChimeraJob(
model_p=str(quantized_onnx_model.model_path), hw_config=hw_config, validate_iss=True
)
## Compile the quantized ONNX model to the specified hardware configuration
cgc_job.compile()
cgc_job.validate_ort_iss(inputs={"input": np.expand_dims(transformed_image.numpy(), 0)})
2026-06-19 04:09 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 04:09 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 04:09 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 04:09 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
----------------------- ---------------------- ---------------------------------------- --------------------
output_DequantizeLinear contrib.epu.dequantize (-23.103130340576172, 22.92263713479042) 26
2026-06-19 04:09 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-06-19 04:09 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-06-19 04:09 - INFO - epu - codegen - START=================================relay_to_tir
2026-06-19 04:09 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-06-19 04:09 - INFO - epu - codegen - START==============================adapt_and_order
2026-06-19 04:09 - INFO - epu - mac_counter -
2026-06-19 04:09 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:09 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-06-19 04:09 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 236,027,904 ops (118,013,952 MACs) - /conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer2/layer2.0/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.0/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer2/layer2.0/downsample/downsample.0/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer3/layer3.0/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.0/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer3/layer3.0/downsample/downsample.0/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer4/layer4.0/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.0/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer4/layer4.0/downsample/downsample.0/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv1/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv2/Conv_quant
2026-06-19 04:09 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-06-19 04:09 - INFO - epu - mac_counter - Total: 3,627,122,688 ops (1,813,561,344 MACs)
2026-06-19 04:09 - INFO - epu - mac_counter - ============================================================
2026-06-19 04:09 - INFO - epu - mac_counter -
2026-06-19 04:09 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:09 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-06-19 04:09 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:09 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-06-19 04:09 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 04:09 - INFO - epu - codegen - START================================lrm_splitting
2026-06-19 04:10 - INFO - epu - codegen - START==============================ext_split_relay
2026-06-19 04:10 - INFO - epu - codegen - START====================================build_tir
2026-06-19 04:10 - INFO - epu - chimera_job - Compilation of resnet18_OpSet16_optimized_sym_int8_q_shaped_QC_U_1d7_2MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful
2026-06-19 04:10 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7661a6d279a0>
2026-06-19 04:10 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7661a6d266e0>
2026-06-19 04:10 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 04:10 - INFO - epu - iss_testing - Done 0:00:00.098784
2026-06-19 04:10 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7661a6d266e0>
FILM 12/12: 100%|███████████████████████████████████████████████████| 12/12 [00:07<00:00, 1.52it/s]
2026-06-19 04:10 - WARNING - epu - iss_testing - /layer3/layer3.1/relu_1/Relu::adapter_tb_strided_tb(tb(0)->tb(0))::/layer4/layer4.0/conv1/Conv_quant:0 not found in the graph that was run in ORT
2026-06-19 04:10 - INFO - epu - iss_testing -
======================================================================
ISS validation results (quantized model, rtol=0.1, atol=-1)
======================================================================
input_QuantizeLinear:0 (PASS, Node, int8)
Bitwise Mismatches 0 / 150528 (0.0000%)
Mismatches > Tol 0 / 150528 (0.0000%)
Thresholds rtol=0.1, atol=-1
RMSE 0
PSNR inf dB
Max Abs Err 0
Max Abs Err / ε 0
Max Err Loc (0, 0, 0, 0)
ORT @ max err -18
ISS @ max err -18
[ERROR] /relu/Relu:0
Error Skipping validation for /relu/Relu:0 dtype mismatch between ISS int8 and ORT float32
/maxpool/MaxPool:0 (PASS, Node, int8x4)
Bitwise Mismatches 0 / 200704 (0.0000%)
Mismatches > Tol 0 / 200704 (0.0000%)
Thresholds rtol=0.1, atol=-1
RMSE 0
PSNR inf dB
Max Abs Err 0
Max Abs Err / ε 0
Max Err Loc (0, 0, 0, 0)
ORT @ max err 6
ISS @ max err 6
[ERROR] /layer1/layer1.0/relu/Relu:0
Error Skipping validation for /layer1/layer1.0/relu/Relu:0 dtype mismatch between ISS int8 and ORT float32
[ERROR] /layer1/layer1.0/relu_1/Relu:0
Error Skipping validation for /layer1/layer1.0/relu_1/Relu:0 dtype mismatch between ISS int8 and ORT float32
[ERROR] /layer1/layer1.1/relu/Relu:0
Error Skipping validation for /layer1/layer1.1/relu/Relu:0 dtype mismatch between ISS int8 and ORT float32
[ERROR] /layer1/layer1.1/relu_1/Relu:0
Error Skipping validation for /layer1/layer1.1/relu_1/Relu:0 dtype mismatch between ISS int8 and ORT float32
[ERROR] /layer2/layer2.1/relu_1/Relu:0
Error Skipping validation for /layer2/layer2.1/relu_1/Relu:0 dtype mismatch between ISS int8 and ORT float32
/avgpool/GlobalAveragePool_quant:0 (PASS, Node, int8)
Bitwise Mismatches 0 / 512 (0.0000%)
Mismatches > Tol 0 / 512 (0.0000%)
Thresholds rtol=0.1, atol=-1
RMSE 0
PSNR inf dB
Max Abs Err 0
Max Abs Err / ε 0
Max Err Loc (0, 0, 0, 0)
ORT @ max err 19
ISS @ max err 19
output_DequantizeLinear:0 (PASS, Node, custom[qfp.26]32)
Bitwise Mismatches 0 / 1000 (0.0000%)
Mismatches > Tol 0 / 1000 (0.0000%)
Thresholds rtol=0.1, atol=-1
RMSE 0
PSNR inf dB
Max Abs Err 0
Max Abs Err / ε 0
Max Err Loc (0, 0)
ORT @ max err -0.5414796
ISS @ max err -0.5414796
output (PASS, Output, custom[qfp.26]32)
Bitwise Mismatches 0 / 1000 (0.0000%)
Mismatches > Tol 0 / 1000 (0.0000%)
Thresholds rtol=0.1, atol=-1
RMSE 0
PSNR inf dB
Max Abs Err 0
Max Abs Err / ε 0
Max Err Loc (0, 0)
ORT @ max err -0.5414796
ISS @ max err -0.5414796
11 entries
[{'S': 'PASS',
'Type': 'Node',
'Name': 'input_QuantizeLinear:0',
'dtype': 'int8',
'total': 150528,
'Bitwise Mismatches': 0,
'Mismatch %': 0.0,
'RMSE': 0.0,
'PSNR (dB)': inf,
'Max Abs Err': 0.0,
'Max Abs Err Loc': (0, 0, 0, 0),
'ORT[loc]': -18,
'ISS[loc]': -18,
'Max Abs Err / ε': 0.0,
'Mismatches Above Tol': 0,
'rtol': 0.1,
'atol': -1,
'Comment': ''},
{'S': 'SKIPPED',
'Type': 'Node',
'Name': '/relu/Relu:0',
'dtype': 'int8x2',
'Mismatches Above Tol': None,
'total': 0,
'rtol': 0.1,
'atol': -1,
'Comment': 'Skipping validation for /relu/Relu:0 dtype mismatch between ISS int8 and ORT float32'},
{'S': 'PASS',
'Type': 'Node',
'Name': '/maxpool/MaxPool:0',
'dtype': 'int8x4',
'total': 200704,
'Bitwise Mismatches': 0,
'Mismatch %': 0.0,
'RMSE': 0.0,
'PSNR (dB)': inf,
'Max Abs Err': 0.0,
'Max Abs Err Loc': (0, 0, 0, 0),
'ORT[loc]': 6,
'ISS[loc]': 6,
'Max Abs Err / ε': 0.0,
'Mismatches Above Tol': 0,
'rtol': 0.1,
'atol': -1,
'Comment': ''},
{'S': 'SKIPPED',
'Type': 'Node',
'Name': '/layer1/layer1.0/relu/Relu:0',
'dtype': 'int8x4',
'Mismatches Above Tol': None,
'total': 0,
'rtol': 0.1,
'atol': -1,
'Comment': 'Skipping validation for /layer1/layer1.0/relu/Relu:0 dtype mismatch between ISS int8 and ORT float32'},
{'S': 'SKIPPED',
'Type': 'Node',
'Name': '/layer1/layer1.0/relu_1/Relu:0',
'dtype': 'int8x4',
'Mismatches Above Tol': None,
'total': 0,
'rtol': 0.1,
'atol': -1,
'Comment': 'Skipping validation for /layer1/layer1.0/relu_1/Relu:0 dtype mismatch between ISS int8 and ORT float32'},
{'S': 'SKIPPED',
'Type': 'Node',
'Name': '/layer1/layer1.1/relu/Relu:0',
'dtype': 'int8x4',
'Mismatches Above Tol': None,
'total': 0,
'rtol': 0.1,
'atol': -1,
'Comment': 'Skipping validation for /layer1/layer1.1/relu/Relu:0 dtype mismatch between ISS int8 and ORT float32'},
{'S': 'SKIPPED',
'Type': 'Node',
'Name': '/layer1/layer1.1/relu_1/Relu:0',
'dtype': 'int8x2',
'Mismatches Above Tol': None,
'total': 0,
'rtol': 0.1,
'atol': -1,
'Comment': 'Skipping validation for /layer1/layer1.1/relu_1/Relu:0 dtype mismatch between ISS int8 and ORT float32'},
{'S': 'SKIPPED',
'Type': 'Node',
'Name': '/layer2/layer2.1/relu_1/Relu:0',
'dtype': 'int8x2',
'Mismatches Above Tol': None,
'total': 0,
'rtol': 0.1,
'atol': -1,
'Comment': 'Skipping validation for /layer2/layer2.1/relu_1/Relu:0 dtype mismatch between ISS int8 and ORT float32'},
{'S': 'PASS',
'Type': 'Node',
'Name': '/avgpool/GlobalAveragePool_quant:0',
'dtype': 'int8',
'total': 512,
'Bitwise Mismatches': 0,
'Mismatch %': 0.0,
'RMSE': 0.0,
'PSNR (dB)': inf,
'Max Abs Err': 0.0,
'Max Abs Err Loc': (0, 0, 0, 0),
'ORT[loc]': 19,
'ISS[loc]': 19,
'Max Abs Err / ε': 0.0,
'Mismatches Above Tol': 0,
'rtol': 0.1,
'atol': -1,
'Comment': ''},
{'S': 'PASS',
'Type': 'Node',
'Name': 'output_DequantizeLinear:0',
'dtype': 'custom[qfp.26]32',
'total': 1000,
'Bitwise Mismatches': 0,
'Mismatch %': 0.0,
'RMSE': 0.0,
'PSNR (dB)': inf,
'Max Abs Err': 0.0,
'Max Abs Err Loc': (0, 0),
'ORT[loc]': -0.5414796,
'ISS[loc]': -0.5414796,
'Max Abs Err / ε': 0.0,
'Mismatches Above Tol': 0,
'rtol': 0.1,
'atol': -1,
'Comment': ''},
{'S': 'PASS',
'Type': 'Output',
'Name': 'output',
'dtype': 'custom[qfp.26]32',
'total': 1000,
'Bitwise Mismatches': 0,
'Mismatch %': 0.0,
'RMSE': 0.0,
'PSNR (dB)': inf,
'Max Abs Err': 0.0,
'Max Abs Err Loc': (0, 0),
'ORT[loc]': -0.5414796,
'ISS[loc]': -0.5414796,
'Max Abs Err / ε': 0.0,
'Mismatches Above Tol': 0,
'rtol': 0.1,
'atol': -1,
'Comment': ''}]
