NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/custom_op/custom_op_tutorial.ipynb.
Tutorial: Custom Op Insertion into a Model
The custom op flow is intended to give the developer a means to define operators and graph structures that CGC (Chimera Graph Compiler) does not have support for yet. The user can represent any operator or structure using CCL (Chimera Compute Language), a C++ API for the Chimera Family of GPNPUs.
In this example, we will simple replace a RELU function with a user-defined CCL version and compare outputs from the two passes.
At a high level we will perform the following steps:
- Load in a float ONNX
- Replace the RELU operations in the ONNX with a Quadric Custom Op Node
- Quantize the ONNX
- Write the custom operation using CCL (Chimera Compute Language)
- Compile the Neural Network with CGC (Chimera Graph Compiler) Passing in the hpp file required to define the replaced node in the graph.
- Compare the results of both networks
First let's set up a few high-level variables. These will be used throughout the tutorial.
import tvm.contrib.epu.graphutils as graphutils
import onnx
from onnxruntime.tools import symbolic_shape_infer
import os, sys, warnings
import logging
from PIL import Image
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from matplotlib.gridspec import GridSpec
%matplotlib inline
Load the float ONNX file
- Begin with a pretain ONNX export from Torch
input_onnx_name = "../common/float_onnx/resnet18_float32.onnx"
custom_onnx_name = "./resnet18_float32_custom.onnx"
## load the onnx
model = onnx.load(input_onnx_name)
Replace RELU operation with Custom Op
Node replacement uses a simple pattern matching API. The patterns are defined using a basic Python API. In this example, we need to replace any node named "Relu" with a QuadricCustomOpNode named "quadricRelu."
## Initialize the filter object and create a
from tvm.contrib.epu.chimera_job.quantize import quadric_quantize
from sdk_cli.utils import model_helpers
fw = graphutils.CustomOpReplacer.FilterWildcard
## Write a basic pattern to search the graph with
relu = graphutils.FilterNode("Relu", [fw], -1)
## Initialize the filter replacer object
replacer = graphutils.CustomOpReplacer(model)
custom_model = replacer.replace_all(relu, ccl_func_name="quadricRelu", element_wise=True)
## Save the ONNX.
onnx.save(custom_model, custom_onnx_name)
Quantize the Network
## Quantize the network
## assumptions for all imagenet-trained models. for user-supplied models change the following:
dataset_mean = [0.485, 0.456, 0.406]
dataset_std = [0.229, 0.224, 0.225]
dataset_input_size = (224, 224) # an (W, H) tuple
## include quadric's cli helpers and instantiate a module to help
mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
calibration_set = r"../common/validation/imagenet-mini-quadric/val"
quantize_result = quadric_quantize(
custom_onnx_name, 100, mh, calibration_set, asymmetric_activation=True
)
/tmp/ipykernel_2661/1084251870.py:9: DeprecationWarning: Call to deprecated class ModelHelper. ('ModelHelper' class is being deprecated. Quadric APIs have been updated to use PyTorch datasets and transforms instead.) -- Deprecated since version 24.01.
mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
2026-06-19 03:53 - INFO - epu - quantize - Collecting calibration data
2026-06-19 03:53 - INFO - epu - quantize - Optimized model to opset
2026-06-19 03:53 - INFO - epu - quantize - Saved optimized model to resnet18_float32_custom_float32_opt.onnx
2026-06-19 03:53 - INFO - epu - quantize - Input shapes: [1, 3, 224, 224]. Input names: input
2026-06-19 03:53 - INFO - epu - quantize - Output shapes: [[1, 1000]]. Output names: ['output']
2026-06-19 03:53 - INFO - epu - quantize - applying calibration data to input: input
2026-06-19 03:53 - INFO - epu - quantize - calibration set size: 100
2026-06-19 03:53 - INFO - epu - quantize - Running real quantization on this input: input with input shape: [1, 3, 224, 224]
2026-06-19 03:53 - DEBUG - epu - quantize - Full exclusion set for quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-06-19 03:53 - DEBUG - epu - quantize - excl_nodes ['CustomOp/quadricRelu16', 'CustomOp/quadricRelu15', 'CustomOp/quadricRelu14', 'CustomOp/quadricRelu13', 'CustomOp/quadricRelu12', 'CustomOp/quadricRelu11', 'CustomOp/quadricRelu10', 'CustomOp/quadricRelu9', 'CustomOp/quadricRelu8', 'CustomOp/quadricRelu7', 'CustomOp/quadricRelu6', 'CustomOp/quadricRelu5', 'CustomOp/quadricRelu4', 'CustomOp/quadricRelu3', 'CustomOp/quadricRelu2', 'CustomOp/quadricRelu1', 'CustomOp/quadricRelu0']
2026-06-19 03:53 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-06-19 03:53 - INFO - epu - quantize - Quantization done succesfully!
2026-06-19 03:53 - INFO - epu - quantize - ONNX full precision model size: 44.59 MB
2026-06-19 03:53 - INFO - epu - quantize - ONNX quantized model size: 11.2 MB
2026-06-19 03:53 - INFO - epu - quantize - Saved quantized model to /quadric/sdk-cli/examples/custom_op/resnet18_custom_opt_asym_int8_q.onnx
2026-06-19 03:53 - INFO - epu - quantize - Saved shape inferenced model to /quadric/sdk-cli/examples/custom_op/resnet18_custom_opt_asym_int8_q.onnx
2026-06-19 03:53 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-06-19 03:53 - INFO - epu - quantize - Model still has FLOAT/FLOAT16 types. Creating ranges for floating point tensors using calibration data
2026-06-19 03:53 - INFO - epu - quantize - Saved tensor ranges to /quadric/sdk-cli/examples/custom_op/resnet18_custom_opt_asym_int8_q.onnx.tranges
Create a CGC Job for the custom op replaced onnx
The ONNX can still be run in ONNX runtime
import tvm.contrib.epu.chimera_job.core as core
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
## custom op replaced onnx
cgc_job_custom = ChimeraJob(
model_p=quantize_result.qmodel_path,
custom_ops="./quadric_relu.hpp",
trange_file=quantize_result.tranges_path,
)
cgc_job_custom.analyze_network()
2026-06-19 03:53 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 03:53 - INFO - epu - chimera_job - Numerical ranges provided
2026-06-19 03:53 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 03:53 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 03:53 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
----------------------------------------------------- ------------------------------------------ --------------------- --------------------
/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-6.5802f, 7.52022f] 27
CustomOp/quadricRelu16 contrib.epu.quadric_custom_op_element_wise [0f, 7.52022f] 27
/layer1/layer1.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-7.18829f, 5.45318f] 28
CustomOp/quadricRelu15 contrib.epu.quadric_custom_op_element_wise [0f, 5.45318f] 28
/layer1/layer1.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-7.78808f, 7.19808f] 28
CustomOp/quadricRelu14 contrib.epu.quadric_custom_op_element_wise [0f, 7.19808f] 28
/layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-6.02743f, 4.15134f] 28
CustomOp/quadricRelu13 contrib.epu.quadric_custom_op_element_wise [0f, 4.15134f] 28
/layer1/layer1.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-9.29728f, 8.00788f] 27
CustomOp/quadricRelu12 contrib.epu.quadric_custom_op_element_wise [0f, 8.00788f] 27
/layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-4.38546f, 3.92383f] 28
CustomOp/quadricRelu11 contrib.epu.quadric_custom_op_element_wise [0f, 3.92383f] 28
/layer2/layer2.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.84398f, 7.2528f] 27
CustomOp/quadricRelu10 contrib.epu.quadric_custom_op_element_wise [0f, 7.2528f] 27
/layer2/layer2.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.49521f, 3.87704f] 28
CustomOp/quadricRelu9 contrib.epu.quadric_custom_op_element_wise [0f, 3.87704f] 28
/layer2/layer2.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.85122f, 6.2975f] 28
CustomOp/quadricRelu8 contrib.epu.quadric_custom_op_element_wise [0f, 6.2975f] 28
/layer3/layer3.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.24033f, 3.82193f] 28
CustomOp/quadricRelu7 contrib.epu.quadric_custom_op_element_wise [0f, 3.82193f] 28
/layer3/layer3.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-4.24109f, 6.42589f] 28
CustomOp/quadricRelu6 contrib.epu.quadric_custom_op_element_wise [0f, 6.42589f] 28
/layer3/layer3.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.49678f, 3.61143f] 28
CustomOp/quadricRelu5 contrib.epu.quadric_custom_op_element_wise [0f, 3.61143f] 28
/layer3/layer3.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.39252f, 11.5455f] 27
CustomOp/quadricRelu4 contrib.epu.quadric_custom_op_element_wise [0f, 11.5455f] 27
/layer4/layer4.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-2.75571f, 2.45431f] 29
CustomOp/quadricRelu3 contrib.epu.quadric_custom_op_element_wise [0f, 2.45431f] 29
/layer4/layer4.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.46417f, 7.38675f] 27
CustomOp/quadricRelu2 contrib.epu.quadric_custom_op_element_wise [0f, 7.38675f] 27
/layer4/layer4.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.89905f, 2.35414f] 29
CustomOp/quadricRelu1 contrib.epu.quadric_custom_op_element_wise [0f, 2.35414f] 29
/layer4/layer4.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-12.8325f, 39.3389f] 25
CustomOp/quadricRelu0 contrib.epu.quadric_custom_op_element_wise [0f, 39.3389f] 25
output_DequantizeLinear contrib.epu.dequantize [-9.74725f, 28.4802f] 26
Analysis of /quadric/sdk-cli/examples/custom_op/resnet18_custom_opt_asym_int8_q.onnx
2026-06-19 03:53 - INFO - epu - chimera_job - Function Prototypes for Custom OPs found
2026-06-19 03:53 - INFO - epu - chimera_job - Please see the file: resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp for a template example of custom op implementation
List of custom ops found in the network:
quadricRelu
2026-06-19 03:53 - INFO - epu - chimera_job - Wrote custom op seed file to resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp.
'resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp'
!cat {cgc_job_custom.name}_analyze_custom_op_template.hpp
qVar_t<FixedPoint32<27>> quadricRelu (FixedPoint32<27> ext_tensor_in0 /* CustomOp/quadricRelu16 */ ) {
/* User implementation of custom op */
}
qVar_t<FixedPoint32<28>> quadricRelu (FixedPoint32<28> ext_tensor_in0 /* CustomOp/quadricRelu15 */ ) {
/* User implementation of custom op */
}
qVar_t<FixedPoint32<29>> quadricRelu (FixedPoint32<29> ext_tensor_in0 /* CustomOp/quadricRelu3 */ ) {
/* User implementation of custom op */
}
qVar_t<FixedPoint32<25>> quadricRelu (FixedPoint32<25> ext_tensor_in0 /* CustomOp/quadricRelu0 */ ) {
/* User implementation of custom op */
}
Review the results
We now have two ONNX files. We can use a graph visualizer such as Netron to open both ONNX files and confirm the Op replacement worked as expected.
Notice that the properties of the QuadricCustomOp include designation that this is an element-wise Custom Operation. 
Write the Custom Op using CCL (Chimera Compute Language)
From Wikipedia: https://en.wikipedia.org/wiki/Rectifier_(neural_networks) a RELU is an Elementwise Linear Recitifier.
We can express this elementwise operation in CCL using the following code:
using namespace chimera;
template <typename T, std::uint8_t numFracBits>
INLINE qVar_t<FixedPoint<T, numFracBits>> quadricRelu(qVar_t<FixedPoint<T, numFracBits>> A) {
if (A > 0)
return A;
else
return 0;
}
Compile the Neural Network with CGC (Chimera Graph Compiler)
In this step, the two different ONNX files: the original ONNX and the ONNX with the custom QuadricRelu function can be lowered.
The input ONNX is lowered with no modifications to the lower_network_to_chimera caller. Where the custom ONNX is passed to lower_network_to_chimera with the custom_op_header optional argument pointing to the full path of the hpp file that contains the definition of the custom op.
cgc_job_custom.compile()
2026-06-19 03:53 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 03:53 - INFO - epu - chimera_job - Numerical ranges provided
2026-06-19 03:53 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 03:53 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 03:53 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
----------------------------------------------------- ------------------------------------------ --------------------- --------------------
/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-6.5802f, 7.52022f] 27
CustomOp/quadricRelu16 contrib.epu.quadric_custom_op_element_wise [0f, 7.52022f] 27
/layer1/layer1.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-7.18829f, 5.45318f] 28
CustomOp/quadricRelu15 contrib.epu.quadric_custom_op_element_wise [0f, 5.45318f] 28
/layer1/layer1.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-7.78808f, 7.19808f] 28
CustomOp/quadricRelu14 contrib.epu.quadric_custom_op_element_wise [0f, 7.19808f] 28
/layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-6.02743f, 4.15134f] 28
CustomOp/quadricRelu13 contrib.epu.quadric_custom_op_element_wise [0f, 4.15134f] 28
/layer1/layer1.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-9.29728f, 8.00788f] 27
CustomOp/quadricRelu12 contrib.epu.quadric_custom_op_element_wise [0f, 8.00788f] 27
/layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-4.38546f, 3.92383f] 28
CustomOp/quadricRelu11 contrib.epu.quadric_custom_op_element_wise [0f, 3.92383f] 28
/layer2/layer2.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.84398f, 7.2528f] 27
CustomOp/quadricRelu10 contrib.epu.quadric_custom_op_element_wise [0f, 7.2528f] 27
/layer2/layer2.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.49521f, 3.87704f] 28
CustomOp/quadricRelu9 contrib.epu.quadric_custom_op_element_wise [0f, 3.87704f] 28
/layer2/layer2.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.85122f, 6.2975f] 28
CustomOp/quadricRelu8 contrib.epu.quadric_custom_op_element_wise [0f, 6.2975f] 28
/layer3/layer3.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.24033f, 3.82193f] 28
CustomOp/quadricRelu7 contrib.epu.quadric_custom_op_element_wise [0f, 3.82193f] 28
/layer3/layer3.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-4.24109f, 6.42589f] 28
CustomOp/quadricRelu6 contrib.epu.quadric_custom_op_element_wise [0f, 6.42589f] 28
/layer3/layer3.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.49678f, 3.61143f] 28
CustomOp/quadricRelu5 contrib.epu.quadric_custom_op_element_wise [0f, 3.61143f] 28
/layer3/layer3.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.39252f, 11.5455f] 27
CustomOp/quadricRelu4 contrib.epu.quadric_custom_op_element_wise [0f, 11.5455f] 27
/layer4/layer4.0/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-2.75571f, 2.45431f] 29
CustomOp/quadricRelu3 contrib.epu.quadric_custom_op_element_wise [0f, 2.45431f] 29
/layer4/layer4.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-5.46417f, 7.38675f] 27
CustomOp/quadricRelu2 contrib.epu.quadric_custom_op_element_wise [0f, 7.38675f] 27
/layer4/layer4.1/conv1/Conv_output_0_DequantizeLinear contrib.epu.qlinear_conv2d [-3.89905f, 2.35414f] 29
CustomOp/quadricRelu1 contrib.epu.quadric_custom_op_element_wise [0f, 2.35414f] 29
/layer4/layer4.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-12.8325f, 39.3389f] 25
CustomOp/quadricRelu0 contrib.epu.quadric_custom_op_element_wise [0f, 39.3389f] 25
output_DequantizeLinear contrib.epu.dequantize [-9.74725f, 28.4802f] 26
2026-06-19 03:53 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-06-19 03:53 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-06-19 03:53 - INFO - epu - codegen - START=================================relay_to_tir
2026-06-19 03:53 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-06-19 03:53 - INFO - epu - codegen - START==============================adapt_and_order
2026-06-19 03:53 - INFO - epu - mac_counter -
2026-06-19 03:53 - INFO - epu - mac_counter - ============================================================
2026-06-19 03:53 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-06-19 03:53 - INFO - epu - mac_counter - ============================================================
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 236,027,904 ops (118,013,952 MACs) - /conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.0/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer2/layer2.0/downsample/downsample.0/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer3/layer3.0/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.0/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer3/layer3.0/downsample/downsample.0/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 115,605,504 ops (57,802,752 MACs) - /layer4/layer4.0/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.0/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 12,845,056 ops (6,422,528 MACs) - /layer4/layer4.0/downsample/downsample.0/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv1/Conv_output_0_DequantizeLinear
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv2/Conv_quant
2026-06-19 03:53 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-06-19 03:53 - INFO - epu - mac_counter - Total: 3,627,122,688 ops (1,813,561,344 MACs)
2026-06-19 03:53 - INFO - epu - mac_counter - ============================================================
2026-06-19 03:53 - INFO - epu - mac_counter -
2026-06-19 03:54 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 03:54 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-06-19 03:54 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 03:54 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-06-19 03:54 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 03:54 - INFO - epu - codegen - START================================lrm_splitting
2026-06-19 03:55 - INFO - epu - codegen - START==============================ext_split_relay
2026-06-19 03:55 - INFO - epu - codegen - START====================================build_tir
2026-06-19 03:55 - INFO - epu - chimera_job - Compilation of resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful
Run the neural network
Now that we have fully-linked CCL representations for each network. We can pass each network representation to the Instruction Set Simulator.
run_folder = r"../common/calibration/raw"
dict_label_path_dataset = r"../common/validation/imagenet_classes.txt"
dict_label = {idx: x.replace("\n", "") for idx, x in enumerate(open(dict_label_path_dataset))}
THREADS = 3
NUM_IMG = 6
allimages = mh.get_images(run_folder, NUM_IMG)
input_images = [{"input": mh.load_image(i)} for i in allimages]
all_results = {}
all_results["iss"] = cgc_job_custom.run_batch_inference_harness(
inputs=input_images, threads=THREADS
)
all_results["ort"] = cgc_job_custom.run_batch_ort_harness(inputs=input_images, threads=THREADS)
Processing: 100%|████████████████████████████████| 6/6 [00:31<00:00, 5.31s/it]
2026-06-19 03:56 - WARNING - epu - chimera_job - ORT is not threadsafe -- forcing single threaded batch execution
100%|████████████████████████████████████████████| 6/6 [00:00<00:00, 6.79it/s]
Compare the Results
for i, inp in enumerate(input_images):
ax = []
fig = plt.figure(constrained_layout=True, figsize=(12, 4))
gs = GridSpec(2, 2, figure=fig)
ax.append(fig.add_subplot(gs[:, 0]))
ax.append(fig.add_subplot(gs[0, 1]))
ax.append(fig.add_subplot(gs[1, 1], sharex=ax[1]))
ax[0].imshow(mpimg.imread(allimages[i]))
ax[0].set_title(os.path.basename(allimages[i]))
ax[0].set_axis_off()
plt.setp(ax[1].get_xticklabels(), visible=False)
for axis, key in enumerate(all_results.keys()):
classify = model_helpers.ClassifyResult(f"{key}", dict_label, softmax=True)
classify.output = all_results[key][i]["output"]
classify.calculate_stats()
classify.add_mpl_fig(ax[axis + 1], "darkviolet" if axis > 0 else None)
plt.show()
print(cgc_job_custom)
cgc_job_custom.plot_run_statistics()
╒═════════════════════╤════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/custom_op/resnet18_custom_opt_asym_int8_q.onnx │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Custom Ops │ /quadric/sdk-cli/examples/custom_op/quadric_relu.hpp │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 16MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 9.749MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 2.375kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 1.814 │
╘═════════════════════╧════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input │ [1, 3, 224, 224] │ tensor[FixedPoint32<29>] │ n/a │
├────┼────────┼────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ output │ [1, 1000] │ tensor[FixedPoint32<26>] │ 0.039 │
╘════╧════════╧════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 0.30 │
├──────────────────────────────────┼─────────┤
│ FPS │ 3350.61 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 1731.31 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 1935.30 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 11.83 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 0.00 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 38.70 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 0.01 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 21.82% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 507,370
[SDK-CLI] : Executions/second: 3,350.61
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 80.205K
data_array : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 81.132K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 255.198K
data_external: ▇ 8.113K
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇ 61.262K
for more information check run directory: /quadric/sdk-cli/examples/custom_op/ccl_build/resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_035602_21d602
2026-06-19 03:56 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/custom_op/ccl_build/resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_035602_21d602/data/resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/custom_op/ccl_build/resnet18_custom_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_035602_21d602/data'
