Unlock this feature

This feature isn’t part of your plan yet

Contact sales to get upgraded to the full DevStudio experience.

Unlock this feature

This feature isn't part of your plan yet.

Tutorial: Custom Operator Insertion into a Model


NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/custom_op/custom_op_tutorial.ipynb.


Replacing an op with a custom kernel through the plugin

A custom op is a CCL function standing in for a subgraph CGC would otherwise lower itself, or cannot lower at all. This notebook takes the float ResNet18 export and replaces all 17 Relu nodes with quadricRelu, an element-wise CCL function in quadric_relu.hpp, then quantizes, compiles, and classifies six images on the ISS and in ONNX Runtime.

The replacement is described once, in the bundle relu_plugins/quadric_relu: the pattern Relu($x), the function name, element_wise: true, and the kernel file. analyze_model applies it to the float graph and writes the substituted model and the combined header. Quantization runs on the substituted graph, so the customOp nodes are quantized like any other node, and the compile takes the header the plugin wrote.

Relu is in CGC's convert map; this notebook shows the mechanism on an op everyone knows, and the bundle's manifest records that.

#partwhat it does
1SetupPaths and imports.
2Substitute on the float graphanalyze_model replaces every Relu and writes the substituted model and the header.
3Quantizequadric_quantize on the substituted graph with 100 ImageNet calibration images.
4CompileA ChimeraJob on the quantized model with the header the plugin wrote.
5Run on the ISS and in ONNX RuntimeSix images through both engines.
6CompareTop-5 classes per image from both engines, the logit correlation and the top-1 agreement.
7ProfileThe job summary and the run statistics plot.

1. Setup

Imports analyze_model from the plugin and the compiler's ChimeraJob, and points at the float ResNet18 export, at relu_plugins/ and at _plugin_out_relu/ beside the notebook, where the substituted model and the header go. ModelHelper.get_images draws its images with random.sample, so the seed fixes which 100 calibration images and which 6 test images every run uses.

License Notice

The images this notebook uses as input data, for calibration, evaluation or training, come from the ImageNet database, whose terms of access state: "Researcher shall use the Database only for non-commercial research and educational purposes." https://www.image-net.org/download.php

%matplotlib inline
import os
import random
from pathlib import Path

import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.gridspec import GridSpec
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from tvm.contrib.epu.chimera_job.quantize import quadric_quantize

from sdk_cli.cgc_plugin import analyze_model
from sdk_cli.utils import model_helpers
from sdk_cli.utils.model_helpers import example_path

float_onnx = example_path("common/float_onnx/resnet18_float32.onnx")
relu_plugins = example_path("custom_op/relu_plugins")
work_dir = example_path("custom_op") / "_plugin_out_relu"
random.seed(0)

2. Substitute on the float graph

The bundle's manifest matches the pattern Relu($x) and binds it to quadricRelu with element_wise: true, which tells CGC the function maps one element to one element, so it is emitted inline over the tensor. The kernel is a template over the fixed-point type that returns its input where positive and zero elsewhere:

using namespace chimera;
template <typename T, std::uint8_t numFracBits>
INLINE qVar_t<FixedPoint<T, numFracBits>> quadricRelu(qVar_t<FixedPoint<T, numFracBits>> A) {
  if (A > 0)
    return A;
  else
    return 0;
}

analyze_model applies the bundle to the float model with the shipped library switched off and writes the substituted model and the combined header into _plugin_out_relu/. The result it prints names the bundle and counts the customOps: 17, one per Relu. The graph in Netron, before and after:

before after

analysis = analyze_model(
    float_onnx, plugins_dir=relu_plugins, out_dir=work_dir, use_default_library=False
)
print(analysis)
cgc plugin: 17 customOp(s) from quadric_relu -> /quadric/sdk-cli/examples/custom_op/_plugin_out_relu

3. Quantize

Builds the ImageNet preprocessing helper (224 by 224, the usual mean and std) and quantizes the substituted model on 100 images from common/validation/imagenet-mini-quadric/val with asymmetric activations. The result holds the quantized model path and the .tranges file.

dataset_mean = [0.485, 0.456, 0.406]
dataset_std = [0.229, 0.224, 0.225]
dataset_input_size = (224, 224)
mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
calibration_set = example_path("common/validation/imagenet-mini-quadric/val")

quantize_result = quadric_quantize(
    analysis.model_path, 100, mh, str(calibration_set), asymmetric_activation=True
)
/tmp/ipykernel_2558/3768456042.py:4: DeprecationWarning: Call to deprecated class ModelHelper. ('ModelHelper' class is being deprecated. Quadric APIs have been updated to use PyTorch datasets and transforms instead.) -- Deprecated since version 24.01.
  mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
2026-10-23 00:18 - INFO - epu - quantize - Collecting calibration data
2026-10-23 00:18 - INFO - epu - quantize - Optimized model to opset
2026-10-23 00:18 - INFO - epu - quantize - Saved optimized model to model_float32_opt.onnx
2026-10-23 00:19 - INFO - epu - quantize - Input shapes: [1, 3, 224, 224]. Input names: input
2026-10-23 00:19 - INFO - epu - quantize - Output shapes: [[1, 1000]]. Output names: ['output']
2026-10-23 00:19 - INFO - epu - quantize - applying calibration data to input: input
2026-10-23 00:19 - INFO - epu - quantize - calibration set size: 100
2026-10-23 00:19 - INFO - epu - quantize - Running real quantization on this input: input with input shape: [1, 3, 224, 224]
2026-10-23 00:19 - DEBUG - epu - quantize - Full exclusion set for quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-10-23 00:19 - DEBUG - epu - quantize - excl_nodes ['CustomOp/quadricRelu16', 'CustomOp/quadricRelu15', 'CustomOp/quadricRelu14', 'CustomOp/quadricRelu13', 'CustomOp/quadricRelu12', 'CustomOp/quadricRelu11', 'CustomOp/quadricRelu10', 'CustomOp/quadricRelu9', 'CustomOp/quadricRelu8', 'CustomOp/quadricRelu7', 'CustomOp/quadricRelu6', 'CustomOp/quadricRelu5', 'CustomOp/quadricRelu4', 'CustomOp/quadricRelu3', 'CustomOp/quadricRelu2', 'CustomOp/quadricRelu1', 'CustomOp/quadricRelu0']
2026-10-23 00:19 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-10-23 00:19 - INFO - epu - quantize - Quantization done succesfully!
2026-10-23 00:19 - INFO - epu - quantize - ONNX full precision model size: 44.59 MB
2026-10-23 00:19 - INFO - epu - quantize - ONNX quantized model size: 11.2 MB
2026-10-23 00:19 - INFO - epu - quantize - Saved quantized model to /quadric/sdk-cli/examples/custom_op/model_opt_asym_int8_q.onnx
2026-10-23 00:19 - INFO - epu - quantize - Saved shape inferenced model to /quadric/sdk-cli/examples/custom_op/model_opt_asym_int8_q.onnx
2026-10-23 00:19 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16/BFLOAT16 types.
2026-10-23 00:19 - INFO - epu - quantize - Model still has FLOAT/FLOAT16/BFLOAT16 types. Creating ranges for floating point tensors using calibration data
2026-10-23 00:19 - INFO - epu - quantize - Saved tensor ranges to /quadric/sdk-cli/examples/custom_op/model_opt_asym_int8_q.onnx.tranges

4. Compile

Builds the job on the quantized model with custom_ops set to the header analyze_model wrote and the .tranges from quantization. analyze_network() writes the prototype CGC expects for the customOp; the cell prints it so the signature can be compared with quadricRelu. Then compiles.

cgc_job = ChimeraJob(
    model_p=quantize_result.qmodel_path,
    custom_ops=analysis.header_path,
    trange_file=quantize_result.tranges_path,
)
print(Path(cgc_job.analyze_network()).read_text())
cgc_job.compile()
2026-10-23 00:19 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-10-23 00:19 - INFO - epu - chimera_job - Numerical ranges provided
2026-10-23 00:19 - INFO - epu - util - START===============================optimize_relay
2026-10-23 00:19 - INFO - epu - util - START====================quantize_to_cpu_runnable_fx
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /conv1/Conv_output_0_DequantizeLinear to static range: (-6.764380931854248, 6.925437927246094) -> (-6.764380931854248, 6.925437826663256) (within tol 0.0536856)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /conv1/Conv_output_0_DequantizeLinear to static range: (-6.764380931854248, 6.925437927246094) -> (-6.764380931854248, 6.925437826663256) (within tol 0.0536856)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.0/Add_output_0_DequantizeLinear to static range: (-5.883411884307861, 7.526526927947998) -> (-5.883411783725023, 7.526526927947998) (within tol 0.0530037)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.0/Add_output_0_DequantizeLinear to static range: (-5.883411884307861, 7.526526927947998) -> (-5.883411783725023, 7.526526927947998) (within tol 0.0530037)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear to static range: (-5.506912708282471, 3.6116764545440674) -> (-5.5069126933813095, 3.6116764545440674) (within tol 0.0357592)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear to static range: (-5.506912708282471, 3.6116764545440674) -> (-5.5069126933813095, 3.6116764545440674) (within tol 0.0357592)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear to static range: (-4.071578502655029, 3.417748212814331) -> (-4.071578372269869, 3.417748212814331) (within tol 0.0297196)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear to static range: (-4.071578502655029, 3.417748212814331) -> (-4.071578372269869, 3.417748212814331) (within tol 0.0297196)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/Add_output_0_DequantizeLinear to static range: (-4.604947566986084, 5.5744099617004395) -> (-4.604947507381439, 5.5744099617004395) (within tol 0.0403943)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/Add_output_0_DequantizeLinear to static range: (-4.604947566986084, 5.5744099617004395) -> (-4.604947507381439, 5.5744099617004395) (within tol 0.0403943)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer3/layer3.1/Add_output_0_DequantizeLinear to static range: (-6.147651195526123, 6.608725070953369) -> (-6.147651195526123, 6.608725067228079) (within tol 0.0512304)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer3/layer3.1/Add_output_0_DequantizeLinear to static range: (-6.147651195526123, 6.608725070953369) -> (-6.147651195526123, 6.608725067228079) (within tol 0.0512304)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer4/layer4.1/Add_output_0_DequantizeLinear to static range: (-12.259681701660156, 30.999481201171875) -> (-12.259681671857834, 30.999481201171875) (within tol 0.175138)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer4/layer4.1/Add_output_0_DequantizeLinear to static range: (-12.259681701660156, 30.999481201171875) -> (-12.259681671857834, 30.999481201171875) (within tol 0.175138)
2026-10-23 00:19 - INFO - epu - fx - 

Source name                                            Op                                          Output 0 Range           Output 0 Frac Bits
-----------------------------------------------------  ------------------------------------------  ---------------------  --------------------
/conv1/Conv_output_0_DequantizeLinear                  contrib.epu.qlinear_conv2d                  [-6.76438f, 6.92544f]                    28
CustomOp/quadricRelu16                                 contrib.epu.quadric_custom_op_element_wise  [0f, 6.92544f]                           28
/layer1/layer1.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-6.77029f, 4.39629f]                    28
CustomOp/quadricRelu15                                 contrib.epu.quadric_custom_op_element_wise  [0f, 4.39629f]                           28
/layer1/layer1.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-5.88341f, 7.52653f]                    27
CustomOp/quadricRelu14                                 contrib.epu.quadric_custom_op_element_wise  [0f, 7.52653f]                           27
/layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-5.50691f, 3.61168f]                    28
CustomOp/quadricRelu13                                 contrib.epu.quadric_custom_op_element_wise  [0f, 3.61168f]                           28
/layer1/layer1.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-8.84499f, 7.67433f]                    27
CustomOp/quadricRelu12                                 contrib.epu.quadric_custom_op_element_wise  [0f, 7.67433f]                           27
/layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-4.07158f, 3.41775f]                    28
CustomOp/quadricRelu11                                 contrib.epu.quadric_custom_op_element_wise  [0f, 3.41775f]                           28
/layer2/layer2.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-4.60495f, 5.57441f]                    28
CustomOp/quadricRelu10                                 contrib.epu.quadric_custom_op_element_wise  [0f, 5.57441f]                           28
/layer2/layer2.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-3.33472f, 4.00751f]                    28
CustomOp/quadricRelu9                                  contrib.epu.quadric_custom_op_element_wise  [0f, 4.00751f]                           28
/layer2/layer2.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-4.62387f, 6.68891f]                    28
CustomOp/quadricRelu8                                  contrib.epu.quadric_custom_op_element_wise  [0f, 6.68891f]                           28
/layer3/layer3.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-2.97226f, 3.65614f]                    28
CustomOp/quadricRelu7                                  contrib.epu.quadric_custom_op_element_wise  [0f, 3.65614f]                           28
/layer3/layer3.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-3.72033f, 4.65911f]                    28
CustomOp/quadricRelu6                                  contrib.epu.quadric_custom_op_element_wise  [0f, 4.65911f]                           28
/layer3/layer3.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-3.16272f, 4.80734f]                    28
CustomOp/quadricRelu5                                  contrib.epu.quadric_custom_op_element_wise  [0f, 4.80734f]                           28
/layer3/layer3.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-6.14765f, 6.60873f]                    28
CustomOp/quadricRelu4                                  contrib.epu.quadric_custom_op_element_wise  [0f, 6.60873f]                           28
/layer4/layer4.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-2.91034f, 2.65423f]                    29
CustomOp/quadricRelu3                                  contrib.epu.quadric_custom_op_element_wise  [0f, 2.65423f]                           29
/layer4/layer4.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-5.64248f, 6.4839f]                     28
CustomOp/quadricRelu2                                  contrib.epu.quadric_custom_op_element_wise  [0f, 6.4839f]                            28
/layer4/layer4.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-3.24364f, 2.57584f]                    29
CustomOp/quadricRelu1                                  contrib.epu.quadric_custom_op_element_wise  [0f, 2.57584f]                           29
/layer4/layer4.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-12.2597f, 30.9995f]                    25
CustomOp/quadricRelu0                                  contrib.epu.quadric_custom_op_element_wise  [0f, 30.9995f]                           25
output_DequantizeLinear                                contrib.epu.dequantize                      [-10.7316f, 27.2763f]                    26



Analysis of /quadric/sdk-cli/examples/custom_op/model_opt_asym_int8_q.onnx


2026-10-23 00:19 - INFO - epu - chimera_job - Function Prototypes for Custom OPs found
2026-10-23 00:19 - INFO - epu - chimera_job - Please see the file: model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp for a template example of custom op implementation


List of custom ops found in the network:
quadricRelu



2026-10-23 00:19 - INFO - epu - chimera_job - Wrote custom op seed file to model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp.


qVar_t<FixedPoint32<28>> quadricRelu (FixedPoint32<28> ext_tensor_in0  /* CustomOp/quadricRelu16 */ ) {
 /* User implementation of custom op */ 

}

qVar_t<FixedPoint32<27>> quadricRelu (FixedPoint32<27> ext_tensor_in0  /* CustomOp/quadricRelu14 */ ) {
 /* User implementation of custom op */ 

}

qVar_t<FixedPoint32<29>> quadricRelu (FixedPoint32<29> ext_tensor_in0  /* CustomOp/quadricRelu3 */ ) {
 /* User implementation of custom op */ 

}

qVar_t<FixedPoint32<25>> quadricRelu (FixedPoint32<25> ext_tensor_in0  /* CustomOp/quadricRelu0 */ ) {
 /* User implementation of custom op */ 

}




2026-10-23 00:19 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-10-23 00:19 - INFO - epu - chimera_job - Numerical ranges provided
2026-10-23 00:19 - INFO - epu - util - START===============================optimize_relay
2026-10-23 00:19 - INFO - epu - util - START====================quantize_to_cpu_runnable_fx
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /conv1/Conv_output_0_DequantizeLinear to static range: (-6.764380931854248, 6.925437927246094) -> (-6.764380931854248, 6.925437826663256) (within tol 0.0536856)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /conv1/Conv_output_0_DequantizeLinear to static range: (-6.764380931854248, 6.925437927246094) -> (-6.764380931854248, 6.925437826663256) (within tol 0.0536856)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.0/Add_output_0_DequantizeLinear to static range: (-5.883411884307861, 7.526526927947998) -> (-5.883411783725023, 7.526526927947998) (within tol 0.0530037)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.0/Add_output_0_DequantizeLinear to static range: (-5.883411884307861, 7.526526927947998) -> (-5.883411783725023, 7.526526927947998) (within tol 0.0530037)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear to static range: (-5.506912708282471, 3.6116764545440674) -> (-5.5069126933813095, 3.6116764545440674) (within tol 0.0357592)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear to static range: (-5.506912708282471, 3.6116764545440674) -> (-5.5069126933813095, 3.6116764545440674) (within tol 0.0357592)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear to static range: (-4.071578502655029, 3.417748212814331) -> (-4.071578372269869, 3.417748212814331) (within tol 0.0297196)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear to static range: (-4.071578502655029, 3.417748212814331) -> (-4.071578372269869, 3.417748212814331) (within tol 0.0297196)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/Add_output_0_DequantizeLinear to static range: (-4.604947566986084, 5.5744099617004395) -> (-4.604947507381439, 5.5744099617004395) (within tol 0.0403943)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer2/layer2.0/Add_output_0_DequantizeLinear to static range: (-4.604947566986084, 5.5744099617004395) -> (-4.604947507381439, 5.5744099617004395) (within tol 0.0403943)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer3/layer3.1/Add_output_0_DequantizeLinear to static range: (-6.147651195526123, 6.608725070953369) -> (-6.147651195526123, 6.608725067228079) (within tol 0.0512304)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer3/layer3.1/Add_output_0_DequantizeLinear to static range: (-6.147651195526123, 6.608725070953369) -> (-6.147651195526123, 6.608725067228079) (within tol 0.0512304)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer4/layer4.1/Add_output_0_DequantizeLinear to static range: (-12.259681701660156, 30.999481201171875) -> (-12.259681671857834, 30.999481201171875) (within tol 0.175138)
2026-10-23 00:19 - INFO - epu - fx - Clamped annotated range on /layer4/layer4.1/Add_output_0_DequantizeLinear to static range: (-12.259681701660156, 30.999481201171875) -> (-12.259681671857834, 30.999481201171875) (within tol 0.175138)
2026-10-23 00:19 - INFO - epu - fx - 

Source name                                            Op                                          Output 0 Range           Output 0 Frac Bits
-----------------------------------------------------  ------------------------------------------  ---------------------  --------------------
/conv1/Conv_output_0_DequantizeLinear                  contrib.epu.qlinear_conv2d                  [-6.76438f, 6.92544f]                    28
CustomOp/quadricRelu16                                 contrib.epu.quadric_custom_op_element_wise  [0f, 6.92544f]                           28
/layer1/layer1.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-6.77029f, 4.39629f]                    28
CustomOp/quadricRelu15                                 contrib.epu.quadric_custom_op_element_wise  [0f, 4.39629f]                           28
/layer1/layer1.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-5.88341f, 7.52653f]                    27
CustomOp/quadricRelu14                                 contrib.epu.quadric_custom_op_element_wise  [0f, 7.52653f]                           27
/layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-5.50691f, 3.61168f]                    28
CustomOp/quadricRelu13                                 contrib.epu.quadric_custom_op_element_wise  [0f, 3.61168f]                           28
/layer1/layer1.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-8.84499f, 7.67433f]                    27
CustomOp/quadricRelu12                                 contrib.epu.quadric_custom_op_element_wise  [0f, 7.67433f]                           27
/layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-4.07158f, 3.41775f]                    28
CustomOp/quadricRelu11                                 contrib.epu.quadric_custom_op_element_wise  [0f, 3.41775f]                           28
/layer2/layer2.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-4.60495f, 5.57441f]                    28
CustomOp/quadricRelu10                                 contrib.epu.quadric_custom_op_element_wise  [0f, 5.57441f]                           28
/layer2/layer2.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-3.33472f, 4.00751f]                    28
CustomOp/quadricRelu9                                  contrib.epu.quadric_custom_op_element_wise  [0f, 4.00751f]                           28
/layer2/layer2.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-4.62387f, 6.68891f]                    28
CustomOp/quadricRelu8                                  contrib.epu.quadric_custom_op_element_wise  [0f, 6.68891f]                           28
/layer3/layer3.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-2.97226f, 3.65614f]                    28
CustomOp/quadricRelu7                                  contrib.epu.quadric_custom_op_element_wise  [0f, 3.65614f]                           28
/layer3/layer3.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-3.72033f, 4.65911f]                    28
CustomOp/quadricRelu6                                  contrib.epu.quadric_custom_op_element_wise  [0f, 4.65911f]                           28
/layer3/layer3.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-3.16272f, 4.80734f]                    28
CustomOp/quadricRelu5                                  contrib.epu.quadric_custom_op_element_wise  [0f, 4.80734f]                           28
/layer3/layer3.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-6.14765f, 6.60873f]                    28
CustomOp/quadricRelu4                                  contrib.epu.quadric_custom_op_element_wise  [0f, 6.60873f]                           28
/layer4/layer4.0/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-2.91034f, 2.65423f]                    29
CustomOp/quadricRelu3                                  contrib.epu.quadric_custom_op_element_wise  [0f, 2.65423f]                           29
/layer4/layer4.0/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-5.64248f, 6.4839f]                     28
CustomOp/quadricRelu2                                  contrib.epu.quadric_custom_op_element_wise  [0f, 6.4839f]                            28
/layer4/layer4.1/conv1/Conv_output_0_DequantizeLinear  contrib.epu.qlinear_conv2d                  [-3.24364f, 2.57584f]                    29
CustomOp/quadricRelu1                                  contrib.epu.quadric_custom_op_element_wise  [0f, 2.57584f]                           29
/layer4/layer4.1/Add_output_0_DequantizeLinear         contrib.epu.dequantize                      [-12.2597f, 30.9995f]                    25
CustomOp/quadricRelu0                                  contrib.epu.quadric_custom_op_element_wise  [0f, 30.9995f]                           25
output_DequantizeLinear                                contrib.epu.dequantize                      [-10.7316f, 27.2763f]                    26

2026-10-23 00:19 - INFO - epu - util - START====================build_cpu_runnable_fx_relay
2026-10-23 00:19 - INFO - epu - util - START=======================quantize_to_chimera_fx
2026-10-23 00:19 - INFO - epu - util - START=================================relay_to_tir
2026-10-23 00:19 - INFO - epu - util - START===========================relay_to_epu_relay
2026-10-23 00:19 - INFO - epu - util - START==============================adapt_and_order
2026-10-23 00:19 - INFO - epu - mac_counter - 
2026-10-23 00:19 - INFO - epu - mac_counter - ============================================================
2026-10-23 00:19 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-10-23 00:19 - INFO - epu - mac_counter - ============================================================
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 236,027,904 ops (118,013,952 MACs) - /conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.0/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer1/layer1.1/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 115,605,504 ops (57,802,752 MACs) - /layer2/layer2.0/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.0/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 12,845,056 ops (6,422,528 MACs) - /layer2/layer2.0/downsample/downsample.0/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer2/layer2.1/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 115,605,504 ops (57,802,752 MACs) - /layer3/layer3.0/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.0/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 12,845,056 ops (6,422,528 MACs) - /layer3/layer3.0/downsample/downsample.0/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer3/layer3.1/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 115,605,504 ops (57,802,752 MACs) - /layer4/layer4.0/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.0/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 12,845,056 ops (6,422,528 MACs) - /layer4/layer4.0/downsample/downsample.0/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv1/Conv_output_0_DequantizeLinear
2026-10-23 00:19 - INFO - epu - mac_counter -   conv2d: 231,211,008 ops (115,605,504 MACs) - /layer4/layer4.1/conv2/Conv_quant
2026-10-23 00:19 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-10-23 00:19 - INFO - epu - mac_counter - Total: 3,627,122,688 ops (1,813,561,344 MACs)
2026-10-23 00:19 - INFO - epu - mac_counter - ============================================================
2026-10-23 00:19 - INFO - epu - mac_counter - 
2026-10-23 00:19 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 00:19 - INFO - epu - util - START=============================plan_lrm_virtual
2026-10-23 00:19 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 00:19 - INFO - epu - util - START===============================lrm_alloc_loop
2026-10-23 00:19 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 00:19 - INFO - epu - util - START================================lrm_splitting
2026-10-23 00:19 - INFO - epu - util - START=====================LrmUsageChecker.__call__
2026-10-23 00:19 - INFO - epu - util - START==============================ext_split_relay
2026-10-23 00:19 - INFO - epu - util - START=========================OcmAllocator.analyze
2026-10-23 00:19 - INFO - epu - util - START===============================Compiler.build
2026-10-23 00:19 - INFO - epu - util - START========================RelayToPrimFunc.build
2026-10-23 00:19 - INFO - epu - util - START=========================OcmAllocator.analyze
2026-10-23 00:19 - INFO - epu - util - START====================================build_tir
2026-10-23 00:20 - INFO - epu - chimera_job - Compilation of model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful

5. Run on the ISS and in ONNX Runtime

Loads six images from common/calibration/raw with the same preprocessing and runs them through the compiled model on the ISS and through ONNX Runtime with the batch harnesses. Both result lists hold one output logit vector per image.

run_folder = example_path("common/calibration/raw")
labels_path = example_path("common/validation/imagenet_classes.txt")
dict_label = {idx: line.strip() for idx, line in enumerate(labels_path.read_text().splitlines())}

THREADS = 3
NUM_IMG = 6
allimages = mh.get_images(str(run_folder), NUM_IMG)
input_images = [{"input": mh.load_image(i)} for i in allimages]
all_results = {}
all_results["iss"] = cgc_job.run_batch_inference_harness(inputs=input_images, threads=THREADS)
all_results["ort"] = cgc_job.run_batch_ort_harness(inputs=input_images, threads=THREADS)
Processing: 100%|████████████████████████████████| 6/6 [00:31<00:00,  5.32s/it]
2026-10-23 00:20 - WARNING - epu - chimera_job - ORT is not threadsafe -- forcing single threaded batch execution
100%|████████████████████████████████████████████| 6/6 [00:01<00:00,  5.56it/s]

6. Compare

For each image draws the picture with the ISS top-5 classes above the ONNX Runtime top-5 (softmax applied to both). Then prints the lowest correlation between the ISS and ONNX Runtime logit vectors over the six images and the count of images whose top-1 agrees. The assertion requires every correlation to be at least 0.99 (measured lowest 0.9959 with the seed above, top-1 agreeing on all six).

for i, inp in enumerate(input_images):
    fig = plt.figure(constrained_layout=True, figsize=(12, 4))
    gs = GridSpec(2, 2, figure=fig)
    ax = [fig.add_subplot(gs[:, 0]), fig.add_subplot(gs[0, 1])]
    ax.append(fig.add_subplot(gs[1, 1], sharex=ax[1]))

    ax[0].imshow(mpimg.imread(allimages[i]))
    ax[0].set_title(os.path.basename(allimages[i]))
    ax[0].set_axis_off()
    plt.setp(ax[1].get_xticklabels(), visible=False)

    for axis, key in enumerate(all_results):
        classify = model_helpers.ClassifyResult(key, dict_label, softmax=True)
        classify.output = all_results[key][i]["output"]
        classify.calculate_stats()
        classify.add_mpl_fig(ax[axis + 1], "darkviolet" if axis > 0 else None)
plt.show()

iss = np.asarray([r["output"] for r in all_results["iss"]], np.float64).reshape(NUM_IMG, -1)
ort = np.asarray([r["output"] for r in all_results["ort"]], np.float64).reshape(NUM_IMG, -1)
corr = [float(np.corrcoef(a, b)[0, 1]) for a, b in zip(iss, ort)]
agree = int((iss.argmax(axis=1) == ort.argmax(axis=1)).sum())
print(f"lowest logits correlation ISS vs ONNX Runtime: {min(corr):.4f}")
print(f"top-1 agreement ISS vs ONNX Runtime: {agree}/{NUM_IMG}")
assert min(corr) >= 0.99, f"ISS logits drifted from ONNX Runtime: corr {min(corr):.4f}"

lowest logits correlation ISS vs ONNX Runtime: 0.9954
top-1 agreement ISS vs ONNX Runtime: 5/6

7. Profile

Prints the job summary and plots the run statistics of the ISS runs.

print(cgc_job)
cgc_job.plot_run_statistics()
╒═════════════════════╤══════════════════════════════════════════════════════════════════════╕
│ Module Name         │ model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ ONNX File           │ /quadric/sdk-cli/examples/custom_op/model_opt_asym_int8_q.onnx       │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ Custom Ops          │ /quadric/sdk-cli/examples/custom_op/_plugin_out_relu/custom_ops.hpp  │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ Product Target      │ QC-U                                                                 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ Number of Cores     │ 1                                                                    │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700                                                                │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ L2M Size            │ 16MB                                                                 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ LRM Size            │ 4kB                                                                  │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ External Read BW    │ 128GBps                                                              │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ External Write BW   │ 128GBps                                                              │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ MACS per PE         │ 16                                                                   │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ Max L2M             │ 9.749MB                                                              │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ Max LRM             │ 2.375kB                                                              │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes  │ 0.000MB                                                              │
├─────────────────────┼──────────────────────────────────────────────────────────────────────┤
│ Network GMACs       │ 1.814                                                                │
╘═════════════════════╧══════════════════════════════════════════════════════════════════════╛

╒════╤════════╤════════╤══════════════════╤══════════════════════════╤═══════╕
│    │ Type   │ Name   │ shape            │ type                     │ mse   │
╞════╪════════╪════════╪══════════════════╪══════════════════════════╪═══════╡
│  0 │ Input  │ input  │ [1, 3, 224, 224] │ tensor[FixedPoint32<29>] │ n/a   │
├────┼────────┼────────┼──────────────────┼──────────────────────────┼───────┤
│  1 │ Output │ output │ [1, 1000]        │ tensor[FixedPoint32<26>] │ 0.064 │
╘════╧════════╧════════╧══════════════════╧══════════════════════════╧═══════╛

Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation: 
╒══════════════════════════════════╤═════════╕
│ Latency (ms)                     │ 0.28    │
├──────────────────────────────────┼─────────┤
│ FPS                              │ 3589.69 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW)   │ 2941.96 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 1220.17 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB)                │ 11.83   │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB)                │ 0.00    │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps)             │ 41.46   │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps)             │ 0.01    │
├──────────────────────────────────┼─────────┤
│ MAC Utilization                  │ 23.38%  │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm

[SDK-CLI] : TotalCycles: 473,578
[SDK-CLI] : Executions/second: 3,589.69

compute      : ▇▇▇▇▇▇▇▇▇▇▇▇▇ 70.954K
data_array   : ▇▇▇▇▇▇▇▇▇▇ 56.096K
mac          : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 256.085K
data_external: ▇ 8.112K
data_ocm     : ▇▇▇▇▇▇▇▇▇▇▇ 60.91K

for more information check run directory: /quadric/sdk-cli/examples/custom_op/ccl_build/model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20261023_002025_aa112c


2026-10-23 00:20 - INFO - epu - chimera_job - Combined plots generated and saved to: 
/quadric/sdk-cli/examples/custom_op/ccl_build/model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20261023_002025_aa112c/data/model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png





'/quadric/sdk-cli/examples/custom_op/ccl_build/model_opt_asym_int8_q_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20261023_002025_aa112c/data'

Sign in to your account

Don't have an account? 
By signing in, you are agreeing to our Terms of Use and Privacy Policy.
Quadric // One architecture. Every algorithm.

Develop.

Simulate.

Profile.

Collaborate.