NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/yolo/yolov4/yolov4.ipynb.
YOLOv4 Object Detection with Custom Head on Chimera GPNPU
YOLOv4 (Bochkovskiy et al., 2020) is a single-stage anchor-based detector that introduced Mish activations, CSPDarknet53 as a backbone, and a "bag of freebies" of training-time augmentations. This notebook takes the pretrained yolov4.onnx from the ONNX Model Zoo, surgically replaces its detection head with the SDK's CCL custom op cgc::yolov4Head, quantizes the result to INT8, and compiles the whole model with the Chimera Graph Compiler (CGC) so backbone
- head run together on the Chimera GPNPU.
Why a custom head?
YOLOv4's head decodes three scales of anchor predictions with per-scale sigmoids and per-anchor shape gymnastics — operator shapes and branches that don't compile cleanly to the GPNPU. Hand-rolled as a CCL kernel, the same logic runs as a single fused op on-chip, and the rest of the graph compiles as ordinary convolutions. Paired with a host-side NMS, we get the whole pipeline end-to-end.
Model: YOLOv4 (COCO, 416×416)
Note — split pipeline.
- On the Chimera GPNPU: the backbone (including a baked-in
cgc::yolov4Headop)- On the host CPU: NMS and bounding-box visualization
The full pipeline can run end-to-end on the GPNPU — see
yolox_e2e_chipy.ipynborretina_net_e2e_chipy.ipynbfor examples that compose CCL custom ops with ChiPy to keep everything on-chip.
1. Setup
Imports and per-run configuration. ONNX download, graph surgery (shape-fix, transpose removal, custom-op head replacement), calibration subset, and bounding-box visualization all live in yolov4_helpers.py.
import gc
import numpy as np
import onnx
from PIL import Image
from examples.models.yolo.yoloutils.yolohelpers import yolov4_iss_extraction
from sdk_cli.lib.inference import InferenceEngine, batch_inference
from sdk_cli.lib.quantize import QuantizedONNXModel, quantize_onnx_model
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from yolov4_helpers import (
DEFAULT_IMAGE_PATHS,
EXCLUDED_NODE_TYPES,
INPUT_SIZE,
MODEL_NAME,
build_calibration_subset,
build_excluded_nodes,
display_detections,
download_model,
fix_input_shape,
load_images,
remove_input_transpose,
replace_head_with_custom_op,
run_postprocess,
)
%matplotlib inline
2. Download Model
The ONNX Model Zoo hosts a pre-trained yolov4.onnx (plus the matching yolov4_anchors.txt). We pull both rather than re-exporting from Darknet.
src_onnx = download_model()
print(f"Downloaded: {src_onnx.name}")
Downloaded: yolov4.onnx
3. Fix Batch Size
The stock ONNX declares the batch as a runtime-unknown dimension (unk__2104) so that it can accept any batch size. CGC needs a fully static input shape, so we rewrite the input tensor to (1, 416, 416, 3) and run onnxsim to propagate the constant everywhere.
shape_fixed_model = fix_input_shape(src_onnx)
onnx.save(shape_fixed_model, "yolov4-shape-fixed.onnx")
print("Batch size fixed")
Batch size fixed
4. Remove Input Transpose
The stock ONNX includes an NHWC→NCHW transpose wired immediately to the first convolution. Dropping the transpose and feeding the input directly as NCHW removes a dynamic-layout branch and lets us re-declare the input shape as (1, 3, 416, 416).
sim_model = remove_input_transpose(shape_fixed_model)
sim_model_file = f"{MODEL_NAME}-sim.onnx"
onnx.save(sim_model, sim_model_file)
print(f"Transpose removed -> {sim_model_file}")
Transpose removed -> yolov4-sim.onnx
5. Quantization
The shape-fixed model is quantized to INT8 with asymmetric activations on a COCO-like calibration subset. Nodes and op types that feed the eventual custom-op head (Mul, Exp, Log, Tanh, and a list of concats / splits / lambda-adds by name) stay in float — the head is replaced after quantization and consumes its inputs via a single dequantize.
calibration_subset = build_calibration_subset(INPUT_SIZE)
quantized_onnx_model: QuantizedONNXModel = quantize_onnx_model(
sim_model_file,
calibration_subset,
onnx_node_types_to_exclude=EXCLUDED_NODE_TYPES,
onnx_node_names_to_exclude=build_excluded_nodes(),
asymmetric_activation=True,
)
print(f"Quantized ONNX: {quantized_onnx_model.model_path.name}")
print(f"Tensor ranges: {quantized_onnx_model.tensor_ranges_path.name}")
gc.collect()
2026-06-19 03:53 - INFO - sdk - quantize - ONNX model to quantize is defined in OpSet 11 , but the Chimera Graph Compiler (CGC) currently only supports models defined in OpSets: [12, 13, 14, 15, 16]. Converting to OpSet 12.
2026-06-19 03:53 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-06-19 03:53 - DEBUG - sdk - quantize - Forcing node types: []
2026-06-19 03:53 - DEBUG - sdk - quantize - ONNX Node types excluded from quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp', 'Mul', 'Exp', 'Log', 'Tanh']
2026-06-19 03:53 - DEBUG - sdk - quantize - ONNX Node names excluded from quantization: ['StatefulPartitionedCall/model/lambda_11/mul', 'StatefulPartitionedCall/model/lambda_28/mul', 'StatefulPartitionedCall/model/lambda_61/Log', 'StatefulPartitionedCall/model/lambda_6/Tanh', 'StatefulPartitionedCall/model/lambda_51/mul', 'StatefulPartitionedCall/model/lambda_51/add', 'StatefulPartitionedCall/model/lambda_35/Log', 'StatefulPartitionedCall/model/lambda_35/mul', 'StatefulPartitionedCall/model/lambda_53/mul', 'StatefulPartitionedCall/model/lambda_12/add', 'StatefulPartitionedCall/model/lambda_17/Tanh', 'StatefulPartitionedCall/model/lambda_53/Exp', 'StatefulPartitionedCall/model/lambda_66/Exp', 'StatefulPartitionedCall/model/lambda_14/Log', 'StatefulPartitionedCall/model/lambda_19/Exp', 'StatefulPartitionedCall/model/lambda/Exp', 'StatefulPartitionedCall/model/lambda_17/add', 'StatefulPartitionedCall/model/lambda_55/Log', 'StatefulPartitionedCall/model/lambda_42/Exp', 'StatefulPartitionedCall/model/lambda_67/mul', 'StatefulPartitionedCall/model/lambda_22/add', 'StatefulPartitionedCall/model/lambda_20/Tanh', 'StatefulPartitionedCall/model/lambda_60/Log', 'StatefulPartitionedCall/model/tf_op_layer_concat_11/concat_11', 'StatefulPartitionedCall/model/lambda_43/Exp', 'StatefulPartitionedCall/model/lambda_27/add', 'StatefulPartitionedCall/model/lambda_14/add', 'StatefulPartitionedCall/model/lambda_47/Log', 'StatefulPartitionedCall/model/lambda_1/mul', 'StatefulPartitionedCall/model/lambda_50/mul', 'StatefulPartitionedCall/model/lambda_66/mul', 'StatefulPartitionedCall/model/lambda_61/mul', 'StatefulPartitionedCall/model/lambda_17/Exp', 'StatefulPartitionedCall/model/tf_op_layer_Sigmoid_5/Sigmoid_5', 'StatefulPartitionedCall/model/lambda_22/Exp', 'StatefulPartitionedCall/model/lambda_2/Exp', 'StatefulPartitionedCall/model/lambda_30/Exp', 'StatefulPartitionedCall/model/lambda/mul', 'StatefulPartitionedCall/model/lambda_70/Exp', 'StatefulPartitionedCall/model/tf_op_layer_concat_12/concat_12', 'StatefulPartitionedCall/model/lambda_42/mul', 'StatefulPartitionedCall/model/lambda_28/add', 'StatefulPartitionedCall/model/lambda_67/add', 'StatefulPartitionedCall/model/lambda_24/add', 'StatefulPartitionedCall/model/lambda_65/add', 'StatefulPartitionedCall/model/lambda_22/Log', 'StatefulPartitionedCall/model/lambda_53/Tanh', 'StatefulPartitionedCall/model/tf_op_layer_concat_10/concat_10', 'StatefulPartitionedCall/model/lambda_50/Tanh', 'StatefulPartitionedCall/model/lambda_43/add', 'StatefulPartitionedCall/model/lambda_49/Tanh', 'StatefulPartitionedCall/model/lambda_4/Tanh', 'StatefulPartitionedCall/model/lambda_27/Tanh', 'StatefulPartitionedCall/model/lambda_18/add', 'StatefulPartitionedCall/model/lambda_21/Exp', 'StatefulPartitionedCall/model/lambda_31/Tanh', 'StatefulPartitionedCall/model/lambda_47/add', 'StatefulPartitionedCall/model/lambda_33/Exp', 'StatefulPartitionedCall/model/lambda_28/Exp', 'StatefulPartitionedCall/model/lambda_4/mul', 'StatefulPartitionedCall/model/lambda_30/mul', 'StatefulPartitionedCall/model/lambda_68/mul', 'StatefulPartitionedCall/model/lambda_12/Log', 'StatefulPartitionedCall/model/lambda_36/add', 'StatefulPartitionedCall/model/lambda_42/Tanh', 'StatefulPartitionedCall/model/lambda_49/Log', 'StatefulPartitionedCall/model/lambda_52/Tanh', 'StatefulPartitionedCall/model/lambda_68/add', 'StatefulPartitionedCall/model/lambda_5/add', 'StatefulPartitionedCall/model/lambda_23/add', 'StatefulPartitionedCall/model/lambda_53/Log', 'StatefulPartitionedCall/model/tf_op_layer_split/split', 'StatefulPartitionedCall/model/lambda_26/Exp', 'StatefulPartitionedCall/model/lambda_54/mul', 'StatefulPartitionedCall/model/lambda_69/add', 'StatefulPartitionedCall/model/lambda_40/mul', 'StatefulPartitionedCall/model/lambda_46/add', 'StatefulPartitionedCall/model/tf_op_layer_split_2/split_2', 'StatefulPartitionedCall/model/lambda_7/mul', 'StatefulPartitionedCall/model/lambda_5/Exp', 'StatefulPartitionedCall/model/lambda_54/Exp', 'StatefulPartitionedCall/model/lambda_39/Log', 'StatefulPartitionedCall/model/lambda_13/Tanh', 'StatefulPartitionedCall/model/lambda_34/mul', 'StatefulPartitionedCall/model/lambda_7/Tanh', 'StatefulPartitionedCall/model/lambda_39/mul', 'StatefulPartitionedCall/model/lambda_58/add', 'StatefulPartitionedCall/model/lambda_59/Tanh', 'StatefulPartitionedCall/model/lambda_23/Exp', 'StatefulPartitionedCall/model/lambda_29/add', 'StatefulPartitionedCall/model/lambda_45/mul', 'StatefulPartitionedCall/model/lambda_65/mul', 'StatefulPartitionedCall/model/lambda_39/add', 'StatefulPartitionedCall/model/lambda_38/add', 'StatefulPartitionedCall/model/lambda_14/Exp', 'StatefulPartitionedCall/model/lambda_7/Log', 'StatefulPartitionedCall/model/lambda_44/Tanh', 'StatefulPartitionedCall/model/lambda_57/Log', 'StatefulPartitionedCall/model/lambda_59/Log', 'StatefulPartitionedCall/model/lambda_31/Log', 'StatefulPartitionedCall/model/lambda_33/mul', 'StatefulPartitionedCall/model/lambda_19/mul', 'StatefulPartitionedCall/model/lambda_16/add', 'StatefulPartitionedCall/model/lambda_14/Tanh', 'StatefulPartitionedCall/model/lambda_13/add', 'StatefulPartitionedCall/model/lambda_64/mul', 'StatefulPartitionedCall/model/lambda_35/Tanh', 'StatefulPartitionedCall/model/lambda_22/mul', 'StatefulPartitionedCall/model/tf_op_layer_split_1/split_1', 'StatefulPartitionedCall/model/lambda_56/add', 'StatefulPartitionedCall/model/lambda_32/Exp', 'StatefulPartitionedCall/model/lambda_2/mul', 'StatefulPartitionedCall/model/lambda_7/Exp', 'StatefulPartitionedCall/model/lambda_51/Log', 'StatefulPartitionedCall/model/lambda_69/Exp', 'StatefulPartitionedCall/model/lambda_46/Log', 'StatefulPartitionedCall/model/lambda_10/Exp', 'StatefulPartitionedCall/model/lambda_37/Tanh', 'StatefulPartitionedCall/model/tf_op_layer_Sigmoid_1/Sigmoid_1', 'StatefulPartitionedCall/model/lambda_70/Tanh', 'StatefulPartitionedCall/model/lambda_26/Tanh', 'StatefulPartitionedCall/model/lambda_15/Log', 'StatefulPartitionedCall/model/lambda_9/Log', 'StatefulPartitionedCall/model/lambda_45/Log', 'StatefulPartitionedCall/model/lambda_64/Tanh', 'StatefulPartitionedCall/model/lambda_47/Exp', 'StatefulPartitionedCall/model/lambda_68/Log', 'StatefulPartitionedCall/model/lambda_47/mul', 'StatefulPartitionedCall/model/lambda_51/Tanh', 'StatefulPartitionedCall/model/lambda_62/Tanh', 'StatefulPartitionedCall/model/lambda_40/Tanh', 'StatefulPartitionedCall/model/lambda_11/Log', 'StatefulPartitionedCall/model/lambda_25/Tanh', 'StatefulPartitionedCall/model/lambda_57/mul', 'StatefulPartitionedCall/model/lambda/Log', 'StatefulPartitionedCall/model/lambda_8/mul', 'StatefulPartitionedCall/model/lambda_58/Log', 'StatefulPartitionedCall/model/lambda_25/Log', 'StatefulPartitionedCall/model/lambda/add', 'StatefulPartitionedCall/model/lambda_63/add', 'StatefulPartitionedCall/model/lambda_47/Tanh', 'StatefulPartitionedCall/model/lambda_45/Tanh', 'StatefulPartitionedCall/model/lambda_55/Exp', 'StatefulPartitionedCall/model/lambda_49/mul', 'StatefulPartitionedCall/model/lambda_21/add', 'StatefulPartitionedCall/model/lambda_4/add', 'StatefulPartitionedCall/model/tf_op_layer_Sigmoid/Sigmoid', 'StatefulPartitionedCall/model/lambda_14/mul', 'StatefulPartitionedCall/model/lambda_36/Log', 'StatefulPartitionedCall/model/lambda_11/add', 'StatefulPartitionedCall/model/lambda_4/Exp', 'StatefulPartitionedCall/model/lambda_70/Log', 'StatefulPartitionedCall/model/lambda_13/mul', 'StatefulPartitionedCall/model/lambda_38/Log', 'StatefulPartitionedCall/model/lambda_41/Tanh', 'StatefulPartitionedCall/model/lambda_42/Log', 'StatefulPartitionedCall/model/lambda_43/mul', 'StatefulPartitionedCall/model/lambda_22/Tanh', 'StatefulPartitionedCall/model/lambda_57/Tanh', 'StatefulPartitionedCall/model/lambda_54/add', 'StatefulPartitionedCall/model/lambda_12/Exp', 'StatefulPartitionedCall/model/lambda_34/Log', 'StatefulPartitionedCall/model/lambda_18/Exp', 'StatefulPartitionedCall/model/lambda_71/add', 'StatefulPartitionedCall/model/lambda_34/Tanh', 'StatefulPartitionedCall/model/lambda_58/mul', 'StatefulPartitionedCall/model/lambda_52/add', 'StatefulPartitionedCall/model/lambda_29/Tanh', 'StatefulPartitionedCall/model/lambda_5/mul', 'StatefulPartitionedCall/model/lambda_16/Exp', 'StatefulPartitionedCall/model/lambda_52/mul', 'StatefulPartitionedCall/model/lambda_68/Tanh', 'StatefulPartitionedCall/model/lambda_31/mul', 'StatefulPartitionedCall/model/lambda_6/Exp', 'StatefulPartitionedCall/model/lambda_32/Tanh', 'StatefulPartitionedCall/model/lambda_17/Log', 'StatefulPartitionedCall/model/lambda_9/Exp', 'StatefulPartitionedCall/model/lambda_15/Exp', 'StatefulPartitionedCall/model/lambda_55/add', 'StatefulPartitionedCall/model/lambda_30/add', 'StatefulPartitionedCall/model/lambda_35/add', 'StatefulPartitionedCall/model/lambda_32/Log', 'StatefulPartitionedCall/model/lambda_9/add', 'StatefulPartitionedCall/model/lambda_36/Tanh', 'StatefulPartitionedCall/model/lambda_16/Tanh', 'StatefulPartitionedCall/model/lambda_33/add', 'StatefulPartitionedCall/model/lambda_38/mul', 'StatefulPartitionedCall/model/lambda_0/add', 'StatefulPartitionedCall/model/lambda_1/Exp', 'StatefulPartitionedCall/model/lambda_41/mul', 'StatefulPartitionedCall/model/lambda_2/Log', 'StatefulPartitionedCall/model/lambda_61/Tanh', 'StatefulPartitionedCall/model/lambda_69/Log', 'StatefulPartitionedCall/model/lambda_70/add', 'StatefulPartitionedCall/model/lambda_3/Exp', 'StatefulPartitionedCall/model/lambda_26/mul', 'StatefulPartitionedCall/model/lambda_71/Exp', 'StatefulPartitionedCall/model/lambda_32/add', 'StatefulPartitionedCall/model/lambda_69/Tanh', 'StatefulPartitionedCall/model/lambda_1/Tanh', 'StatefulPartitionedCall/model/lambda_28/Tanh', 'StatefulPartitionedCall/model/lambda_15/Tanh', 'StatefulPartitionedCall/model/lambda_32/mul', 'StatefulPartitionedCall/model/lambda_19/Log', 'StatefulPartitionedCall/model/lambda_30/Tanh', 'StatefulPartitionedCall/model/lambda_35/Exp', 'StatefulPartitionedCall/model/lambda_36/mul', 'StatefulPartitionedCall/model/lambda_59/Exp', 'StatefulPartitionedCall/model/lambda_50/Log', 'StatefulPartitionedCall/model/lambda_68/Exp', 'StatefulPartitionedCall/model/lambda_62/mul', 'StatefulPartitionedCall/model/lambda_70/mul', 'StatefulPartitionedCall/model/lambda_20/mul', 'StatefulPartitionedCall/model/lambda_48/Exp', 'StatefulPartitionedCall/model/lambda_8/Log', 'StatefulPartitionedCall/model/lambda_21/Log', 'StatefulPartitionedCall/model/lambda_46/Tanh', 'StatefulPartitionedCall/model/lambda_25/Exp', 'StatefulPartitionedCall/model/lambda_59/add', 'StatefulPartitionedCall/model/lambda_40/Log', 'StatefulPartitionedCall/model/lambda_64/add', 'StatefulPartitionedCall/model/lambda_21/Tanh', 'StatefulPartitionedCall/model/tf_op_layer_Sigmoid_3/Sigmoid_3', 'StatefulPartitionedCall/model/lambda_48/Log', 'StatefulPartitionedCall/model/lambda_6/mul', 'StatefulPartitionedCall/model/lambda_10/mul', 'StatefulPartitionedCall/model/lambda_6/Log', 'StatefulPartitionedCall/model/lambda_18/Tanh', 'StatefulPartitionedCall/model/lambda_61/Exp', 'StatefulPartitionedCall/model/lambda_41/add', 'StatefulPartitionedCall/model/lambda_25/add', 'StatefulPartitionedCall/model/lambda_44/mul', 'StatefulPartitionedCall/model/lambda_5/Log', 'StatefulPartitionedCall/model/lambda_27/mul', 'StatefulPartitionedCall/model/lambda_31/add', 'StatefulPartitionedCall/model/lambda_40/add', 'StatefulPartitionedCall/model/lambda_16/mul', 'StatefulPartitionedCall/model/lambda_37/Log', 'StatefulPartitionedCall/model/lambda_50/add', 'StatefulPartitionedCall/model/lambda_5/Tanh', 'StatefulPartitionedCall/model/lambda_55/mul', 'StatefulPartitionedCall/model/lambda_60/Exp', 'StatefulPartitionedCall/model/lambda_27/Log', 'StatefulPartitionedCall/model/lambda_24/Exp', 'StatefulPartitionedCall/model/lambda_56/Tanh', 'StatefulPartitionedCall/model/lambda_7/add', 'StatefulPartitionedCall/model/lambda_38/Tanh', 'StatefulPartitionedCall/model/lambda_37/mul', 'StatefulPartitionedCall/model/lambda_23/Tanh', 'StatefulPartitionedCall/model/lambda_12/Tanh', 'StatefulPartitionedCall/model/lambda_31/Exp', 'StatefulPartitionedCall/model/lambda_63/mul', 'StatefulPartitionedCall/model/lambda_11/Tanh', 'StatefulPartitionedCall/model/lambda_69/mul', 'StatefulPartitionedCall/model/lambda_63/Log', 'StatefulPartitionedCall/model/lambda_23/mul', 'StatefulPartitionedCall/model/lambda_65/Tanh', 'StatefulPartitionedCall/model/lambda_37/add', 'StatefulPartitionedCall/model/lambda_71/Log', 'StatefulPartitionedCall/model/lambda_10/Log', 'StatefulPartitionedCall/model/lambda_56/Log', 'StatefulPartitionedCall/model/lambda_20/add', 'StatefulPartitionedCall/model/lambda/Tanh', 'StatefulPartitionedCall/model/lambda_46/mul', 'StatefulPartitionedCall/model/lambda_3/Log', 'StatefulPartitionedCall/model/lambda_51/Exp', 'StatefulPartitionedCall/model/lambda_60/mul', 'StatefulPartitionedCall/model/lambda_49/Exp', 'StatefulPartitionedCall/model/lambda_66/Log', 'StatefulPartitionedCall/model/lambda_67/Log', 'StatefulPartitionedCall/model/lambda_20/Log', 'StatefulPartitionedCall/model/lambda_30/Log', 'StatefulPartitionedCall/model/lambda_29/mul', 'StatefulPartitionedCall/model/lambda_48/Tanh', 'StatefulPartitionedCall/model/lambda_1/add', 'StatefulPartitionedCall/model/lambda_34/Exp', 'StatefulPartitionedCall/model/lambda_8/add', 'StatefulPartitionedCall/model/tf_op_layer_Sigmoid_4/Sigmoid_4', 'StatefulPartitionedCall/model/lambda_57/Exp', 'StatefulPartitionedCall/model/lambda_64/Exp', 'StatefulPartitionedCall/model/lambda_24/mul', 'StatefulPartitionedCall/model/lambda_36/Exp', 'StatefulPartitionedCall/model/lambda_63/Tanh', 'StatefulPartitionedCall/model/lambda_19/add', 'StatefulPartitionedCall/model/lambda_58/Tanh', 'StatefulPartitionedCall/model/lambda_46/Exp', 'StatefulPartitionedCall/model/lambda_71/mul', 'StatefulPartitionedCall/model/lambda_9/mul', 'StatefulPartitionedCall/model/lambda_33/Log', 'StatefulPartitionedCall/model/lambda_54/Tanh', 'StatefulPartitionedCall/model/lambda_64/Log', 'StatefulPartitionedCall/model/lambda_53/add', 'StatefulPartitionedCall/model/lambda_58/Exp', 'StatefulPartitionedCall/model/lambda_45/add', 'StatefulPartitionedCall/model/lambda_2/Tanh', 'StatefulPartitionedCall/model/lambda_65/Exp', 'StatefulPartitionedCall/model/lambda_29/Exp', 'StatefulPartitionedCall/model/lambda_52/Exp', 'StatefulPartitionedCall/model/lambda_2/add', 'StatefulPartitionedCall/model/lambda_27/Exp', 'StatefulPartitionedCall/model/lambda_13/Log', 'StatefulPartitionedCall/model/lambda_8/Exp', 'StatefulPartitionedCall/model/lambda_57/add', 'StatefulPartitionedCall/model/lambda_18/Log', 'StatefulPartitionedCall/model/lambda_60/add', 'StatefulPartitionedCall/model/lambda_33/Tanh', 'StatefulPartitionedCall/model/lambda_6/add', 'StatefulPartitionedCall/model/lambda_62/add', 'StatefulPartitionedCall/model/lambda_43/Log', 'StatefulPartitionedCall/model/lambda_15/add', 'StatefulPartitionedCall/model/lambda_67/Exp', 'StatefulPartitionedCall/model/lambda_65/Log', 'StatefulPartitionedCall/model/lambda_48/mul', 'StatefulPartitionedCall/model/lambda_3/add', 'StatefulPartitionedCall/model/lambda_16/Log', 'StatefulPartitionedCall/model/lambda_66/add', 'StatefulPartitionedCall/model/lambda_39/Tanh', 'StatefulPartitionedCall/model/lambda_43/Tanh', 'StatefulPartitionedCall/model/lambda_3/Tanh', 'StatefulPartitionedCall/model/lambda_67/Tanh', 'StatefulPartitionedCall/model/lambda_38/Exp', 'StatefulPartitionedCall/model/lambda_24/Log', 'StatefulPartitionedCall/model/lambda_49/add', 'StatefulPartitionedCall/model/lambda_71/Tanh', 'StatefulPartitionedCall/model/lambda_41/Log', 'StatefulPartitionedCall/model/lambda_20/Exp', 'StatefulPartitionedCall/model/lambda_42/add', 'StatefulPartitionedCall/model/lambda_17/mul', 'StatefulPartitionedCall/model/lambda_26/Log', 'StatefulPartitionedCall/model/lambda_61/add', 'StatefulPartitionedCall/model/lambda_62/Exp', 'StatefulPartitionedCall/model/lambda_12/mul', 'StatefulPartitionedCall/model/lambda_4/Log', 'StatefulPartitionedCall/model/lambda_37/Exp', 'StatefulPartitionedCall/model/tf_op_layer_Sigmoid_2/Sigmoid_2', 'StatefulPartitionedCall/model/lambda_26/add', 'StatefulPartitionedCall/model/lambda_18/mul', 'StatefulPartitionedCall/model/lambda_34/add', 'StatefulPartitionedCall/model/lambda_1/Log', 'StatefulPartitionedCall/model/lambda_54/Log', 'StatefulPartitionedCall/model/lambda_60/Tanh', 'StatefulPartitionedCall/model/lambda_40/Exp', 'StatefulPartitionedCall/model/lambda_24/Tanh', 'StatefulPartitionedCall/model/lambda_56/Exp', 'StatefulPartitionedCall/model/lambda_10/add', 'StatefulPartitionedCall/model/lambda_28/Log', 'StatefulPartitionedCall/model/lambda_13/Exp', 'StatefulPartitionedCall/model/lambda_44/Log', 'StatefulPartitionedCall/model/lambda_41/Exp', 'StatefulPartitionedCall/model/lambda_11/Exp', 'StatefulPartitionedCall/model/lambda_44/add', 'StatefulPartitionedCall/model/lambda_39/Exp', 'StatefulPartitionedCall/model/lambda_62/Log', 'StatefulPartitionedCall/model/lambda_21/mul', 'StatefulPartitionedCall/model/lambda_15/mul', 'StatefulPartitionedCall/model/lambda_10/Tanh', 'StatefulPartitionedCall/model/lambda_19/Tanh', 'StatefulPartitionedCall/model/lambda_50/Exp', 'StatefulPartitionedCall/model/lambda_23/Log', 'StatefulPartitionedCall/model/lambda_45/Exp', 'StatefulPartitionedCall/model/lambda_52/Log', 'StatefulPartitionedCall/model/lambda_3/mul', 'StatefulPartitionedCall/model/lambda_9/Tanh', 'StatefulPartitionedCall/model/lambda_66/Tanh', 'StatefulPartitionedCall/model/lambda_55/Tanh', 'StatefulPartitionedCall/model/lambda_8/Tanh', 'StatefulPartitionedCall/model/lambda_56/mul', 'StatefulPartitionedCall/model/lambda_59/mul', 'StatefulPartitionedCall/model/lambda_63/Exp', 'StatefulPartitionedCall/model/lambda_25/mul', 'StatefulPartitionedCall/model/lambda_48/add', 'StatefulPartitionedCall/model/lambda_29/Log', 'StatefulPartitionedCall/model/lambda_44/Exp']
2026-06-19 03:53 - INFO - sdk - quantize - Starting quantization...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-06-19 03:54 - INFO - sdk - quantize - Quantization completed! Quantized model saved to /quadric/sdk-cli/examples/models/yolo/yolov4/yolov4-sim_OpSet12_optimized_asym_int8_q.onnx
2026-06-19 03:54 - INFO - sdk - quantize - ONNX full precision model size: 245.58MB
2026-06-19 03:54 - INFO - sdk - quantize - ONNX quantized model size: 61.82MB
2026-06-19 03:54 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-06-19 03:54 - INFO - sdk - quantize - ONNX Model with well-defined shapes has been saved at `/quadric/sdk-cli/examples/models/yolo/yolov4/yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.onnx`.
2026-06-19 03:54 - DEBUG - sdk - quantize - Checking for FLOAT/FLOAT16 types...
2026-06-19 03:54 - INFO - sdk - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-06-19 03:54 - INFO - sdk - quantize - Model still has FLOAT/FLOAT16 types after quantization. Creating ranges for floating point tensors using calibration data...
2026-06-19 03:54 - INFO - sdk - quantize - Saved computed tensor ranges to /quadric/sdk-cli/examples/models/yolo/yolov4/yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.tranges.
2026-06-19 03:54 - INFO - sdk - quantize -
╒═══════════════════════════════════════════════════════════════════════════════════════════════════╤══════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Quantized ONNX Model │ Tensor Ranges File │
╞═══════════════════════════════════════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ /quadric/sdk-cli/examples/models/yolo/yolov4/yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.onnx │ /quadric/sdk-cli/examples/models/yolo/yolov4/yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.tranges │
╘═══════════════════════════════════════════════════════════════════════════════════════════════════╧══════════════════════════════════════════════════════════════════════════════════════════════════════╛
Quantized ONNX: yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.onnx
Tensor ranges: yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.tranges
0
6. Replace Heads with cgc::yolov4Head Custom Op
Each of YOLOv4's three detection heads expresses itself as the subgraph DequantizeLinear → Transpose → Reshape → Split → (Sigmoid × 2) → Concat. CustomOpReplacer matches that pattern with a filter tree and rewrites every occurrence as a single call to cgc::yolov4Head<22> — a fused CCL kernel that handles anchor decoding, per-scale sigmoids, and coordinate scaling on the GPNPU.
custom_onnx_path = replace_head_with_custom_op(quantized_onnx_model.model_path)
print(f"Custom-op heads wired -> {custom_onnx_path.name}")
Custom-op heads wired -> yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.onnx
7. Compilation
ChimeraJob wraps the full compile pipeline: graph analyze, kernel selection (including the custom-op heads), memory planning, and code generation. The backbone and all three custom-op heads end up on the GPNPU in the same compiled job.
cgc_job = ChimeraJob(model_p=str(custom_onnx_path))
cgc_job.compile(quiet=True)
8. Inference
batch_inference runs the compiled model against multiple images in parallel through two engines: CHIMERA_ORT_INT8 (ONNX Runtime reference) and CHIMERA_ISS_INT8 (the Chimera GPNPU cycle-accurate simulator). The yolov4_iss_extraction helper reshapes the ISS outputs so they match the layout the postprocess expects.
all_image_paths, all_images = load_images(INPUT_SIZE, DEFAULT_IMAGE_PATHS)
num_images = len(all_images)
engines = {
InferenceEngine.CHIMERA_ORT_INT8: cgc_job,
InferenceEngine.CHIMERA_ISS_INT8: cgc_job,
}
outputs_per_engine = {}
threads = min(num_images, 6)
for engine, job in engines.items():
outputs_per_engine[engine] = batch_inference(
engine,
job,
all_images,
iss_extraction=yolov4_iss_extraction,
threads=threads,
)
2026-06-19 04:01 - WARNING - epu - chimera_job - ORT is not threadsafe -- forcing single threaded batch execution
100%|████████████████████████████████████████████| 3/3 [00:03<00:00, 1.06s/it]
Processing: 100%|████████████████████████████████| 3/3 [04:27<00:00, 89.07s/it]
9. Run Statistics
plot_run_statistics() renders the per-kernel cycle breakdown. The custom-op heads appear alongside the CSPDarknet53 backbone kernels.
print(cgc_job)
cgc_job.plot_run_statistics()
╒═════════════════════╤═══════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ yolov4_sim_OpSet12_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/models/yolo/yolov4/yolov4-sim_OpSet12_optimized_asym_int8_q_shaped.onnx │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 16MB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 15.818MB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 3.000kB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 3.288MB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 30.052 │
╘═════════════════════╧═══════════════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤══════════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪══════════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input_1:0 │ [1, 3, 416, 416] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼──────────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ Identity:0 │ [1, 2704, 3, 85] │ tensor[FixedPoint32<16>] │ 0.003 │
├────┼────────┼──────────────┼──────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ Identity_1:0 │ [1, 676, 3, 85] │ tensor[FixedPoint32<16>] │ 0.002 │
├────┼────────┼──────────────┼──────────────────┼──────────────────────────┼───────┤
│ 3 │ Output │ Identity_2:0 │ [1, 169, 3, 85] │ tensor[FixedPoint32<16>] │ 0.001 │
╘════╧════════╧══════════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 3.81 │
├──────────────────────────────────┼─────────┤
│ FPS │ 262.36 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 1985.28 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 132.15 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 64.70 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 4.34 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 16.58 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 1.11 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 28.31% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 6,479,616
[SDK-CLI] : Executions/second: 262.36
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1.233M
data_array : ▇▇▇▇▇▇▇▇▇▇▇▇▇ 788.564K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 3.019M
data_external: ▇ 112.809K
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1.325M
for more information check run directory: /quadric/sdk-cli/examples/models/yolo/yolov4/ccl_build/yolov4_sim_OpSet12_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040153_882e3a
2026-06-19 04:06 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/models/yolo/yolov4/ccl_build/yolov4_sim_OpSet12_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040153_882e3a/data/yolov4_sim_OpSet12_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/models/yolo/yolov4/ccl_build/yolov4_sim_OpSet12_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_040153_882e3a/data'
Decode the network outputs and overlay the detections on the input image.
detections_per_engine = run_postprocess(outputs_per_engine, all_image_paths, INPUT_SIZE)
display_detections(all_image_paths, detections_per_engine)
Summary
| Model | YOLOv4 (COCO, 416×416) |
| Pipeline | ONNX download → batch-size fix → transpose removal → quantize (INT8 asymmetric) → replace 3 heads with cgc::yolov4Head → CGC compile → ISS + ORT inference → YOLOv4 NMS → bbox overlay |
| Compiled on GPNPU | CSPDarknet53 backbone + three cgc::yolov4Head custom ops |
| On host CPU | NMS, bounding-box visualization |
| Custom Op | cgc::yolov4Head<22> — anchor decode + per-scale sigmoid + box scaling |
| Calibration | QuadricCalibration COCO-like subset |
Key takeaways
- YOLOv4's three-scale detection heads all share the same
DequantizeLinear → Transpose → Reshape → Split → Sigmoid → Concatpattern; a singleCustomOpReplacerfilter rewrites all three in one pass. - Graph surgery (batch-size fix, transpose removal) reshapes the onnx/models-hosted checkpoint into something CGC can compile without dynamic-layout branches.
- The excluded-node list during quantization keeps the operators that feed the custom-op head in float so the head can dequantize once and operate natively on its fused CCL path.
Citation
@article{bochkovskiy2020yolov4,
title = {YOLOv4: Optimal Speed and Accuracy of Object Detection},
author = {Bochkovskiy, Alexey and Wang, Chien-Yao and Liao, Hong-Yuan Mark},
journal = {arXiv preprint arXiv:2004.10934},
year = {2020}
}
