NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/yolo/yolov5_seg/yolov5_seg.ipynb.
YOLOv5 Instance Segmentation on Chimera GPNPU
YOLOv5-seg is Ultralytics' v7.0 instance-segmentation extension of YOLOv5 — the same backbone with an extra mask-prototype head, producing a tuple of bounding boxes plus per-instance segmentation masks. This notebook compiles the yolov5n-seg variant end-to-end: download the pre-built backbone + head ONNX, quantize the backbone to INT8, compile with the Chimera Graph Compiler (CGC), and run ISS + ORT INT8 inferences side-by-side with mask + box overlays.
Why split the graph?
The detection head and mask-prototype head together mix anchor decoding, per-scale sigmoids, and prototype combination — operators that are cheaper to keep on the host CPU. The backbone (with its convolutional mask-prototype trunk) compiles well to the Chimera GPNPU. The ONNX is pre-split at the four head-boundary edges — CGC handles the convolutional bulk and ONNX Runtime handles the classical tail.
Model: YOLOv5n-seg (COCO, 640×640, 2.0M params, 7.1 GFLOPs)
Note — split pipeline.
- On the Chimera GPNPU: the backbone
- On the host CPU: the detection head ONNX, mask coefficient fusion, NMS, and visualization
The full pipeline can run end-to-end on the GPNPU — see
yolox_e2e_chipy.ipynborretina_net_e2e_chipy.ipynbfor examples that compose CCL custom ops with ChiPy to keep everything on-chip.
1. Setup
Imports and per-run configuration. The S3 ONNX download, calibration subset, input loading, postprocess, and mask + box visualization all live in yolov5_seg_helpers.py.
import gc
import onnx
from onnxruntime import InferenceSession
from sdk_cli.lib.inference import InferenceEngine, batch_inference
from sdk_cli.lib.quantize import QuantizedONNXModel, quantize_onnx_model
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from yolov5_seg_helpers import (
DEFAULT_IMAGE_PATHS,
INPUT_SIZE,
build_calibration_subset,
display_detections,
download_model,
load_images,
run_postprocess,
)
%matplotlib inline
2. Model Download
download_model() fetches the pre-built YOLOv5n-seg backbone + head ONNX from the public sdk-cli-models S3 bucket. The backbone covers the convolutional stack plus the mask-prototype trunk; the head holds the anchor-decode + per-scale detection ops and emits two outputs — outputs0 (boxes + scores + labels + mask coefficients) and outputs1 (mask prototypes).
backbone_onnx_file, head_onnx_file = download_model()
Downloading yolov5n-seg-backbone.onnx ...
Backbone: yolov5n-seg-backbone.onnx (8.0 MB)
Downloading yolov5n-seg-head.onnx ...
Head: yolov5n-seg-head.onnx (0.4 MB)
3. Quantization
The backbone is quantized to INT8 with asymmetric activations using the COCO-like subset of QuadricCalibration for tensor-range statistics. The QuantizedONNXModel return value carries both the quantized ONNX and its companion .tranges file.
calibration_subset = build_calibration_subset(INPUT_SIZE)
quantized_onnx_model: QuantizedONNXModel = quantize_onnx_model(
str(backbone_onnx_file),
calibration_subset,
asymmetric_activation=True,
)
print(f"Quantized ONNX: {quantized_onnx_model.model_path.name}")
print(f"Tensor ranges: {quantized_onnx_model.tensor_ranges_path.name}")
gc.collect()
2026-09-22 02:41 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-09-22 02:41 - DEBUG - sdk - quantize - Forcing node types: []
2026-09-22 02:41 - DEBUG - sdk - quantize - ONNX Node types excluded from quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-09-22 02:41 - DEBUG - sdk - quantize - ONNX Node names excluded from quantization: ['/model.0/act/Sigmoid', '/model.4/m/m.0/cv2/act/Mul', '/model.0/act/Mul', '/model.4/m/m.1/cv1/act/Sigmoid', '/model.23/cv1/act/Mul', '/model.4/m/m.0/cv1/act/Sigmoid', '/model.1/act/Sigmoid', '/model.2/cv1/act/Sigmoid', '/model.14/act/Mul', '/model.8/m/m.0/cv1/act/Sigmoid', '/model.9/cv2/act/Mul', '/model.24/proto/cv2/act/Mul', '/model.20/cv3/act/Sigmoid', '/model.20/m/m.0/cv1/act/Mul', '/model.17/m/m.0/cv2/act/Sigmoid', '/model.17/cv3/act/Mul', '/model.8/cv2/act/Mul', '/model.4/m/m.1/cv2/act/Sigmoid', '/model.6/m/m.2/cv2/act/Sigmoid', '/model.13/m/m.0/cv1/act/Sigmoid', '/model.17/cv1/act/Sigmoid', '/model.2/m/m.0/cv2/act/Sigmoid', '/model.2/cv2/act/Sigmoid', '/model.1/act/Mul', '/model.6/cv2/act/Mul', '/model.8/cv3/act/Sigmoid', '/model.13/m/m.0/cv2/act/Sigmoid', '/model.17/cv3/act/Sigmoid', '/model.4/m/m.0/cv1/act/Mul', '/model.23/cv2/act/Mul', '/model.4/m/m.1/cv2/act/Mul', '/model.17/cv2/act/Mul', '/model.6/m/m.2/cv2/act/Mul', '/model.6/cv3/act/Sigmoid', '/model.20/cv3/act/Mul', '/model.20/m/m.0/cv2/act/Mul', '/model.17/cv1/act/Mul', '/model.6/m/m.0/cv2/act/Sigmoid', '/model.2/m/m.0/cv1/act/Sigmoid', '/model.2/m/m.0/cv2/act/Mul', '/model.4/cv1/act/Sigmoid', '/model.8/cv3/act/Mul', '/model.20/cv2/act/Mul', '/model.13/cv2/act/Mul', '/model.6/m/m.1/cv2/act/Mul', '/model.18/act/Sigmoid', '/model.4/cv2/act/Sigmoid', '/model.23/m/m.0/cv2/act/Mul', '/model.23/cv1/act/Sigmoid', '/model.8/m/m.0/cv2/act/Mul', '/model.23/m/m.0/cv1/act/Mul', '/model.6/m/m.1/cv2/act/Sigmoid', '/model.20/m/m.0/cv1/act/Sigmoid', '/model.23/cv2/act/Sigmoid', '/model.2/cv2/act/Mul', '/model.6/m/m.0/cv1/act/Sigmoid', '/model.7/act/Sigmoid', '/model.4/cv1/act/Mul', '/model.13/cv1/act/Mul', '/model.14/act/Sigmoid', '/model.10/act/Sigmoid', '/model.6/m/m.0/cv2/act/Mul', '/model.23/m/m.0/cv2/act/Sigmoid', '/model.21/act/Mul', '/model.2/m/m.0/cv1/act/Mul', '/model.20/cv2/act/Sigmoid', '/model.6/cv2/act/Sigmoid', '/model.4/cv3/act/Mul', '/model.17/cv2/act/Sigmoid', '/model.20/cv1/act/Sigmoid', '/model.24/proto/cv1/act/Sigmoid', '/model.9/cv1/act/Mul', '/model.13/m/m.0/cv1/act/Mul', '/model.13/cv3/act/Mul', '/model.4/cv3/act/Sigmoid', '/model.8/cv2/act/Sigmoid', '/model.13/m/m.0/cv2/act/Mul', '/model.23/m/m.0/cv1/act/Sigmoid', '/model.24/proto/cv1/act/Mul', '/model.6/m/m.2/cv1/act/Sigmoid', '/model.6/cv1/act/Mul', '/model.20/cv1/act/Mul', '/model.17/m/m.0/cv2/act/Mul', '/model.10/act/Mul', '/model.13/cv2/act/Sigmoid', '/model.4/cv2/act/Mul', '/model.23/cv3/act/Sigmoid', '/model.18/act/Mul', '/model.8/m/m.0/cv1/act/Mul', '/model.6/cv1/act/Sigmoid', '/model.6/m/m.1/cv1/act/Mul', '/model.23/cv3/act/Mul', '/model.8/cv1/act/Sigmoid', '/model.17/m/m.0/cv1/act/Mul', '/model.9/cv1/act/Sigmoid', '/model.4/m/m.1/cv1/act/Mul', '/model.21/act/Sigmoid', '/model.5/act/Mul', '/model.20/m/m.0/cv2/act/Sigmoid', '/model.4/m/m.0/cv2/act/Sigmoid', '/model.6/m/m.0/cv1/act/Mul', '/model.6/m/m.2/cv1/act/Mul', '/model.7/act/Mul', '/model.24/proto/cv2/act/Sigmoid', '/model.8/cv1/act/Mul', '/model.5/act/Sigmoid', '/model.2/cv1/act/Mul', '/model.2/cv3/act/Mul', '/model.3/act/Mul', '/model.3/act/Sigmoid', '/model.6/cv3/act/Mul', '/model.13/cv3/act/Sigmoid', '/model.8/m/m.0/cv2/act/Sigmoid', '/model.6/m/m.1/cv1/act/Sigmoid', '/model.2/cv3/act/Sigmoid', '/model.13/cv1/act/Sigmoid', '/model.17/m/m.0/cv1/act/Sigmoid', '/model.9/cv2/act/Sigmoid']
2026-09-22 02:41 - INFO - sdk - quantize - Starting quantization...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-09-22 02:41 - INFO - sdk - quantize - Quantization completed! Quantized model saved to /quadric/sdk-cli/examples/models/yolo/yolov5_seg/yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q.onnx
2026-09-22 02:41 - INFO - sdk - quantize - ONNX full precision model size: 7.63MB
2026-09-22 02:41 - INFO - sdk - quantize - ONNX quantized model size: 2.03MB
2026-09-22 02:41 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-09-22 02:41 - INFO - sdk - quantize - ONNX Model with well-defined shapes has been saved at `/quadric/sdk-cli/examples/models/yolo/yolov5_seg/yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx`.
2026-09-22 02:41 - DEBUG - sdk - quantize - Checking for FLOAT/FLOAT16 types...
2026-09-22 02:41 - INFO - sdk - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-09-22 02:41 - INFO - sdk - quantize - Model still has FLOAT/FLOAT16 types after quantization. Creating ranges for floating point tensors using calibration data...
2026-09-22 02:41 - INFO - sdk - quantize - Saved computed tensor ranges to /quadric/sdk-cli/examples/models/yolo/yolov5_seg/yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges.
2026-09-22 02:41 - INFO - sdk - quantize -
╒═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╤════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Quantized ONNX Model │ Tensor Ranges File │
╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╪════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ /quadric/sdk-cli/examples/models/yolo/yolov5_seg/yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx │ /quadric/sdk-cli/examples/models/yolo/yolov5_seg/yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges │
╘═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╧════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╛
Quantized ONNX: yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx
Tensor ranges: yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges
1563
4. Compilation
ChimeraJob wraps the full compile pipeline: graph analyze, kernel selection, memory planning, and code generation. Both the detection backbone and the mask-prototype trunk compile into the same CGC job.
cgc_job = ChimeraJob(model_p=str(quantized_onnx_model.model_path))
cgc_job.compile(quiet=True)
5. Inference
batch_inference runs the compiled backbone against multiple images in parallel through two engines: CHIMERA_ORT_INT8 (ONNX Runtime reference) and CHIMERA_ISS_INT8 (the Chimera GPNPU cycle-accurate simulator). Both drive the same head ONNX on the host for the classical tail.
all_image_paths, all_images = load_images(INPUT_SIZE, DEFAULT_IMAGE_PATHS)
num_images = len(all_images)
engines = {
InferenceEngine.CHIMERA_ORT_INT8: cgc_job,
InferenceEngine.CHIMERA_ISS_INT8: cgc_job,
}
head_session = InferenceSession(onnx.load(str(head_onnx_file)).SerializeToString())
outputs_per_engine = {}
threads = min(num_images, 6)
for engine, job in engines.items():
outputs_per_engine[engine] = batch_inference(
engine, job, all_images, head_session=head_session, threads=threads
)
2026-09-22 02:49 - WARNING - epu - chimera_job - ORT is not threadsafe -- forcing single threaded batch execution
100%|████████████████████████████████████████████| 3/3 [00:01<00:00, 2.75it/s]
Processing: 100%|████████████████████████████████| 3/3 [01:21<00:00, 27.13s/it]
6. Run Statistics
plot_run_statistics() renders the per-kernel cycle breakdown for the compiled backbone — useful for spotting hot kernels when tuning the target configuration.
print(cgc_job)
cgc_job.plot_run_statistics()
╒═════════════════════╤═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ yolov5n_seg_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/models/yolo/yolov5_seg/yolov5n-seg-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 16MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 10.624MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 1.684kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 3.535 │
╘═════════════════════╧═════════════════════════════════════════════════════════════════════════════════════════════════════════════════╛
NOTE: CGC has used general convolution for some nodes, which may result in suboptimal performance. For performance-critical applications, please contact Quadric support to explore optimization strategies.
General convolution nodes: /model.0/conv/Conv_quant
For more details, see: https://app.quadric.ai/docs/latest/chimera-software-user-guide/chimera-graph-compiler-cgc/graph-optimizations-performed-by-cgc#general-convolution
╒════╤════════╤════════════════════════════════════════╤═══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪════════════════════════════════════════╪═══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ inputs0 │ [1, 3, 640, 640] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼────────────────────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ /model.24/m.0/Conv_output_0 │ [1, 351, 80, 80] │ tensor[FixedPoint32<26>] │ 0.008 │
├────┼────────┼────────────────────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ /model.24/m.1/Conv_output_0 │ [1, 351, 40, 40] │ tensor[FixedPoint32<26>] │ 0.006 │
├────┼────────┼────────────────────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 3 │ Output │ /model.24/m.2/Conv_output_0 │ [1, 351, 20, 20] │ tensor[FixedPoint32<27>] │ 0.003 │
├────┼────────┼────────────────────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 4 │ Output │ /model.24/proto/cv3/conv/Conv_output_0 │ [1, 32, 160, 160] │ tensor[FixedPoint32<28>] │ 0.004 │
╘════╧════════╧════════════════════════════════════════╧═══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 2.48 │
├──────────────────────────────────┼─────────┤
│ FPS │ 403.22 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 2042.93 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 197.37 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 6.67 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 14.37 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 2.63 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 5.66 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 5.12% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 4,216,092
[SDK-CLI] : Executions/second: 403.22
compute : ▇▇▇▇▇▇▇▇▇▇▇▇ 665.351K
data_array : ▇▇▇▇ 215.001K
mac : ▇▇▇▇▇▇▇▇ 465.783K
data_external: ▇▇▇▇ 241.48K
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 2.627M
for more information check run directory: /quadric/sdk-cli/examples/models/yolo/yolov5_seg/ccl_build/yolov5n_seg_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_024940_7f6c82
2026-09-22 02:51 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/models/yolo/yolov5_seg/ccl_build/yolov5n_seg_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_024940_7f6c82/data/yolov5n_seg_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/models/yolo/yolov5_seg/ccl_build/yolov5n_seg_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_024940_7f6c82/data'
Decode the network outputs and overlay the detections + masks on the input image.
detections_per_engine, postprocess_handle = run_postprocess(
outputs_per_engine, all_image_paths, INPUT_SIZE
)
display_detections(all_image_paths, detections_per_engine, postprocess_handle)
Summary
| Model | YOLOv5n-seg (COCO, 640×640, 2.0M params, 7.1 GFLOPs) |
| Pipeline | ONNX download (pre-split, two outputs) → quantize (INT8 asymmetric) → CGC compile → ISS + ORT inference → YOLOv5-seg NMS + mask decode → bbox + mask overlay |
| Compiled on GPNPU | Convolutional backbone + mask-prototype trunk |
| On host CPU | Detection head ONNX, mask coefficient fusion, NMS, visualization |
| Calibration | QuadricCalibration COCO-like subset |
Key takeaways
- Splitting the ONNX at the three detection-head edges plus the mask prototype edge keeps CGC focused on GPNPU-friendly convolutional ops while the anchor-decode + mask coefficient fusion runs on the host.
- The same
ChimeraJobhandle drives bothCHIMERA_ORT_INT8andCHIMERA_ISS_INT8, so side-by-side validation of detection + mask quality across paths is a one-dict change. - Asymmetric INT8 quantization preserves both detection (mAP@box) and instance segmentation (mAP@mask) quality at the configured score / NMS thresholds.
Citation
@software{yolov5_ultralytics,
title = {YOLOv5 by Ultralytics},
author = {Glenn Jocher},
year = {2020},
version = {7.0},
license = {AGPL-3.0},
url = {https://github.com/ultralytics/yolov5},
doi = {10.5281/zenodo.3908559}
}
