NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/yolo/yolov7/yolov7.ipynb.
YOLOv7 Object Detection on Chimera GPNPU
YOLOv7 (Wang, Bochkovskiy & Liao, 2022) is an anchor-based single-stage detector that introduced reparameterized E-ELAN blocks and extended training-time augmentation to outperform prior YOLO variants at matched inference cost. This notebook compiles the yolov7-tiny variant end-to-end: pull the v0.1 source, export to ONNX via torch.onnx, split at the head boundary, quantize the backbone to INT8, compile the backbone with the Chimera Graph Compiler (CGC), and run ISS + ORT INT8 inferences side-by-side.
Why split the graph?
YOLOv7's head mixes anchor decoding, per-scale detection ops, and per-anchor shape gymnastics that are cheaper to keep on the host CPU; the backbone is a dense stack of convolutions that compiles well to the Chimera GPNPU. Splitting the ONNX at the head boundary lets CGC focus on the convolutional backbone and lets ONNX Runtime handle the classical tail — each operator runs where it belongs.
Model: YOLOv7-tiny (COCO, 640×640)
Note — split pipeline.
- On the Chimera GPNPU: the backbone
- On the host CPU: the detection head ONNX, NMS, and bounding-box visualization
The full pipeline can run end-to-end on the GPNPU — see
yolox_e2e_chipy.ipynborretina_net_e2e_chipy.ipynbfor examples that compose CCL custom ops with ChiPy to keep every stage on-chip.
1. Setup
Imports and per-run configuration. The YOLOv7 source-tree download/patch, ONNX export + split, calibration dataset, and bounding-box visualization all live in yolov7_helpers.py.
import gc
import os
## Suppress Ultralytics' auto-pip-install for optional extras. The SDK container
## exposes only ``pip3`` — ultralytics invokes ``pip`` and fails with exit 127,
## polluting the published markdown with warnings.
os.environ.setdefault("YOLO_AUTOINSTALL", "False")
import numpy as np
import onnx
from onnxruntime import InferenceSession
from PIL import Image
from sdk_cli.lib.inference import InferenceEngine, batch_inference
from sdk_cli.lib.quantize import QuantizedONNXModel, quantize_onnx_model
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from yolov7_helpers import (
DEFAULT_IMAGE_PATHS,
INPUT_SIZE,
MODEL_NAME,
build_calibration_dataset,
display_detections,
export_onnx,
load_images,
run_postprocess,
split_backbone_and_head,
)
%matplotlib inline
2. Model Generation and ONNX Export
export_onnx() fetches the WongKinYiu YOLOv7 v0.1 source (downloading and patching it if needed), loads the pretrained yolov7-tiny.pt weights, and writes yolov7-tiny.onnx via torch.onnx.export at the SDK's default opset.
onnx_file = export_onnx(MODEL_NAME)
print(f"Exported: {onnx_file.name}")
Fusing layers...
/usr/local/lib/python3.10/dist-packages/torch/functional.py:539: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /pytorch/aten/src/ATen/native/TensorShape.cpp:3637.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/quadric/sdk-cli/examples/models/yolo/yolov7/models/yolo.py:302: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if augment:
/quadric/sdk-cli/examples/models/yolo/yolov7/models/yolo.py:334: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if profile:
/quadric/sdk-cli/examples/models/yolo/yolov7/models/yolo.py:349: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if profile:
Exported: yolov7-tiny.onnx
3. Split ONNX
YOLOv7 splits into a backbone (convolutional, GPNPU-friendly) and a head (anchor decode + per-scale detection ops, better on the host CPU). We cut the graph at the three head-boundary edges and keep the backbone for CGC compilation; the head piece is run as an ONNX-Runtime reference.
backbone_onnx_file, head_onnx_file = split_backbone_and_head(onnx_file, MODEL_NAME)
print(f"Backbone: {backbone_onnx_file.name}")
print(f"Head: {head_onnx_file.name}")
Backbone: yolov7-tiny-backbone.onnx
Head: yolov7-tiny-head.onnx
4. Quantization
The backbone is quantized to INT8 with asymmetric activations using the COCO-like QuadricCalibration dataset for tensor-range statistics. The QuantizedONNXModel return value carries both the quantized ONNX and its companion .tranges file.
calibration_dataset = build_calibration_dataset(INPUT_SIZE)
quantized_onnx_model: QuantizedONNXModel = quantize_onnx_model(
str(backbone_onnx_file),
calibration_dataset,
asymmetric_activation=True,
)
print(f"Quantized ONNX: {quantized_onnx_model.model_path.name}")
print(f"Tensor ranges: {quantized_onnx_model.tensor_ranges_path.name}")
gc.collect()
2026-09-22 02:58 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-09-22 02:58 - DEBUG - sdk - quantize - Forcing node types: []
2026-09-22 02:58 - DEBUG - sdk - quantize - ONNX Node types excluded from quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-09-22 02:58 - DEBUG - sdk - quantize - ONNX Node names excluded from quantization: []
2026-09-22 02:58 - INFO - sdk - quantize - Starting quantization...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-09-22 02:58 - INFO - sdk - quantize - Quantization completed! Quantized model saved to /quadric/sdk-cli/examples/models/yolo/yolov7/yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q.onnx
2026-09-22 02:58 - INFO - sdk - quantize - ONNX full precision model size: 23.76MB
2026-09-22 02:58 - INFO - sdk - quantize - ONNX quantized model size: 6.04MB
2026-09-22 02:58 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-09-22 02:58 - INFO - sdk - quantize - ONNX Model with well-defined shapes has been saved at `/quadric/sdk-cli/examples/models/yolo/yolov7/yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx`.
2026-09-22 02:58 - DEBUG - sdk - quantize - Checking for FLOAT/FLOAT16 types...
2026-09-22 02:58 - INFO - sdk - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-09-22 02:58 - INFO - sdk - quantize - Model still has FLOAT/FLOAT16 types after quantization. Creating ranges for floating point tensors using calibration data...
2026-09-22 02:58 - INFO - sdk - quantize - Saved computed tensor ranges to /quadric/sdk-cli/examples/models/yolo/yolov7/yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges.
2026-09-22 02:58 - INFO - sdk - quantize -
╒═════════════════════════════════════════════════════════════════════════════════════════════════════════════╤════════════════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Quantized ONNX Model │ Tensor Ranges File │
╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════╪════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ /quadric/sdk-cli/examples/models/yolo/yolov7/yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx │ /quadric/sdk-cli/examples/models/yolo/yolov7/yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges │
╘═════════════════════════════════════════════════════════════════════════════════════════════════════════════╧════════════════════════════════════════════════════════════════════════════════════════════════════════════════╛
Quantized ONNX: yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx
Tensor ranges: yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges
6955
5. Compilation
ChimeraJob wraps the full compile pipeline: graph analyze, kernel selection, memory planning, and code generation. Printing the job after ISS has run renders a summary table that includes the target configuration and cycle counts.
cgc_job = ChimeraJob(model_p=str(quantized_onnx_model.model_path))
cgc_job.compile(quiet=True)
6. Inference
batch_inference runs the compiled backbone against multiple images in parallel through two engines: CHIMERA_ORT_INT8 (ONNX Runtime reference) and CHIMERA_ISS_INT8 (the Chimera GPNPU cycle-accurate simulator). Both drive the same head ONNX on the host for the classical tail.
all_image_paths, all_images = load_images(INPUT_SIZE, DEFAULT_IMAGE_PATHS)
num_images = len(all_images)
engines = {
InferenceEngine.CHIMERA_ORT_INT8: cgc_job,
InferenceEngine.CHIMERA_ISS_INT8: cgc_job,
}
head_session = InferenceSession(onnx.load(str(head_onnx_file)).SerializeToString())
outputs_per_engine = {}
threads = min(num_images, 6)
for engine, job in engines.items():
outputs_per_engine[engine] = batch_inference(
engine, job, all_images, head_session=head_session, threads=threads
)
2026-09-22 03:02 - WARNING - epu - chimera_job - ORT is not threadsafe -- forcing single threaded batch execution
100%|████████████████████████████████████████████| 3/3 [00:00<00:00, 4.42it/s]
Processing: 100%|████████████████████████████████| 3/3 [00:29<00:00, 9.73s/it]
7. Run Statistics
plot_run_statistics() renders the per-kernel cycle breakdown for the compiled backbone — useful for spotting hot kernels when tuning the target configuration.
## Re-enable inline display in case YOLOv7's utils/plots.py switched it to Agg.
%matplotlib inline
print(cgc_job)
cgc_job.plot_run_statistics()
╒═════════════════════╤═════════════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ yolov7_tiny_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/models/yolo/yolov7/yolov7-tiny-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 16MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 8.356MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 2.000kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 6.850 │
╘═════════════════════╧═════════════════════════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤═════════════════════════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪═════════════════════════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ inputs0 │ [1, 3, 640, 640] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼─────────────────────────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ /model.77/m.0/Conv_output_0 │ [1, 255, 80, 80] │ tensor[FixedPoint32<25>] │ 0.028 │
├────┼────────┼─────────────────────────────┼──────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ /model.77/m.1/Conv_output_0 │ [1, 255, 40, 40] │ tensor[FixedPoint32<26>] │ 0.021 │
├────┼────────┼─────────────────────────────┼──────────────────┼──────────────────────────┼───────┤
│ 3 │ Output │ /model.77/m.2/Conv_output_0 │ [1, 255, 20, 20] │ tensor[FixedPoint32<26>] │ 0.022 │
╘════╧════════╧═════════════════════════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 1.14 │
├──────────────────────────────────┼─────────┤
│ FPS │ 877.18 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 3945.18 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 222.34 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 10.74 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 8.17 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 9.20 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 7.00 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 21.57% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 1,938,025
[SDK-CLI] : Executions/second: 877.18
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 501.338K
data_array : ▇▇▇▇▇▇ 124.931K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 952.442K
data_external: ▇▇▇▇▇▇ 118.221K
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇ 238.793K
for more information check run directory: /quadric/sdk-cli/examples/models/yolo/yolov7/ccl_build/yolov7_tiny_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_030233_56553f
2026-09-22 03:03 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/models/yolo/yolov7/ccl_build/yolov7_tiny_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_030233_56553f/data/yolov7_tiny_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/models/yolo/yolov7/ccl_build/yolov7_tiny_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_030233_56553f/data'
Decode the network outputs and overlay the detections on the input image.
detections_per_engine = run_postprocess(outputs_per_engine, all_image_paths, INPUT_SIZE)
display_detections(all_image_paths, detections_per_engine)
Summary
| Model | YOLOv7-tiny (COCO, 640×640) |
| Pipeline | ONNX export (torch.onnx) → split → quantize (INT8 asymmetric) → CGC compile → ISS + ORT inference → YOLOv7 NMS → bbox overlay |
| Compiled on GPNPU | Convolutional backbone |
| On host CPU | YOLOv7 head ONNX, classical NMS, bounding-box visualization |
| Calibration | QuadricCalibration COCO-like dataset |
Key takeaways
- Splitting the ONNX at the head boundary keeps CGC focused on GPNPU-friendly convolutional ops while the anchor-decode + NMS tail runs on the host.
- The SDK exposes the same
ChimeraJobhandle to bothCHIMERA_ORT_INT8(ONNX Runtime reference) andCHIMERA_ISS_INT8(ISS), making side-by-side validation a one-dict change. - Asymmetric INT8 quantization on a COCO-like calibration dataset preserves detection quality at the default YOLOv7 score threshold.
Citation
@article{wang2022yolov7,
title = {YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors},
author = {Wang, Chien-Yao and Bochkovskiy, Alexey and Liao, Hong-Yuan Mark},
journal = {arXiv preprint arXiv:2207.02696},
year = {2022}
}
