NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/detr/mixed_precision/detr_mixed_precision.ipynb.
DETR-R101 Mixed-Precision Object Detection on Chimera GPNPU
DETR (DEtection TRansformer) frames object detection as direct set prediction: a ResNet-101 backbone feeds a transformer encoder–decoder that emits 100 box/class predictions with no anchors and no non-maximum suppression. This notebook exports the public DETR-R101 checkpoint at 750×800, compiles it with a mixed-precision recipe — INT8 for the backbone and the FFNs, FP16 for the transformer's projections and attention — and runs the full network on the Chimera GPNPU. In ONNX Runtime, with the transformer's projections and attention in FP32, the quantized graph is within 1% of the FP32 model's COCO mAP.
DETR has two notebooks. detr_e2e.ipynb compiles the 384×960 Hugging Face export in INT8 with every operator native to CGC (Chimera Graph Compiler); this one exports the facebookresearch/detr checkpoint at 750×800 and runs the transformer as SDK kernels: the projections and attention in FP16, and each FFN block in INT8 with CGC's own quantization arithmetic.
Why mixed precision?
The two halves of DETR are bound by different resources. The ResNet-101 backbone is ~94 GMAC at 750×800 and compute-bound, so it runs INT8: 16,384 MACs per cycle on QC-U, twice the FP16 rate. The transformer is data-movement-bound — its cycles go to DDR and OCM (On-Chip Memory) traffic, not to MACs — so FP16 projections and attention cost few cycles there while keeping the precision an all-INT8 transformer gives up. The FFN linears quantize to INT8 with no measurable loss. An SDK kernel runs each FFN block with the immediates CGC derives for its ops, so it computes the native graph's bits while keeping the block's intermediates on chip.
| Part | Precision | Runs as |
|---|---|---|
| ResNet-101 backbone | INT8, per-channel weights, asymmetric post-ReLU activations | CGC |
| input projection (2048 → 256) | FP16 weights, token-major, from the INT8 layer4 map | SDK kernel nn::detrFp16LinearDq |
| attention projections (Q|K|V, K|V, Q, out) | FP16 weights | SDK kernel nn::detrFp16Linear |
| 17 attention cores | FP16 flash attention | SDK kernels nn::detrSelfAttention, nn::detrCrossAttention, nn::detrCrossAttentionConstQ |
| 12 FFN blocks: residual Add → LayerNorm → 2 FFN linears → residual Add → LayerNorm | INT8 per-tensor linears, int32 bias (QGemm); FixedPoint32 Adds and LayerNorms | SDK kernel nn::detrFfnBlock |
| decoder self-attention residual Adds and LayerNorms | FixedPoint32 | CGC |
| class / box heads | FP16 | CGC |
| Graph | I/O |
|---|---|
| backbone + input projection + encoder + decoder + heads | input [1,3,750,800] → pred_logits [1,100,92], pred_boxes [1,100,4] |
Model: DETR-R101 from facebookresearch/detr (ResNet-101 backbone, COCO, 750×800 input, ~60 M params, 6 encoder + 6 decoder layers)
1. Setup
Imports, then the example's helper modules: graph_rewrites.py (the FP32 graph rewrites), quantize_recipe.py (the precision recipe), custom_op_match.py (the SDK custom ops, with CGC's fixed-point rules in cgc_fixed_point.py) and detr_mp_helpers.py (files and the hardware target). The IoU validation and the side-by-side plot are the ones detr_e2e.ipynb uses, from ../e2e/. The target is QC-U with one core, 8 MB OCM, 4 kB LRM, 16 MACs per PE, 1.50 GHz and 32 GB/s DDR.
import json
import sys
from collections import Counter
from pathlib import Path
import onnx
import onnxruntime as ort
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
import custom_op_match
import detr_mp_helpers as helpers
import graph_rewrites
import quantize_recipe
sys.path.insert(0, str(Path.cwd().parent / "e2e")) # shared with detr_e2e.ipynb
import detr_viz # noqa: E402
import iou_validation # noqa: E402
%matplotlib inline
OUTPUT_DIR = Path("onnx")
OUTPUT_DIR.mkdir(exist_ok=True)
IMG_W, IMG_H = quantize_recipe.INPUT_W, quantize_recipe.INPUT_H # model input 800 x 750
hw_config = helpers.hw_config()
print(hw_config)
QC-U_1d5_8MB_4kB_32GBps_32GBps_16_OFF_x1_x1
2. Model and images
export_fp32_model downloads the public DETR-R101 checkpoint and the facebookresearch/detr model code at a pinned commit, and exports the network to ONNX (opset 13, input 750×800, simplified with onnxsim). The export runs once; later runs reuse onnx/detr_r101_750x800_fp32.onnx. Every activation scale and tensor range is calibrated on 12 COCO train2017 images; COCO val2017 image 632 is the validation input. quantize_recipe.preprocess is the model's input transform: resize to 800×750, scale to [0, 1], ImageNet mean/std normalization.
FP32_ONNX = helpers.export_fp32_model(OUTPUT_DIR / "detr_r101_750x800_fp32.onnx")
calib_images = helpers.fetch_coco_images(helpers.CALIBRATION_IDS, "train2017", "coco/train2017")
val_img = helpers.fetch_coco_images([helpers.VALIDATION_ID], "val2017", "coco/val2017")[0]
print(f"FP32 graph: {FP32_ONNX.name}")
print(f"{len(calib_images)} calibration images; validation image {val_img.name}")
Downloading detr-r101-2c7b67e5.pth (243 MB) ...
FP32 graph: detr_r101_750x800_fp32.onnx
12 calibration images; validation image 000000000632.jpg
3. Prepare the graph
Six rewrites change how the FP32 graph expresses DETR, not what it computes; prepare_graph checks the result against the export in ONNX Runtime on a calibration image.
| Rewrite | Change | For |
|---|---|---|
| opset 17 | standard LayerNormalization | CGC's native LayerNorm |
unfuse_gemm | 17 out-projection Gemms → MatMul + bias Add | one projection pattern for the custom ops |
batch_first | transformer streams [L, 1, C] → [1, L, C] (same memory) | tokens as rows for LayerNorm |
fuse_attention_projections | position embeddings fold into the Q/K biases; Q|K|V and K|V fuse | one fused input per attention op |
heads_last_layer | heads on the last decoder layer only | a sixth of the head compute |
ffn_gemm | FFN MatMul + bias Add → 2-D Gemm | an int32-bias QGemm once INT8 |
PREPARED = graph_rewrites.prepare_graph(
FP32_ONNX, OUTPUT_DIR / "detr_prepared.onnx", quantize_recipe.preprocess(calib_images[0])
)
2026-10-23 01:05 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-10-23 01:05 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-10-23 01:05 - INFO - sdk - quantize - ONNX Model with well-defined shapes has been saved at `onnx/detr_prepared.onnx`.
17 out-projection Gemms split
5 constants made batch-first
22 attention projection groups fused
24 FFN linears as 2-D Gemm
max |output difference| vs the export: 6.18e-05
4. Quantize
All quantization is static: ONNX Runtime calibrates every scale offline on the 12 images, so nothing is computed at run time.
Backbone. quantize_backbone_and_heads quantizes the ResNet-101 backbone to INT8 with per-channel weights and percentile (99.999) calibration. Every backbone activation that comes out of a ReLU becomes asymmetric — zero point -128, scale s·127/255 — so saturating at -128 is the ReLU and the non-negative range keeps all 256 INT8 codes. The input projection then runs token-major ([600, 2048] × [2048, 256]) from the INT8 layer4 output, and the four head MatMuls run in FP16.
Memory. This calibration is the notebook's peak and needs about 32 GB of RAM: ONNX Runtime's histogram calibrator keeps every intermediate tensor of all 12 images before it builds the histograms (quadric-io/tvm#3945 tracks lower-memory calibration).
INT8_BACKBONE = quantize_recipe.quantize_backbone_and_heads(
PREPARED, OUTPUT_DIR / "detr_int8_backbone.onnx", calib_images
)
Collecting tensor data and making histogram ...
Finding optimal threshold for each tensor using 'percentile' algorithm ...
Number of tensors : 753
Number of histogram bins : 2048
Percentile : (0.0010000000000047748,99.999)
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
/quadric/sdk-cli/examples/models/detr/mixed_precision/quantize_recipe.py:200: RuntimeWarning: invalid value encountered in cast
node.input[8] = graph.set_init(f"{node.input[8]}__asym", np.rint(bias).astype(np.int32))
INT8 backbone: 100 asymmetric post-ReLU activations
104 INT8 convs; input_proj token-major over 600 tokens; 4 head MatMuls FP16
FFN linears. quantize_ffn makes the 24 FFN linears static INT8 QGemms — symmetric per-tensor weights, an int32 bias in the accumulator — calibrated on the graph whose backbone is already INT8, so the scales see the activations the compiled network produces. compute_tranges then records the MinMax range of every tensor; CGC sizes each FixedPoint32 format from it. The result, detr_mixed.onnx, runs in ONNX Runtime and is the reference for the ISS run.
MIXED = quantize_recipe.quantize_ffn(INT8_BACKBONE, OUTPUT_DIR / "detr_mixed.onnx", calib_images)
TRANGES = quantize_recipe.compute_tranges(MIXED, OUTPUT_DIR / "detr_mixed.tranges", calib_images)
ops = Counter(node.op_type for node in onnx.load(str(MIXED)).graph.node)
for op in ("QLinearConv", "QLinearAdd", "QGemm", "MatMul", "Softmax", "LayerNormalization"):
print(f" {op:20s} {ops[op]}")
Collecting tensor data and making histogram ...
Finding optimal threshold for each tensor using 'percentile' algorithm ...
Number of tensors : 231
Number of histogram bins : 2048
Percentile : (0.0010000000000047748,99.999)
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
WARNING:root:Please check if the model is already quantized. Note you don't need to quantize a QAT model. OnnxRuntime support to run QAT model directly.
2026-10-23 01:06 - INFO - sdk - quantize - ONNX model shapes inferred.
FFN: 24 static INT8 QGemm (per-tensor weights, int32 bias)
2026-10-23 01:06 - INFO - sdk - quantize - Saved computed tensor ranges to onnx/detr_mixed.tranges.
QLinearConv 104
QLinearAdd 33
QGemm 24
MatMul 78
Softmax 17
LayerNormalization 35
5. SDK custom ops for the transformer
The transformer runs as kernels from the SDK's DETR workload (src/workloads/detr/detr.hpp). Each entry point takes CGC's custom-op argument order — data inputs, constants, data output, OCM allocator, ext scratch — so detr_ops.hpp only includes the workload header, and custom_op_match.replace_transformer binds every region with CustomOpReplacer:
| SDK kernel | Replaces | Count |
|---|---|---|
nn::detrFp16Linear<outFracBits, relu> | a projection MatMul + bias Add: Q|K|V, K|V, Q, out | 39 |
nn::detrFp16LinearDq<outFracBits, relu, …> | the input projection with its DequantizeLinear → Reshape → Transpose: it reads the INT8 layer4 map | 1 |
nn::detrSelfAttention<256, 8, L, 14, …> | fused Q|K|V → context (encoder L = 600, decoder L = 100) | 11 |
nn::detrCrossAttention<256, 8, 100, 600, 14, …> | Q and the fused encoder K|V → context | 5 |
nn::detrCrossAttentionConstQ<…> | decoder layer 0, whose query folds to a constant | 1 |
nn::detrFfnBlock<…> | residual Add → LayerNormalization → QuantizeLinear → QGemm → DequantizeLinear → Relu → QuantizeLinear → QGemm → DequantizeLinear → residual Add → LayerNormalization | 12 |
A projection's bias is part of its op, and each projection and attention op writes FixedPoint32 at its calibrated range's fit minus 2 bits of headroom (at most 24 fraction bits). A projection hands the attention op it feeds FP16, and so does each decoder attention op to its out projection (a Cast pair joins the two ops): the consumer reads the values it would have narrowed itself, from half the bytes. Each FFN block's op takes the formats and multipliers CGC derives for the native ops from the same graph and ranges (cgc_fixed_point.py), so it computes the same bits. The decoder's self-attention residual Adds and LayerNorms, and the heads, stay with CGC.
print(Path(custom_op_match.DETR_OPS_HEADER).read_text().split("\n\n")[-1])
tranges = json.loads(Path(TRANGES).read_text())
model, tranges = custom_op_match.replace_transformer(onnx.load(str(MIXED)), tranges)
CUSTOM_ONNX = OUTPUT_DIR / "detr_mixed_custom_op.onnx"
CUSTOM_TRANGES = OUTPUT_DIR / "detr_mixed_custom_op.tranges"
onnx.save(model, str(CUSTOM_ONNX))
CUSTOM_TRANGES.write_text(json.dumps(tranges))
#include <quadric/qil.h>
#include <workloads/detr/detr.hpp>
1 x nn::detrFp16LinearDq
39 x nn::detrFp16Linear
11 x nn::detrSelfAttention
1 x nn::detrCrossAttentionConstQ
5 x nn::detrCrossAttention
12 x nn::detrFfnBlock
51211
6. Compile with CGC
One ChimeraJob compiles the whole network: CGC compiles the INT8 backbone, the decoder's self-attention residual Adds and LayerNorms and the FP16 heads, and calls the SDK kernels for the 69 custom ops through custom_ops=detr_ops.hpp. write_directives writes one CGC directive, BUFFERED_CONSTANTS, for eight convs at the end of layer3 and the start of layer4: CGC double-buffers their weights in L2, so the weight streams overlap the convs.
DIRECTIVES = helpers.write_directives(CUSTOM_ONNX, OUTPUT_DIR / "detr_directives.json")
job = ChimeraJob(
str(CUSTOM_ONNX),
hw_config=hw_config,
trange_file=str(CUSTOM_TRANGES),
target_lang="ASM",
custom_ops=custom_op_match.DETR_OPS_HEADER,
directives=str(DIRECTIVES),
)
job.compile()
2026-10-23 01:06 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-10-23 01:06 - INFO - epu - chimera_job - Numerical ranges provided
2026-10-23 01:07 - INFO - epu - util - START===============================optimize_relay
2026-10-23 01:07 - INFO - epu - util - START====================quantize_to_cpu_runnable_fx
2026-10-23 01:07 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ----------------------------- ---------------------- --------------------
CustomOp/nn::detrFp16LinearDq<22, false, 27, 4, 108486512, -128>0 contrib.epu.quadric_custom_op [-41.7654f, 67.4873f] 22
CustomOp/nn::detrSelfAttention<256, 8, 600, 14>40 contrib.epu.quadric_custom_op [-3.54624f, 3.58214f] 24
CustomOp/nn::detrFp16Linear<24, false>2 contrib.epu.quadric_custom_op [-3.66227f, 3.29602f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<2, 0>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1282365123>, nn::DetrQGemmScale<31, 9182304>, nn::DetrDequant<27, 4, 190630656, 0>, nn::DetrQuant<31, 22, 1468521838>, nn::DetrQGemmScale<31, 1185681>, nn::DetrDequant<25, 6, 950293312, 0>, nn::DetrAddShifts<3, 1>, nn::DetrLayerNorm<30, 27>>57 contrib.epu.quadric_custom_op [-7.05256f, 10.6376f] 27
CustomOp/nn::detrSelfAttention<256, 8, 600, 14>41 contrib.epu.quadric_custom_op [-2.82922f, 2.25089f] 24
CustomOp/nn::detrFp16Linear<24, false>4 contrib.epu.quadric_custom_op [-2.2709f, 2.34404f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1220637555>, nn::DetrQGemmScale<31, 11051052>, nn::DetrDequant<27, 4, 160385632, 0>, nn::DetrQuant<31, 22, 1339756036>, nn::DetrQGemmScale<31, 3246576>, nn::DetrDequant<26, 5, 439758976, 0>, nn::DetrAddShifts<2, 1>, nn::DetrLayerNorm<30, 27>>58 contrib.epu.quadric_custom_op [-5.87514f, 9.8673f] 27
CustomOp/nn::detrSelfAttention<256, 8, 600, 14>42 contrib.epu.quadric_custom_op [-2.42419f, 1.65777f] 24
CustomOp/nn::detrFp16Linear<24, false>6 contrib.epu.quadric_custom_op [-1.97739f, 1.85522f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 26>, nn::DetrQuant<31, 23, 1773573290>, nn::DetrQGemmScale<31, 16643754>, nn::DetrDequant<27, 4, 169193456, 0>, nn::DetrQuant<31, 22, 1891187278>, nn::DetrQGemmScale<31, 1813401>, nn::DetrDequant<26, 5, 466645376, 0>, nn::DetrAddShifts<1, 1>, nn::DetrLayerNorm<30, 27>>59 contrib.epu.quadric_custom_op [-6.381f, 12.468f] 27
CustomOp/nn::detrSelfAttention<256, 8, 600, 14>43 contrib.epu.quadric_custom_op [-3.10795f, 2.57492f] 24
CustomOp/nn::detrFp16Linear<24, false>8 contrib.epu.quadric_custom_op [-2.93452f, 3.20187f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 26>, nn::DetrQuant<31, 23, 1466558899>, nn::DetrQGemmScale<31, 17270776>, nn::DetrDequant<27, 4, 170211072, 0>, nn::DetrQuant<31, 22, 1589646902>, nn::DetrQGemmScale<31, 900199>, nn::DetrDequant<24, 7, 1529457536, 0>, nn::DetrAddShifts<2, 0>, nn::DetrLayerNorm<30, 27>>60 contrib.epu.quadric_custom_op [-4.52354f, 8.97418f] 27
CustomOp/nn::detrSelfAttention<256, 8, 600, 14>44 contrib.epu.quadric_custom_op [-1.65261f, 1.74644f] 24
CustomOp/nn::detrFp16Linear<24, false>10 contrib.epu.quadric_custom_op [-2.05472f, 2.45916f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 26>, nn::DetrQuant<31, 23, 1623144563>, nn::DetrQGemmScale<31, 16750892>, nn::DetrDequant<27, 4, 189608240, 0>, nn::DetrQuant<31, 21, 1073920926>, nn::DetrQGemmScale<31, 3515005>, nn::DetrDequant<27, 4, 223992432, 0>, nn::DetrAddShifts<1, 2>, nn::DetrLayerNorm<30, 27>>61 contrib.epu.quadric_custom_op [-5.70239f, 7.81518f] 27
CustomOp/nn::detrSelfAttention<256, 8, 600, 14>45 contrib.epu.quadric_custom_op [-1.89166f, 2.58921f] 24
CustomOp/nn::detrFp16Linear<24, false>12 contrib.epu.quadric_custom_op [-2.15759f, 2.34716f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1211972122>, nn::DetrQGemmScale<31, 12398790>, nn::DetrDequant<27, 4, 159455536, 0>, nn::DetrQuant<31, 21, 1351989980>, nn::DetrQGemmScale<31, 5043087>, nn::DetrDequant<28, 3, 97893848, 0>, nn::DetrAddShifts<1, 2>, nn::DetrLayerNorm<30, 27>>62 contrib.epu.quadric_custom_op [-5.83407f, 7.25772f] 27
CustomOp/nn::detrFp16Linear<24, false>19 contrib.epu.quadric_custom_op [-16.9832f, 21.2901f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 0>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1693247436>, nn::DetrQGemmScale<31, 18815642>, nn::DetrDequant<27, 4, 163986112, 0>, nn::DetrQuant<31, 22, 1845108967>, nn::DetrQGemmScale<31, 4641763>, nn::DetrDequant<26, 5, 302625824, 0>, nn::DetrAddShifts<1, 0>, nn::DetrLayerNorm<30, 27>>63 contrib.epu.quadric_custom_op [-7.16079f, 7.72301f] 27
/transformer/decoder/norm/Mul/LayerNormFusion/ nn.layer_norm [-8.41658f, 9.3433f] 27
/transformer/decoder/Unsqueeze expand_dims [-8.41658f, 9.3433f] 27
CustomOp/nn::detrFp16Linear<24, false>21 contrib.epu.quadric_custom_op [-1.64221f, 1.63054f] 24
/transformer/decoder/layers.1/Add_1 add [-7.21587f, 7.78399f] 24
/transformer/decoder/layers.1/norm1/Mul/LayerNormFusion/ nn.layer_norm [-15.2501f, 15.4379f] 26
CustomOp/nn::detrFp16Linear<24, false>23 contrib.epu.quadric_custom_op [-4.99229f, 5.11015f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 2>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1780659378>, nn::DetrQGemmScale<31, 8560102>, nn::DetrDequant<27, 4, 170173504, 0>, nn::DetrQuant<31, 22, 2050462819>, nn::DetrQGemmScale<31, 4840581>, nn::DetrDequant<27, 4, 135883424, 0>, nn::DetrAddShifts<1, 1>, nn::DetrLayerNorm<30, 27>>64 contrib.epu.quadric_custom_op [-8.91403f, 8.6944f] 27
/transformer/decoder/norm_1/Mul/LayerNormFusion/ nn.layer_norm [-9.31676f, 8.90368f] 27
/transformer/decoder/Unsqueeze_1 expand_dims [-9.31676f, 8.90368f] 27
CustomOp/nn::detrFp16Linear<24, false>25 contrib.epu.quadric_custom_op [-1.25123f, 1.58678f] 24
/transformer/decoder/layers.2/Add_1 add [-8.93146f, 9.95304f] 24
/transformer/decoder/layers.2/norm1/Mul/LayerNormFusion/ nn.layer_norm [-16.3636f, 15.4004f] 26
CustomOp/nn::detrFp16Linear<24, false>27 contrib.epu.quadric_custom_op [-3.47813f, 3.14704f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 2>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1869955973>, nn::DetrQGemmScale<31, 7504808>, nn::DetrDequant<27, 4, 179692496, 0>, nn::DetrQuant<31, 21, 1446057718>, nn::DetrQGemmScale<31, 5070777>, nn::DetrDequant<28, 3, 83060584, 0>, nn::DetrAddShifts<0, 1>, nn::DetrLayerNorm<30, 27>>65 contrib.epu.quadric_custom_op [-9.5007f, 8.6273f] 27
/transformer/decoder/norm_2/Mul/LayerNormFusion/ nn.layer_norm [-9.32758f, 8.62317f] 27
/transformer/decoder/Unsqueeze_2 expand_dims [-9.32758f, 8.62317f] 27
CustomOp/nn::detrFp16Linear<24, false>29 contrib.epu.quadric_custom_op [-1.24536f, 1.18176f] 24
/transformer/decoder/layers.3/Add_1 add [-9.45327f, 8.64872f] 24
/transformer/decoder/layers.3/norm1/Mul/LayerNormFusion/ nn.layer_norm [-14.8137f, 13.4298f] 27
CustomOp/nn::detrFp16Linear<24, false>31 contrib.epu.quadric_custom_op [-2.18947f, 2.48249f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1851064293>, nn::DetrQGemmScale<31, 7934342>, nn::DetrDequant<27, 4, 168926432, 0>, nn::DetrQuant<31, 21, 1809985153>, nn::DetrQGemmScale<31, 6463304>, nn::DetrDequant<29, 2, 52053072, 0>, nn::DetrAddShifts<0, 2>, nn::DetrLayerNorm<30, 27>>66 contrib.epu.quadric_custom_op [-8.93156f, 8.00151f] 27
/transformer/decoder/norm_3/Mul/LayerNormFusion/ nn.layer_norm [-9.12056f, 8.31811f] 27
/transformer/decoder/Unsqueeze_3 expand_dims [-9.12056f, 8.31811f] 27
CustomOp/nn::detrFp16Linear<24, false>33 contrib.epu.quadric_custom_op [-1.00865f, 0.69051f] 24
/transformer/decoder/layers.4/Add_1 add [-8.91111f, 8.02116f] 24
/transformer/decoder/layers.4/norm1/Mul/LayerNormFusion/ nn.layer_norm [-12.8401f, 11.4509f] 27
CustomOp/nn::detrFp16Linear<24, false>35 contrib.epu.quadric_custom_op [-1.60102f, 1.56437f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 1827922425>, nn::DetrQGemmScale<31, 7749272>, nn::DetrDequant<27, 4, 164779584, 0>, nn::DetrQuant<31, 20, 1082039243>, nn::DetrQGemmScale<31, 9446074>, nn::DetrDequant<30, 1, 23980838, 0>, nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 27>>67 contrib.epu.quadric_custom_op [-8.06311f, 7.19139f] 27
/transformer/decoder/norm_4/Mul/LayerNormFusion/ nn.layer_norm [-8.99097f, 8.11467f] 27
/transformer/decoder/Unsqueeze_4 expand_dims [-8.99097f, 8.11467f] 27
CustomOp/nn::detrFp16Linear<24, false>37 contrib.epu.quadric_custom_op [-0.308205f, 0.26387f] 24
/transformer/decoder/layers.5/Add_1 add [-8.06536f, 7.14836f] 24
/transformer/decoder/layers.5/norm1/Mul/LayerNormFusion/ nn.layer_norm [-10.9401f, 9.96122f] 27
CustomOp/nn::detrFp16Linear<24, false>39 contrib.epu.quadric_custom_op [-0.699257f, 0.79747f] 24
CustomOp/nn::detrFfnBlock<nn::DetrAddShifts<0, 3>, nn::DetrLayerNorm<30, 27>, nn::DetrQuant<31, 23, 2074427211>, nn::DetrQGemmScale<31, 6489467>, nn::DetrDequant<27, 4, 156155664, 0>, nn::DetrQuant<31, 20, 1459243937>, nn::DetrQGemmScale<31, 13065720>, nn::DetrDequant<31, 0, 12949098, 0>, nn::DetrAddShifts<0, 4>, nn::DetrLayerNorm<30, 27>>68 contrib.epu.quadric_custom_op [-8.07257f, 7.49196f] 27
/transformer/decoder/norm_5/Mul/LayerNormFusion/ nn.layer_norm [-8.7712f, 7.9072f] 27
/transformer/decoder/Unsqueeze_5 expand_dims [-8.7712f, 7.9072f] 27
/transformer/decoder/Concat concatenate [-9.32758f, 9.3433f] 27
/heads/Gather_last_decoder_layer take [-8.7712f, 7.9072f] 27
/class_embed/Add add [-20.8878f, 14.2947f] 15
/bbox_embed/layers.0/Add add [-9.71581f, 7.49946f] 15
/bbox_embed/Relu nn.relu [0f, 7.49946f] 15
/bbox_embed/layers.1/Add add [-27.5865f, 10.9518f] 15
/bbox_embed/Relu_1 nn.relu [0f, 10.9518f] 15
/bbox_embed/layers.2/Add add [-5.87925f, 29.5201f] 15
/Sigmoid sigmoid [0.00278911f, 1f] -
2026-10-23 01:07 - INFO - epu - util - START====================build_cpu_runnable_fx_relay
2026-10-23 01:07 - INFO - epu - util - START=======================quantize_to_chimera_fx
2026-10-23 01:07 - INFO - epu - util - START=================================relay_to_tir
2026-10-23 01:07 - INFO - epu - util - START===========================relay_to_epu_relay
2026-10-23 01:07 - INFO - epu - util - START==============================adapt_and_order
2026-10-23 01:07 - INFO - epu - mac_counter -
2026-10-23 01:07 - INFO - epu - mac_counter - ============================================================
2026-10-23 01:07 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-10-23 01:07 - INFO - epu - mac_counter - ============================================================
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,822,400,000 ops (1,411,200,000 MACs) - /backbone/backbone.0/body/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 308,019,200 ops (154,009,600 MACs) - /backbone/backbone.0/body/layer1/layer1.0/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer1/layer1.0/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer1/layer1.0/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer1/layer1.0/downsample/downsample.0/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer1/layer1.1/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer1/layer1.1/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer1/layer1.1/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer1/layer1.2/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer1/layer1.2/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer1/layer1.2/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,464,153,600 ops (1,232,076,800 MACs) - /backbone/backbone.0/body/layer2/layer2.0/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer2/layer2.0/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer2/layer2.0/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,464,153,600 ops (1,232,076,800 MACs) - /backbone/backbone.0/body/layer2/layer2.0/downsample/downsample.0/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer2/layer2.1/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer2/layer2.1/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer2/layer2.1/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer2/layer2.2/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer2/layer2.2/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer2/layer2.2/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer2/layer2.3/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer2/layer2.3/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer2/layer2.3/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,464,153,600 ops (1,232,076,800 MACs) - /backbone/backbone.0/body/layer3/layer3.0/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.0/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.0/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,464,153,600 ops (1,232,076,800 MACs) - /backbone/backbone.0/body/layer3/layer3.0/downsample/downsample.0/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.1/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.1/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.1/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.2/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.2/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.2/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.3/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.3/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.3/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.4/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.4/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.4/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.5/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.5/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.5/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.6/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.6/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.6/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.7/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.7/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.7/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.8/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.8/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.8/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.9/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.9/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.9/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.10/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.10/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.10/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.11/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.11/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.11/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.12/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.12/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.12/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.13/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.13/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.13/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.14/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.14/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.14/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.15/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.15/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.15/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.16/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.16/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.16/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.17/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.17/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.17/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.18/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.18/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.18/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.19/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.19/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.19/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.20/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.20/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.20/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.21/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.21/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.21/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.22/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,772,172,800 ops (1,386,086,400 MACs) - /backbone/backbone.0/body/layer3/layer3.22/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,232,076,800 ops (616,038,400 MACs) - /backbone/backbone.0/body/layer3/layer3.22/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,464,153,600 ops (1,232,076,800 MACs) - /backbone/backbone.0/body/layer4/layer4.0/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,831,155,200 ops (1,415,577,600 MACs) - /backbone/backbone.0/body/layer4/layer4.0/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,258,291,200 ops (629,145,600 MACs) - /backbone/backbone.0/body/layer4/layer4.0/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,516,582,400 ops (1,258,291,200 MACs) - /backbone/backbone.0/body/layer4/layer4.0/downsample/downsample.0/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,258,291,200 ops (629,145,600 MACs) - /backbone/backbone.0/body/layer4/layer4.1/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,831,155,200 ops (1,415,577,600 MACs) - /backbone/backbone.0/body/layer4/layer4.1/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,258,291,200 ops (629,145,600 MACs) - /backbone/backbone.0/body/layer4/layer4.1/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,258,291,200 ops (629,145,600 MACs) - /backbone/backbone.0/body/layer4/layer4.2/conv1/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 2,831,155,200 ops (1,415,577,600 MACs) - /backbone/backbone.0/body/layer4/layer4.2/conv2/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - conv2d: 1,258,291,200 ops (629,145,600 MACs) - /backbone/backbone.0/body/layer4/layer4.2/conv3/Conv_quant
2026-10-23 01:07 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-10-23 01:07 - INFO - epu - mac_counter - Total: 187,378,329,600 ops (93,689,164,800 MACs)
2026-10-23 01:07 - INFO - epu - mac_counter - ============================================================
2026-10-23 01:07 - INFO - epu - mac_counter -
2026-10-23 01:07 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 01:07 - INFO - epu - util - START=============================plan_lrm_virtual
2026-10-23 01:07 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 01:07 - INFO - epu - util - START===============================lrm_alloc_loop
2026-10-23 01:07 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 01:07 - INFO - epu - util - START================================lrm_splitting
2026-10-23 01:07 - INFO - epu - util - START=====================LrmUsageChecker.__call__
2026-10-23 01:07 - INFO - epu - util - START==============================ext_split_relay
2026-10-23 01:07 - WARNING - epu - external_split - for iterator_region 20: all tensors fit in OCM but bypassing early return because external_split_type directive forces ExternalSplitType.BUFFERED_CONSTANTS
2026-10-23 01:07 - INFO - epu - util - START=========================OcmAllocator.analyze
2026-10-23 01:07 - INFO - epu - util - START===============================Compiler.build
2026-10-23 01:07 - INFO - epu - util - START========================RelayToPrimFunc.build
2026-10-23 01:07 - INFO - epu - util - START=========================OcmAllocator.analyze
2026-10-23 01:07 - INFO - epu - util - START====================================build_tir
2026-10-23 01:08 - INFO - epu - chimera_job - Compilation of detr_mixed_custom_op_QC_U_1d5_8MB_4kB_32GBps_32GBps_16_OFF_x1_x1 successful
7. Run on the ISS
Run the compiled network on the ISS (Instruction Set Simulator) with the target's DDR model (4 KB bursts, 200-cycle average latency), and the same image through ONNX Runtime on detr_mixed.onnx: the graph before custom-op replacement, where the transformer's projections and attention run in FP32. The job summary below includes the ISS run's profile.
img = quantize_recipe.preprocess(val_img)
iss_out = job.run_inference_harness(
inputs={"input": img},
compare_ort=False,
additional_archsim_options=helpers.ARCHSIM_OPTIONS,
)
sess = ort.InferenceSession(str(MIXED), providers=["CPUExecutionProvider"])
ort_out = {o.name: v for o, v in zip(sess.get_outputs(), sess.run(None, {"input": img}))}
print(job)
2026-10-23 01:08 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x7968805f89d0>
FILM 115/115: 100%|███████████████████████████████████████████████| 115/115 [07:13<00:00, 3.77s/it]
╒═════════════════════╤════════════════════════════════════════════════════════════════════╕
│ Module Name │ detr_mixed_custom_op_QC_U_1d5_8MB_4kB_32GBps_32GBps_16_OFF_x1_x1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ ONNX File │ onnx/detr_mixed_custom_op.onnx │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Custom Ops │ /quadric/sdk-cli/examples/models/detr/mixed_precision/detr_ops.hpp │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.500 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 32GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 32GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 7.992MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 3.000kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 26.936MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 96.954 │
╘═════════════════════╧════════════════════════════════════════════════════════════════════╛
╒════╤════════╤═════════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪═════════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input │ [1, 3, 750, 800] │ tensor[FixedPoint32<29>] │ n/a │
├────┼────────┼─────────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ pred_logits │ [1, 100, 92] │ tensor[FixedPoint32<15>] │ n/a │
├────┼────────┼─────────────┼──────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ pred_boxes │ [1, 100, 4] │ tensor[FixedPoint32<31>] │ n/a │
╘════╧════════╧═════════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.5 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 24.07 │
├──────────────────────────────────┼─────────┤
│ FPS │ 41.54 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 2775.07 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 14.97 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 225.08 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 93.24 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 9.13 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 3.78 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 16.39% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 36,111,116
[SDK-CLI] : Executions/second: 41.54
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 7.723M
data_array : ▇▇▇▇▇▇ 1.671M
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 12.448M
data_external: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 9.105M
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 4.462M
for more information check run directory: /quadric/sdk-cli/examples/models/detr/mixed_precision/ccl_build/detr_mixed_custom_op_QC_U_1d5_8MB_4kB_32GBps_32GBps_16_OFF_x1_x1/run/20261023_010820_c83488
8. Compare against ONNX Runtime
Decode both outputs (best class per query, no-object dropped), keep the top 10 queries by score, and match each ONNX Runtime detection to an ISS detection of the same class with IoU > 0.5. The gate: at least 80% of the ONNX Runtime detections are matched.
iou_metrics = iou_validation.run_iou_validation(
ort_out=ort_out,
iss_out=iss_out,
img_w=IMG_W,
img_h=IMG_H,
pass_threshold=80.0,
)
assert iou_metrics[
"passed"
], f"ISS-vs-ORT agreement {iou_metrics['match_pct']:.1f}% < 80% threshold"
DETR IoU validation (top-10 by score, IoU>0.5)
ORT detections : 10
ISS detections : 10
Matched : 9/10
Mean IoU : 0.9714
[PASS] ISS-vs-ORT agreement = 90.00% (threshold 80%)
Draw the top-10 detections above a 0.3 score on the validation image — ONNX Runtime (left), Chimera GPNPU on the ISS (right).
fig = detr_viz.compare_detections(
iss_out,
ort_out,
str(val_img),
img_w=IMG_W,
img_h=IMG_H,
top_k=10,
score_thr=0.3,
out_png="detr_mixed_precision_detections.png",
title="DETR-R101 Mixed Precision: ORT (CPU) vs Chimera GPNPU",
)
Saved detr_mixed_precision_detections.png
Summary
| Model | DETR-R101, facebookresearch/detr checkpoint (ResNet-101 backbone, COCO, 750×800, ~60 M params) |
| Pipeline | exact FP32 rewrites → static mixed-precision quantization → SDK custom ops for the transformer → one CGC compile |
| Target | QC-U, 1 core, 8 MB OCM, 4 kB LRM, 16 MACs/PE, 1.50 GHz, 32 GB/s DDR |
| Quantization | INT8 backbone (per-channel weights, asymmetric post-ReLU activations) and FFNs; FP16 projections, attention and heads; 12 calibration images |
| Custom ops | 39 × nn::detrFp16Linear, 1 × nn::detrFp16LinearDq, 11 × nn::detrSelfAttention, 5 × nn::detrCrossAttention, 1 × nn::detrCrossAttentionConstQ, 12 × nn::detrFfnBlock (SDK src/workloads/detr/) |
| Accuracy | COCO val2017 bbox mAP (5000 images, ONNX Runtime on detr_mixed.onnx, transformer projections and attention in FP32): 0.401 mixed precision vs 0.404 FP32 |
| Validation | ISS vs ONNX Runtime on detr_mixed.onnx (transformer FP32 in ORT): top-10 box agreement |
Key takeaways
- Precision follows the roofline: INT8 where DETR is compute-bound (the backbone), FP16 where it is bound by data movement (the transformer), so the FP16 transformer adds few cycles.
- The recipe stays within 1% of the FP32 model's COCO mAP (ONNX Runtime, transformer projections and attention in FP32) with static quantization only — every scale is calibrated offline.
- The SDK's DETR kernels take CGC's custom-op argument order, so one
CustomOpReplacerpass and a two-line header bring them into the same compile as the natively compiled backbone. - A custom op can take over native INT8 ops without changing their bits: each FFN block's op takes the immediates CGC derives for them (
cgc_fixed_point.py) and keeps the block's intermediates on chip.
Citation
@inproceedings{carion2020detr,
title = {End-to-End Object Detection with Transformers},
author = {Carion, Nicolas and Massa, Francisco and Synnaeve, Gabriel and Usunier, Nicolas and Kirillov, Alexander and Zagoruyko, Sergey},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2020}
}
