NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/detr/e2e/detr_e2e.ipynb.
DETR-R101 End-to-End Object Detection on Chimera GPNPU
DETR (DEtection TRansformer) frames object detection as direct set prediction: a ResNet-101 backbone feeds a transformer encoder–decoder that emits 100 box/class predictions with no anchors and no non-maximum suppression. This notebook compiles the whole DETR graph as one unit and runs it on the Chimera GPNPU, validating the result against ONNX Runtime with box-level IoU.
Why end-to-end?
The entire network — backbone, input projection, transformer encoder/decoder, and prediction heads — is quantized once and compiled as one CGC (Chimera Graph Compiler) unit that runs on the ISS (Instruction Set Simulator). Compiling end-to-end keeps one calibration, one tensor-range set, and one graph — no split and no intermediate handoff.
| Graph | I/O |
|---|---|
| backbone + input projection + encoder + decoder + heads | pixel_values [1,3,384,960] → pred_logits [1,100,92], pred_boxes [1,100,4] |
Model: DETR-R101 (ResNet-101 backbone, COCO, 384×960 input, ~60 M params, 6 encoder + 6 decoder layers)
1. Setup
Imports first; then paths, calibration constants, and the hardware target.
from pathlib import Path
import numpy as np
import onnxruntime as ort
from sdk_cli.utils.model_helpers import ModelHelper
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from tvm.contrib.epu.chimera_job.hw_config import HWConfig
from tvm.contrib.epu.chimera_job.quantize import quadric_quantize
import detr_viz
import iou_validation
from model_download import ensure_calibration_images
from model_download import main as download_model
FULL_FP = "onnx/detr_full.onnx" # float32 full DETR graph (downloaded below)
CUSTOM_OP_ONNX = (
"onnx/detr_full_customop.onnx" # quantized full graph after decoder-attention custom-op
)
KERNEL_HEADER = (
"detr_attention_core.hpp" # CCL kernel for the decoder-attention custom op (attn_stub_src_path)
)
OUTPUT_DIR = "onnx"
Path(OUTPUT_DIR).mkdir(exist_ok=True)
CALIB_DIR = Path("calib_images") # COCO val2017 calibration images
NUM_CALIB_IMAGES = 100 # COCO images used for INT8 calibration
IMG_H, IMG_W = 384, 960
DETR_MEAN = [0.485, 0.456, 0.406]
DETR_STD = [0.229, 0.224, 0.225]
hw_config = HWConfig(
product="QC-U",
ocm_size="8MB",
lrm_size="4kB",
macs_per_pe=16,
clock_freq_ghz=1.56,
ext_rd_bw="64GBps",
ext_wr_bw="64GBps",
num_cores=1,
)
2. Download & export the model
Download facebook/detr-resnet-101 and export the full inference DETR graph to onnx/detr_full.onnx. The implementation lives in model_download.py — it exports the full graph only (no sub-graph extraction). This cell runs the download only if the graph is not already present in onnx/.
if Path(FULL_FP).exists():
print(f"Found cached graph: {FULL_FP}")
else:
download_model()
Downloading facebook/detr-resnet-101 from HuggingFace -> onnx/facebook-detr-resnet-101 ...
Saved to onnx/facebook-detr-resnet-101
Model ready.
Exporting full DETR graph ...
Exporting detr_full.onnx ...
/usr/local/lib/python3.10/dist-packages/transformers/models/detr/modeling_detr.py:568: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_weights.size() != (batch_size * self.num_heads, target_len, source_len):
/usr/local/lib/python3.10/dist-packages/transformers/models/detr/modeling_detr.py:599: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if attn_output.size() != (batch_size * self.num_heads, target_len, self.head_dim):
in pixel_values: [1, 3, 384, 960]
out pred_logits: [1, 100, 92]
out pred_boxes: [1, 100, 4]
[PASS] onnx/detr_full.onnx
in [[1, 3, 384, 960]]
out [[1, 100, 92], [1, 100, 4]]
3. Build the COCO calibration set
Fetch 100 COCO val2017 images into calib_images/ and calibrate on them. ModelHelper resizes and normalizes each image into pixel_values [1, 3, 384, 960]. A broad calibration set produces representative INT8 tensor ranges across the full graph.
mh = ModelHelper(size=(IMG_W, IMG_H), mean=DETR_MEAN, std=DETR_STD)
## Fetch the COCO val2017 calibration images into calib_images/.
ensure_calibration_images(out_dir=str(CALIB_DIR), num_images=NUM_CALIB_IMAGES)
image_paths = sorted(
p for p in CALIB_DIR.iterdir() if p.suffix.lower() in (".jpg", ".jpeg", ".png")
)[:NUM_CALIB_IMAGES]
assert image_paths, f"No images found in {CALIB_DIR}"
NUM_CALIB = len(image_paths)
print(f"Calibrating on {NUM_CALIB} COCO images from {CALIB_DIR}/")
print(f"Validation image (in calibration set): {image_paths[0].name}")
/tmp/ipykernel_24081/1729535217.py:1: DeprecationWarning: Call to deprecated class ModelHelper. ('ModelHelper' class is being deprecated. Quadric APIs have been updated to use PyTorch datasets and transforms instead.) -- Deprecated since version 24.01.
mh = ModelHelper(size=(IMG_W, IMG_H), mean=DETR_MEAN, std=DETR_STD)
Downloading COCO calibration images -> calib_images/ ...
Calibration set ready: 100 images in calib_images/
Calibrating on 100 COCO images from calib_images/
Validation image (in calibration set): 000000000139.jpg
4. Quantize the full graph
One INT8 calibration pass over the whole DETR graph produces the quantized model and its tensor-range file for the end-to-end compile.
qfull = quadric_quantize(
FULL_FP,
num_images=NUM_CALIB,
mh=mh,
calibration_folder=str(CALIB_DIR),
synthetic_input=False,
output_folder=OUTPUT_DIR,
)
QMODEL = qfull.qmodel_path
TRANGES = qfull.tranges_path
print(f"Quantized full model: {QMODEL}")
print(f"Tensor ranges: {TRANGES}")
2026-07-26 21:19 - INFO - epu - quantize - Collecting calibration data
2026-07-26 21:19 - INFO - epu - quantize - Optimized model to opset
2026-07-26 21:19 - INFO - epu - quantize - Saved optimized model to detr_full_float32_opt.onnx
2026-07-26 21:19 - INFO - epu - quantize - Input shapes: [1, 3, 384, 960]. Input names: pixel_values
2026-07-26 21:19 - INFO - epu - quantize - Output shapes: [[1, 100, 92], [1, 100, 4]]. Output names: ['pred_logits', 'pred_boxes']
2026-07-26 21:19 - INFO - epu - quantize - applying calibration data to input: pixel_values
2026-07-26 21:19 - INFO - epu - quantize - calibration set size: 100
2026-07-26 21:19 - INFO - epu - quantize - Running real quantization on this input: pixel_values with input shape: [1, 3, 384, 960]
2026-07-26 21:19 - DEBUG - epu - quantize - Full exclusion set for quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-07-26 21:19 - DEBUG - epu - quantize - excl_nodes ['/encoder/layers.0/self_attn/Softmax', '/encoder/layers.1/self_attn/Softmax', '/encoder/layers.2/self_attn/Softmax', '/encoder/layers.3/self_attn/Softmax', '/encoder/layers.4/self_attn/Softmax', '/encoder/layers.5/self_attn/Softmax', '/decoder/layers.0/encoder_attn/Softmax', '/decoder/layers.1/self_attn/Softmax', '/decoder/layers.1/encoder_attn/Softmax', '/decoder/layers.2/self_attn/Softmax', '/decoder/layers.2/encoder_attn/Softmax', '/decoder/layers.3/self_attn/Softmax', '/decoder/layers.3/encoder_attn/Softmax', '/decoder/layers.4/self_attn/Softmax', '/decoder/layers.4/encoder_attn/Softmax', '/decoder/layers.5/self_attn/Softmax', '/decoder/layers.5/encoder_attn/Softmax', '/Sigmoid', '/encoder/layers.0/self_attn_layer_norm/ReduceMean', '/encoder/layers.0/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.0/self_attn_layer_norm/Pow', '/encoder/layers.0/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.0/self_attn_layer_norm/Add', '/encoder/layers.0/self_attn_layer_norm/Sqrt', '/encoder/layers.0/self_attn_layer_norm/Div', 'encoder.layers.0.self_attn_layer_norm.weight', '/encoder/layers.0/self_attn_layer_norm/Mul', 'encoder.layers.0.self_attn_layer_norm.bias', '/encoder/layers.0/self_attn_layer_norm/Add_1', '/encoder/layers.0/final_layer_norm/ReduceMean', '/encoder/layers.0/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.0/final_layer_norm/Pow', '/encoder/layers.0/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.0/final_layer_norm/Add', '/encoder/layers.0/final_layer_norm/Sqrt', '/encoder/layers.0/final_layer_norm/Div', 'encoder.layers.0.final_layer_norm.weight', '/encoder/layers.0/final_layer_norm/Mul', 'encoder.layers.0.final_layer_norm.bias', '/encoder/layers.0/final_layer_norm/Add_1', '/encoder/layers.1/self_attn_layer_norm/ReduceMean', '/encoder/layers.1/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.1/self_attn_layer_norm/Pow', '/encoder/layers.1/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.1/self_attn_layer_norm/Add', '/encoder/layers.1/self_attn_layer_norm/Sqrt', '/encoder/layers.1/self_attn_layer_norm/Div', 'encoder.layers.1.self_attn_layer_norm.weight', '/encoder/layers.1/self_attn_layer_norm/Mul', 'encoder.layers.1.self_attn_layer_norm.bias', '/encoder/layers.1/self_attn_layer_norm/Add_1', '/encoder/layers.1/final_layer_norm/ReduceMean', '/encoder/layers.1/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.1/final_layer_norm/Pow', '/encoder/layers.1/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.1/final_layer_norm/Add', '/encoder/layers.1/final_layer_norm/Sqrt', '/encoder/layers.1/final_layer_norm/Div', 'encoder.layers.1.final_layer_norm.weight', '/encoder/layers.1/final_layer_norm/Mul', 'encoder.layers.1.final_layer_norm.bias', '/encoder/layers.1/final_layer_norm/Add_1', '/encoder/layers.2/self_attn_layer_norm/ReduceMean', '/encoder/layers.2/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.2/self_attn_layer_norm/Pow', '/encoder/layers.2/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.2/self_attn_layer_norm/Add', '/encoder/layers.2/self_attn_layer_norm/Sqrt', '/encoder/layers.2/self_attn_layer_norm/Div', 'encoder.layers.2.self_attn_layer_norm.weight', '/encoder/layers.2/self_attn_layer_norm/Mul', 'encoder.layers.2.self_attn_layer_norm.bias', '/encoder/layers.2/self_attn_layer_norm/Add_1', '/encoder/layers.2/final_layer_norm/ReduceMean', '/encoder/layers.2/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.2/final_layer_norm/Pow', '/encoder/layers.2/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.2/final_layer_norm/Add', '/encoder/layers.2/final_layer_norm/Sqrt', '/encoder/layers.2/final_layer_norm/Div', 'encoder.layers.2.final_layer_norm.weight', '/encoder/layers.2/final_layer_norm/Mul', 'encoder.layers.2.final_layer_norm.bias', '/encoder/layers.2/final_layer_norm/Add_1', '/encoder/layers.3/self_attn_layer_norm/ReduceMean', '/encoder/layers.3/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.3/self_attn_layer_norm/Pow', '/encoder/layers.3/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.3/self_attn_layer_norm/Add', '/encoder/layers.3/self_attn_layer_norm/Sqrt', '/encoder/layers.3/self_attn_layer_norm/Div', 'encoder.layers.3.self_attn_layer_norm.weight', '/encoder/layers.3/self_attn_layer_norm/Mul', 'encoder.layers.3.self_attn_layer_norm.bias', '/encoder/layers.3/self_attn_layer_norm/Add_1', '/encoder/layers.3/final_layer_norm/ReduceMean', '/encoder/layers.3/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.3/final_layer_norm/Pow', '/encoder/layers.3/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.3/final_layer_norm/Add', '/encoder/layers.3/final_layer_norm/Sqrt', '/encoder/layers.3/final_layer_norm/Div', 'encoder.layers.3.final_layer_norm.weight', '/encoder/layers.3/final_layer_norm/Mul', 'encoder.layers.3.final_layer_norm.bias', '/encoder/layers.3/final_layer_norm/Add_1', '/encoder/layers.4/self_attn_layer_norm/ReduceMean', '/encoder/layers.4/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.4/self_attn_layer_norm/Pow', '/encoder/layers.4/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.4/self_attn_layer_norm/Add', '/encoder/layers.4/self_attn_layer_norm/Sqrt', '/encoder/layers.4/self_attn_layer_norm/Div', 'encoder.layers.4.self_attn_layer_norm.weight', '/encoder/layers.4/self_attn_layer_norm/Mul', 'encoder.layers.4.self_attn_layer_norm.bias', '/encoder/layers.4/self_attn_layer_norm/Add_1', '/encoder/layers.4/final_layer_norm/ReduceMean', '/encoder/layers.4/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.4/final_layer_norm/Pow', '/encoder/layers.4/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.4/final_layer_norm/Add', '/encoder/layers.4/final_layer_norm/Sqrt', '/encoder/layers.4/final_layer_norm/Div', 'encoder.layers.4.final_layer_norm.weight', '/encoder/layers.4/final_layer_norm/Mul', 'encoder.layers.4.final_layer_norm.bias', '/encoder/layers.4/final_layer_norm/Add_1', '/encoder/layers.5/self_attn_layer_norm/ReduceMean', '/encoder/layers.5/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.5/self_attn_layer_norm/Pow', '/encoder/layers.5/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.5/self_attn_layer_norm/Add', '/encoder/layers.5/self_attn_layer_norm/Sqrt', '/encoder/layers.5/self_attn_layer_norm/Div', 'encoder.layers.5.self_attn_layer_norm.weight', '/encoder/layers.5/self_attn_layer_norm/Mul', 'encoder.layers.5.self_attn_layer_norm.bias', '/encoder/layers.5/self_attn_layer_norm/Add_1', '/encoder/layers.5/final_layer_norm/ReduceMean', '/encoder/layers.5/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.5/final_layer_norm/Pow', '/encoder/layers.5/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.5/final_layer_norm/Add', '/encoder/layers.5/final_layer_norm/Sqrt', '/encoder/layers.5/final_layer_norm/Div', 'encoder.layers.5.final_layer_norm.weight', '/encoder/layers.5/final_layer_norm/Mul', 'encoder.layers.5.final_layer_norm.bias', '/encoder/layers.5/final_layer_norm/Add_1', '/decoder/layers.0/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.0/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.0/encoder_attn_layer_norm/Pow', '/decoder/layers.0/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.0/encoder_attn_layer_norm/Add', '/decoder/layers.0/encoder_attn_layer_norm/Sqrt', '/decoder/layers.0/encoder_attn_layer_norm/Div', 'decoder.layers.0.encoder_attn_layer_norm.weight', '/decoder/layers.0/encoder_attn_layer_norm/Mul', 'decoder.layers.0.encoder_attn_layer_norm.bias', '/decoder/layers.0/encoder_attn_layer_norm/Add_1', '/decoder/layers.0/final_layer_norm/ReduceMean', '/decoder/layers.0/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.0/final_layer_norm/Pow', '/decoder/layers.0/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.0/final_layer_norm/Add', '/decoder/layers.0/final_layer_norm/Sqrt', '/decoder/layers.0/final_layer_norm/Div', 'decoder.layers.0.final_layer_norm.weight', '/decoder/layers.0/final_layer_norm/Mul', 'decoder.layers.0.final_layer_norm.bias', '/decoder/layers.0/final_layer_norm/Add_1', '/decoder/layers.1/self_attn_layer_norm/ReduceMean', '/decoder/layers.1/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.1/self_attn_layer_norm/Pow', '/decoder/layers.1/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.1/self_attn_layer_norm/Add', '/decoder/layers.1/self_attn_layer_norm/Sqrt', '/decoder/layers.1/self_attn_layer_norm/Div', 'decoder.layers.1.self_attn_layer_norm.weight', '/decoder/layers.1/self_attn_layer_norm/Mul', 'decoder.layers.1.self_attn_layer_norm.bias', '/decoder/layers.1/self_attn_layer_norm/Add_1', '/decoder/layers.1/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.1/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.1/encoder_attn_layer_norm/Pow', '/decoder/layers.1/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.1/encoder_attn_layer_norm/Add', '/decoder/layers.1/encoder_attn_layer_norm/Sqrt', '/decoder/layers.1/encoder_attn_layer_norm/Div', 'decoder.layers.1.encoder_attn_layer_norm.weight', '/decoder/layers.1/encoder_attn_layer_norm/Mul', 'decoder.layers.1.encoder_attn_layer_norm.bias', '/decoder/layers.1/encoder_attn_layer_norm/Add_1', '/decoder/layers.1/final_layer_norm/ReduceMean', '/decoder/layers.1/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.1/final_layer_norm/Pow', '/decoder/layers.1/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.1/final_layer_norm/Add', '/decoder/layers.1/final_layer_norm/Sqrt', '/decoder/layers.1/final_layer_norm/Div', 'decoder.layers.1.final_layer_norm.weight', '/decoder/layers.1/final_layer_norm/Mul', 'decoder.layers.1.final_layer_norm.bias', '/decoder/layers.1/final_layer_norm/Add_1', '/decoder/layers.2/self_attn_layer_norm/ReduceMean', '/decoder/layers.2/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.2/self_attn_layer_norm/Pow', '/decoder/layers.2/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.2/self_attn_layer_norm/Add', '/decoder/layers.2/self_attn_layer_norm/Sqrt', '/decoder/layers.2/self_attn_layer_norm/Div', 'decoder.layers.2.self_attn_layer_norm.weight', '/decoder/layers.2/self_attn_layer_norm/Mul', 'decoder.layers.2.self_attn_layer_norm.bias', '/decoder/layers.2/self_attn_layer_norm/Add_1', '/decoder/layers.2/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.2/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.2/encoder_attn_layer_norm/Pow', '/decoder/layers.2/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.2/encoder_attn_layer_norm/Add', '/decoder/layers.2/encoder_attn_layer_norm/Sqrt', '/decoder/layers.2/encoder_attn_layer_norm/Div', 'decoder.layers.2.encoder_attn_layer_norm.weight', '/decoder/layers.2/encoder_attn_layer_norm/Mul', 'decoder.layers.2.encoder_attn_layer_norm.bias', '/decoder/layers.2/encoder_attn_layer_norm/Add_1', '/decoder/layers.2/final_layer_norm/ReduceMean', '/decoder/layers.2/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.2/final_layer_norm/Pow', '/decoder/layers.2/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.2/final_layer_norm/Add', '/decoder/layers.2/final_layer_norm/Sqrt', '/decoder/layers.2/final_layer_norm/Div', 'decoder.layers.2.final_layer_norm.weight', '/decoder/layers.2/final_layer_norm/Mul', 'decoder.layers.2.final_layer_norm.bias', '/decoder/layers.2/final_layer_norm/Add_1', '/decoder/layers.3/self_attn_layer_norm/ReduceMean', '/decoder/layers.3/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.3/self_attn_layer_norm/Pow', '/decoder/layers.3/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.3/self_attn_layer_norm/Add', '/decoder/layers.3/self_attn_layer_norm/Sqrt', '/decoder/layers.3/self_attn_layer_norm/Div', 'decoder.layers.3.self_attn_layer_norm.weight', '/decoder/layers.3/self_attn_layer_norm/Mul', 'decoder.layers.3.self_attn_layer_norm.bias', '/decoder/layers.3/self_attn_layer_norm/Add_1', '/decoder/layers.3/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.3/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.3/encoder_attn_layer_norm/Pow', '/decoder/layers.3/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.3/encoder_attn_layer_norm/Add', '/decoder/layers.3/encoder_attn_layer_norm/Sqrt', '/decoder/layers.3/encoder_attn_layer_norm/Div', 'decoder.layers.3.encoder_attn_layer_norm.weight', '/decoder/layers.3/encoder_attn_layer_norm/Mul', 'decoder.layers.3.encoder_attn_layer_norm.bias', '/decoder/layers.3/encoder_attn_layer_norm/Add_1', '/decoder/layers.3/final_layer_norm/ReduceMean', '/decoder/layers.3/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.3/final_layer_norm/Pow', '/decoder/layers.3/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.3/final_layer_norm/Add', '/decoder/layers.3/final_layer_norm/Sqrt', '/decoder/layers.3/final_layer_norm/Div', 'decoder.layers.3.final_layer_norm.weight', '/decoder/layers.3/final_layer_norm/Mul', 'decoder.layers.3.final_layer_norm.bias', '/decoder/layers.3/final_layer_norm/Add_1', '/decoder/layers.4/self_attn_layer_norm/ReduceMean', '/decoder/layers.4/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.4/self_attn_layer_norm/Pow', '/decoder/layers.4/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.4/self_attn_layer_norm/Add', '/decoder/layers.4/self_attn_layer_norm/Sqrt', '/decoder/layers.4/self_attn_layer_norm/Div', 'decoder.layers.4.self_attn_layer_norm.weight', '/decoder/layers.4/self_attn_layer_norm/Mul', 'decoder.layers.4.self_attn_layer_norm.bias', '/decoder/layers.4/self_attn_layer_norm/Add_1', '/decoder/layers.4/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.4/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.4/encoder_attn_layer_norm/Pow', '/decoder/layers.4/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.4/encoder_attn_layer_norm/Add', '/decoder/layers.4/encoder_attn_layer_norm/Sqrt', '/decoder/layers.4/encoder_attn_layer_norm/Div', 'decoder.layers.4.encoder_attn_layer_norm.weight', '/decoder/layers.4/encoder_attn_layer_norm/Mul', 'decoder.layers.4.encoder_attn_layer_norm.bias', '/decoder/layers.4/encoder_attn_layer_norm/Add_1', '/decoder/layers.4/final_layer_norm/ReduceMean', '/decoder/layers.4/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.4/final_layer_norm/Pow', '/decoder/layers.4/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.4/final_layer_norm/Add', '/decoder/layers.4/final_layer_norm/Sqrt', '/decoder/layers.4/final_layer_norm/Div', 'decoder.layers.4.final_layer_norm.weight', '/decoder/layers.4/final_layer_norm/Mul', 'decoder.layers.4.final_layer_norm.bias', '/decoder/layers.4/final_layer_norm/Add_1', '/decoder/layers.5/self_attn_layer_norm/ReduceMean', '/decoder/layers.5/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.5/self_attn_layer_norm/Pow', '/decoder/layers.5/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.5/self_attn_layer_norm/Add', '/decoder/layers.5/self_attn_layer_norm/Sqrt', '/decoder/layers.5/self_attn_layer_norm/Div', 'decoder.layers.5.self_attn_layer_norm.weight', '/decoder/layers.5/self_attn_layer_norm/Mul', 'decoder.layers.5.self_attn_layer_norm.bias', '/decoder/layers.5/self_attn_layer_norm/Add_1', '/decoder/layers.5/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.5/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.5/encoder_attn_layer_norm/Pow', '/decoder/layers.5/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.5/encoder_attn_layer_norm/Add', '/decoder/layers.5/encoder_attn_layer_norm/Sqrt', '/decoder/layers.5/encoder_attn_layer_norm/Div', 'decoder.layers.5.encoder_attn_layer_norm.weight', '/decoder/layers.5/encoder_attn_layer_norm/Mul', 'decoder.layers.5.encoder_attn_layer_norm.bias', '/decoder/layers.5/encoder_attn_layer_norm/Add_1', '/decoder/layers.5/final_layer_norm/ReduceMean', '/decoder/layers.5/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.5/final_layer_norm/Pow', '/decoder/layers.5/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.5/final_layer_norm/Add', '/decoder/layers.5/final_layer_norm/Sqrt', '/decoder/layers.5/final_layer_norm/Div', 'decoder.layers.5.final_layer_norm.weight', '/decoder/layers.5/final_layer_norm/Mul', 'decoder.layers.5.final_layer_norm.bias', '/decoder/layers.5/final_layer_norm/Add_1', '/decoder/layernorm/ReduceMean', '/decoder/layernorm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layernorm/Pow', '/decoder/layernorm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layernorm/Add', '/decoder/layernorm/Sqrt', '/decoder/layernorm/Div', 'decoder.layernorm.weight', '/decoder/layernorm/Mul', 'decoder.layernorm.bias', '/decoder/layernorm/Add_1', '/encoder/layers.0/self_attn_layer_norm/ReduceMean', '/encoder/layers.0/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.0/self_attn_layer_norm/Pow', '/encoder/layers.0/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.0/self_attn_layer_norm/Add', '/encoder/layers.0/self_attn_layer_norm/Sqrt', '/encoder/layers.0/self_attn_layer_norm/Div', '/encoder/layers.0/final_layer_norm/ReduceMean', '/encoder/layers.0/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.0/final_layer_norm/Pow', '/encoder/layers.0/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.0/final_layer_norm/Add', '/encoder/layers.0/final_layer_norm/Sqrt', '/encoder/layers.0/final_layer_norm/Div', '/encoder/layers.1/self_attn_layer_norm/ReduceMean', '/encoder/layers.1/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.1/self_attn_layer_norm/Pow', '/encoder/layers.1/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.1/self_attn_layer_norm/Add', '/encoder/layers.1/self_attn_layer_norm/Sqrt', '/encoder/layers.1/self_attn_layer_norm/Div', '/encoder/layers.1/final_layer_norm/ReduceMean', '/encoder/layers.1/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.1/final_layer_norm/Pow', '/encoder/layers.1/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.1/final_layer_norm/Add', '/encoder/layers.1/final_layer_norm/Sqrt', '/encoder/layers.1/final_layer_norm/Div', '/encoder/layers.2/self_attn_layer_norm/ReduceMean', '/encoder/layers.2/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.2/self_attn_layer_norm/Pow', '/encoder/layers.2/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.2/self_attn_layer_norm/Add', '/encoder/layers.2/self_attn_layer_norm/Sqrt', '/encoder/layers.2/self_attn_layer_norm/Div', '/encoder/layers.2/final_layer_norm/ReduceMean', '/encoder/layers.2/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.2/final_layer_norm/Pow', '/encoder/layers.2/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.2/final_layer_norm/Add', '/encoder/layers.2/final_layer_norm/Sqrt', '/encoder/layers.2/final_layer_norm/Div', '/encoder/layers.3/self_attn_layer_norm/ReduceMean', '/encoder/layers.3/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.3/self_attn_layer_norm/Pow', '/encoder/layers.3/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.3/self_attn_layer_norm/Add', '/encoder/layers.3/self_attn_layer_norm/Sqrt', '/encoder/layers.3/self_attn_layer_norm/Div', '/encoder/layers.3/final_layer_norm/ReduceMean', '/encoder/layers.3/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.3/final_layer_norm/Pow', '/encoder/layers.3/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.3/final_layer_norm/Add', '/encoder/layers.3/final_layer_norm/Sqrt', '/encoder/layers.3/final_layer_norm/Div', '/encoder/layers.4/self_attn_layer_norm/ReduceMean', '/encoder/layers.4/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.4/self_attn_layer_norm/Pow', '/encoder/layers.4/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.4/self_attn_layer_norm/Add', '/encoder/layers.4/self_attn_layer_norm/Sqrt', '/encoder/layers.4/self_attn_layer_norm/Div', '/encoder/layers.4/final_layer_norm/ReduceMean', '/encoder/layers.4/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.4/final_layer_norm/Pow', '/encoder/layers.4/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.4/final_layer_norm/Add', '/encoder/layers.4/final_layer_norm/Sqrt', '/encoder/layers.4/final_layer_norm/Div', '/encoder/layers.5/self_attn_layer_norm/ReduceMean', '/encoder/layers.5/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.5/self_attn_layer_norm/Pow', '/encoder/layers.5/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.5/self_attn_layer_norm/Add', '/encoder/layers.5/self_attn_layer_norm/Sqrt', '/encoder/layers.5/self_attn_layer_norm/Div', '/encoder/layers.5/final_layer_norm/ReduceMean', '/encoder/layers.5/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/encoder/layers.5/final_layer_norm/Pow', '/encoder/layers.5/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/encoder/layers.5/final_layer_norm/Add', '/encoder/layers.5/final_layer_norm/Sqrt', '/encoder/layers.5/final_layer_norm/Div', '/decoder/layers.0/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.0/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.0/encoder_attn_layer_norm/Pow', '/decoder/layers.0/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.0/encoder_attn_layer_norm/Add', '/decoder/layers.0/encoder_attn_layer_norm/Sqrt', '/decoder/layers.0/encoder_attn_layer_norm/Div', '/decoder/layers.0/final_layer_norm/ReduceMean', '/decoder/layers.0/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.0/final_layer_norm/Pow', '/decoder/layers.0/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.0/final_layer_norm/Add', '/decoder/layers.0/final_layer_norm/Sqrt', '/decoder/layers.0/final_layer_norm/Div', '/decoder/layers.1/self_attn_layer_norm/ReduceMean', '/decoder/layers.1/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.1/self_attn_layer_norm/Pow', '/decoder/layers.1/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.1/self_attn_layer_norm/Add', '/decoder/layers.1/self_attn_layer_norm/Sqrt', '/decoder/layers.1/self_attn_layer_norm/Div', '/decoder/layers.1/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.1/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.1/encoder_attn_layer_norm/Pow', '/decoder/layers.1/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.1/encoder_attn_layer_norm/Add', '/decoder/layers.1/encoder_attn_layer_norm/Sqrt', '/decoder/layers.1/encoder_attn_layer_norm/Div', '/decoder/layers.1/final_layer_norm/ReduceMean', '/decoder/layers.1/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.1/final_layer_norm/Pow', '/decoder/layers.1/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.1/final_layer_norm/Add', '/decoder/layers.1/final_layer_norm/Sqrt', '/decoder/layers.1/final_layer_norm/Div', '/decoder/layers.2/self_attn_layer_norm/ReduceMean', '/decoder/layers.2/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.2/self_attn_layer_norm/Pow', '/decoder/layers.2/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.2/self_attn_layer_norm/Add', '/decoder/layers.2/self_attn_layer_norm/Sqrt', '/decoder/layers.2/self_attn_layer_norm/Div', '/decoder/layers.2/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.2/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.2/encoder_attn_layer_norm/Pow', '/decoder/layers.2/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.2/encoder_attn_layer_norm/Add', '/decoder/layers.2/encoder_attn_layer_norm/Sqrt', '/decoder/layers.2/encoder_attn_layer_norm/Div', '/decoder/layers.2/final_layer_norm/ReduceMean', '/decoder/layers.2/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.2/final_layer_norm/Pow', '/decoder/layers.2/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.2/final_layer_norm/Add', '/decoder/layers.2/final_layer_norm/Sqrt', '/decoder/layers.2/final_layer_norm/Div', '/decoder/layers.3/self_attn_layer_norm/ReduceMean', '/decoder/layers.3/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.3/self_attn_layer_norm/Pow', '/decoder/layers.3/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.3/self_attn_layer_norm/Add', '/decoder/layers.3/self_attn_layer_norm/Sqrt', '/decoder/layers.3/self_attn_layer_norm/Div', '/decoder/layers.3/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.3/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.3/encoder_attn_layer_norm/Pow', '/decoder/layers.3/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.3/encoder_attn_layer_norm/Add', '/decoder/layers.3/encoder_attn_layer_norm/Sqrt', '/decoder/layers.3/encoder_attn_layer_norm/Div', '/decoder/layers.3/final_layer_norm/ReduceMean', '/decoder/layers.3/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.3/final_layer_norm/Pow', '/decoder/layers.3/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.3/final_layer_norm/Add', '/decoder/layers.3/final_layer_norm/Sqrt', '/decoder/layers.3/final_layer_norm/Div', '/decoder/layers.4/self_attn_layer_norm/ReduceMean', '/decoder/layers.4/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.4/self_attn_layer_norm/Pow', '/decoder/layers.4/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.4/self_attn_layer_norm/Add', '/decoder/layers.4/self_attn_layer_norm/Sqrt', '/decoder/layers.4/self_attn_layer_norm/Div', '/decoder/layers.4/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.4/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.4/encoder_attn_layer_norm/Pow', '/decoder/layers.4/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.4/encoder_attn_layer_norm/Add', '/decoder/layers.4/encoder_attn_layer_norm/Sqrt', '/decoder/layers.4/encoder_attn_layer_norm/Div', '/decoder/layers.4/final_layer_norm/ReduceMean', '/decoder/layers.4/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.4/final_layer_norm/Pow', '/decoder/layers.4/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.4/final_layer_norm/Add', '/decoder/layers.4/final_layer_norm/Sqrt', '/decoder/layers.4/final_layer_norm/Div', '/decoder/layers.5/self_attn_layer_norm/ReduceMean', '/decoder/layers.5/self_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.5/self_attn_layer_norm/Pow', '/decoder/layers.5/self_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.5/self_attn_layer_norm/Add', '/decoder/layers.5/self_attn_layer_norm/Sqrt', '/decoder/layers.5/self_attn_layer_norm/Div', '/decoder/layers.5/encoder_attn_layer_norm/ReduceMean', '/decoder/layers.5/encoder_attn_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.5/encoder_attn_layer_norm/Pow', '/decoder/layers.5/encoder_attn_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.5/encoder_attn_layer_norm/Add', '/decoder/layers.5/encoder_attn_layer_norm/Sqrt', '/decoder/layers.5/encoder_attn_layer_norm/Div', '/decoder/layers.5/final_layer_norm/ReduceMean', '/decoder/layers.5/final_layer_norm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layers.5/final_layer_norm/Pow', '/decoder/layers.5/final_layer_norm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layers.5/final_layer_norm/Add', '/decoder/layers.5/final_layer_norm/Sqrt', '/decoder/layers.5/final_layer_norm/Div', '/decoder/layernorm/ReduceMean', '/decoder/layernorm/Sub', '/encoder/layers.0/self_attn_layer_norm/Constant_output_0', '/decoder/layernorm/Pow', '/decoder/layernorm/ReduceMean_1', '/encoder/layers.0/self_attn_layer_norm/Constant_1_output_0', '/decoder/layernorm/Add', '/decoder/layernorm/Sqrt', '/decoder/layernorm/Div']
2026-07-26 21:19 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-07-26 21:21 - INFO - epu - quantize - Quantization done succesfully!
2026-07-26 21:21 - INFO - epu - quantize - ONNX full precision model size: 230.66 MB
2026-07-26 21:21 - INFO - epu - quantize - ONNX quantized model size: 58.4 MB
2026-07-26 21:21 - INFO - epu - quantize - Saved quantized model to onnx/detr_full_opt_sym_int8_q.onnx
2026-07-26 21:21 - INFO - epu - quantize - Saved shape inferenced model to onnx/detr_full_opt_sym_int8_q.onnx
2026-07-26 21:21 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-07-26 21:21 - INFO - epu - quantize - Model still has FLOAT/FLOAT16 types. Creating ranges for floating point tensors using calibration data
2026-07-26 21:23 - INFO - epu - quantize - Saved tensor ranges to onnx/detr_full_opt_sym_int8_q.onnx.tranges
Quantized full model: onnx/detr_full_opt_sym_int8_q.onnx
Tensor ranges: onnx/detr_full_opt_sym_int8_q.onnx.tranges
5. Custom-op the decoder attention
The decoder attention — self- and cross-attention across all 6 decoder layers (11 cores) — is implemented with a custom op backed by nn::multiheadAttention (int8 KCache/VCache SDPA). custom_op_attention.py splices the 11 decoder cores on the quantized graph, and detr_attention_core.hpp is the CCL (Chimera Compute Language) kernel: nn::detrAttentionCore for the runtime-Q cores and nn::detrAttentionCoreConstQ for the layer-0 cross-attention, whose Q is a compile-time constant. The backbone, encoder, and prediction heads run on the native path.
## Custom-op the decoder attention (self + cross, 11 cores): route it through nn::multiheadAttention.
## detrAttentionCore handles the runtime-Q cores; detrAttentionCoreConstQ the layer-0 constant-Q cross-attn.
## The backbone, encoder, and heads run on the native path.
from custom_op_attention import replace_attention_custom_ops
replace_attention_custom_ops(QMODEL, CUSTOM_OP_ONNX)
Graph: QUANTIZED (onnx/detr_full_opt_sym_int8_q.onnx); 11 attention cores:
[cross] /decoder/layers.0/encoder_attn seq_q=100 seq_k=360
[cross] /decoder/layers.1/encoder_attn seq_q=100 seq_k=360
[self ] /decoder/layers.1/self_attn seq_q=100 seq_k=100
[cross] /decoder/layers.2/encoder_attn seq_q=100 seq_k=360
[self ] /decoder/layers.2/self_attn seq_q=100 seq_k=100
[cross] /decoder/layers.3/encoder_attn seq_q=100 seq_k=360
[self ] /decoder/layers.3/self_attn seq_q=100 seq_k=100
[cross] /decoder/layers.4/encoder_attn seq_q=100 seq_k=360
[self ] /decoder/layers.4/self_attn seq_q=100 seq_k=100
[cross] /decoder/layers.5/encoder_attn seq_q=100 seq_k=360
[self ] /decoder/layers.5/self_attn seq_q=100 seq_k=100
replaced -> nn::detrAttentionCoreConstQ<8,32,100,360,14,31,23> @ /decoder/layers.0/encoder_attn
replaced -> nn::detrAttentionCore<8,32,100,360,14,31,23> @ /decoder/layers.1/encoder_attn
replaced -> nn::detrAttentionCore<8,32,100,100,14,31,23> @ /decoder/layers.1/self_attn
replaced -> nn::detrAttentionCore<8,32,100,360,14,31,23> @ /decoder/layers.2/encoder_attn
replaced -> nn::detrAttentionCore<8,32,100,100,14,31,23> @ /decoder/layers.2/self_attn
replaced -> nn::detrAttentionCore<8,32,100,360,14,31,23> @ /decoder/layers.3/encoder_attn
replaced -> nn::detrAttentionCore<8,32,100,100,14,31,23> @ /decoder/layers.3/self_attn
replaced -> nn::detrAttentionCore<8,32,100,360,14,31,23> @ /decoder/layers.4/encoder_attn
replaced -> nn::detrAttentionCore<8,32,100,100,14,31,23> @ /decoder/layers.4/self_attn
replaced -> nn::detrAttentionCore<8,32,100,360,14,31,23> @ /decoder/layers.5/encoder_attn
replaced -> nn::detrAttentionCore<8,32,100,100,14,31,23> @ /decoder/layers.5/self_attn
Wrote onnx/detr_full_customop.onnx (replaced 11, skipped 0)
(11, 0)
6. Compile the full graph to Chimera ASM
Compile the entire quantized DETR graph as one ChimeraJob — backbone, input projection, encoder, decoder, and heads in one unit. compile() emits Chimera ASM and prepares the unit for the ISS run in the next section.
job = ChimeraJob(
CUSTOM_OP_ONNX,
hw_config=hw_config,
trange_file=TRANGES,
attn_stub_src_path=str(
Path(KERNEL_HEADER).resolve()
), # CCL kernel for the decoder-attention custom op
target_lang="ASM",
validate_iss=True,
)
job.compile()
print("DETR compiled as one unit: backbone + input projection + encoder + decoder + heads")
2026-07-26 21:23 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-07-26 21:23 - INFO - epu - chimera_job - Numerical ranges provided
2026-07-26 21:26 - INFO - epu - codegen - START===============================optimize_relay
2026-07-26 21:26 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-07-26 21:26 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
------------------------------------------------------------------ ----------------------------- ------------------------- --------------------
/encoder/layers.0/self_attn/MatMul_output_0_DequantizeLinear contrib.epu.dequantize [-928.459f, 40.7219f] 20
/encoder/layers.0/self_attn/Softmax nn.softmax [0f, 1f] 20
/encoder/layers.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-52.8879f, 67.1676f] 24
/encoder/layers.0/self_attn_layer_norm/Add_1 nn.layer_norm [-10.6562f, 14.3276f] 27
/encoder/layers.0/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-68.2148f, 68.2148f] 24
/encoder/layers.0/final_layer_norm/Add_1 nn.layer_norm [-7.48073f, 10.7392f] 27
/encoder/layers.1/self_attn/MatMul_output_0_DequantizeLinear contrib.epu.dequantize [-11.5462f, 21.5643f] 26
/encoder/layers.1/self_attn/Softmax nn.softmax [1.16433e-11f, 0.966332f] 26
/encoder/layers.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-7.71115f, 10.9391f] 27
/encoder/layers.1/self_attn_layer_norm/Add_1 nn.layer_norm [-13.3774f, 14.8253f] 27
/encoder/layers.1/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-18.4685f, 34.4561f] 25
/encoder/layers.1/final_layer_norm/Add_1 nn.layer_norm [-6.27449f, 10.3709f] 27
/encoder/layers.2/self_attn/MatMul_output_0_DequantizeLinear contrib.epu.dequantize [-18.6902f, 23.9763f] 26
/encoder/layers.2/self_attn/Softmax nn.softmax [4.05337e-15f, 0.885612f] 26
/encoder/layers.2/Add_output_0_DequantizeLinear contrib.epu.dequantize [-6.24802f, 10.3586f] 27
/encoder/layers.2/self_attn_layer_norm/Add_1 nn.layer_norm [-9.48785f, 18.1524f] 26
/encoder/layers.2/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-34.7632f, 60.4784f] 25
/encoder/layers.2/final_layer_norm/Add_1 nn.layer_norm [-6.26767f, 11.9776f] 27
/encoder/layers.3/self_attn/MatMul_output_0_DequantizeLinear contrib.epu.dequantize [-42.238f, 48.3817f] 25
/encoder/layers.3/self_attn/Softmax nn.softmax [2.53344e-31f, 0.971664f] 25
/encoder/layers.3/Add_output_0_DequantizeLinear contrib.epu.dequantize [-6.32148f, 11.6352f] 27
/encoder/layers.3/self_attn_layer_norm/Add_1 nn.layer_norm [-7.33951f, 22.111f] 26
/encoder/layers.3/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-66.827f, 146.328f] 23
/encoder/layers.3/final_layer_norm/Add_1 nn.layer_norm [-4.69213f, 8.91069f] 27
/encoder/layers.4/self_attn/MatMul_output_0_DequantizeLinear contrib.epu.dequantize [-22.8891f, 26.1884f] 26
/encoder/layers.4/self_attn/Softmax nn.softmax [1.05028e-20f, 0.936618f] 26
/encoder/layers.4/Add_output_0_DequantizeLinear contrib.epu.dequantize [-4.77218f, 9.18283f] 27
/encoder/layers.4/self_attn_layer_norm/Add_1 nn.layer_norm [-8.20218f, 20.1111f] 26
/encoder/layers.4/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-13.1065f, 33.0393f] 25
/encoder/layers.4/final_layer_norm/Add_1 nn.layer_norm [-5.8982f, 7.76882f] 27
/encoder/layers.5/self_attn/MatMul_output_0_DequantizeLinear contrib.epu.dequantize [-32.3585f, 30.0833f] 25
/encoder/layers.5/self_attn/Softmax nn.softmax [3.05479e-21f, 0.937707f] 25
/encoder/layers.5/Add_output_0_DequantizeLinear contrib.epu.dequantize [-6.12066f, 8.09712f] 27
/encoder/layers.5/self_attn_layer_norm/Add_1 nn.layer_norm [-9.88279f, 14.1164f] 27
/encoder/layers.5/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-11.6545f, 15.1388f] 27
/encoder/layers.5/final_layer_norm/Add_1 nn.layer_norm [-6.60448f, 7.47337f] 27
CustomOp/nn::detrAttentionCoreConstQ<8, 32, 100, 360, 14, 31, 23>0 contrib.epu.quadric_custom_op [-5.08429f, 4.88569f] 14
/decoder/layers.0/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-21.9025f, 30.5672f] 26
/decoder/layers.0/encoder_attn_layer_norm/Add_1 nn.layer_norm [-7.69445f, 10.0638f] 27
/decoder/layers.0/Add_2_output_0_DequantizeLinear contrib.epu.dequantize [-21.3602f, 27.0411f] 26
/decoder/layers.0/final_layer_norm/Add_1 nn.layer_norm [-8.20018f, 7.84764f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 100, 14, 31, 23>2 contrib.epu.quadric_custom_op [-3.94156f, 3.94156f] 14
/decoder/layers.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-8.21843f, 7.8974f] 27
/decoder/layers.1/self_attn_layer_norm/Add_1 nn.layer_norm [-17.1152f, 15.705f] 26
CustomOp/nn::detrAttentionCore<8, 32, 100, 360, 14, 31, 23>1 contrib.epu.quadric_custom_op [-3.60811f, 3.66584f] 14
/decoder/layers.1/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-16.845f, 16.187f] 26
/decoder/layers.1/encoder_attn_layer_norm/Add_1 nn.layer_norm [-9.08264f, 9.79273f] 27
/decoder/layers.1/Add_2_output_0_DequantizeLinear contrib.epu.dequantize [-12.5053f, 20.3611f] 26
/decoder/layers.1/final_layer_norm/Add_1 nn.layer_norm [-9.255f, 9.35414f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 100, 14, 31, 23>4 contrib.epu.quadric_custom_op [-3.09733f, 3.14852f] 14
/decoder/layers.2/Add_output_0_DequantizeLinear contrib.epu.dequantize [-9.35218f, 10.0716f] 27
/decoder/layers.2/self_attn_layer_norm/Add_1 nn.layer_norm [-17.3578f, 15.8053f] 26
CustomOp/nn::detrAttentionCore<8, 32, 100, 360, 14, 31, 23>3 contrib.epu.quadric_custom_op [-3.31137f, 3.09945f] 14
/decoder/layers.2/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-17.1577f, 15.9513f] 26
/decoder/layers.2/encoder_attn_layer_norm/Add_1 nn.layer_norm [-9.45886f, 8.60461f] 27
/decoder/layers.2/Add_2_output_0_DequantizeLinear contrib.epu.dequantize [-11.7887f, 11.7887f] 27
/decoder/layers.2/final_layer_norm/Add_1 nn.layer_norm [-9.83173f, 9.01512f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 100, 14, 31, 23>6 contrib.epu.quadric_custom_op [-3.33742f, 2.7638f] 14
/decoder/layers.3/Add_output_0_DequantizeLinear contrib.epu.dequantize [-9.75193f, 9.06625f] 27
/decoder/layers.3/self_attn_layer_norm/Add_1 nn.layer_norm [-15.1917f, 13.8236f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 360, 14, 31, 23>5 contrib.epu.quadric_custom_op [-2.7165f, 2.75996f] 14
/decoder/layers.3/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-15.3203f, 13.884f] 27
/decoder/layers.3/encoder_attn_layer_norm/Add_1 nn.layer_norm [-8.91601f, 9.902f] 27
/decoder/layers.3/Add_2_output_0_DequantizeLinear contrib.epu.dequantize [-9.54903f, 11.8278f] 27
/decoder/layers.3/final_layer_norm/Add_1 nn.layer_norm [-9.41138f, 8.43573f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 100, 14, 31, 23>8 contrib.epu.quadric_custom_op [-2.62837f, 2.93075f] 14
/decoder/layers.4/Add_output_0_DequantizeLinear contrib.epu.dequantize [-9.23462f, 8.44102f] 27
/decoder/layers.4/self_attn_layer_norm/Add_1 nn.layer_norm [-13.0399f, 11.9727f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 360, 14, 31, 23>7 contrib.epu.quadric_custom_op [-2.24147f, 2.81848f] 14
/decoder/layers.4/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-13.1938f, 13.0908f] 27
/decoder/layers.4/encoder_attn_layer_norm/Add_1 nn.layer_norm [-9.38106f, 9.9824f] 27
/decoder/layers.4/Add_2_output_0_DequantizeLinear contrib.epu.dequantize [-9.49118f, 10.98f] 27
/decoder/layers.4/final_layer_norm/Add_1 nn.layer_norm [-8.45807f, 7.37254f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 100, 14, 31, 23>10 contrib.epu.quadric_custom_op [-1.00316f, 0.995318f] 14
/decoder/layers.5/Add_output_0_DequantizeLinear contrib.epu.dequantize [-8.40672f, 7.35588f] 27
/decoder/layers.5/self_attn_layer_norm/Add_1 nn.layer_norm [-11.0395f, 10.311f] 27
CustomOp/nn::detrAttentionCore<8, 32, 100, 360, 14, 31, 23>9 contrib.epu.quadric_custom_op [-2.18123f, 2.0449f] 14
/decoder/layers.5/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-11.1308f, 10.696f] 27
/decoder/layers.5/encoder_attn_layer_norm/Add_1 nn.layer_norm [-8.47405f, 8.63597f] 27
/decoder/layers.5/Add_2_output_0_DequantizeLinear contrib.epu.dequantize [-8.42067f, 8.916f] 27
/decoder/layers.5/final_layer_norm/Add_1 nn.layer_norm [-8.40978f, 7.43264f] 27
/decoder/layernorm/Add_1 nn.layer_norm [-8.90096f, 8.0754f] 27
pred_logits_DequantizeLinear contrib.epu.dequantize [-23.437f, 14.0622f] 26
/bbox_pred/layers.2/Add_output_0_DequantizeLinear contrib.epu.dequantize [-6.36507f, 19.2881f] 26
/Sigmoid sigmoid [0.00171766f, 1f] -
2026-07-26 21:26 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-07-26 21:26 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-07-26 21:26 - INFO - epu - codegen - START=================================relay_to_tir
2026-07-26 21:26 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-07-26 21:26 - INFO - epu - codegen - START==============================adapt_and_order
2026-07-26 21:26 - INFO - epu - mac_counter -
2026-07-26 21:26 - INFO - epu - mac_counter - ============================================================
2026-07-26 21:26 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-07-26 21:26 - INFO - epu - mac_counter - ============================================================
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,734,082,560 ops (867,041,280 MACs) - /conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 188,743,680 ops (94,371,840 MACs) - /layer1/layer1.0/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer1/layer1.0/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer1/layer1.0/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer1/layer1.0/downsample/downsample.0/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer1/layer1.1/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer1/layer1.1/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer1/layer1.1/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer1/layer1.2/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer1/layer1.2/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer1/layer1.2/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,509,949,440 ops (754,974,720 MACs) - /layer2/layer2.0/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer2/layer2.0/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer2/layer2.0/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,509,949,440 ops (754,974,720 MACs) - /layer2/layer2.0/downsample/downsample.0/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer2/layer2.1/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer2/layer2.1/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer2/layer2.1/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer2/layer2.2/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer2/layer2.2/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer2/layer2.2/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer2/layer2.3/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer2/layer2.3/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer2/layer2.3/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,509,949,440 ops (754,974,720 MACs) - /layer3/layer3.0/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.0/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.0/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,509,949,440 ops (754,974,720 MACs) - /layer3/layer3.0/downsample/downsample.0/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.1/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.1/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.1/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.2/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.2/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.2/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.3/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.3/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.3/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.4/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.4/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.4/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.5/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.5/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.5/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.6/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.6/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.6/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.7/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.7/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.7/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.8/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.8/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.8/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.9/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.9/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.9/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.10/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.10/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.10/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.11/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.11/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.11/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.12/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.12/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.12/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.13/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.13/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.13/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.14/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.14/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.14/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.15/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.15/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.15/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.16/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.16/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.16/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.17/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.17/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.17/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.18/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.18/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.18/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.19/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.19/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.19/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.20/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.20/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.20/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.21/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.21/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.21/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.22/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer3/layer3.22/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer3/layer3.22/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,509,949,440 ops (754,974,720 MACs) - /layer4/layer4.0/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer4/layer4.0/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer4/layer4.0/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,509,949,440 ops (754,974,720 MACs) - /layer4/layer4.0/downsample/downsample.0/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer4/layer4.1/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer4/layer4.1/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer4/layer4.1/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer4/layer4.2/conv1/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 1,698,693,120 ops (849,346,560 MACs) - /layer4/layer4.2/conv2/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 754,974,720 ops (377,487,360 MACs) - /layer4/layer4.2/conv3/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /input_proj/Conv_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /encoder/layers.0/self_attn/out_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.0/fc1/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.0/fc2/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /encoder/layers.1/self_attn/out_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.1/fc1/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.1/fc2/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /encoder/layers.2/self_attn/out_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.2/fc1/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.2/fc2/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /encoder/layers.3/self_attn/out_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.3/fc1/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.3/fc2/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /encoder/layers.4/self_attn/out_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.4/fc1/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.4/fc2/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /encoder/layers.5/self_attn/out_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.5/fc1/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 377,487,360 ops (188,743,680 MACs) - /encoder/layers.5/fc2/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.0/encoder_attn/k_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.0/encoder_attn/v_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.1/encoder_attn/k_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.1/encoder_attn/v_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.2/encoder_attn/k_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.2/encoder_attn/v_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.3/encoder_attn/k_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.3/encoder_attn/v_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.4/encoder_attn/k_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.4/encoder_attn/v_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.5/encoder_attn/k_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - conv2d: 47,185,920 ops (23,592,960 MACs) - /decoder/layers.5/encoder_attn/v_proj/MatMul_quant
2026-07-26 21:26 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-07-26 21:26 - INFO - epu - mac_counter - Total: 120,359,485,440 ops (60,179,742,720 MACs)
2026-07-26 21:26 - INFO - epu - mac_counter - ============================================================
2026-07-26 21:26 - INFO - epu - mac_counter -
2026-07-26 21:27 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-07-26 21:27 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-07-26 21:29 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-07-26 21:29 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-07-26 21:31 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-07-26 21:31 - INFO - epu - codegen - START================================lrm_splitting
2026-07-26 21:34 - INFO - epu - codegen - START==============================ext_split_relay
2026-07-26 21:36 - INFO - epu - codegen - File copied from /quadric/sdk-cli/examples/models/detr/e2e/detr_attention_core.hpp to /quadric/sdk-cli/examples/models/detr/e2e/ccl_build/detr_full_customop_QC_U_1d56_8MB_4kB_64GBps_64GBps_16_OFF_x1_x1/attention_stubs.hpp
2026-07-26 21:36 - INFO - epu - codegen - START====================================build_tir
2026-07-26 21:37 - INFO - epu - chimera_job - Compilation of detr_full_customop_QC_U_1d56_8MB_4kB_64GBps_64GBps_16_OFF_x1_x1 successful
DETR compiled as one unit: backbone + input projection + encoder + decoder + heads
7. Run on ISS + validate (ISS vs ORT)
Run the compiled unit on the ISS, then report, in the cells below: the run's profiling and box-level IoU agreement against the same quantized graph on ONNX Runtime (run_iou_validation, boxes matched by IoU + class label).
val_img = image_paths[0]
img = mh.load_image(str(val_img)).astype(np.float32)
print(f"Validating with image: {val_img.name} shape={img.shape}")
## ISS on the custom-op graph; ORT reference on the quantized full graph (keyed to the ISS
## output names so the downstream IoU/visualization line up per output).
iss_out = job.run_inference_harness(inputs={"pixel_values": img}, compare_ort=False)
sess = ort.InferenceSession(QMODEL, providers=["CPUExecutionProvider"])
ort_out = dict(zip(iss_out.keys(), sess.run(None, {"pixel_values": img})))
## Job summary, includes the ISS run's profiling.
print(job)
Validating with image: 000000000139.jpg shape=(1, 3, 384, 960)
2026-07-26 21:37 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x77610ce3a890>
FILM 188/188: 100%|███████████████████████████████████████████████| 188/188 [05:07<00:00, 1.63s/it]
╒═════════════════════╤═════════════════════════════════════════════════════════════════╕
│ Module Name │ detr_full_customop_QC_U_1d56_8MB_4kB_64GBps_64GBps_16_OFF_x1_x1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ ONNX File │ onnx/detr_full_customop.onnx │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.560 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ External Read BW │ 64GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ External Write BW │ 64GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Max L2M │ 7.941MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Max LRM │ 3.000kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 15.469MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 61.987 │
╘═════════════════════╧═════════════════════════════════════════════════════════════════╛
╒════╤════════╤══════════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪══════════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ pixel_values │ [1, 3, 384, 960] │ tensor[FixedPoint32<29>] │ n/a │
├────┼────────┼──────────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ pred_logits │ [1, 100, 92] │ tensor[FixedPoint32<26>] │ n/a │
├────┼────────┼──────────────┼──────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ pred_boxes │ [1, 100, 4] │ tensor[FixedPoint32<31>] │ n/a │
╘════╧════════╧══════════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.56 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 16.58 │
├──────────────────────────────────┼─────────┤
│ FPS │ 60.30 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 1944.36 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 31.01 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 103.67 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 31.37 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 6.10 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 1.85 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 14.62% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 25,869,661
[SDK-CLI] : Executions/second: 60.30
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 10.175M
data_array : ▇▇▇▇▇▇▇▇ 1.745M
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 6.722M
data_external: ▇▇▇▇▇▇▇▇ 1.816M
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 5.265M
for more information check run directory: /quadric/sdk-cli/examples/models/detr/e2e/ccl_build/detr_full_customop_QC_U_1d56_8MB_4kB_64GBps_64GBps_16_OFF_x1_x1/build
iou_metrics = iou_validation.run_iou_validation(
ort_out=ort_out,
iss_out=iss_out,
img_w=IMG_W,
img_h=IMG_H,
pass_threshold=80.0,
)
assert iou_metrics[
"passed"
], f"IoU regression: ISS-vs-ORT agreement {iou_metrics['match_pct']:.1f}% < 80% threshold"
DETR IoU validation (top-10 by score, IoU>0.5)
ORT detections : 10
ISS detections : 10
Matched : 8/10
Mean IoU : 0.8484
[PASS] ISS-vs-ORT agreement = 80.00% (threshold 80%)
8. Visualize detections (ORT vs ISS)
Draw the decoded detections side-by-side — ORT reference (left) vs Chimera GPNPU / ISS (right) — on the validation image, mirroring the YoloX e2e notebook, and save detr_e2e_detections.png. Each panel shows the top-10 boxes above the score threshold with class + score; boxes that line up across the two panels indicate ISS-vs-ORT agreement.
## Side-by-side ORT reference vs Chimera GPNPU (ISS) detections on the validation image, saved to a
## PNG -- same style as the YoloX e2e notebook (compare_detections).
detr_viz.compare_detections(
iss_out,
ort_out,
str(val_img),
img_w=IMG_W,
img_h=IMG_H,
top_k=10,
score_thr=0.3,
out_png="detr_e2e_detections.png",
)
Saved detr_e2e_detections.png
Summary
| Model | DETR-R101 (ResNet-101 backbone, COCO, 384×960, ~60 M params) |
| Pipeline | full graph quantized once and compiled as one end-to-end Chimera unit |
| Target | QC-U, 8 MB OCM (On-Chip Memory), 4 kB LRM, 16 MACs/PE, 1.56 GHz |
| Quantization | symmetric INT8, one COCO calibration pass |
| Validation | box IoU: ISS unit vs full-graph ONNX Runtime |
Key takeaways
- DETR runs on the Chimera GPNPU as a full-graph compilation, end-to-end:
pixel_values→pred_logits,pred_boxes. - One INT8 calibration pass over the whole graph keeps quantization simple and matches the full graph in ONNX Runtime.
- The ResNet backbone and the transformer encoder/decoder compile together as one unit — no split, no boundary handoff.
'%3e%3crect%20width='512'%20height='512'%20rx='60'%3e%3c/rect%3e%3cpath%20fill='%23fff'%20d='M355.904%20100H408.832L293.2%20232.16L429.232%20412H322.72L239.296%20302.928L143.84%20412H90.8805L214.56%20270.64L84.0645%20100H193.28L268.688%20199.696L355.904%20100ZM337.328%20380.32H366.656L177.344%20130.016H145.872L337.328%20380.32Z'%3e%3c/path%3e%3c/g%3e%3cdefs%3e%3cclipPath%20id='clip0_84_15698'%3e%3crect%20width='512'%20height='512'%20fill='%23fff'%3e%3c/rect%3e%3c/clipPath%3e%3c/defs%3e%3c/svg%3e)
