Chimera Software User GuideTutorials & Model DemosModel DemosModel Demo: RetinaNet End-to-End Detection

Model Demo: RetinaNet End-to-End Detection

NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/retina_net/retina_net_e2e_chipy.ipynb.

RetinaNet End-to-End on Chimera GPNPU

This notebook runs the entire RetinaNet detection pipeline on the Chimera GPNPU — image preprocessing, backbone inference, anchor decoding, and NMS — in a single compiled program.

Why end-to-end?

Most detection demos split the pipeline: the backbone runs on the accelerator while postprocessing (decode + NMS) runs on the host CPU. Some even split preprocessing to the host. This creates round-trip penalties and leaves the CPU as a bottleneck.

Here we use ChiPy to compose CCL preprocessing kernels (crop, resize, normalize) with the CGC-compiled backbone and a CCL postprocessing kernel (topK + sigmoid + anchor decode + NMS). The result is a single GPNPU program with zero host intervention from raw image to final detections.

Pipeline

RetinaNet E2E Pipeline

Model: RetinaNet (ResNet-50 + FPN, COCO 80-class, 480x640, torchvision)

1. Setup

from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
from PIL import Image

from examples.models.retina_net.retina_net_chipy import (
    retinanet_end_to_end,
    run_end_to_end,
)
from examples.models.retina_net.retina_net_helpers import class_names, visualize

%matplotlib inline

2. Model

We use retinanet-cut.onnx — a pre-quantized RetinaNet backbone (ResNet-50 + FPN) cut at the 10 FPN head outputs (5 classification + 5 regression heads), quantized to asymmetric int8.

The model is already checked into this directory. For details on how to export, quantize, and cut the model, see the original retina_net.ipynb notebook in this directory's git history.

model_path = Path("retinanet-cut.onnx")
assert model_path.exists(), f"Model not found: {model_path}"
print(f"Model: {model_path} ({model_path.stat().st_size / 1e6:.1f} MB)")

Model: retinanet-cut.onnx (57.2 MB)

3. The CCL Postprocessing Kernel

The decode + NMS pipeline is implemented as a single CCL kernel in retinanet_postprocess.hpp.

Stage	What happens
topK per head	`nn::topK` selects the top-1000 scoring proposals from each FPN head
Dequant + sigmoid	`nn::dequantizeLinear` + `nn::sigmoid` converts int8 scores to Q0.15 probabilities
Anchor decode	`decodeBoxes` maps regression offsets to absolute coordinates using pre-computed anchors
Class-aware NMS	`nn::nms<100>` suppresses overlapping detections per-class, all in OCM

All 5 heads are processed sequentially (1000 proposals each), with temporary buffers freed between heads. The 5x1000 decoded proposals, scores, and classes fit in OCM for the final NMS pass.

Preprocessing kernels

Unlike YOLOX E2E, RetinaNet also runs preprocessing on-chip:

Kernel	CCL function	What it does
crop	`image::crop`	Center-crop to 4:3 aspect ratio
resize	`image::resizeImage`	Bilinear interpolation to 480x640
normalize	`image::channelNorm`	ImageNet mean/std normalization

hpp = Path("retinanet_postprocess.hpp").read_text()

## Show the main postprocess function signature and NMS call
print("// === retinanetPostprocess: topK + sigmoid + decode + NMS ===")
nms_start = hpp.find("// nms")
nms_end = hpp.find("ocmMem.free(ocmOutputClasses);") + len("ocmMem.free(ocmOutputClasses);")
print(hpp[nms_start:nms_end].strip())

print("\n// === anchor-based box decode ===")
dec_start = hpp.find("INLINE void decodeBoxes")
dec_end = hpp.find("writeFlow.write(qBoxOut);") + len("writeFlow.write(qBoxOut);")
## Just show the coordinate math
math_start = hpp.find("qVar_t<FxPt32_20> qWidth", dec_start)
print(hpp[math_start:dec_end].strip())

// === retinanetPostprocess: topK + sigmoid + decode + NMS ===
// nms
  // Make sure the clear output scores with zeros. Otherwise, we may get false
  // positives if there are not enough detections raued out to fill the output
  // tensor.
  fillOcmTensor(ocmOutputScores, 0);
  EmptyType ocmOutputIdx;
  nn::nms<100>(ocmBoxes, ocmScores, ocmClasses, ocmOutputBoxes, ocmOutputScores,
               ocmOutputClasses, ocmOutputIdx, iou_threshold, score_threshold,
               ocmMem);

  memCpy(ocmOutputBoxes, ddrOutputBoxes);
  memCpy(ocmOutputScores, ddrOutputScores);
  memCpy(ocmOutputClasses, ddrOutputClasses);
  ocmMem.free(ocmScores);
  ocmMem.free(ocmBoxes);
  ocmMem.free(ocmClasses);
  ocmMem.free(ocmOutputBoxes);
  ocmMem.free(ocmOutputScores);
  ocmMem.free(ocmOutputClasses);

// === anchor-based box decode ===
qVar_t<FxPt32_20> qWidth =
        qBaseAnchors[qAnchorId * 4 + 2] - qBaseAnchors[qAnchorId * 4] + 1;
    qVar_t<FxPt32_20> qHeight =
        qBaseAnchors[qAnchorId * 4 + 3] - qBaseAnchors[qAnchorId * 4 + 1] + 1;

    qVar_t<FxPt32_20> qCtrX =
        qBaseAnchors[qAnchorId * 4] + qX * strideWidth + (qWidth / 2);
    qVar_t<FxPt32_20> qCtrY =
        qBaseAnchors[qAnchorId * 4 + 1] + qY * strideHeight + (qHeight / 2);

    qVar_t<FxPt32_20> qPredCtrX = FxPt32_20(qRelCodes[0]) * qWidth + qCtrX;
    qVar_t<FxPt32_20> qPredCtrY = FxPt32_20(qRelCodes[1]) * qHeight + qCtrY;
    qVar_t<FxPt32_20> qPredW =
        math::exp<inFracBits, math::ExpMethod::REMEZ, 20>(qRelCodes[2]) *
        qWidth;
    qVar_t<FxPt32_20> qPredH =
        math::exp<inFracBits, math::ExpMethod::REMEZ, 20>(qRelCodes[3]) *
        qHeight;

    qBoxOut[0] = math::clip<OutT>(qPredCtrX - (qPredW / 2), FxPt32_20(0),
                                  FxPt32_20(imageW - 1));
    qBoxOut[1] = math::clip<OutT>(qPredCtrY - (qPredH / 2), FxPt32_20(0),
                                  FxPt32_20(imageH - 1));
    qBoxOut[2] = math::clip<OutT>(qPredCtrX + (qPredW / 2) - 1, FxPt32_20(0),
                                  FxPt32_20(imageW - 1));
    qBoxOut[3] = math::clip<OutT>(qPredCtrY + (qPredH / 2) - 1, FxPt32_20(0),
                                  FxPt32_20(imageH - 1));
    writeFlow.write(qBoxOut);

4. ChiPy End-to-End Pipeline

The full pipeline is defined in retina_net_chipy.py. ChiPy composes 4 CCL custom ops (crop, resize, normalize, postprocess) with the CGC-compiled backbone into a single GPNPU program:

@chipy.func()
def retinanet_end_to_end(model_path, inp, ...):
    cropped = crop(inp, target_height, target_width)
    resized = resize_image(cropped, target_height, target_width)
    normalized = normalize(resized, dataset_mean, dataset_std)
    model_outp = chipy.infer_onnx(model_path, normalized)
    return retinanet_postprocess(model_outp[0], ..., base_anchors, ...)

Each @chipy.ccl_custom_op has a Python body (CPU reference) and maps to a C++ CCL kernel in retinanet_postprocess.hpp. The @chipy.func chains them all.

## Show the ChiPy pipeline source directly from the file
import re

src = Path("retina_net_chipy.py").read_text()

## Extract the retinanet_end_to_end function
match = re.search(
    r"(@chipy\.func\(\)\ndef retinanet_end_to_end\b.*?)(?=\n\ndef |\nif __name__|\Z)",
    src,
    re.DOTALL,
)
if match:
    print("# --- retinanet_end_to_end ---")
    print(match.group(1).rstrip())

## Show the custom op decorators (just signatures)
print("\n# --- CCL custom ops ---")
for op_match in re.finditer(r"(@chipy\.ccl_custom_op\([^)]+\)\ndef \w+\([^)]+\):)", src):
    print(op_match.group(1))
    print("    ...")
    print()

## --- retinanet_end_to_end ---
@chipy.func()
def retinanet_end_to_end(
    model_path,
    inp,
    dataset_mean,
    dataset_std,
    base_anchors,
    scale0,
    scale1,
    scale2,
    scale3,
    scale4,
    iou_threshold,
    score_threshold,
    target_height,
    target_width,
):  # noqa: D103
    #########################################################
    # We create a ChiPy function to combine the retinanet backbone
    # with our postprocessing op into one CCL kernel.
    #
    # ChiPy embeds compile-time constants in the generated IR. Use chipy.const()
    # for tensor constants (like dataset_mean, dataset_std, base_anchors).
    # Python scalars (like scale0-4, iou_threshold, score_threshold) are
    # automatically converted to constants. ChiPy chooses the correct amount
    # of FixedPoint fractional bits for any floating point constants.
    #########################################################
    # preprocessing steps
    cropped = crop(inp, target_height, target_width)
    resized = resize_image(cropped, target_height, target_width)
    normalized = normalize(resized, dataset_mean, dataset_std)

    # run model
    model_outp = chipy.infer_onnx(model_path, normalized)

    return retinanet_postprocess(
        model_outp[0],
        model_outp[1],
        model_outp[3],
        model_outp[4],
        model_outp[2],
        model_outp[5],
        model_outp[6],
        model_outp[8],
        model_outp[9],
        model_outp[7],
        base_anchors,
        scale0,
        scale1,
        scale2,
        scale3,
        scale4,
        iou_threshold,
        score_threshold,
    )

## --- CCL custom ops ---
@chipy.ccl_custom_op(
    frac_bits=[0],
    ccl_func_name="crop",
    reserved_l2m=0,
    reserved_ext=0,
)
def crop(input_image, target_height, target_width):
    ...

@chipy.ccl_custom_op(
    frac_bits=[0],
    ccl_func_name="resizeImage",
    reserved_l2m=0,
    reserved_ext=0,
)
def resize_image(input_image, target_height, target_width):
    ...

@chipy.ccl_custom_op(
    frac_bits=[16],
    ccl_func_name="channelNorm",
    reserved_l2m=0,
    reserved_ext=0,
)
def normalize(input_image, dataset_mean, dataset_std):
    ...

@chipy.ccl_custom_op(
    frac_bits=[20, 31, 0],
    ccl_func_name="retinanetPostprocess",
    reserved_l2m=2500000,  # this value can be lowered in the future.
    reserved_ext=0,
    io_in_ext_mem=True,
)
def retinanet_postprocess(
    cls_head0,
    cls_head1,
    cls_head2,
    cls_head3,
    cls_head4,
    box_head0,
    box_head1,
    box_head2,
    box_head3,
    box_head4,
    base_anchors,
    scale0,
    scale1,
    scale2,
    scale3,
    scale4,
    iou_threshold,
    score_threshold,
):
    ...

5. Compile & Run

We run the pipeline twice:

CPU — numpy preprocessing + ORT backbone + numpy postprocess (float reference)
GPNPU ISS — Fully compiled (CCL preprocessing + CGC backbone + CCL postprocess), with region profiling

run_end_to_end() from retina_net_chipy.py handles both runs and compares results.

import logging

from tvm.relay.backend.contrib.epu.util import logger as tvm_logger

tvm_logger.setLevel(logging.INFO)

image_files = [
    str(Path("../../common/calibration/coco-like/33887522274_eebd074106_k.jpeg")),
]

results = run_end_to_end(image_files, num_images=1)

Inference on ../../common/calibration/coco-like/33887522274_eebd074106_k.jpeg


2026-07-18 12:13 - INFO - epu - codegen - START==================================build_relay
2026-07-18 12:13 - INFO - epu - codegen - START===============================optimize_relay
2026-07-18 12:13 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-07-18 12:13 - INFO - epu - fx - 

Source name                Op                             Output 0 Range                               Output 0 Frac Bits  Output 1 Range    Output 1 Frac Bits    Output 2 Range    Output 2 Frac Bits
-------------------------  -----------------------------  -----------------------------------------  --------------------  ----------------  --------------------  ----------------  --------------------
                           contrib.epu.quadric_custom_op  (-32768.0, 32767.99998474121)                                16  N/A               N/A                   N/A               N/A
output6_DequantizeLinear   contrib.epu.dequantize         (-0.7297837734222412, 0.72408233769238)                      31  N/A               N/A                   N/A               N/A
output7_DequantizeLinear   contrib.epu.dequantize         (-0.6624605059623718, 0.6572850332595408)                    31  N/A               N/A                   N/A               N/A
output8_DequantizeLinear   contrib.epu.dequantize         (-0.6554427742958069, 0.6503221276216209)                    31  N/A               N/A                   N/A               N/A
output9_DequantizeLinear   contrib.epu.dequantize         (-0.6816837191581726, 0.6763580651022494)                    31  N/A               N/A                   N/A               N/A
output10_DequantizeLinear  contrib.epu.dequantize         (-0.5943658947944641, 0.5897224112413824)                    31  N/A               N/A                   N/A               N/A
                           contrib.epu.quadric_custom_op  (-2048.0, 2047.9999990463257)                                20  N/A               31                    N/A               0

2026-07-18 12:13 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-07-18 12:13 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-07-18 12:13 - INFO - epu - codegen - START=================================relay_to_tir
2026-07-18 12:13 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-07-18 12:13 - INFO - epu - codegen - START==============================adapt_and_order
2026-07-18 12:13 - INFO - epu - mac_counter - 
2026-07-18 12:13 - INFO - epu - mac_counter - ============================================================
2026-07-18 12:13 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-07-18 12:13 - INFO - epu - mac_counter - ============================================================
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,445,068,800 ops (722,534,400 MACs) - Conv_0_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 157,286,400 ops (78,643,200 MACs) - Conv_4_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_7_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_10_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_12_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_16_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_19_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_22_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_26_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_29_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_32_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,258,291,200 ops (629,145,600 MACs) - Conv_36_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_39_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_42_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,258,291,200 ops (629,145,600 MACs) - Conv_44_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_48_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_51_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_54_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_58_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_61_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_64_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_68_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_71_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_74_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,258,291,200 ops (629,145,600 MACs) - Conv_78_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_81_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_84_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,258,291,200 ops (629,145,600 MACs) - Conv_86_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_90_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_93_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_96_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_100_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_103_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_106_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_110_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_113_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_116_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_120_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_123_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_126_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_130_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_133_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_136_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_140_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_143_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_146_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_150_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_153_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_156_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_160_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_163_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_166_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_170_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_173_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_176_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_180_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_183_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_186_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_190_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_193_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_196_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_200_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_203_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_206_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_210_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_213_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_216_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_220_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_223_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_226_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_230_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_233_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_236_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_240_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_243_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_246_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_250_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_253_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_256_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_260_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_263_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_266_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_270_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_273_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_276_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_280_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_283_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_286_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_290_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_293_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_296_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_300_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_303_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_306_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,258,291,200 ops (629,145,600 MACs) - Conv_310_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_313_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_316_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,258,291,200 ops (629,145,600 MACs) - Conv_318_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_322_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_325_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_328_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_332_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_335_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_338_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 314,572,800 ops (157,286,400 MACs) - Conv_342_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 629,145,600 ops (314,572,800 MACs) - Conv_343_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,258,291,200 ops (629,145,600 MACs) - Conv_346_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_352_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_355_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_357_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_359_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_361_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 15,925,248,000 ops (7,962,624,000 MACs) - Conv_363_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_353_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_364_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_366_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_368_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_370_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 3,981,312,000 ops (1,990,656,000 MACs) - Conv_372_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_354_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_373_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_375_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_377_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_379_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 995,328,000 ops (497,664,000 MACs) - Conv_381_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 754,974,720 ops (377,487,360 MACs) - Conv_349_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_382_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_384_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_386_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_388_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 265,420,800 ops (132,710,400 MACs) - Conv_390_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_351_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_391_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_393_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_395_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_397_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 66,355,200 ops (33,177,600 MACs) - Conv_399_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_400_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_402_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_404_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 5,662,310,400 ops (2,831,155,200 MACs) - Conv_406_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 796,262,400 ops (398,131,200 MACs) - Conv_408_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_409_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_411_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_413_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 1,415,577,600 ops (707,788,800 MACs) - Conv_415_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 199,065,600 ops (99,532,800 MACs) - Conv_417_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_418_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_420_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_422_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 353,894,400 ops (176,947,200 MACs) - Conv_424_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 49,766,400 ops (24,883,200 MACs) - Conv_426_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_427_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_429_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_431_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 94,371,840 ops (47,185,920 MACs) - Conv_433_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 13,271,040 ops (6,635,520 MACs) - Conv_435_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_436_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_438_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_440_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 23,592,960 ops (11,796,480 MACs) - Conv_442_quant
2026-07-18 12:13 - INFO - epu - mac_counter -   conv2d: 3,317,760 ops (1,658,880 MACs) - Conv_444_quant
2026-07-18 12:13 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-07-18 12:13 - INFO - epu - mac_counter - Total: 188,608,020,480 ops (94,304,010,240 MACs)
2026-07-18 12:13 - INFO - epu - mac_counter - ============================================================
2026-07-18 12:13 - INFO - epu - mac_counter - 
2026-07-18 12:13 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-07-18 12:13 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-07-18 12:14 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-07-18 12:14 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-07-18 12:15 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-07-18 12:15 - INFO - epu - codegen - START================================lrm_splitting
2026-07-18 12:17 - INFO - epu - codegen - START==============================ext_split_relay
2026-07-18 12:21 - INFO - epu - codegen - START====================================build_tir
2026-07-18 12:22 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x75f623fa0250>
2026-07-18 12:22 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x75f623fa0250>
2026-07-18 12:22 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x75f623fa0250>
2026-07-18 12:22 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x75f623fa0250>
/usr/local/lib/python3.10/dist-packages/tvm/relay/backend/contrib/epu/iss_testing.py:119: RuntimeWarning: invalid value encountered in cast
  input_tensor = (input_tensor * (1 << epu_fx_util.frac_bits_from_range(trange))).astype(
2026-07-18 12:22 - INFO - epu - iss_testing - Started Running Graph on Chimera ISS...
2026-07-18 12:27 - INFO - epu - iss_testing - Done 0:05:13.492931


5 GPNPU # boxes out
[[0.18964843 0.60807294 0.35078126 0.77421874]
 [0.86503905 0.5989583  0.8990234  0.7348958 ]
 [0.44941407 0.40807292 0.6419922  0.72760415]
 [0.8298828  0.59583336 0.8611328  0.72760415]
 [0.8023437  0.5932292  0.8324219  0.7299479 ]] GPNPU BOXES
[0.8920593  0.8304138  0.79681396 0.7125244  0.7125244 ] GPNPU scores
[2 0 5 0 0] GPNPU classes
5 CPU # boxes out
[[0.86503494 0.5989798  0.8991488  0.7349063 ]
 [0.8023225  0.5930594  0.8322307  0.7305687 ]
 [0.8291357  0.5951826  0.8608902  0.7295444 ]
 [0.18977667 0.6068192  0.35089526 0.7739308 ]
 [0.44950995 0.40818793 0.6420829  0.7278008 ]] CPU Boxes
[0.8304184  0.7352141  0.7125434  0.89207125 0.79683614] CPU scores
[0 0 0 2 5] CPU classes

6. Visualization

Side-by-side comparison of CPU (float reference) vs GPNPU (fully on-chip) detections.

for result in results:
    fig = visualize(result)
    print(f"Mean IoU (CPU vs GPNPU): {result['mean_iou']:.4f}")

Mean IoU (CPU vs GPNPU): 0.9828

Summary


Model	RetinaNet (ResNet-50 + FPN, 480x640, COCO 80-class, torchvision)
Pipeline	Crop + Resize + Normalize (CCL) + Backbone (CGC) + topK + Decode + NMS (CCL)
Target	Chimera GPNPU (32 cores, 8 MACs/PE, OCM)
Quantization	Asymmetric int8 activations, int8 weights
Custom Ops	`crop`, `resizeImage`, `channelNorm`, `retinanetPostprocess`

Key takeaways

Preprocessing on-chip eliminates host-side image manipulation entirely
ChiPy composes 4 CCL kernels + 1 CGC backbone into a single GPNPU program
Anchor-based decode with nn::topK + nn::sigmoid + nn::nms runs fully in OCM
End-to-end on GPNPU means raw uint8 image in, final detections out — zero host round-trips

Citation

@inproceedings{lin2017focal,
  title     = {Focal Loss for Dense Object Detection},
  author    = {Lin, Tsung-Yi and Goyal, Priya and Girshick, Ross and He, Kaiming and Doll{\'a}r, Piotr},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  year      = {2017}
}

RetinaNet End-to-End on Chimera GPNPU Why end-to-end?Pipeline 1. Setup 2. Model 3. The CCL Postprocessing Kernel Preprocessing kernels 4. ChiPy End-to-End Pipeline 5. Compile & Run 6. Visualization Summary Key takeaways Citation

Model Demo: Centernet Detection

Model Demo: Blazepose Pose Estimation

Table of Contents

Home

What's New

Chimera Software User Guide

Introduction to the Chimera SDK

Chimera SDK Quick Start Guide

Chimera SDK Command Line Interface (CLI)

CGC API Reference

Tutorial: Using SDK as a Library

Tutorials & Model Demos

Quantization Tutorials

Custom Op Tutorials

SDK Tutorials

Model Demos

Model Demo: Llama-2 15M (Baby Llama-2)

Model Demo: QWEN3 8B End-to-End CGC and ISS Execution

Model Demo: QWEN3 Prefill All Decoders

Model Demo: DeepSeek-R1-Distill-Qwen-1.5B End-to-End CGC and ISS Execution

Model Demo: QWEN3 Single Decoder

Model Demo: Qwen2.5-0.5B INT8 Quantization Pipeline

Model Demo: ConvNeXt Detection

Model Demo: QWEN3 Prefill Decoder Validation

Model Demo: ConvNeXt Segmentation

Model Demo: Classifiers Zoo

Model Demo: Detectors Zoo - MMDetection

Model Demo: Segmentors Zoo - MMSegmentation

Model Demo: Pose Estimators Zoo - MMPose

Model Demo: Detectors3D Zoo - MMDetection3D

MODEL Demo: Optical Character Recognition (OCR) Zoo - MMOCR

Model Demo: YOLOv3 Object Detection

Model Demo: YOLOv4 Object Detection

Model Demo: YOLOv5 Detection

Model Demo: YOLOv5 Detection and Segmentation

Model Demo: YOLOR Detection

Model Demo: YOLOX End-to-End Detection

Model Demo: YOLOv7 Detection

Model Demo: YOLOv8 Detection

Model Demo: YOLOv8 Pose Estimation

Model Demo: YOLOP Detection and Segmentation

Model Demo: QAT Vision Transformer (ViT)

Model Demo: QAT Swin Transformer

Model Demo: Mediapipe Face Pipeline

Demo: DOOM Renderer on Chimera GPNPU

Model Demo: Mediapipe Hand Pipeline

Model Demo: Whisper Tiny (Encoder + Decoder)

Model Demo: L2CS Fine-Grained Gaze Estimation

Model Demo: ASVspoof2021 LA Anti-Spoofing (LFCC-LCNN-BiLSTM)

Model Demo: UNET Tumor Segmentation

Model Demo: DETR Encoder

Model Demo: FFNet Segmentation

Model Demo: Centernet Detection

Model Demo: RetinaNet End-to-End Detection

Model Demo: Blazepose Pose Estimation

Model Demo: Pose Resnet Human Pose Estimation

Model Demo: MaskRCNN Detection and Segmentation

Model Demo: Keypoint R-CNN

Model Demo: Faster R-CNN Detection

Model Demo: FCOS Detection

Model Demo: DDRNet Classificationls

Model Demo: PI0.5 End-to-End VLA Inference

Model Demo: BEVFormer End-to-End 3D Detection

Model Demo: SegFormer Semantic Segmentation

Model Demo: DETR Object Detection

Multicore Demo

Chimera Graph Compiler (CGC)

Chimera Compute Library (CCL) API

Chimera LLVM C++ Compiler

Chimera Instruction Set Simulator (ISS)

Debugging

Chimera SDK Licensing Policy Documentation

Glossary

Datasheet - Chimera QC Series GPNPU Processors

All systems operational

270 East Lane
Burlingame, CA 94010
USA

1-844-GPNPU-00
(1-844-476-7800)

PRODUCTS

COMPANY

NEWS + INSIGHTS

This documentation is preliminary and confidential. It is subject to change. Quadric does not give any warranty express or implied that the contents will be complete or accurate or up to date. The company shall not be liable for any loss, actions, claims, proceedings, demands or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

UH-OH

It looks like you don’t have access to that feature yet

UH-OH

Model Demo: RetinaNet End-to-End Detection

RetinaNet End-to-End on Chimera GPNPU

Why end-to-end?

Pipeline

1. Setup

2. Model

3. The CCL Postprocessing Kernel

Preprocessing kernels

4. ChiPy End-to-End Pipeline

5. Compile & Run

6. Visualization

Summary

Key takeaways

Citation

PRODUCTS

COMPANY

NEWS + INSIGHTS

Sign in to your account