Unlock this feature

This feature isn’t part of your plan yet

Contact sales to get upgraded to the full DevStudio experience.

Unlock this feature

This feature isn't part of your plan yet.

Model Demo: Centernet Detection


NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/centernet/centernet.ipynb.


from sdk_cli.visualizers.centernet import CenternetVisualizer
from sdk_cli.node_builtins.outputs.centernet_visualization import centernet

c = CenternetVisualizer(centernet.LABELS["VOCO"])
Length of classes: 20
import onnx
from sdk_cli.utils import model_helpers
import onnxruntime as ort
import numpy as np
import matplotlib.pyplot as plt

fp_onnx_name = f"centernet_float32.onnx"
from tvm.contrib.epu.chimera_job.quantize import quadric_quantize

images_path = "../../common/calibration/coco-like"
## include quadric's cli helpers and instantiate a module to help

model = onnx.load(fp_onnx_name)
_input = model.graph.input[0]
_input_shape = [dim_value.dim_value for dim_value in _input.type.tensor_type.shape.dim]
print(f"NCHW: {_input_shape}")
dataset_input_size = (_input_shape[-1], _input_shape[-2])  # an (W, H) tuple

dataset_mean = [0.5, 0.5, 0.5]
dataset_std = [0.5, 0.5, 0.5]
mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
NCHW: [1, 3, 544, 544]


/tmp/ipykernel_2637/1708068201.py:21: DeprecationWarning: Call to deprecated class ModelHelper. ('ModelHelper' class is being deprecated. Quadric APIs have been updated to use PyTorch datasets and transforms instead.) -- Deprecated since version 24.01.
  mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
quantize_result = quadric_quantize(fp_onnx_name, 100, mh, images_path)
2026-10-23 00:50 - INFO - epu - quantize - Collecting calibration data
2026-10-23 00:50 - INFO - epu - quantize - Optimized model to opset
2026-10-23 00:50 - INFO - epu - quantize - Converted model to opset 12
2026-10-23 00:50 - INFO - epu - quantize - Saved optimized model to centernet_float32_float32_opt.onnx
2026-10-23 00:50 - INFO - epu - quantize - Input shapes: [1, 3, 544, 544]. Input names: input
2026-10-23 00:50 - INFO - epu - quantize - Output shapes: [[1, 20, 136, 136], [1, 2, 136, 136], [1, 2, 136, 136]]. Output names: ['output', '387', '390']
2026-10-23 00:50 - INFO - epu - quantize - applying calibration data to input: input
2026-10-23 00:50 - INFO - epu - quantize - calibration set size: 6
2026-10-23 00:50 - INFO - epu - quantize - Running real quantization on this input: input with input shape: [1, 3, 544, 544]
2026-10-23 00:50 - DEBUG - epu - quantize - Full exclusion set for quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-10-23 00:50 - DEBUG - epu - quantize - excl_nodes ['Sigmoid_98']
2026-10-23 00:50 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-10-23 00:50 - INFO - epu - quantize - Quantization done succesfully!
2026-10-23 00:50 - INFO - epu - quantize - ONNX full precision model size: 102.27 MB
2026-10-23 00:50 - INFO - epu - quantize - ONNX quantized model size: 25.65 MB
2026-10-23 00:50 - INFO - epu - quantize - Saved quantized model to /quadric/sdk-cli/examples/models/centernet/centernet_opt_sym_int8_q.onnx
2026-10-23 00:50 - INFO - epu - quantize - Saved shape inferenced model to /quadric/sdk-cli/examples/models/centernet/centernet_opt_sym_int8_q.onnx
2026-10-23 00:50 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16/BFLOAT16 types.
2026-10-23 00:50 - INFO - epu - quantize - Model still has FLOAT/FLOAT16/BFLOAT16 types. Creating ranges for floating point tensors using calibration data


Custom quantization code for ConvTranspose
Custom quantization code for ConvTranspose
Custom quantization code for ConvTranspose


2026-10-23 00:50 - INFO - epu - quantize - Saved tensor ranges to /quadric/sdk-cli/examples/models/centernet/centernet_opt_sym_int8_q.onnx.tranges

Run the heatmap conv in FP16

For CenterNet-style models, the conv directly before the Sigmoid head runs in FP16 and the rest of the model stays INT8. The next cell finds the Sigmoid in the FP32 model, takes the conv that feeds it, and passes that name to convert_to_fp16.

convert_to_fp16 replaces each conv in convs with DequantizeLinear -> Cast(fp16) -> Conv(fp16) -> Cast(fp32), takes the conv weights from the FP32 model, and adds tranges for the new tensors from the calibration images. To run more convs in FP16, add their FP32 node names to convs.

from sdk_cli.utils.fp16_conv import convert_to_fp16

## Find the Sigmoid head in the FP32 model and the conv that feeds it.
fp_model = onnx.load(fp_onnx_name)
producers = {out: node for node in fp_model.graph.node for out in node.output}
sigmoid = next(node for node in fp_model.graph.node if node.op_type == "Sigmoid")
heatmap_conv = producers[sigmoid.input[0]]
print(f"{heatmap_conv.name} ({heatmap_conv.op_type}) -> {sigmoid.name} ({sigmoid.op_type})")

calib_inputs = ({"input": mh.load_image(p)} for p in mh.get_images(images_path))
fp16_model_path, fp16_tranges_path = convert_to_fp16(
    quantize_result.qmodel_path,
    fp_onnx_name,
    quantize_result.tranges_path,
    calib_inputs,
    convs=[heatmap_conv.name],
)
Conv_97 (Conv) -> Sigmoid_98 (Sigmoid)
def run_onnx_all_images(images_path, onnx_name):
    vis_image_list = []
    allimages = mh.get_images(images_path)
    ort_sess = ort.InferenceSession(onnx_name)
    for image_p in allimages:
        image = mh.load_image(image_p)
        onnx_output = ort_sess.run([], {"input": image})
        output_img = CenternetVisualizer.input_img_to_display(image)
        vis_img = c.draw_boxes(onnx_output, output_img, threshold=0.5, scale=4)
        vis_image_list.append(vis_img)
    return vis_image_list


list_fp = run_onnx_all_images(images_path, fp_onnx_name)
print(list_fp)
[<PIL.PngImagePlugin.PngImageFile image mode=RGBA size=640x480 at 0x7D79B49C6BC0>, <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=640x480 at 0x7D79AE8CFEE0>, <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=640x480 at 0x7D79AF05B6A0>, <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=640x480 at 0x7D79AE8FEC20>, <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=640x480 at 0x7D79AEF37610>, <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=640x480 at 0x7D79AEFBD480>]

from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob

cgc_job = ChimeraJob(model_p=fp16_model_path, trange_file=fp16_tranges_path)
cgc_job.compile(quiet=True)
THREADS = 2

all_images = mh.get_images(images_path, THREADS)
all_inputs = [{"input": mh.load_image(image)} for image in all_images]
all_results = cgc_job.run_batch_inference_harness(inputs=all_inputs, threads=THREADS)
Processing: 100%|███████████████████████████████| 2/2 [05:52<00:00, 176.25s/it]
for i, result in enumerate(all_results):
    image = mh.load_image(all_images[i])
    output_img = CenternetVisualizer.input_img_to_display(image)
    list_output = [result["output"], result["387"], result["390"]]
    img = c.draw_boxes(list_output, output_img, threshold=0.5, scale=4)
plt.show()

print(cgc_job)
cgc_job.plot_run_statistics();
╒═════════════════════╤═══════════════════════════════════════════════════════════════════════════════════╕
│ Module Name         │ centernet_opt_sym_int8_q_fp16conv_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1  │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File           │ /quadric/sdk-cli/examples/models/centernet/centernet_opt_sym_int8_q_fp16conv.onnx │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ Product Target      │ QC-U                                                                              │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores     │ 1                                                                                 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700                                                                             │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size            │ 16MB                                                                              │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size            │ 4kB                                                                               │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW    │ 128GBps                                                                           │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW   │ 128GBps                                                                           │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE         │ 16                                                                                │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M             │ 15.268MB                                                                          │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM             │ 2.750kB                                                                           │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes  │ 0.000MB                                                                           │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs       │ 52.861                                                                            │
╘═════════════════════╧═══════════════════════════════════════════════════════════════════════════════════╛

╒════╤════════╤════════╤═══════════════════╤══════════════════════════╤═══════╕
│    │ Type   │ Name   │ shape             │ type                     │ mse   │
╞════╪════════╪════════╪═══════════════════╪══════════════════════════╪═══════╡
│  0 │ Input  │ input  │ [1, 3, 544, 544]  │ tensor[FixedPoint32<30>] │ n/a   │
├────┼────────┼────────┼───────────────────┼──────────────────────────┼───────┤
│  1 │ Output │ output │ [1, 20, 136, 136] │ tensor[FixedPoint32<31>] │ 0.000 │
├────┼────────┼────────┼───────────────────┼──────────────────────────┼───────┤
│  2 │ Output │ 387    │ [1, 2, 136, 136]  │ tensor[FixedPoint32<24>] │ 1.282 │
├────┼────────┼────────┼───────────────────┼──────────────────────────┼───────┤
│  3 │ Output │ 390    │ [1, 2, 136, 136]  │ tensor[FixedPoint32<30>] │ 0.000 │
╘════╧════════╧════════╧═══════════════════╧══════════════════════════╧═══════╛

Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation: 
╒══════════════════════════════════╤═════════╕
│ Latency (ms)                     │ 5.54    │
├──────────────────────────────────┼─────────┤
│ FPS                              │ 180.43  │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW)   │ 3202.37 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 56.34   │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB)                │ 29.10   │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB)                │ 1.69    │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps)             │ 5.13    │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps)             │ 0.30    │
├──────────────────────────────────┼─────────┤
│ MAC Utilization                  │ 34.24%  │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm

[SDK-CLI] : TotalCycles: 9,421,878
[SDK-CLI] : Executions/second: 180.43

compute      : ▇▇▇▇▇▇▇▇ 1.233M
data_array   : ▇▇▇ 437.671K
mac          : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 7.041M
data_external: ▏ 68.145K
data_ocm     : ▇▇▇ 525.516K

for more information check run directory: /quadric/sdk-cli/examples/models/centernet/ccl_build/centernet_opt_sym_int8_q_fp16conv_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20261023_005212_5532ce


2026-10-23 00:58 - INFO - epu - chimera_job - Combined plots generated and saved to: 
/quadric/sdk-cli/examples/models/centernet/ccl_build/centernet_opt_sym_int8_q_fp16conv_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20261023_005212_5532ce/data/centernet_opt_sym_int8_q_fp16conv_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png

## Run the full INT8 model on one image: CI tracks its cycles, and the next cell compares it.
int8_job = ChimeraJob(model_p=quantize_result.qmodel_path, trange_file=quantize_result.tranges_path)
int8_job.compile(quiet=True)
int8_result = int8_job.run_inference_harness(inputs=all_inputs[0], quiet=True)
2026-10-23 00:58 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x7d79b4aa4190>
2026-10-23 01:01 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x7d798c8d1fc0>
2026-10-23 01:01 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-10-23 01:01 - INFO - epu - iss_testing - Done 0:00:01.098587

Compare with ONNX Runtime FP32

The next cell runs the FP32 model and the INT8 model on ONNX Runtime for the same images, and computes the cosine similarity of each output with the FP32 output. output is the class heatmap after the Sigmoid, 387 the box sizes and 390 the center offsets. The ISS INT8 row is the first image only.

The check passes when, on every image, the ISS FP16-conv heatmap cosine is at least HEATMAP_MIN_COSINE and the box size and offset cosines are at least REGRESSION_MIN_COSINE.

On this model the INT8 and FP16-conv runs agree with FP32 about equally. The next part uses a model where INT8 loses accuracy in the heatmap and the FP16 conv recovers it.

HEATMAP_MIN_COSINE = 0.95
REGRESSION_MIN_COSINE = 0.995
OUTPUT_NAMES = ["output", "387", "390"]


def cosine(first, second):
    first = np.asarray(first, dtype=np.float64).ravel()
    second = np.asarray(second, dtype=np.float64).ravel()
    return float(first @ second / (np.linalg.norm(first) * np.linalg.norm(second)))


fp32_sess = ort.InferenceSession(fp_onnx_name)
int8_sess = ort.InferenceSession(quantize_result.qmodel_path)
rows = []
for i, (inputs, result) in enumerate(zip(all_inputs, all_results)):
    ref = dict(zip(OUTPUT_NAMES, fp32_sess.run(OUTPUT_NAMES, inputs)))
    ort_int8 = dict(zip(OUTPUT_NAMES, int8_sess.run(OUTPUT_NAMES, inputs)))
    rows.append((f"image {i} ISS FP16-conv", [cosine(result[n], ref[n]) for n in OUTPUT_NAMES]))
    rows.append((f"image {i} ORT INT8", [cosine(ort_int8[n], ref[n]) for n in OUTPUT_NAMES]))
    if i == 0:
        rows.append(("image 0 ISS INT8", [cosine(int8_result[n], ref[n]) for n in OUTPUT_NAMES]))

print(f"{'cosine vs ORT FP32':<26}" + "".join(f"{n:>10}" for n in OUTPUT_NAMES))
for label, values in rows:
    print(f"{label:<26}" + "".join(f"{v:>10.4f}" for v in values))

fp16_rows = [values for label, values in rows if label.endswith("ISS FP16-conv")]
heatmap_min = min(values[0] for values in fp16_rows)
regression_min = min(min(values[1:]) for values in fp16_rows)
assert heatmap_min >= HEATMAP_MIN_COSINE, f"heatmap cosine {heatmap_min:.4f} < {HEATMAP_MIN_COSINE}"
assert (
    regression_min >= REGRESSION_MIN_COSINE
), f"box cosine {regression_min:.4f} < {REGRESSION_MIN_COSINE}"
print(
    f"[PASS] ISS FP16-conv vs ORT FP32: heatmap cosine {heatmap_min:.4f} >= {HEATMAP_MIN_COSINE}, "
    f"box cosine {regression_min:.4f} >= {REGRESSION_MIN_COSINE}"
)
cosine vs ORT FP32            output       387       390
image 0 ISS FP16-conv         0.9581    0.9996    0.9990
image 0 ORT INT8              0.9616    0.9996    0.9990
image 0 ISS INT8              0.9562    0.9996    0.9990
image 1 ISS FP16-conv         0.9906    0.9997    0.9993
image 1 ORT INT8              0.9852    0.9997    0.9993
[PASS] ISS FP16-conv vs ORT FP32: heatmap cosine 0.9581 >= 0.95, box cosine 0.9990 >= 0.995

CenterNet-R18 on COCO: INT8 and FP16 heatmap conv

centernet_r18.onnx is OpenMMLab's CenterNet with a ResNet-18 backbone, trained on COCO (80 classes), with a 512x512 input. centernet_r18_decode.onnx is the box decode from the same export: it takes the three head outputs and returns the top 100 boxes and labels. It runs on ONNX Runtime.

In this model INT8 loses accuracy at the heatmap conv. Its output is quantized to one 8-bit grid, so many detections get the same score and their order is lost. Running that conv in FP16 keeps the scores distinct. The next cells quantize the model, convert the heatmap conv to FP16 with convert_to_fp16, and run both the INT8 and the FP16-conv model on the ISS.

r18_onnx_name = "centernet_r18.onnx"
r18_mh = model_helpers.ModelHelper((512, 512), [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
r18_quant = quadric_quantize(r18_onnx_name, 100, r18_mh, images_path)

r18_model = onnx.load(r18_onnx_name)
r18_producers = {out: node for node in r18_model.graph.node for out in node.output}
r18_sigmoid = next(node for node in r18_model.graph.node if node.op_type == "Sigmoid")
r18_heatmap_conv = r18_producers[r18_sigmoid.input[0]]
print(f"{r18_heatmap_conv.name} ({r18_heatmap_conv.op_type})", end=" -> ")
print(f"{r18_sigmoid.name} ({r18_sigmoid.op_type})")

r18_fp16_path, r18_fp16_tranges_path = convert_to_fp16(
    r18_quant.qmodel_path,
    r18_onnx_name,
    r18_quant.tranges_path,
    ({"input": r18_mh.load_image(p)} for p in r18_mh.get_images(images_path)),
    convs=[r18_heatmap_conv.name],
)
2026-10-23 01:01 - INFO - epu - quantize - Collecting calibration data
2026-10-23 01:01 - INFO - epu - quantize - Optimized model to opset
2026-10-23 01:01 - INFO - epu - quantize - Saved optimized model to centernet_r18_float32_opt.onnx
2026-10-23 01:01 - INFO - epu - quantize - Input shapes: [1, 3, 512, 512]. Input names: input
2026-10-23 01:01 - INFO - epu - quantize - Output shapes: [[1, 80, 128, 128], [1, 2, 128, 128], [1, 2, 128, 128]]. Output names: ['bbox_head_output0', 'bbox_head_output1', 'bbox_head_output2']
2026-10-23 01:01 - INFO - epu - quantize - applying calibration data to input: input
2026-10-23 01:01 - INFO - epu - quantize - calibration set size: 6
2026-10-23 01:01 - INFO - epu - quantize - Running real quantization on this input: input with input shape: [1, 3, 512, 512]
2026-10-23 01:01 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.


Custom quantization code for ConvTranspose
Custom quantization code for ConvTranspose
Custom quantization code for ConvTranspose


2026-10-23 01:01 - INFO - epu - quantize - Quantization done succesfully!
2026-10-23 01:01 - INFO - epu - quantize - ONNX full precision model size: 54.25 MB
2026-10-23 01:01 - INFO - epu - quantize - ONNX quantized model size: 13.64 MB
2026-10-23 01:01 - INFO - epu - quantize - Saved quantized model to /quadric/sdk-cli/examples/models/centernet/centernet_r18_opt_sym_int8_q.onnx
2026-10-23 01:01 - INFO - epu - quantize - Saved shape inferenced model to /quadric/sdk-cli/examples/models/centernet/centernet_r18_opt_sym_int8_q.onnx
2026-10-23 01:01 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16/BFLOAT16 types.
2026-10-23 01:01 - INFO - epu - quantize - Model still has FLOAT/FLOAT16/BFLOAT16 types. Creating ranges for floating point tensors using calibration data
2026-10-23 01:01 - INFO - epu - quantize - Saved tensor ranges to /quadric/sdk-cli/examples/models/centernet/centernet_r18_opt_sym_int8_q.onnx.tranges


/bbox_head/heatmap_head/heatmap_head.2/Conv (Conv) -> /bbox_head/Sigmoid (Sigmoid)
r18_inputs = [{"input": r18_mh.load_image(p)} for p in sorted(r18_mh.get_images(images_path))]
r18_jobs = {
    "ISS INT8": ChimeraJob(model_p=r18_quant.qmodel_path, trange_file=r18_quant.tranges_path),
    "ISS FP16 conv": ChimeraJob(model_p=r18_fp16_path, trange_file=r18_fp16_tranges_path),
}
r18_results = {}
for label, job in r18_jobs.items():
    job.compile(quiet=True)
    r18_results[label] = job.run_batch_inference_harness(inputs=r18_inputs, threads=THREADS)
Processing: 100%|████████████████████████████████| 6/6 [01:24<00:00, 14.08s/it]
Processing: 100%|████████████████████████████████| 6/6 [01:27<00:00, 14.51s/it]

Compare with ONNX Runtime FP32

For each image, the next cell decodes the FP32 ONNX Runtime outputs and the two ISS runs into boxes. The FP32 detections scoring above SCORE_MIN are the reference. For each run it reports:

  • matched: reference detections that have a detection of the same class at IoU above 0.5;
  • levels: the number of distinct heatmap scores at the FP32 heatmap pixels above SCORE_MIN. FP32 has a different score at almost every pixel; a lower count means tied scores.

The check passes when, over all images, the ISS FP16-conv run matches at least MATCH_MIN of the reference detections and has at least LEVELS_RATIO_MIN times as many score levels as the ISS INT8 run.

SCORE_MIN = 0.3
MATCH_MIN = 0.9
LEVELS_RATIO_MIN = 3.0
HEAD_OUTPUTS = ["bbox_head_output0", "bbox_head_output1", "bbox_head_output2"]

decode_sess = ort.InferenceSession("centernet_r18_decode.onnx")
r18_fp32_sess = ort.InferenceSession(r18_onnx_name)


def decode(outputs):
    feeds = {name: np.asarray(out, dtype=np.float32) for name, out in zip(HEAD_OUTPUTS, outputs)}
    dets, labels = decode_sess.run(None, feeds)
    return dets[0], labels[0]


def box_iou(box, boxes):
    top_left = np.maximum(box[:2], boxes[:, :2])
    bottom_right = np.minimum(box[2:4], boxes[:, 2:4])
    inter = np.prod(np.clip(bottom_right - top_left, 0, None), axis=1)
    areas = np.prod(boxes[:, 2:4] - boxes[:, :2], axis=1)
    return inter / (np.prod(box[2:4] - box[:2]) + areas - inter)


def count_matched(ref_dets, ref_labels, dets, labels):
    matched = 0
    for box, label in zip(ref_dets, ref_labels):
        same_class = dets[labels == label]
        if len(same_class) and box_iou(box, same_class).max() > 0.5:
            matched += 1
    return matched


run_labels = ["ORT FP32", *r18_results]
totals = {label: [0, 0] for label in run_labels}
num_ref = 0
print(
    f"{'':<10}{'FP32 dets':>10}"
    + "".join(f"{label + ' matched/levels':>30}" for label in run_labels)
)
for i, inputs in enumerate(r18_inputs):
    fp32_outputs = r18_fp32_sess.run(HEAD_OUTPUTS, inputs)
    ref_dets, ref_labels = decode(fp32_outputs)
    keep = ref_dets[:, 4] > SCORE_MIN
    ref_dets, ref_labels = ref_dets[keep], ref_labels[keep]
    peaks = fp32_outputs[0] > SCORE_MIN
    num_ref += len(ref_dets)
    runs = {"ORT FP32": fp32_outputs}
    for label, results in r18_results.items():
        runs[label] = [results[i][name] for name in HEAD_OUTPUTS]
    row = f"image {i:<4}{len(ref_dets):>10}"
    for label, outputs in runs.items():
        matched = count_matched(ref_dets, ref_labels, *decode(outputs))
        levels = len(np.unique(np.asarray(outputs[0])[peaks]))
        totals[label][0] += matched
        totals[label][1] += levels
        row += f"{f'{matched}/{levels}':>30}"
    print(row)

fp16_matched, fp16_levels = totals["ISS FP16 conv"]
int8_matched, int8_levels = totals["ISS INT8"]
match_fraction = fp16_matched / num_ref
levels_ratio = fp16_levels / int8_levels
print(f"ISS INT8: matched {int8_matched}/{num_ref}, levels {int8_levels}")
print(f"ISS FP16 conv: matched {fp16_matched}/{num_ref}, levels {fp16_levels}")
assert match_fraction >= MATCH_MIN, f"matched {match_fraction:.3f} < {MATCH_MIN}"
assert levels_ratio >= LEVELS_RATIO_MIN, f"levels ratio {levels_ratio:.2f} < {LEVELS_RATIO_MIN}"
print(
    f"[PASS] CenterNet-R18 ISS FP16 conv vs ORT FP32: matched {match_fraction:.3f} >= {MATCH_MIN}, "
    f"score levels {levels_ratio:.2f}x ISS INT8 >= {LEVELS_RATIO_MIN}x"
)
           FP32 dets       ORT FP32 matched/levels       ISS INT8 matched/levels  ISS FP16 conv matched/levels
image 0            1                          1/18                           1/4                          1/18
image 1           17                         17/76                         17/12                         16/76
image 2           12                         12/45                          12/8                         12/45
image 3            8                          8/86                          8/11                          8/86
image 4            5                          5/30                           5/8                          5/30
image 5            2                          2/15                           2/9                          2/15
ISS INT8: matched 45/45, levels 52
ISS FP16 conv: matched 44/45, levels 270
[PASS] CenterNet-R18 ISS FP16 conv vs ORT FP32: matched 0.978 >= 0.9, score levels 5.19x ISS INT8 >= 3.0x

Sign in to your account

Don't have an account? 
By signing in, you are agreeing to our Terms of Use and Privacy Policy.
Quadric // One architecture. Every algorithm.

Develop.

Simulate.

Profile.

Collaborate.