Model Demo: FCOS Detection

NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/fcos/fcos.ipynb.

FCOS: Fully Convolutional One-Stage Object Detection

Abstract

FCOS is an anchor-box free, proposal free, single-stage object detection model. By eliminating the predefined set of anchor boxes, FCOS avoids computation related to anchor boxes such as calculating overlapping during training. It also avoids all hyper-parameters related to anchor boxes, which are often very sensitive to the final detection performance.

The following figure shows the network architecture of FCOS, where C3, C4, and C5 denote the feature maps of the backbone network and P3 to P7 are the feature levels used for the final prediction. H × W is the height and width of feature maps. ‘/s’ (s = 8, 16, ..., 128) is the downsampling ratio of the feature maps at the level to the input image. As an example, all the numbers are computed with an 800 × 1024 input. (Excerpted from the paper)

Further etails can be found in the paper.

• Zhi Tian, Chunhua Shen, Hao Chen, Tong He, FCOS: Fully Convolutional One-Stage Object Detection

Preparation

onnx-graphsurgeon is used surgically operate onnx files. nvidia-pyindex is its dependency.

!pip3 install nvidia-pyindex onnx-graphsurgeon -q

DEPRECATION: The HTML index page being used (https://developer.nvidia.com/w/compute/redist/onnx-graphsurgeon/) is not a proper HTML 5 document. This is in violation of PEP 503 which requires these pages to be well-formed HTML 5 documents. Please reach out to the owners of this index page, and ask them to update this index page to a valid HTML 5 document. pip 22.2 will enforce this behaviour change. Discussion can be found at https://github.com/pypa/pip/issues/10825
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv


import os
import sys
from pathlib import Path
import numpy as np
import torch
import torchvision
from onnxruntime import InferenceSession
import sdk_cli.lib.quantize as quant
import tvm.contrib.epu.chimera_job.constants as sdk_constants
from onnxsim import simplify
import onnx
import onnxsim
import tvm.contrib.epu.graphutils as gutils
import warnings

FILE_PATH = Path(os.path.abspath(""))
sys.path.append(f"{FILE_PATH.parent}")

from yolo.yoloutils.yolohelpers import onnx_check_and_sim, ModelHelperYolo  # noqa: E402

Model Load

At first, the model is loaded from torchvision.

• Model: torchvision.models.detection.fcos_resnet50_fpn
• Weights: torchvision.models.detection.FCOS_ResNet50_FPN_Weights.DEFAULT
  • Trained with COCO dataset with 91 classes

Visit torchvision site for more information.

Loaded model (FCOS) is used later for post-processing.

FCOS = torchvision.models.detection.fcos_resnet50_fpn(
    weights=torchvision.models.detection.FCOS_ResNet50_FPN_Weights.DEFAULT
)

ONNX Export

device = torch.device("cpu")

FCOS.to(device)
FCOS.eval()

onnx_file = "FCOS_ResNet50_FPN.onnx"

model_input_size = (1344, 800)  # width x height
x = torch.randn(1, 3, *model_input_size[::-1], requires_grad=True)

## Export the model
torch.onnx.export(
    FCOS,  # model being run
    x,  # model input (or a tuple for multiple inputs)
    onnx_file,  # where to save the model (can be a file or file-like object)
    export_params=True,  # store the trained parameter weights inside the model file
    do_constant_folding=True,  # whether to execute constant folding for optimization
    opset_version=sdk_constants.DEFAULT_ONNX_OPSET,
    input_names=["inputs0"],  # the model's input names
    output_names=["outputs0", "outputs1", "outputs2"],
)  # the model's output names

onnx_file = onnx_check_and_sim(onnx_file)
print(f"FCOS ONNX is exported as {onnx_file}")

/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py:4624: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  * torch.tensor(scale_factors[i], dtype=torch.float32)
/usr/local/lib/python3.10/dist-packages/torchvision/ops/boxes.py:166: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  boxes_x = torch.min(boxes_x, torch.tensor(width, dtype=boxes.dtype, device=boxes.device))
/usr/local/lib/python3.10/dist-packages/torchvision/ops/boxes.py:168: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  boxes_y = torch.min(boxes_y, torch.tensor(height, dtype=boxes.dtype, device=boxes.device))
/usr/local/lib/python3.10/dist-packages/torchvision/models/detection/transform.py:308: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  torch.tensor(s, dtype=torch.float32, device=boxes.device)
/usr/local/lib/python3.10/dist-packages/torchvision/models/detection/transform.py:309: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  / torch.tensor(s_orig, dtype=torch.float32, device=boxes.device)


simplified onnx is saved as FCOS_ResNet50_FPN-sim.onnx
FCOS ONNX is exported as FCOS_ResNet50_FPN-sim.onnx

Operate on ONNX

The following operations are being done for further experiment.

• Remove transform nodes
• Fix the output boxes number
• Replace the nms with the custom ops
• Exclude GroupNorm from quantization

Remove Transform Nodes

FCOS includes image transformation at the beginnning of this model. It accepts images normalized with the following.

• mean: [0, 0, 0]
• std: [1, 1, 1]
• value range: 0 to 1

These images are transformed into the following in the model.

• mean: [0.485, 0.456, 0.406]
• std: [0.229, 0.224, 0.225]

In this experiment, image transformation nodes are skipped, and image normalization is done outside of the net. Therefore, a model helper (mh_fcos) is defined with mean = [0.485, 0.456, 0.406] and std = [0.485, 0.456, 0.406].

model = onnx.load(onnx_file)

input_edges = ["/transform/Unsqueeze_12_output_0"]
output_edges = [
    "outputs0",
    "outputs1",
    "outputs2",
]

extractor = onnx.utils.Extractor(model)

fcos_onnx = Path("fcos.onnx")
fcos_model = extractor.extract_model(input_edges, output_edges)

onnx.checker.check_model(fcos_model)
fcos_model, check = onnxsim.simplify(fcos_model)
assert check, "Simplified ONNX model could not be validated"

onnx.save(fcos_model, fcos_onnx.name)
print(f"ONNX for FCOS is generated as {fcos_onnx.name}.")

ONNX for FCOS is generated as fcos.onnx.

Fix Output Boxes Number

The original Pytorch FCOS generates variable number of output boxes. Here we are going to fix this number.

from fcosutils import pad_nms_output
import onnx_graphsurgeon as gs

new_model = onnx.load(fcos_onnx.name)
onnx.checker.check_model(new_model)

## Taken from self.detections_per_img in the python code
max_output_boxes = 100

pad_nms_output(
    new_model,
    max_output_boxes,
    "outputs0",
    num_dims=2,
    dtype=onnx.TensorProto.FLOAT,
)
pad_nms_output(
    new_model,
    max_output_boxes,
    "outputs1",
    num_dims=1,
    dtype=onnx.TensorProto.FLOAT,
)
pad_nms_output(
    new_model,
    max_output_boxes,
    "outputs2",
    num_dims=1,
    dtype=onnx.TensorProto.INT64,
)

## Try simplifying again
new_model, success = onnxsim.simplify(new_model)
## clean up model
g = gs.import_onnx(new_model)
g.cleanup(remove_unused_node_outputs=True)
new_model = gs.export_onnx(g)
onnx.checker.check_model(new_model)

Replace NMS with Custom OPS

The original nms is replaced with cgc::fcosNms.

custom_op_input_edges = [
    "/head/classification_head/Concat_10_output_0",
    "/head/regression_head/Concat_21_output_0",
    "/head/regression_head/Concat_20_output_0",
]
custom_op_output_edges = [
    "sliced_tensor_outputs0",
    "sliced_tensor_outputs1",
    "sliced_tensor_outputs2",
]
replacer = gutils.CustomOpReplacer(new_model)
out_frac_bits = [18, 31, 0]
dequant_frac_bits = 18

ccl_func_name = "cgc::fcosNms<800, 1344, 793, 1333, 60, 20>"
_, new_model = replacer.replace_subgraph_by_edges(
    custom_op_output_edges,
    custom_op_input_edges,
    ccl_func_name,
    element_wise=False,
    fixed_point_frac_bits=out_frac_bits,
    keep_constants=[],
)

new_base_name = f"{fcos_onnx.stem}-sim{fcos_onnx.suffix}"
sim_model_file = str(fcos_onnx.with_name(new_base_name))
onnx.save(new_model, sim_model_file)
onnx.checker.check_model(new_model)

Quantization

from torch.utils.data import Subset
from torchvision.transforms import Compose, Normalize, ToTensor
from sdk_cli.utils.transforms import ResizePad
from sdk_cli.lib.quantize import (
    QuantizedONNXModel,
    quantize_onnx_model,
)
from sdk_cli.utils.datasets import QuadricCalibration

dataset_mean, dataset_std = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]

transforms = Compose(
    [
        ResizePad(model_input_size),
        ToTensor(),
        Normalize(dataset_mean, dataset_std),
    ]
)

## Path to directory containing data to use for for numerical range calibration during quantization
## Data is used also used to compare accuracy of fp32 and int8 models
dataset = QuadricCalibration.Dataset(transform=transforms)

## NOTE: `coco-like` is the 0th index target for `QuadricCalibration.Dataset`
coco_like_data_indices = [index for index, target in enumerate(dataset.targets) if target == 0]
coco_like_subset_of_dataset = Subset(dataset, coco_like_data_indices)

quantized_onnx_model: QuantizedONNXModel = quantize_onnx_model(
    sim_model_file,
    coco_like_subset_of_dataset,
    onnx_node_types_to_exclude=["Concat"],
    asymmetric_activation=True,
    skip_optimization=False,
)

quantized_model = onnx.load(quantized_onnx_model.model_path)

## Move dequantize inside the custom-op
replacer = gutils.CustomOpReplacer(quantized_model)
replacer.fold_quant_nodes_into_custom_ops()
onnx.checker.check_model(quantized_model)

quant_onnx = f"{fcos_onnx.stem}-custom_quant{fcos_onnx.suffix}"
onnx.save(quantized_model, quant_onnx)
print(f"Quant ONNX: {quant_onnx}, Tranges File: {str(quantized_onnx_model.tensor_ranges_path)}")

2026-06-19 03:54 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-06-19 03:54 - DEBUG - sdk - quantize - Forcing node types: []
2026-06-19 03:54 - DEBUG - sdk - quantize - ONNX Node types excluded from quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp', 'Concat']
2026-06-19 03:54 - DEBUG - sdk - quantize - ONNX Node names excluded from quantization: ['/head/classification_head/conv/conv.4_2/Reshape_1', '/head/regression_head/conv/conv.7/Reshape_1', '/head/regression_head/conv/conv.7_3/Reshape', '/head/classification_head/conv/conv.1_3/Reshape_1', '/head/classification_head/conv/conv.10/Reshape', '/head/regression_head/conv/conv.4_4/Reshape_1', '/head/classification_head/conv/conv.4_4/InstanceNormalization', '/head/regression_head/conv/conv.4/Mul', '/head/classification_head/conv/conv.1_2/InstanceNormalization', '/head/regression_head/conv/conv.7_2/InstanceNormalization', '/head/classification_head/conv/conv.7_2/InstanceNormalization', '_v_1655', 'onnx::Add_4462', '/head/classification_head/conv/conv.4/Reshape_1', '/head/regression_head/conv/conv.1_3/Reshape', '/head/regression_head/conv/conv.7_2/Reshape_1', '/head/regression_head/conv/conv.7_4/Reshape', '/head/classification_head/conv/conv.4_4/Reshape', '/head/classification_head/conv/conv.4_1/Reshape', 'CustomOp/cgc::fcosNms<800, 1344, 793, 1333, 60, 20>0', '/head/classification_head/conv/conv.7/Add', '/head/regression_head/conv/conv.7_1/Mul', '/head/classification_head/conv/conv.10_2/Reshape_1', '/head/regression_head/conv/conv.10_4/Reshape_1', '/head/classification_head/conv/conv.4_1/Add', '/head/regression_head/conv/conv.10_1/Mul', '/head/classification_head/conv/conv.7_2/Add', '/head/classification_head/conv/conv.1_3/Add', '/head/regression_head/conv/conv.1_3/Reshape_1', '/head/classification_head/conv/conv.7_4/Reshape', 'onnx::Add_4402', 'onnx::Add_4468', '/head/classification_head/conv/conv.7_3/Add', '/head/regression_head/conv/conv.7_1/Add', '/head/regression_head/conv/conv.1_3/Add', '/head/regression_head/conv/conv.7_4/InstanceNormalization', '/head/regression_head/conv/conv.4_3/Mul', '/head/regression_head/conv/conv.10_4/Add', '/head/regression_head/conv/conv.4_2/InstanceNormalization', 'onnx::Add_4464', '/head/classification_head/conv/conv.10_4/InstanceNormalization', '/head/regression_head/conv/conv.10_2/Add', '/head/classification_head/conv/conv.4_2/Mul', '/head/regression_head/conv/conv.1_1/InstanceNormalization', '/head/regression_head/conv/conv.4_1/Reshape_1', 'onnx::Mul_4407', '/head/classification_head/conv/conv.7_4/Mul', '/head/classification_head/conv/conv.1_4/Mul', '/head/regression_head/conv/conv.10_2/InstanceNormalization', '/head/regression_head/conv/conv.7_2/Mul', '/head/classification_head/conv/conv.4_3/Mul', '/head/regression_head/conv/conv.1_1/Reshape', '/head/regression_head/conv/conv.10_1/Add', '/head/regression_head/conv/conv.1_3/InstanceNormalization', '/head/classification_head/conv/conv.10_4/Reshape_1', '/head/classification_head/conv/conv.10_3/InstanceNormalization', '/head/regression_head/Concat_20', '/head/classification_head/conv/conv.1_3/InstanceNormalization', '/head/regression_head/conv/conv.1/Reshape', '/head/classification_head/conv/conv.10_1/Reshape_1', 'onnx::Mul_4401', '/head/regression_head/conv/conv.1/Add', '/head/classification_head/conv/conv.1_1/Mul', '/head/classification_head/conv/conv.4/InstanceNormalization', '_v_1653', '/head/classification_head/conv/conv.7_4/InstanceNormalization', 'onnx::Mul_4463', '/head/regression_head/conv/conv.7_1/Reshape_1', '/head/classification_head/conv/conv.4/Reshape', '/head/regression_head/conv/conv.7/Reshape', '/head/regression_head/conv/conv.4/Reshape', '/head/regression_head/conv/conv.4_2/Reshape', '/head/regression_head/conv/conv.4_1/Mul', '/head/regression_head/conv/conv.10/Reshape', '/head/regression_head/conv/conv.10_2/Reshape', '/head/classification_head/conv/conv.1_1/Reshape', '/head/classification_head/conv/conv.7/Reshape_1', '/head/classification_head/conv/conv.7_3/Reshape_1', '/head/regression_head/conv/conv.4_2/Mul', '/head/classification_head/conv/conv.1_4/Reshape_1', '/head/regression_head/conv/conv.4/Add', '/head/classification_head/conv/conv.1/Reshape_1', '/head/regression_head/conv/conv.1_2/Reshape_1', '/head/regression_head/conv/conv.4_2/Reshape_1', '/head/regression_head/conv/conv.10/Mul', '/head/classification_head/conv/conv.4_3/Reshape', '/head/regression_head/conv/conv.7_4/Mul', '/head/regression_head/conv/conv.7/Mul', '/head/classification_head/conv/conv.1/Reshape', '/head/regression_head/conv/conv.10_1/Reshape', '/head/classification_head/conv/conv.1_4/Reshape', '/head/classification_head/conv/conv.4_4/Reshape_1', '/head/classification_head/conv/conv.10/InstanceNormalization', '/head/regression_head/conv/conv.1/InstanceNormalization', '/head/regression_head/conv/conv.1/Reshape_1', '/head/classification_head/conv/conv.7_4/Reshape_1', '/head/classification_head/conv/conv.1_4/Add', '/head/classification_head/conv/conv.7_4/Add', 'onnx::Mul_4405', '/head/regression_head/conv/conv.1_4/InstanceNormalization', '/head/classification_head/conv/conv.7_2/Mul', '/head/regression_head/conv/conv.1_2/Add', 'onnx::Mul_4465', '/head/regression_head/conv/conv.10/Reshape_1', '/head/regression_head/conv/conv.4_2/Add', '/head/classification_head/conv/conv.10_3/Mul', '/head/classification_head/conv/conv.10_2/Reshape', '/head/classification_head/conv/conv.10_4/Add', '/head/regression_head/conv/conv.10_4/Mul', '/head/classification_head/conv/conv.10/Add', '/head/regression_head/conv/conv.1_1/Reshape_1', '/head/classification_head/conv/conv.7_3/Reshape', '/head/classification_head/conv/conv.1_2/Reshape', '/head/regression_head/conv/conv.7/Add', '/backbone/fpn/Concat_1_output_0', '/head/regression_head/conv/conv.1_1/Add', '/head/classification_head/conv/conv.10_1/Add', '/head/classification_head/conv/conv.1/Constant_2_output_0', 'onnx::Add_4408', '_v_1651', '/head/regression_head/conv/conv.1_4/Add', '/head/classification_head/conv/conv.4_2/Reshape', '/head/classification_head/conv/conv.1_1/Reshape_1', '/head/classification_head/conv/conv.4_4/Add', '/head/regression_head/Concat_21', '/head/classification_head/conv/conv.10/Mul', '/head/classification_head/conv/conv.4_2/InstanceNormalization', '/head/regression_head/conv/conv.10/InstanceNormalization', '/head/regression_head/conv/conv.4_1/InstanceNormalization', '/head/classification_head/conv/conv.7/Mul', '/head/regression_head/conv/conv.4_1/Reshape', '/head/regression_head/conv/conv.7_1/Reshape', '/head/regression_head/conv/conv.10_2/Mul', '/head/regression_head/conv/conv.7_3/Mul', '/head/regression_head/conv/conv.7_2/Add', '/head/classification_head/conv/conv.10_2/Mul', '/head/regression_head/conv/conv.4_1/Add', '/head/classification_head/conv/conv.4/Mul', '/head/classification_head/conv/conv.7_1/Add', '/head/classification_head/conv/conv.10_4/Mul', '/head/regression_head/conv/conv.4_3/Add', '/head/regression_head/conv/conv.4_4/Add', '/head/classification_head/conv/conv.1/Constant_1_output_0', '/backbone/fpn/Concat_3_output_0', '/head/regression_head/conv/conv.10_3/Add', '/head/classification_head/conv/conv.10_1/Mul', '/head/classification_head/conv/conv.7_1/Reshape', 'onnx::Mul_4467', '/head/classification_head/conv/conv.1_4/InstanceNormalization', '/head/regression_head/conv/conv.4_3/Reshape', '/head/regression_head/conv/conv.10_4/Reshape', '/head/regression_head/conv/conv.7_1/InstanceNormalization', '/head/regression_head/conv/conv.4_4/InstanceNormalization', '/head/regression_head/conv/conv.4_4/Reshape', '/head/regression_head/conv/conv.1_3/Mul', '/head/regression_head/conv/conv.10_2/Reshape_1', '/head/classification_head/conv/conv.1_1/InstanceNormalization', '/head/classification_head/conv/conv.1_2/Mul', '/head/regression_head/conv/conv.4_3/InstanceNormalization', '/head/regression_head/conv/conv.1_1/Mul', '/head/classification_head/conv/conv.1_3/Mul', '/head/classification_head/conv/conv.4_1/Reshape_1', '/head/regression_head/conv/conv.10_1/InstanceNormalization', '/head/regression_head/conv/conv.7_3/InstanceNormalization', '/head/regression_head/conv/conv.1_4/Reshape_1', '/head/regression_head/conv/conv.7/InstanceNormalization', '/head/regression_head/conv/conv.4/Reshape_1', '/head/classification_head/conv/conv.10_1/InstanceNormalization', '/head/classification_head/conv/conv.1_1/Add', '/head/classification_head/conv/conv.10_2/Add', '/head/classification_head/Concat_10', '/head/regression_head/conv/conv.4_4/Mul', '/head/classification_head/conv/conv.7/InstanceNormalization', '/head/regression_head/conv/conv.4/InstanceNormalization', '/head/regression_head/conv/conv.10_3/Reshape_1', '/head/classification_head/conv/conv.1/Constant_output_0', '/head/classification_head/conv/conv.7_3/InstanceNormalization', '/head/regression_head/conv/conv.10_1/Reshape_1', '/head/classification_head/conv/conv.10_3/Reshape_1', '/head/regression_head/conv/conv.10_3/Mul', '/head/classification_head/conv/conv.4_2/Add', '/head/regression_head/conv/conv.1_4/Reshape', '/head/regression_head/conv/conv.10_3/InstanceNormalization', '/head/regression_head/conv/conv.10_4/InstanceNormalization', 'onnx::Mul_4403', '/head/regression_head/conv/conv.1/Mul', '/head/regression_head/conv/conv.1_2/Reshape', '/head/classification_head/conv/conv.4/Add', '/head/classification_head/conv/conv.4_1/InstanceNormalization', 'onnx::Mul_4461', '/head/regression_head/conv/conv.1_4/Mul', '/head/regression_head/conv/conv.1_2/InstanceNormalization', '/head/classification_head/conv/conv.7/Reshape', '/head/regression_head/conv/conv.7_4/Reshape_1', '/head/classification_head/conv/conv.1_2/Reshape_1', '/head/regression_head/conv/conv.7_3/Add', '/head/classification_head/conv/conv.10/Reshape_1', '/head/classification_head/conv/conv.4_3/Add', '/head/classification_head/conv/conv.10_3/Add', '/head/classification_head/conv/conv.4_3/Reshape_1', '/head/classification_head/conv/conv.10_3/Reshape', '/head/classification_head/conv/conv.4_1/Mul', '/head/classification_head/conv/conv.4_3/InstanceNormalization', 'onnx::Add_4404', '/head/classification_head/conv/conv.7_1/Mul', '/head/regression_head/conv/conv.7_3/Reshape_1', '/head/regression_head/conv/conv.10_3/Reshape', '/head/classification_head/conv/conv.1_3/Reshape', '/head/regression_head/conv/conv.4_3/Reshape_1', '/head/classification_head/conv/conv.1_2/Add', '/head/regression_head/conv/conv.7_4/Add', '/head/classification_head/conv/conv.7_1/Reshape_1', '/head/regression_head/conv/conv.1_2/Mul', '/head/classification_head/conv/conv.1/Add', '/head/regression_head/conv/conv.10/Add', '/head/classification_head/conv/conv.1/InstanceNormalization', '/head/classification_head/conv/conv.7_2/Reshape', '/head/classification_head/conv/conv.1/Mul', '/head/classification_head/conv/conv.4_4/Mul', '/head/regression_head/conv/conv.7_2/Reshape', '/head/classification_head/conv/conv.7_3/Mul', '/head/classification_head/conv/conv.7_1/InstanceNormalization', '/head/classification_head/conv/conv.10_2/InstanceNormalization', '/head/classification_head/conv/conv.10_4/Reshape', '/head/classification_head/conv/conv.10_1/Reshape', 'onnx::Add_4466', 'onnx::Add_4406', '/head/classification_head/conv/conv.7_2/Reshape_1']
2026-06-19 03:54 - INFO - sdk - quantize - Starting quantization...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-06-19 03:54 - INFO - sdk - quantize - Quantization completed! Quantized model saved to /quadric/sdk-cli/examples/models/fcos/fcos-sim_OpSet16_optimized_asym_int8_q.onnx
2026-06-19 03:54 - INFO - sdk - quantize - ONNX full precision model size: 124.22MB
2026-06-19 03:54 - INFO - sdk - quantize - ONNX quantized model size: 31.57MB
2026-06-19 03:54 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-06-19 03:54 - INFO - sdk - quantize - ONNX Model with well-defined shapes has been saved at `/quadric/sdk-cli/examples/models/fcos/fcos-sim_OpSet16_optimized_asym_int8_q_shaped.onnx`.
2026-06-19 03:54 - DEBUG - sdk - quantize - Checking for FLOAT/FLOAT16 types...
2026-06-19 03:54 - INFO - sdk - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-06-19 03:54 - INFO - sdk - quantize - Model still has FLOAT/FLOAT16 types after quantization. Creating ranges for floating point tensors using calibration data...
2026-06-19 03:55 - INFO - sdk - quantize - Saved computed tensor ranges to /quadric/sdk-cli/examples/models/fcos/fcos-sim_OpSet16_optimized_asym_int8_q_shaped.tranges.
2026-06-19 03:55 - INFO - sdk - quantize - 
╒══════════════════════════════════════════════════════════════════════════════════════════╤═════════════════════════════════════════════════════════════════════════════════════════════╕
│ Quantized ONNX Model                                                                     │ Tensor Ranges File                                                                          │
╞══════════════════════════════════════════════════════════════════════════════════════════╪═════════════════════════════════════════════════════════════════════════════════════════════╡
│ /quadric/sdk-cli/examples/models/fcos/fcos-sim_OpSet16_optimized_asym_int8_q_shaped.onnx │ /quadric/sdk-cli/examples/models/fcos/fcos-sim_OpSet16_optimized_asym_int8_q_shaped.tranges │
╘══════════════════════════════════════════════════════════════════════════════════════════╧═════════════════════════════════════════════════════════════════════════════════════════════╛


Quant ONNX: fcos-custom_quant.onnx, Tranges File: /quadric/sdk-cli/examples/models/fcos/fcos-sim_OpSet16_optimized_asym_int8_q_shaped.tranges

Compilation

from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from tvm.contrib.epu.chimera_job.hw_config import HWConfig

trange_file = str(quantized_onnx_model.tensor_ranges_path)
hw_config = HWConfig(ocm_size="32MB")
cgc_job = ChimeraJob(
    model_p=quant_onnx,
    hw_config=hw_config,
    trange_file=trange_file,
)
cgc_job.compile(quiet=True)
print(cgc_job)

╒═════════════════════╤══════════════════════════════════════════════════════════════════╕
│ Module Name         │ fcos_custom_quant_QC_U_1d7_32MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ ONNX File           │ fcos-custom_quant.onnx                                           │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ Product Target      │ QC-U                                                             │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ Number of Cores     │ 1                                                                │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700                                                            │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ L2M Size            │ 32MB                                                             │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ LRM Size            │ 4kB                                                              │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ External Read BW    │ 128GBps                                                          │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ External Write BW   │ 128GBps                                                          │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ MACS per PE         │ 16                                                               │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ Max L2M             │ 32.000MB                                                         │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ Max LRM             │ 3.000kB                                                          │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes  │ 50.513MB                                                         │
├─────────────────────┼──────────────────────────────────────────────────────────────────┤
│ Network GMACs       │ 215.298                                                          │
╘═════════════════════╧══════════════════════════════════════════════════════════════════╛

╒════╤════════╤══════════════════════════════════╤═══════════════════╤══════════════════════════╤═══════╕
│    │ Type   │ Name                             │ shape             │ type                     │ mse   │
╞════╪════════╪══════════════════════════════════╪═══════════════════╪══════════════════════════╪═══════╡
│  0 │ Input  │ /transform/Unsqueeze_12_output_0 │ [1, 3, 800, 1344] │ tensor[FixedPoint32<29>] │ n/a   │
├────┼────────┼──────────────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  1 │ Output │ sliced_tensor_outputs0           │ [100, 4]          │ tensor[FixedPoint32<18>] │ n/a   │
├────┼────────┼──────────────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  2 │ Output │ sliced_tensor_outputs1           │ [100]             │ tensor[FixedPoint32<31>] │ n/a   │
├────┼────────┼──────────────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  3 │ Output │ sliced_tensor_outputs2           │ [100]             │ tensor[int32]            │ n/a   │
╘════╧════════╧══════════════════════════════════╧═══════════════════╧══════════════════════════╧═══════╛

Demo

Inference

from onnxruntime import InferenceSession
from PIL import Image
import torch
from sdk_cli.lib.inference import InferenceEngine, single_inference
from sdk_cli.node_builtins.classical.resize_bounding_boxes import resize_bounding_boxes

run_folder = "../../common/calibration/coco-like/"

engines = {
    InferenceEngine.CHIMERA_ORT_INT8: cgc_job,
    InferenceEngine.CHIMERA_ISS_INT8: cgc_job,
}

image_file = Path(run_folder) / "33823288584_1d21cf0a26_k.jpeg"
org_shape = Image.open(image_file).size

score_threshold = 0.3

all_detections = {}

for inference_engine, engine in engines.items():
    image_loaded = np.expand_dims(transforms(Image.open(image_file)), axis=0)
    outputs = single_inference(inference_engine, engine, image_loaded)

    boxes = torch.from_numpy(outputs[0])
    scores = torch.from_numpy(outputs[1].reshape(-1, 1))
    labels = torch.from_numpy(outputs[2].reshape(-1, 1))
    detections = torch.cat([boxes, scores, labels], dim=1)

    if len(detections):
        detections = detections[detections[:, 4] > score_threshold]
        detections[:, :4] = resize_bounding_boxes(
            model_input_size, org_shape, detections[:, :4]
        ).round()

    all_detections[inference_engine] = detections

2026-06-19 04:02 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x79f91a4a5bd0>
2026-06-19 04:02 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 04:02 - INFO - epu - iss_testing - Done 0:00:02.042674
2026-06-19 04:02 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x79f91a4a6080>
FILM 111/111: 100%|███████████████████████████████████████████████| 111/111 [14:18<00:00,  7.73s/it]
2026-06-19 04:16 - INFO - epu - iss_testing - Found tranges for input: <tvm.contrib.epu.interval.Interval object at 0x79f91a2d5300>
2026-06-19 04:16 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 04:16 - INFO - epu - iss_testing - Done 0:00:01.620106

Display Results

import matplotlib.pyplot as plt
from sdk_cli.node_builtins.outputs.bbox_label_visualizer import draw_bbox
from sdk_cli.utils.datasets.COCO import COCO91CLASSES

%matplotlib inline

MODEL_NAME = "FCOS"

pic_len = len(engines) + 1

ax, idx = {}, 1
fig = plt.figure(figsize=(6 * pic_len, 6), tight_layout=True)

ax[idx] = fig.add_subplot(int("1%s%s" % (pic_len, idx)))
ax[idx].imshow(Image.open(image_file))
ax[idx].set_title("Original Image")
ax[idx].axis("off")

for inference_engine, detections in all_detections.items():
    idx += 1
    ax[idx] = fig.add_subplot(int("1%s%s" % (pic_len, idx)))
    ax[idx].imshow(draw_bbox(np.array(Image.open(image_file)), detections, classes=COCO91CLASSES))
    ax[idx].set_title(f"{MODEL_NAME}: {str(inference_engine)}")
    ax[idx].axis("off")

fig.show()