Tutorial: Fixed Point Support on GPNPU

NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/ort_fixed_point/ort_fixed_point.ipynb.

Running ORT in fixed-point/gpnpu mode

Setup

Quadric's fork of onnxruntime allows for running onnx graphs in fixed point. Here we showcase its usage with resnet.

First, make sure that you have quadric's latest ORT.

• visit https://github.com/quadric-io/onnxruntime/releases and download the appropriate .whl from stable tagged release and run pip install <.whl file>
• or, you can pip install -r requirements_quadric.txt within root of tvm repo.

from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
import numpy as np

Create a fixed-point graph with ChimeraJob

## float and quantized graph
model_path = "./onnx/resnet_18_float.onnx"

## fixed-point graph
fxpt_model_path = "./onnx/resnet_18_fixed.onnx"

cgc_job = ChimeraJob(model_p=model_path, macs_per_pe=8, quiet_iss=False)

## Builds fixedpoint graph, stores it at specified file path, stores internally self.fxpt_model, and returns it.
fx_model = cgc_job.convert_to_fxpt_onnx(fxpt_model_path=fxpt_model_path)

/tmp/ipykernel_20913/1754330126.py:7: DeprecationWarning: Specifying hardware configuration through individual parameters is deprecated. Please use hw_config parameter instead. Example: hw_cfg = HWConfig(product='QC-U', ocm_size='16MB'); ChimeraJob('model.onnx', hw_config=hw_cfg)
  cgc_job = ChimeraJob(model_p=model_path, macs_per_pe=8, quiet_iss=False)
2026-06-19 03:57 - INFO - epu - chimera_job - Converting ./onnx/resnet_18_float.onnx to fixed-point version ...
2026-06-19 03:57 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 03:57 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 03:57 - INFO - epu - fx - 

Source name              Op                      Output 0 Range                             Output 0 Frac Bits
-----------------------  ----------------------  ---------------------------------------  --------------------
output_DequantizeLinear  contrib.epu.dequantize  (-10.437687635421753, 33.9224848151207)                    25

2026-06-19 03:57 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-06-19 03:57 - INFO - epu - chimera_job - Successfully converted onnx file ./onnx/resnet_18_float.onnx to fixed-point version ./onnx/resnet_18_fixed.onnx.

Inference

import random

seed = 0
np.random.seed(seed)

## create input
input_shape = (1, 3, 224, 224)
input_data = np.random.rand(*input_shape).astype(np.float32)
input_dict = {"input": input_data}
output_name = "output"

Option 1: Run with ChimeraJob method run_onnx_inf_session

## This option is a wrapper around onnxruntime which includes float-to-fixed point conversions
## at inputs and outputs.
#
## This can actually run without needing to explicitly call convert_to_fxpt_onnx prior as done in this
## notebook. It will call it internally if it hasn't been called before.
output_ort_1 = cgc_job.run_onnx_inf_session(
    input_dict, enable_gpnpu_emulation=True  # <--- make sure to set this to True
)[output_name]

2026-06-19 03:57 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 03:57 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7015285e1f60>
2026-06-19 03:57 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:57 - INFO - epu - chimera_job - running ort reference with fixed-point model: ./onnx/resnet_18_fixed.onnx
2026-06-19 03:57 - INFO - epu - iss_testing - Done 0:00:00.465890

Option 2: Run directly with Onnxruntime

import onnxruntime as ort

## By inspecting fixed-point graph, gather the following information:
input_frac_bits = 27
output_frac_bits = 25

## GPNPU Session
session_options = ort.SessionOptions()
session_options.add_session_config_entry(
    "session.enable_gpnpu", "1"
)  # <-- This is critical step to enable
session = ort.InferenceSession(
    fxpt_model_path,  # <-- supply fixed-point graph converted earlier with ChimeraJob
    sess_options=session_options,
    providers=["CPUExecutionProvider"],
)

## Extract input and output names
input_name_fxpt = session.get_inputs()[0].name
output_name_fxpt = session.get_outputs()[0].name

## Run ORT inference
input_dict_fxpt = {input_name_fxpt: np.int32(input_data * 2**input_frac_bits)}
output_fxpt = session.run(
    [output_name_fxpt],
    input_dict_fxpt,
)[0]

## Convert from fixed-point to float
output_ort_2 = (output_fxpt / (1 << output_frac_bits)).astype(np.float32)


## Confirming that both options give the exact same results:
both_options_match = np.array_equal(output_ort_1, output_ort_2)
print(f"Both options match: {both_options_match}")

Both options match: True

Compare with CGC

## Run inference
cgc_job.analyze_network()
cgc_job.compile(quiet=True)
output_cgc = cgc_job.run_inference_harness(inputs=input_dict)[output_name]


## Compare ORT to CGC. We expect an exact match here.
max_diff = np.max(np.abs(output_ort_1 - output_cgc))
print(f"Max difference between ORT and CGC outputs: {max_diff}")
exact_match = np.array_equal(output_ort_1, output_cgc)
print(f"Exact match: {exact_match}")

2026-06-19 03:57 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 03:57 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 03:57 - INFO - epu - fx - 

Source name              Op                      Output 0 Range                             Output 0 Frac Bits
-----------------------  ----------------------  ---------------------------------------  --------------------
output_DequantizeLinear  contrib.epu.dequantize  (-10.437687635421753, 33.9224848151207)                    25



Analysis of ./onnx/resnet_18_float.onnx


2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7013efe63580>
FILM 12/12: 100%|███████████████████████████████████████████████████| 12/12 [00:16<00:00,  1.34s/it]
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7013efe63580>
2026-06-19 03:58 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:58 - INFO - epu - iss_testing - Done 0:00:00.237801


Max difference between ORT and CGC outputs: 0.0
Exact match: True