NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/waveformer/waveformer.ipynb.
WaveFormer sEMG Gesture Classification on Chimera GPNPU
End-to-end walkthrough of taking the INT8 WaveFormer encoder onto the Chimera GPNPU with the Chimera Graph Compiler (CGC), following the same shape as the other sdk-cli model notebooks:
- Model Setup - export + INT8-quantize (
export_quant.py) / inspect the ONNX, and set up the SDK env + QC-NanoHWConfig. - Prepare inputs & calibration - a real sEMG window + ORT reference, and the
.tranges(compute_tensor_ranges). - Compile through CGC - compile the graph (only
waveAttention x6stays a custom op; everything else - patch conv, LayerNorms, residual adds, the FFN/projectionQGemms, the CLS head - compiles natively). - Build the EPU binary (
sdk source) - compile the CGC-generated CCL C++ + thewaveAttentionkernel into the EPU binary / host executable via Quadric LLVM: (4a) prepare the input, then (4b) compile + run on the ISS. - Validate against ORT - decode the EPU logits and check argmax + logit closeness vs ONNX Runtime.
What WaveFormer is. An sEMG gesture classifier (EPN612: 8 channels, 6 classes, ~3.1M params) - a ViT-style encoder (6 blocks, 8 heads, head_dim 32, embed_dim 256, seq 161 = 160 patch tokens + 1 CLS, mlp_ratio=1). Input [1,1,8,1000] fp32 (8-channel x 1000-sample signal) -> output [1,6] class logits.
Target HW. QC-Nano: 8x8 PE array, 4 MB L2/OCM, 4 kB LRM, 8 MACs/PE, single core.
1. Model Setup
export_quant.py is the checkpoint -> INT8-ONNX conversion (no external download; built from the local waveformer_epn612.pth). As a process it is a 4-stage pipeline:
Stage 1 - Rebuild the trained model in PyTorch. Loads waveformer_epn612.pth into a Waveformer_base (EPN612 config), then applies two structural patches so it exports cleanly: RoPE attention is rewritten to run qkv/proj on 2D tensors (and drop a zero-width slice that breaks ORT's calibrator), and each block's MLP is wrapped so fc1/fc2 collapse into a single Gemm. Both exist so the linears export as Gemm -> QGemm.
Stage 2 - Export to fp32 ONNX. torch.onnx.export(..., opset_version=17) traces the model on a dummy [1,1,8,1000] input -> waveformer_fp32.onnx (output [1,6]); quant_pre_process then runs shape-inference / cleanup into a temporary _prep.onnx. Nothing is quantized yet.
Stage 3 - Calibrate + decide the fp32/int8 split. Feeds data/calib.npy (representative sEMG windows) through the graph via a CalibrationDataReader, collecting per-tensor min/max ranges (MinMax) that set each int8 scale. Builds a nodes_to_exclude list that keeps every /attn/ node fp32 except qkv, proj, and /attn/MatMul_1 - i.e. quantize the projections + the A.V matmul, keep Q.K + softmax + RoPE in fp32 (softmax probs in [0,1] map poorly onto symmetric int8).
Stage 4 - Quantize to INT8 ONNX. quantize_static(...) inserts the int8 kernels using the calibrated scales. Key settings: QuantFormat.QOperator (fused int8 ops - QGemm, QLinearAdd, QLinearConv - not QDQ pairs), QInt8 activations + weights, per_channel=False (per-tensor scale), ActivationSymmetric=True + WeightSymmetric=True (zero-point 0, which the Quadric matmul path requires). Writes waveformer_int8.onnx and removes the temp _prep.onnx.
Result: waveformer_epn612.pth -> waveformer_fp32.onnx (float) -> waveformer_int8.onnx (symmetric per-tensor int8, QOperator; projections + A.V quantized, Q.K / softmax / RoPE fp32) - the graph the later steps calibrate (.tranges) and compile through CGC. This step does not produce the .tranges (that is step 2); the ranges computed here are baked into the ONNX quant scales.
The cell below sets up the SDK env (imports + the QC-Nano HWConfig), then loads and inspects waveformer_int8.onnx - regenerating it via export_quant.py only if missing (needs torch/timm + the checkpoint). Run inside the Quadric SDK env (tvm.contrib.epu must be importable).
from collections import Counter
from pathlib import Path
import numpy as np
import onnx
import onnxruntime as ort
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from tvm.contrib.epu.chimera_job.hw_config import HWConfig
## --- paths ---
INT8_ONNX = "waveformer_int8.onnx"
TRANGES = "waveformer_int8.tranges"
CUSTOM_OP_ONNX = "waveformer_int8_custom_op.onnx"
TEST_X = "data/test_x.npy" # real sEMG windows [N,1,8,1000]
TEST_Y = "data/test_y.npy" # labels
## --- WaveFormer config (fixed by the EPN612 / Waveformer_base checkpoint) ---
N_BLOCKS, NUM_HEADS, HEAD_DIM, EMBED_DIM, SEQ_LEN = 6, 8, 32, 256, 161
CLASSES = [
"hand relax",
"hand close",
"wrist flexion",
"wrist extension",
"ulnar dev.",
"radial dev.",
]
## --- QC-Nano hardware target---
HW = HWConfig(
product="QC-N",
ocm_size="4MB",
macs_per_pe=8,
num_cores=1,
lrm_size="4kB",
ext_rd_bw="8GBps",
ext_wr_bw="8GBps",
clock_freq_ghz=0.5,
) # QC-Nano: 8 GB/s DDR, 500 MHz
print("HW target:", HW, "| onnxruntime", ort.__version__, "| onnx", onnx.__version__)
## --- load + inspect the committed INT8 model ---
model = onnx.load(INT8_ONNX)
g = model.graph
in_name = g.input[0].name
out_name = g.output[0].name
def _shape(v):
return [d.dim_value for d in v.type.tensor_type.shape.dim]
ops = Counter(n.op_type for n in g.node)
print(f"model : {INT8_ONNX} ({len(g.node)} nodes)")
print(f"input : {in_name} {_shape(g.input[0])}")
print(f"output : {out_name} {_shape(g.output[0])}")
print(
f"encoder : {N_BLOCKS} blocks x (LayerNorm -> attention -> residual -> LayerNorm -> MLP -> residual)"
)
print(f"op types : {dict(sorted(ops.items()))}")
HW target: QC-N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1 | onnxruntime 1.20.0 | onnx 1.16.2
model : waveformer_int8.onnx (449 nodes)
input : input [1, 1, 8, 1000]
output : logits [1, 6]
encoder : 6 blocks x (LayerNorm -> attention -> residual -> LayerNorm -> MLP -> residual)
op types : {'Add': 12, 'Concat': 12, 'DequantizeLinear': 39, 'Div': 7, 'Erf': 7, 'Gather': 43, 'LayerNormalization': 15, 'MatMul': 6, 'Mul': 54, 'QGemm': 25, 'QLinearAdd': 19, 'QLinearConcat': 1, 'QLinearConv': 1, 'QLinearMatMul': 6, 'QLinearMul': 14, 'QuantizeLinear': 46, 'Reshape': 61, 'Softmax': 6, 'Sub': 12, 'Transpose': 39, 'Unsqueeze': 24}
2. Prepare inputs & calibration
Two things: (a) a real sEMG window to run plus its ORT golden; (b) the per-tensor ranges (.tranges) CGC needs to assign fixed-point fractional bits.
The tranges come from 3_make_tranges.py via sdk_cli.lib.quantize.compute_tensor_ranges (the ORT MinMax calibrator over data/calib.npy) - not sdk graph quantize, whose opset down-convert dies on the opset-17 LayerNormalization. The cell regenerates them only if missing.
## --- (a) a real sEMG input window + ORT golden ---
X = np.load(TEST_X).astype(np.float32)
Y = np.load(TEST_Y) if Path(TEST_Y).exists() else None
SAMPLE = 0
x = X[SAMPLE : SAMPLE + 1] # [1,1,8,1000]
print(f"input window : {x.shape} dtype={x.dtype}")
if Y is not None:
print(f"true label : {int(Y[SAMPLE])} ({CLASSES[int(Y[SAMPLE])]})")
sess = ort.InferenceSession(INT8_ONNX, providers=["CPUExecutionProvider"])
ort_logits = sess.run(None, {in_name: x})[0] # [1,6]
pred = int(np.argmax(ort_logits))
print(f"ORT logits : {np.round(ort_logits, 3).tolist()}")
print(f"ORT pred : {pred} ({CLASSES[pred]})")
## --- (b) tensor-range calibration (.tranges) ---
import json
print(f"[tranges] {TRANGES}: {len(json.load(open(TRANGES)))} entries")
input window : (1, 1, 8, 1000) dtype=float32
true label : 5 (radial dev.)
ORT logits : [[-2.7939999103546143, -0.847000002861023, -4.317999839782715, 5.926000118255615, -4.656000137329102, 4.572000026702881]]
ORT pred : 3 (wrist extension)
[tranges] waveformer_int8.tranges: 328 entries
3. Compile through CGC
custom_op_match.py replaces each block's head-split + RoPE + flash-attention core with a waveAttention custom op (its [1,161,3,8,32] tensors are 5-D, so the attention core stays a custom CCL kernel). Everything else compiles natively - the patch conv, all LayerNorms, both residual QLinearAdds per block, the qkv/proj/fc1/fc2 QGemms, the erf-GELU (as a qlut), and the CLS head.
The cell calls ChimeraJob(...).compile() - the same single call the other model notebooks use - which runs the CGC front/middle end + back end + Quadric LLVM build in one shot and emits the generated CCL C++ for the whole graph.
import custom_op_match
## waveAttention x6; everything else native
custom_op_match.prepare_custom_ops_graph(INT8_ONNX, CUSTOM_OP_ONNX, to_print=True)
co = Counter(n.op_type for n in onnx.load(CUSTOM_OP_ONNX).graph.node)
n_custom = sum(v for k, v in co.items() if k.startswith("QuadricCustomOp"))
native = {k: v for k, v in sorted(co.items()) if not k.startswith("QuadricCustomOp")}
print(f"custom ops : {n_custom} (waveAttention x{N_BLOCKS})")
print(f"native (CGC) : {native}\n")
## compile through CGC; custom_ops = local waveformer_ops.hpp providing nn::waveAttentionOp
cgc_job = ChimeraJob(
CUSTOM_OP_ONNX,
hw_config=HW,
trange_file=TRANGES,
target_lang="ASM",
custom_ops=custom_op_match.WAVE_ATTENTION_HEADER,
validate_iss=False,
)
cgc_job.compile()
print(cgc_job)
[match] loading waveformer_int8.onnx
[match] 11 custom ops; CGC lowers natively: {'DequantizeLinear': 28, 'Div': 2, 'Erf': 2, 'Gather': 1, 'LayerNormalization': 15, 'QGemm': 15, 'QLinearAdd': 14, 'QLinearConcat': 1, 'QLinearConv': 1, 'QLinearMul': 4, 'QuantizeLinear': 24, 'Reshape': 15, 'Transpose': 3}
[match] wrote waveformer_int8_custom_op.onnx
custom ops : 11 (waveAttention x6)
native (CGC) : {'DequantizeLinear': 28, 'Div': 2, 'Erf': 2, 'Gather': 1, 'LayerNormalization': 15, 'QGemm': 15, 'QLinearAdd': 14, 'QLinearConcat': 1, 'QLinearConv': 1, 'QLinearMul': 4, 'QuantizeLinear': 24, 'Reshape': 15, 'Transpose': 3}
2026-08-20 14:34 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-08-20 14:34 - INFO - epu - chimera_job - Numerical ranges provided
2026-08-20 14:34 - INFO - epu - codegen - START===============================optimize_relay
2026-08-20 14:34 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-08-20 14:34 - INFO - epu - fx - Clamped annotated range on /blocks.1/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-14.693208694458008, 15.295390129089355) -> (-14.693208694458008, 15.295389704406261) (within tol 0.120436)
2026-08-20 14:34 - INFO - epu - fx - Clamped annotated range on /blocks.2/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-17.228588104248047, 18.862335205078125) -> (-17.228588104248047, 18.862334311008453) (within tol 0.148522)
2026-08-20 14:34 - INFO - epu - fx - Clamped annotated range on /blocks.3/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-15.205657958984375, 16.93963623046875) -> (-15.205657958984375, 16.939636066555977) (within tol 0.133383)
2026-08-20 14:34 - INFO - epu - fx - Clamped annotated range on /blocks.5/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-13.569168090820312, 14.360703468322754) -> (-13.569168090820312, 14.360703274607658) (within tol 0.113076)
2026-08-20 14:34 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
------------------------------------------------- ----------------------------- ---------------------- --------------------
/patch_embed/Transpose_output_0_DequantizeLinear contrib.epu.dequantize [-1.54369f, 1.38568f] 30
/patch_embed/norm/LayerNormalization nn.layer_norm [-4.8421f, 4.83072f] 28
/Concat_output_0_DequantizeLinear contrib.epu.dequantize [-0.189842f, 4.82199f] 28
/blocks.0/norm1/LayerNormalization nn.layer_norm [-3.98641f, 7.42318f] 27
/blocks.0/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-9.86747f, 10.191f] 27
CustomOp/waveAttention0 contrib.epu.quadric_custom_op [-5.80482f, 7.42834f] 16
/blocks.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-16.4873f, 16.6182f] 26
/blocks.0/norm2/LayerNormalization nn.layer_norm [-4.29883f, 4.49691f] 28
/blocks.0/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-25.4366f, 28.588f] 26
/blocks.1/norm1/LayerNormalization nn.layer_norm [-4.55638f, 4.75343f] 28
/blocks.1/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-14.6932f, 15.2954f] 27
CustomOp/waveAttention1 contrib.epu.quadric_custom_op [-6.26072f, 5.9012f] 16
/blocks.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-35.7952f, 31.2498f] 25
/blocks.1/norm2/LayerNormalization nn.layer_norm [-5.17243f, 4.5562f] 28
/blocks.1/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-43.3882f, 40.2891f] 25
/blocks.2/norm1/LayerNormalization nn.layer_norm [-5.59709f, 4.7339f] 28
/blocks.2/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-17.2286f, 18.8623f] 26
CustomOp/waveAttention2 contrib.epu.quadric_custom_op [-7.09626f, 6.56859f] 16
/blocks.2/Add_output_0_DequantizeLinear contrib.epu.dequantize [-49.343f, 44.2922f] 25
/blocks.2/norm2/LayerNormalization nn.layer_norm [-5.38715f, 4.78134f] 28
/blocks.2/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-50.7397f, 47.1155f] 25
/blocks.3/norm1/LayerNormalization nn.layer_norm [-5.57496f, 5.23384f] 28
/blocks.3/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-15.2057f, 16.9396f] 26
CustomOp/waveAttention3 contrib.epu.quadric_custom_op [-6.79083f, 6.45284f] 16
/blocks.3/Add_output_0_DequantizeLinear contrib.epu.dequantize [-51.7598f, 49.6894f] 25
/blocks.3/norm2/LayerNormalization nn.layer_norm [-5.60962f, 5.0564f] 28
/blocks.3/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-51.7394f, 49.6698f] 25
/blocks.4/norm1/LayerNormalization nn.layer_norm [-5.45329f, 4.92244f] 28
/blocks.4/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-14.9916f, 14.5194f] 27
CustomOp/waveAttention4 contrib.epu.quadric_custom_op [-10.1218f, 10.8138f] 16
/blocks.4/Add_output_0_DequantizeLinear contrib.epu.dequantize [-55.8718f, 54.9849f] 25
/blocks.4/norm2/LayerNormalization nn.layer_norm [-5.38267f, 5.16993f] 28
/blocks.4/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-54.7764f, 54.3417f] 25
/blocks.5/norm1/LayerNormalization nn.layer_norm [-5.79005f, 5.02069f] 28
/blocks.5/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-13.5692f, 14.3607f] 27
CustomOp/waveAttention5 contrib.epu.quadric_custom_op [-9.16577f, 8.51346f] 16
/blocks.5/Add_output_0_DequantizeLinear contrib.epu.dequantize [-52.2163f, 51.8083f] 25
/blocks.5/norm2/LayerNormalization nn.layer_norm [-5.40404f, 4.50069f] 28
/blocks.5/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-66.6578f, 65.0955f] 24
/norm/LayerNormalization nn.layer_norm [-5.00211f, 4.32799f] 28
/Gather take [-3.8103f, 3.45511f] 28
/fc_norm/LayerNormalization nn.layer_norm [-3.87842f, 3.78483f] 29
logits_DequantizeLinear contrib.epu.dequantize [-8.80456f, 10.7517f] 27
2026-08-20 14:34 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-08-20 14:34 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-08-20 14:34 - INFO - epu - codegen - START=================================relay_to_tir
2026-08-20 14:34 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-08-20 14:34 - INFO - epu - codegen - START==============================adapt_and_order
2026-08-20 14:34 - INFO - epu - mac_counter -
2026-08-20 14:34 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:34 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-08-20 14:34 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:34 - INFO - epu - mac_counter - conv2d: 4,096,000 ops (2,048,000 MACs) - /patch_embed/proj/Conv_quant
2026-08-20 14:34 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-08-20 14:34 - INFO - epu - mac_counter - Total: 4,096,000 ops (2,048,000 MACs)
2026-08-20 14:34 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:34 - INFO - epu - mac_counter -
2026-08-20 14:34 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:35 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-08-20 14:36 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:36 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-08-20 14:37 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:37 - INFO - epu - codegen - START================================lrm_splitting
2026-08-20 14:39 - INFO - epu - fuse_weights - No optimization for node /patch_embed/proj/Conv_quant with properties { kernel_size: [1, 50], channels: 256, strides:[1, 50], padding:[0, 0, 0, 0], data_layout:NCHW, dilation:[1, 1], groups:1, kernel_layout:OIHW, out_dtype:int8, out_layout: }.Applying General Convolution algorithm. Performance for this node can be improved significantly. Please refer to the documentation or contact Quadric Support.
2026-08-20 14:39 - INFO - epu - codegen - START==============================ext_split_relay
2026-08-20 14:40 - INFO - epu - codegen - START====================================build_tir
2026-08-20 14:40 - INFO - epu - chimera_job - StackOverflow detected. Disabling LUT-init hoisting and retrying compilation.
2026-08-20 14:40 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-08-20 14:40 - INFO - epu - chimera_job - Numerical ranges provided
2026-08-20 14:41 - INFO - epu - codegen - START===============================optimize_relay
2026-08-20 14:41 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-08-20 14:41 - INFO - epu - fx - Clamped annotated range on /blocks.1/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-14.693208694458008, 15.295390129089355) -> (-14.693208694458008, 15.295389704406261) (within tol 0.120436)
2026-08-20 14:41 - INFO - epu - fx - Clamped annotated range on /blocks.2/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-17.228588104248047, 18.862335205078125) -> (-17.228588104248047, 18.862334311008453) (within tol 0.148522)
2026-08-20 14:41 - INFO - epu - fx - Clamped annotated range on /blocks.3/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-15.205657958984375, 16.93963623046875) -> (-15.205657958984375, 16.939636066555977) (within tol 0.133383)
2026-08-20 14:41 - INFO - epu - fx - Clamped annotated range on /blocks.5/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-13.569168090820312, 14.360703468322754) -> (-13.569168090820312, 14.360703274607658) (within tol 0.113076)
2026-08-20 14:41 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
------------------------------------------------- ----------------------------- ---------------------- --------------------
/patch_embed/Transpose_output_0_DequantizeLinear contrib.epu.dequantize [-1.54369f, 1.38568f] 30
/patch_embed/norm/LayerNormalization nn.layer_norm [-4.8421f, 4.83072f] 28
/Concat_output_0_DequantizeLinear contrib.epu.dequantize [-0.189842f, 4.82199f] 28
/blocks.0/norm1/LayerNormalization nn.layer_norm [-3.98641f, 7.42318f] 27
/blocks.0/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-9.86747f, 10.191f] 27
CustomOp/waveAttention0 contrib.epu.quadric_custom_op [-5.80482f, 7.42834f] 16
/blocks.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-16.4873f, 16.6182f] 26
/blocks.0/norm2/LayerNormalization nn.layer_norm [-4.29883f, 4.49691f] 28
/blocks.0/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-25.4366f, 28.588f] 26
/blocks.1/norm1/LayerNormalization nn.layer_norm [-4.55638f, 4.75343f] 28
/blocks.1/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-14.6932f, 15.2954f] 27
CustomOp/waveAttention1 contrib.epu.quadric_custom_op [-6.26072f, 5.9012f] 16
/blocks.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-35.7952f, 31.2498f] 25
/blocks.1/norm2/LayerNormalization nn.layer_norm [-5.17243f, 4.5562f] 28
/blocks.1/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-43.3882f, 40.2891f] 25
/blocks.2/norm1/LayerNormalization nn.layer_norm [-5.59709f, 4.7339f] 28
/blocks.2/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-17.2286f, 18.8623f] 26
CustomOp/waveAttention2 contrib.epu.quadric_custom_op [-7.09626f, 6.56859f] 16
/blocks.2/Add_output_0_DequantizeLinear contrib.epu.dequantize [-49.343f, 44.2922f] 25
/blocks.2/norm2/LayerNormalization nn.layer_norm [-5.38715f, 4.78134f] 28
/blocks.2/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-50.7397f, 47.1155f] 25
/blocks.3/norm1/LayerNormalization nn.layer_norm [-5.57496f, 5.23384f] 28
/blocks.3/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-15.2057f, 16.9396f] 26
CustomOp/waveAttention3 contrib.epu.quadric_custom_op [-6.79083f, 6.45284f] 16
/blocks.3/Add_output_0_DequantizeLinear contrib.epu.dequantize [-51.7598f, 49.6894f] 25
/blocks.3/norm2/LayerNormalization nn.layer_norm [-5.60962f, 5.0564f] 28
/blocks.3/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-51.7394f, 49.6698f] 25
/blocks.4/norm1/LayerNormalization nn.layer_norm [-5.45329f, 4.92244f] 28
/blocks.4/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-14.9916f, 14.5194f] 27
CustomOp/waveAttention4 contrib.epu.quadric_custom_op [-10.1218f, 10.8138f] 16
/blocks.4/Add_output_0_DequantizeLinear contrib.epu.dequantize [-55.8718f, 54.9849f] 25
/blocks.4/norm2/LayerNormalization nn.layer_norm [-5.38267f, 5.16993f] 28
/blocks.4/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-54.7764f, 54.3417f] 25
/blocks.5/norm1/LayerNormalization nn.layer_norm [-5.79005f, 5.02069f] 28
/blocks.5/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-13.5692f, 14.3607f] 27
CustomOp/waveAttention5 contrib.epu.quadric_custom_op [-9.16577f, 8.51346f] 16
/blocks.5/Add_output_0_DequantizeLinear contrib.epu.dequantize [-52.2163f, 51.8083f] 25
/blocks.5/norm2/LayerNormalization nn.layer_norm [-5.40404f, 4.50069f] 28
/blocks.5/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-66.6578f, 65.0955f] 24
/norm/LayerNormalization nn.layer_norm [-5.00211f, 4.32799f] 28
/Gather take [-3.8103f, 3.45511f] 28
/fc_norm/LayerNormalization nn.layer_norm [-3.87842f, 3.78483f] 29
logits_DequantizeLinear contrib.epu.dequantize [-8.80456f, 10.7517f] 27
2026-08-20 14:41 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-08-20 14:41 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-08-20 14:41 - INFO - epu - codegen - START=================================relay_to_tir
2026-08-20 14:41 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-08-20 14:41 - INFO - epu - codegen - START==============================adapt_and_order
2026-08-20 14:41 - INFO - epu - mac_counter -
2026-08-20 14:41 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:41 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-08-20 14:41 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:41 - INFO - epu - mac_counter - conv2d: 4,096,000 ops (2,048,000 MACs) - /patch_embed/proj/Conv_quant
2026-08-20 14:41 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-08-20 14:41 - INFO - epu - mac_counter - Total: 4,096,000 ops (2,048,000 MACs)
2026-08-20 14:41 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:41 - INFO - epu - mac_counter -
2026-08-20 14:41 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:41 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-08-20 14:41 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:41 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-08-20 14:42 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:42 - INFO - epu - codegen - START================================lrm_splitting
2026-08-20 14:44 - INFO - epu - fuse_weights - No optimization for node /patch_embed/proj/Conv_quant with properties { kernel_size: [1, 50], channels: 256, strides:[1, 50], padding:[0, 0, 0, 0], data_layout:NCHW, dilation:[1, 1], groups:1, kernel_layout:OIHW, out_dtype:int8, out_layout: }.Applying General Convolution algorithm. Performance for this node can be improved significantly. Please refer to the documentation or contact Quadric Support.
2026-08-20 14:44 - INFO - epu - codegen - START==============================ext_split_relay
2026-08-20 14:45 - INFO - epu - codegen - START====================================build_tir
2026-08-20 14:45 - INFO - epu - chimera_job - StackOverflow detected. Disabling qlut optimizations and retrying compilation.
2026-08-20 14:45 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-08-20 14:45 - INFO - epu - chimera_job - Numerical ranges provided
2026-08-20 14:45 - INFO - epu - codegen - START===============================optimize_relay
2026-08-20 14:46 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-08-20 14:46 - INFO - epu - fx - Clamped annotated range on /patch_embed/act_ft/Div to static range: (-3.423882007598877, 3.415832281112671) -> (-3.423882007598877, 3.4096643847449313) (within tol 0.0268478)
2026-08-20 14:46 - INFO - epu - fx - Clamped annotated range on /blocks.1/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-14.693208694458008, 15.295390129089355) -> (-14.693208694458008, 15.295389704406261) (within tol 0.120436)
2026-08-20 14:46 - INFO - epu - fx - Clamped annotated range on /blocks.2/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-17.228588104248047, 18.862335205078125) -> (-17.228588104248047, 18.862334311008453) (within tol 0.148522)
2026-08-20 14:46 - INFO - epu - fx - Clamped annotated range on /blocks.3/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-15.205657958984375, 16.93963623046875) -> (-15.205657958984375, 16.939636066555977) (within tol 0.133383)
2026-08-20 14:46 - INFO - epu - fx - Clamped annotated range on /blocks.5/attn/qkv/Gemm_output_0_DequantizeLinear to static range: (-13.569168090820312, 14.360703468322754) -> (-13.569168090820312, 14.360703274607658) (within tol 0.113076)
2026-08-20 14:46 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
---------------------------------------------------- ----------------------------- --------------------------------------- --------------------
/patch_embed/Transpose_output_0_DequantizeLinear contrib.epu.dequantize [-1.54369f, 1.38568f] 30
/patch_embed/norm/LayerNormalization nn.layer_norm [-4.8421f, 4.83072f] 28
contrib.epu.dequantize (-4.859961986541748, 4.821993533521891) 28
/patch_embed/act_ft/Div divide [-3.42388f, 3.41583f] 29
/patch_embed/act_ft/Erf erf [-0.999999f, 0.999999f] 30
/Concat_output_0_DequantizeLinear contrib.epu.dequantize [-0.189842f, 4.82199f] 28
/blocks.0/norm1/LayerNormalization nn.layer_norm [-3.98641f, 7.42318f] 27
/blocks.0/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-9.86747f, 10.191f] 27
CustomOp/waveAttention0 contrib.epu.quadric_custom_op [-5.80482f, 7.42834f] 16
/blocks.0/Add_output_0_DequantizeLinear contrib.epu.dequantize [-16.4873f, 16.6182f] 26
/blocks.0/norm2/LayerNormalization nn.layer_norm [-4.29883f, 4.49691f] 28
/blocks.0/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-25.4366f, 28.588f] 26
/blocks.1/norm1/LayerNormalization nn.layer_norm [-4.55638f, 4.75343f] 28
/blocks.1/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-14.6932f, 15.2954f] 27
CustomOp/waveAttention1 contrib.epu.quadric_custom_op [-6.26072f, 5.9012f] 16
/blocks.1/Add_output_0_DequantizeLinear contrib.epu.dequantize [-35.7952f, 31.2498f] 25
/blocks.1/norm2/LayerNormalization nn.layer_norm [-5.17243f, 4.5562f] 28
/blocks.1/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-43.3882f, 40.2891f] 25
/blocks.2/norm1/LayerNormalization nn.layer_norm [-5.59709f, 4.7339f] 28
/blocks.2/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-17.2286f, 18.8623f] 26
CustomOp/waveAttention2 contrib.epu.quadric_custom_op [-7.09626f, 6.56859f] 16
/blocks.2/Add_output_0_DequantizeLinear contrib.epu.dequantize [-49.343f, 44.2922f] 25
/blocks.2/norm2/LayerNormalization nn.layer_norm [-5.38715f, 4.78134f] 28
/blocks.2/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-50.7397f, 47.1155f] 25
/blocks.3/norm1/LayerNormalization nn.layer_norm [-5.57496f, 5.23384f] 28
/blocks.3/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-15.2057f, 16.9396f] 26
CustomOp/waveAttention3 contrib.epu.quadric_custom_op [-6.79083f, 6.45284f] 16
/blocks.3/Add_output_0_DequantizeLinear contrib.epu.dequantize [-51.7598f, 49.6894f] 25
/blocks.3/norm2/LayerNormalization nn.layer_norm [-5.60962f, 5.0564f] 28
/blocks.3/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-51.7394f, 49.6698f] 25
/blocks.4/norm1/LayerNormalization nn.layer_norm [-5.45329f, 4.92244f] 28
/blocks.4/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-14.9916f, 14.5194f] 27
CustomOp/waveAttention4 contrib.epu.quadric_custom_op [-10.1218f, 10.8138f] 16
/blocks.4/Add_output_0_DequantizeLinear contrib.epu.dequantize [-55.8718f, 54.9849f] 25
/blocks.4/norm2/LayerNormalization nn.layer_norm [-5.38267f, 5.16993f] 28
/blocks.4/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-54.7764f, 54.3417f] 25
/blocks.5/norm1/LayerNormalization nn.layer_norm [-5.79005f, 5.02069f] 28
/blocks.5/attn/qkv/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-13.5692f, 14.3607f] 27
CustomOp/waveAttention5 contrib.epu.quadric_custom_op [-9.16577f, 8.51346f] 16
/blocks.5/Add_output_0_DequantizeLinear contrib.epu.dequantize [-52.2163f, 51.8083f] 25
/blocks.5/norm2/LayerNormalization nn.layer_norm [-5.40404f, 4.50069f] 28
/blocks.5/mlp/mod/fc1/Gemm_output_0_DequantizeLinear contrib.epu.dequantize [-13.0614f, 13.1659f] 27
/blocks.5/mlp/mod/act/Div divide [-9.23584f, 9.30973f] 27
/blocks.5/mlp/mod/act/Erf erf [-1f, 1f] 30
/blocks.5/Add_1_output_0_DequantizeLinear contrib.epu.dequantize [-66.6578f, 65.0955f] 24
/norm/LayerNormalization nn.layer_norm [-5.00211f, 4.32799f] 28
/Gather take [-3.8103f, 3.45511f] 28
/fc_norm/LayerNormalization nn.layer_norm [-3.87842f, 3.78483f] 29
logits_DequantizeLinear contrib.epu.dequantize [-8.80456f, 10.7517f] 27
2026-08-20 14:46 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-08-20 14:46 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-08-20 14:46 - INFO - epu - codegen - START=================================relay_to_tir
2026-08-20 14:46 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-08-20 14:46 - INFO - epu - codegen - START==============================adapt_and_order
2026-08-20 14:46 - INFO - epu - mac_counter -
2026-08-20 14:46 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:46 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-08-20 14:46 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:46 - INFO - epu - mac_counter - conv2d: 4,096,000 ops (2,048,000 MACs) - /patch_embed/proj/Conv_quant
2026-08-20 14:46 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-08-20 14:46 - INFO - epu - mac_counter - Total: 4,096,000 ops (2,048,000 MACs)
2026-08-20 14:46 - INFO - epu - mac_counter - ============================================================
2026-08-20 14:46 - INFO - epu - mac_counter -
2026-08-20 14:46 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:46 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-08-20 14:46 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:46 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-08-20 14:47 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-08-20 14:47 - INFO - epu - codegen - START================================lrm_splitting
2026-08-20 14:48 - INFO - epu - fuse_weights - No optimization for node /patch_embed/proj/Conv_quant with properties { kernel_size: [1, 50], channels: 256, strides:[1, 50], padding:[0, 0, 0, 0], data_layout:NCHW, dilation:[1, 1], groups:1, kernel_layout:OIHW, out_dtype:int8, out_layout: }.Applying General Convolution algorithm. Performance for this node can be improved significantly. Please refer to the documentation or contact Quadric Support.
2026-08-20 14:48 - INFO - epu - codegen - START==============================ext_split_relay
2026-08-20 14:49 - INFO - epu - codegen - START====================================build_tir
2026-08-20 14:50 - INFO - epu - chimera_job - Compilation of waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1 successful
╒═════════════════════╤════════════════════════════════════════════════════════════════════╕
│ Module Name │ waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ ONNX File │ waveformer_int8_custom_op.onnx │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Custom Ops │ /quadric/sdk-cli/examples/models/waveformer/waveformer_ops.hpp │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-N │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 0.500 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 4MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 8GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 8GBps │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 8 │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 0.903MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 0.299kB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.629MB │
├─────────────────────┼────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 0.356 │
╘═════════════════════╧════════════════════════════════════════════════════════════════════╛
NOTE: CGC has used general convolution for some nodes, which may result in suboptimal performance. For performance-critical applications, please contact Quadric support to explore optimization strategies.
General convolution nodes: /patch_embed/proj/Conv_quant, /patch_embed/proj/Conv_quant, /patch_embed/proj/Conv_quant
For more details, see: https://app.quadric.ai/docs/latest/chimera-software-user-guide/chimera-graph-compiler-cgc/graph-optimizations-performed-by-cgc#general-convolution
╒════╤════════╤════════╤═════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪════════╪═════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input │ [1, 1, 8, 1000] │ tensor[FixedPoint32<30>] │ n/a │
├────┼────────┼────────┼─────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ logits │ [1, 6] │ tensor[FixedPoint32<27>] │ n/a │
╘════╧════════╧════════╧═════════════════╧══════════════════════════╧═══════╛
4. Build the EPU binary (sdk source)
Section 3 runs CGC and emits the generated CCL C++ for the whole graph under ccl_build/waveformer_int8_custom_op_QC_N_.../ - the nn::waveAttentionOp calls (one per block) plus every natively-compiled op. This step compiles that generated .cpp into the EPU binary + host executable with the SDK's sdk source tool (Quadric LLVM + the Chimera runtime + the waveAttention kernel), the same explicit build step the qwen / whisper notebooks use.
--include-cgc-headerspulls in CGC's generated headers (the custom-op prototypes / const buffers).- The
--target QC-N/ cores / OCM / MAC / bandwidth flags mirror theHWConfigused to compile.
Steps 4a-4b stage the input and run that build; section 5 validates the result against the ORT golden.
4a. Prepare input
The generated host reads its input as a raw input.bin from the build dir. This cell converts the sEMG window x from fp32 to fixed-point.
## Stage input.bin (sEMG window as FixedPoint32<30>) next to const_tensor_data.bin, where the host reads it.
INPUT_FRAC_BITS = 30
_BASE = "waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1"
input_fx = np.round(x.astype(np.float64) * (1 << INPUT_FRAC_BITS)).astype(np.int32)
input_fx.tofile(f"ccl_build/{_BASE}/build/input.bin")
print(
f"[input.bin] FixedPoint32<{INPUT_FRAC_BITS}> {tuple(input_fx.shape)} -> ccl_build/{_BASE}/build/"
)
[input.bin] FixedPoint32<30> (1, 1, 8, 1000) -> ccl_build/waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1/build/
4b. Compile and run on the ISS
sdk source compiles the CGC-generated .cpp with the Quadric LLVM toolchain and runs it on the ISS.
%%bash
set -euo pipefail
BASE=waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1
BD=${BASE}_QC-N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1
## wipe the build dir so find_package reconfigures against the current SDK_INSTALL_PATH (a stale
## CMakeCache would otherwise pin QuadricSdk_DIR to the previous SDK).
rm -rf "$BD"
## generated host reads input.bin + const_tensor_data.bin from its run dir (BD/output); stage them there.
mkdir -p "$BD/output"
cp "ccl_build/$BASE/build/input.bin" "ccl_build/$BASE/build/const_tensor_data.bin" "$BD/output/"
## --clock-freq-ghz 0.5 = 500 MHz (needs ALLOWED_CLK floor <= 0.5 in tvm hw_config.py -> dir 0d5).
sdk source ccl_build/$BASE/$BASE.cpp \
--include-cgc-headers \
--target QC-N \
--num-cores 1 \
--ocm-size 4MB \
--macs-per-pe 8 \
--clock-freq-ghz 0.5 \
--ext-read-bw 8GBps \
--ext-write-bw 8GBps \
--ddr-axi-width 256 \
--quiet
2026-08-20 14:50 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/models/waveformer/waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1_QC-N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1/build -DNUM_GPNPUS=1 -DNUM_CORES=8 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=4096 -DNUM_PE_MACS=8 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DSTACKOVERFLOW_ERROR=OFF -DHARDWARE_TARGET=OFF
2026-08-20 14:50 - DEBUG - sdk - cli - Executing command: make -j8
[SDK-CLI] : Executing on QC-N simulator
2026-08-20 14:50 - DEBUG - sdk - cli - Executing command: ./waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1_host -c --ddrRdBwTotal 65536.0 --ddrWrBwTotal 65536.0 --ddrAxiWidth 256 --instMemDepth 1310720 --ocmSize 4194304 --cycleTimeNS 2.0 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
[SDK-CLI] : TotalCycles: 8,681,144
[SDK-CLI] : Executions/second: 58
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇���▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 4.07M
data_array : ▇▇▇▇���▇▇▇▇ 795.54K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1.186M
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 2.068M
data_external: ▇▇▇▇▇▇ 523.706K
[SDK-CLI] : Execution completed.
5. Validate against ORT
Model output. The model produces [1,6] class logits - one score per gesture class - and its prediction is the argmax over them.
EPU vs ORT. This cell decodes the EPU logits from logits.bin and runs the same INT8 ONNX through ORT on the identical window, then compares the two.
Threshold. Both paths dequantize the same int8 output tensor, so their differences are whole multiples of the model's logits_scale (~0.0847) - on this window the per-class errors are exactly 1, 2, 2, 3, 4 and 5 of those steps. The threshold is therefore expressed in output steps rather than as an absolute constant: the cell asserts the argmax classes match and that max|EPU-ORT| stays within TOL_LSB = 6 steps (0.508), against a measured max of 5 steps (0.423). A clean run prints VALIDATED.
For scale: the logit span is ~10.8 (128 steps) and the argmax margin is ~1.6 (19 steps), so a 5-step error is ~3.9% of full scale and would have to grow ~2x to flip the prediction. Most of the residual is int8 quantization of the attention P.V product, which is design-inherent rather than a defect.
## Decode the EPU logits (FixedPoint32<27>) and validate vs ORT: argmax AND logit closeness.
import numpy as np
import onnx
import onnxruntime as ort
from onnx import numpy_helper
CLASSES = [
"hand relax",
"hand close",
"wrist flexion",
"wrist extension",
"ulnar dev.",
"radial dev.",
]
## EPU and ORT land on the same int8 output grid: every observed difference is a whole
## multiple of the model's own logits_scale (1-5 steps on this window). Bounding in grid
## units ties the threshold to the model's quantization instead of an arbitrary constant.
## An absolute 0.2 would be ~2.4 steps -- below the noise floor of a 6-block int8
## requantization chain; a broken RoPE/layout is >50 steps.
_g = onnx.load("waveformer_int8.onnx").graph
_init = {i.name: numpy_helper.to_array(i) for i in _g.initializer}
_deq = next(n for n in _g.node if _g.output[0].name in n.output) # logits DequantizeLinear
LOGIT_LSB = float(_init[_deq.input[1]]) # int8 output step (logits_scale), ~0.0847
TOL_LSB = 6 # 6 output steps = 0.508; measured max is 5 steps (0.423)
TOL = TOL_LSB * LOGIT_LSB
_BD = "waveformer_int8_custom_op_QC_N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1_QC-N_0d5_4MB_4kB_8GBps_8GBps_8_OFF_x1_x1"
epu = np.fromfile(f"{_BD}/output/logits.bin", dtype=np.int32).astype(np.float64) / (1 << 27)
_x = np.load("data/test_x.npy").astype(np.float32)[0:1]
_ort = (
ort.InferenceSession("waveformer_int8.onnx", providers=["CPUExecutionProvider"])
.run(None, {"input": _x})[0]
.flatten()
)
ep, op = int(np.argmax(epu)), int(np.argmax(_ort))
max_diff = float(np.abs(epu - _ort).max())
_top2 = np.sort(_ort)[::-1]
margin = float(_top2[0] - _top2[1])
print(f"EPU logits : {np.round(epu, 3).tolist()} -> {ep} ({CLASSES[ep]})")
print(f"ORT logits : {np.round(_ort, 3).tolist()} -> {op} ({CLASSES[op]})")
print(
f"argmax {'match' if ep == op else 'MISMATCH'} "
f"max|EPU-ORT| = {max_diff:.3f} = {max_diff / LOGIT_LSB:.1f} LSB "
f"(tol {TOL:.3f} = {TOL_LSB} LSB)"
)
print(
f"argmax margin : {margin:.3f} ({margin / LOGIT_LSB:.1f} LSB) -- a flip needs {margin / 2:.3f}"
)
assert ep == op, f"argmax mismatch: EPU {ep} vs ORT {op}"
assert max_diff <= TOL, (
f"logits diverge: max|EPU-ORT|={max_diff:.3f} ({max_diff / LOGIT_LSB:.1f} LSB) "
f"> {TOL:.3f} ({TOL_LSB} LSB) (argmax-correct-by-luck?)"
)
print("VALIDATED")
EPU logits : [-2.878, -0.762, -4.487, 6.095, -4.656, 4.318] -> 3 (wrist extension)
ORT logits : [-2.7939999103546143, -0.847000002861023, -4.317999839782715, 5.926000118255615, -4.656000137329102, 4.572000026702881] -> 3 (wrist extension)
argmax match max|EPU-ORT| = 0.254 = 3.0 LSB (tol 0.508 = 6 LSB)
argmax margin : 1.355 (16.0 LSB) -- a flip needs 0.677
VALIDATED
'%3e%3crect%20width='512'%20height='512'%20rx='60'%3e%3c/rect%3e%3cpath%20fill='%23fff'%20d='M355.904%20100H408.832L293.2%20232.16L429.232%20412H322.72L239.296%20302.928L143.84%20412H90.8805L214.56%20270.64L84.0645%20100H193.28L268.688%20199.696L355.904%20100ZM337.328%20380.32H366.656L177.344%20130.016H145.872L337.328%20380.32Z'%3e%3c/path%3e%3c/g%3e%3cdefs%3e%3cclipPath%20id='clip0_84_15698'%3e%3crect%20width='512'%20height='512'%20fill='%23fff'%3e%3c/rect%3e%3c/clipPath%3e%3c/defs%3e%3c/svg%3e)
