Tutorial: Voxel Pooling

NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/custom_op/voxel_pooling/voxel_pooling.ipynb.

Voxel pooling with the SDK kernel the autofinder finds

Voxel pooling is the scatter step of BEVDepth-style networks: every one of the 473,088 camera points carries 80 feature channels and a grid cell, and the channels of the points landing in the same cell are summed into a 128 by 128 grid. It has no native PyTorch implementation and is usually a CUDA kernel. The SDK ships a CCL version of it, nn::voxelPooling, exercised by the sweep test sdk/tests/sweep_tests/voxel_pooling/.

The model here, voxel_original.onnx, was exported from PyTorch with a symbolic function so the exporter emits one VoxelPooling node with two inputs. That node has no ONNX schema and no entry in CGC's convert map, so CGC cannot compile the model as exported.

class VoxelPoolingFunction(torch.autograd.Function):
    @staticmethod
    def forward(ctx, img_features, geom):
        return img_features + geom  # stand-in; the ONNX export only needs `symbolic`

    @staticmethod
    def symbolic(g, img_features, geom):
        return g.op("mydomain::VoxelPooling", img_features, geom)

This notebook uses the plugin's autofinder to find the SDK kernel for that node, generates a bundle whose adapter forwards to it, compiles the model with one ChimeraJob(...).compile(), runs it on the ISS with the recorded inputs, and checks the grid against the NumPy reference the SDK sweep test uses. Nothing here is written by hand: no replacer call, no header. For the step-by-step dry run with the reference statistics, which stops before the compile, see the autofinder tutorial.

1. Setup

Imports the plugin and its autofinder. load_reference() reads sdk_sweep_reference.json from the installed sdk_cli package; staleness_warning returns a message when the SDK commit that reference was built from differs from the toolchain in use. Creates _plugin_out/ beside the notebook, rebuilt on every run.

import glob
import importlib.util
import shutil
import time
from pathlib import Path

import numpy as np
import onnx

from sdk_cli.cgc_plugin import cgc_native_ops, discover_candidate_ops, pattern_from_nodes
from sdk_cli.cgc_plugin.autofinder.adapter import SweepCall, generate_bundle
from sdk_cli.cgc_plugin.autofinder.propose_bundles import propose
from sdk_cli.cgc_plugin.autofinder.reference import (
    describe_reference,
    load_reference,
    staleness_warning,
)
from sdk_cli.cgc_plugin.chimera_job import ChimeraJob
from sdk_cli.utils.model_helpers import example_path

example_dir = example_path("custom_op/voxel_pooling")
model_path = example_dir / "voxel_original.onnx"
inputs_npy = example_dir / "voxel_pooling_inputs.npy"

work_dir = example_dir / "_plugin_out"
shutil.rmtree(work_dir, ignore_errors=True)
work_dir.mkdir()

reference = load_reference()
print(f"reference : {describe_reference(reference)}")
print(
    f"toolchain : {staleness_warning(reference) or 'built from the same SDK the reference describes'}"
)
print(f"model     : {model_path}")

reference : sdk@3f1b472d0 on devel (detached), 275 sweep-test dirs
toolchain : the shipped sweep reference describes sdk@3f1b472d0 but the toolchain in use is sdk@8b91cc4; kernels may have moved. Rebuild it: python -m sdk_cli.cgc_plugin.autofinder.build_reference
model     : /quadric/sdk-cli/examples/custom_op/voxel_pooling/voxel_original.onnx

2. The op CGC cannot lower

cgc_native_ops() returns the op-types in CGC's convert map. discover_candidate_ops groups the model's nodes by op-type and keeps the types not in that set. pattern_from_nodes prints the topology pattern written from the VoxelPooling node. The assertion pins it as the only such op-type.

model = onnx.load(str(model_path))
native_ops = cgc_native_ops()
candidates = discover_candidate_ops(model, native_ops=native_ops)

print(f"op-types in the model    : {sorted({n.op_type for n in model.graph.node})}")
print(f"not in CGC's convert map : {list(candidates)}")
pattern, output = pattern_from_nodes(model, candidates["VoxelPooling"][:1])
print(f"\npattern (output = {output}):\n{pattern}")

assert list(candidates) == ["VoxelPooling"], candidates

op-types in the model    : ['Cast', 'Constant', 'DequantizeLinear', 'Identity', 'QuantizeLinear', 'VoxelPooling']
not in CGC's convert map : ['VoxelPooling']

pattern (output = voxelpooling0):
in0 = *
in1 = *
voxelpooling0 = VoxelPooling(in0, in1)

3. Find the SDK kernel

propose(model_path, out_dir, None, False) is a dry run: for each op-type CGC cannot lower it prints the pattern, the proposed bundle name and whether the reference holds a sweep test for it. The assertion pins the answer to the voxel_pooling sweep directory. SweepCall.from_dict rebuilds the kernel call parsed from that sweep test, and the cell prints its symbol, its arguments, and for each tensor argument the role, dtype and shape the sweep generator declares.

proposals = propose(model_path, work_dir / "proposed", None, False)

proposal = proposals[0]
assert proposal["op_type"] == "VoxelPooling", proposals
assert proposal["sdk_kernel"] and proposal["sdk_kernel"]["dir"] == "voxel_pooling", proposal

call = SweepCall.from_dict(proposal["sdk_kernel"]["call"])
print(f"\nsweep_tests/{call.sweep_dir}: {call.symbol}({', '.join(call.args)})")
for arg in call.args:
    if arg == call.allocator:
        print(f"  {arg:12} OCM allocator")
        continue
    dtype, dims = call.meta[call.roles[arg]]
    print(f"  {arg:12} {call.roles[arg]:14} {dtype:24} {list(dims)}")

model            : /quadric/sdk-cli/examples/custom_op/voxel_pooling/voxel_original.onnx
sdk reference    : sdk@3f1b472d0 on devel (detached), 275 sweep-test dirs


the shipped sweep reference describes sdk@3f1b472d0 but the toolchain in use is sdk@8b91cc4; kernels may have moved. Rebuild it: python -m sdk_cli.cgc_plugin.autofinder.build_reference


!! the shipped sweep reference describes sdk@3f1b472d0 but the toolchain in use is sdk@8b91cc4; kernels may have moved. Rebuild it: python -m sdk_cli.cgc_plugin.autofinder.build_reference
sweep tests known: 275 kernel dirs

op-types CGC cannot lower: {'VoxelPooling': 1}

==============================================================================
VoxelPooling  x1  ->  bundle 'voxelpooling'
==============================================================================
pattern (by example, from the first occurrence):
    in0 = *
    in1 = *
    voxelpooling0 = VoxelPooling(in0, in1)

SDK kernel : EXISTS -- sweep_tests/voxel_pooling
             symbols in it: voxelPooling
             -> FORWARD to it. That test shows the call, the tensor types it
                expects and the staging. Do not reimplement it.

(dry run -- pass --write to scaffold this bundle)

sweep_tests/voxel_pooling: nn::voxelPooling(ddrGeom, ddrFeatures, ddrOut, ocmMem)
  ddrGeom      GeomShape      DataType(EPU_TYPE.INT16) [1, 1, None, 3]
  ddrFeatures  FeaturesShape  test_case.in_type        [1, 1, None, None]
  ddrOut       OutShape       test_case.out_type       [1, None, None, None]
  ocmMem       OCM allocator

4. Generate the bundle

generate_bundle assigns the node's inputs to the kernel's arguments by rank and literal dims (the geometry's trailing 3 fixes the order), renders the forwarding adapter quadric_plugin::voxelPoolingStub, and adds an eliminate_identity post-pass because an Identity feeds the node and CGC rejects an integer copy. options carries the two values the sweep test cannot know: allow_io_in_l2_mem and a 1 MB reserved_ocm rule clamped to a quarter of the part's OCM. Prints the generated files.

plugins_dir = work_dir / "plugins"
bundle = generate_bundle(
    model,
    candidates["VoxelPooling"],
    call,
    plugins_dir,
    name="voxel_pooling",
    options={
        "allow_io_in_l2_mem": True,
        "reserved_ocm": {"builtin": "fixed", "params": {"bytes": 1048576, "clamp": "quarter"}},
    },
    description=(
        "VoxelPooling -> quadric_plugin::voxelPoolingStub -> nn::voxelPooling, the kernel the SDK "
        "sweep test voxel_pooling/ exercises. Adapter generated by voxel_pooling.ipynb."
    ),
)
print(f"bundle: {bundle}  files: {sorted(p.name for p in bundle.iterdir())}\n")
print("--- kernel.hpp ---")
print((bundle / "kernel.hpp").read_text())
print("--- plugin.yaml ---")
print((bundle / "plugin.yaml").read_text())

bundle: /quadric/sdk-cli/examples/custom_op/voxel_pooling/_plugin_out/plugins/voxel_pooling  files: ['kernel.hpp', 'plugin.yaml']

--- kernel.hpp ---
// VoxelPooling: adapter over nn::voxelPooling, generated from the SDK sweep test.
// Forwards only. One overload per operand order the sweep shapes allow; C++ picks the
// one CGC binds.
using namespace chimera;

namespace quadric_plugin {

template <std::uint32_t d0_2, typename T0, std::uint32_t d1_2, std::uint32_t d1_3, typename T1, typename OutTensor, typename OcmAllocatorType>
INLINE void voxelPoolingStub(DdrTensor<T0, 1, 1, d0_2, 3>& in0, DdrTensor<T1, 1, 1, d1_2, d1_3>& in1, OutTensor& out, OcmAllocatorType& ocmMem) {
  nn::voxelPooling(in0, in1, out, ocmMem);
}

}  // namespace quadric_plugin

--- plugin.yaml ---
name: voxel_pooling
description: VoxelPooling -> quadric_plugin::voxelPoolingStub -> nn::voxelPooling, the kernel the SDK
  sweep test voxel_pooling/ exercises. Adapter generated by voxel_pooling.ipynb.
impl_type: ccl
kernel:
  file: kernel.hpp
  ccl_func_name: quadric_plugin::voxelPoolingStub
  element_wise: false
match:
  mode: topology
  output: voxelpooling0
  topology: |
    in0 = *
    in1 = *
    voxelpooling0 = VoxelPooling(in0, in1)
options:
  allow_io_in_l2_mem: true
  reserved_ocm:
    builtin: fixed
    params:
      bytes: 1048576
      clamp: quarter
postprocess:
  reason: Identity ['geom_xyz_identity'] feeds VoxelPooling; CGC's fx canonicalizer cannot range-propagate
    an int copy
  onnxoptimizer_passes:
  - eliminate_identity

5. Compile

Removes stale ccl_build/voxel_original* folders, builds the plugin's ChimeraJob on the original model with plugins_dir pointing at the generated bundle, and compiles. The constructor replaces the VoxelPooling node with a customOp bound to the adapter and hands CGC the substituted graph and the header; region_timeout is raised for slow machines. Prints the compile time, plugin_result, the op-types of the substituted graph, and the line in the emitted CCL that calls the adapter. The assertions pin the matched bundle and exactly one such call.

for stale_build in glob.glob("ccl_build/voxel_original*"):
    shutil.rmtree(stale_build, ignore_errors=True)

compile_start = time.perf_counter()
cgc_job = ChimeraJob(
    str(model_path),
    plugins_dir=str(plugins_dir),
    plugin_out_dir=str(work_dir / "compiled"),
    region_timeout=3600,
)
cgc_job.compile(quiet=True)
print(f"compiled in {time.perf_counter() - compile_start:.1f}s")
print(f"plugin_result: {cgc_job.plugin_result}")
assert cgc_job.plugin_result.matched_plugins == ("voxel_pooling",), cgc_job.plugin_result

substituted = onnx.load(cgc_job.plugin_result.model_path)
print(f"op-types after substitution: {sorted({n.op_type for n in substituted.graph.node})}")

generated_ccl = next(Path(p) for p in glob.glob("ccl_build/voxel_original*/*.cpp"))
stub_calls = [
    line.strip()
    for line in generated_ccl.read_text().splitlines()
    if "quadric_plugin::voxelPoolingStub(" in line
]
print(f"{generated_ccl.name}: {stub_calls[0]}")
assert len(stub_calls) == 1, stub_calls

WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: output
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: scale
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: zero_point
/usr/local/lib/python3.10/dist-packages/tvm/relay/frontend/onnx.py:6495: UserWarning: No Op registered for VoxelPooling with domain_version of 16

==> Context: Bad node spec for node. Name: CustomOp/quadric_plugin::voxelPoolingStub0 OpType: QuadricCustomOp
  warnings.warn(str(e))


compiled in 59.5s
plugin_result: cgc plugin: 1 customOp(s) from voxel_pooling -> /quadric/sdk-cli/examples/custom_op/voxel_pooling/_plugin_out/compiled
op-types after substitution: ['Cast', 'Constant', 'DequantizeLinear', 'QuadricCustomOp', 'QuantizeLinear']
voxel_original_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.cpp: quadric_plugin::voxelPoolingStub(input_geom_xyz, ext_tensor_0, ext_tensor_1, mem_allocator) /* CustomOp/quadric_plugin::voxelPoolingStub0 */ ;

6. Run on the ISS

Loads voxel_pooling_inputs.npy (473,088 points with 80 feature channels into a 128 by 128 grid), reshapes the two inputs to 4-D, and runs run_inference_harness with compare_ort=False, because ONNX Runtime has no VoxelPooling implementation. Plots the run statistics and prints the job summary; the region holding the kernel call dominates the cycle count.

data = np.load(str(inputs_npy), allow_pickle=True).item()
geom_xyz = data["geom_xyz"].reshape(1, 1, 473088, 3)
img_feat = data["img_feat_with_depth"].reshape(1, 1, 473088, 80)

iss_outputs = cgc_job.run_inference_harness(
    compare_ort=False, inputs={"geom_xyz": geom_xyz, "img_feat_with_depth": img_feat}
)
cgc_job.plot_run_statistics()
print(cgc_job)

2026-10-23 00:14 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x78c0cc7e7d60>
2026-10-23 00:14 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x78c0cc7e4760>
FILM 3/3: 100%|██████████████████████████████████████████████████████| 3/3 [17:49<00:00, 356.55s/it]
2026-10-23 00:34 - INFO - epu - chimera_job - Combined plots generated and saved to: 
/quadric/sdk-cli/examples/custom_op/voxel_pooling/ccl_build/voxel_original_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20261023_001428_af6d15/data/voxel_original_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png



╒═════════════════════╤════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name         │ voxel_original_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1                              │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File           │ /quadric/sdk-cli/examples/custom_op/voxel_pooling/_plugin_out/compiled/voxel_original.onnx │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Custom Ops          │ /quadric/sdk-cli/examples/custom_op/voxel_pooling/_plugin_out/compiled/custom_ops.hpp      │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target      │ QC-U                                                                                       │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores     │ 1                                                                                          │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700                                                                                      │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size            │ 16MB                                                                                       │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size            │ 4kB                                                                                        │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW    │ 128GBps                                                                                    │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW   │ 128GBps                                                                                    │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE         │ 16                                                                                         │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M             │ 6.250MB                                                                                    │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM             │ 0.391kB                                                                                    │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes  │ 37.344MB                                                                                   │
├─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs       │                                                                                            │
╘═════════════════════╧════════════════════════════════════════════════════════════════════════════════════════════╛

╒════╤════════╤═════════════════════╤════════════════════╤══════════════════════════╤═══════╕
│    │ Type   │ Name                │ shape              │ type                     │ mse   │
╞════╪════════╪═════════════════════╪════════════════════╪══════════════════════════╪═══════╡
│  0 │ Input  │ img_feat_with_depth │ [1, 1, 473088, 80] │ tensor[FixedPoint32<27>] │ n/a   │
├────┼────────┼─────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  1 │ Input  │ geom_xyz            │ [1, 1, 473088, 3]  │ tensor[int32]            │ n/a   │
├────┼────────┼─────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  2 │ Output │ output              │ [1, 80, 128, 128]  │ tensor[int32]            │ n/a   │
╘════╧════════╧═════════════════════╧════════════════════╧══════════════════════════╧═══════╛

Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation: 
╒══════════════════════════════════╤═════════╕
│ Latency (ms)                     │   39.71 │
├──────────────────────────────────┼─────────┤
│ FPS                              │   25.18 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW)   │ 3716.98 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │    6.77 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB)                │  216.01 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB)                │   42.34 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps)             │    5.31 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps)             │    1.04 │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm

[SDK-CLI] : TotalCycles: 67,508,159
[SDK-CLI] : Executions/second: 25.18

compute      : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 36.725M
data_array   : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 10.303M
mac          :  0
data_external: ▏ 105.682K
data_ocm     : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 20.371M

for more information check run directory: /quadric/sdk-cli/examples/custom_op/voxel_pooling/ccl_build/voxel_original_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20261023_001428_af6d15

7. Accuracy against the SDK's reference

Runs the graph's own QuantizeLinear node (with its Constant operands) through ONNX Runtime to get the int8 features the kernel consumed. Loads voxel_pooling_train_forward from voxel_pooling_oracle.py, a verbatim copy of the function in the SDK's sweep_runner.py, and computes the expected grid. The graph's output is int8 (its DequantizeLinear has scale 1), so a voxel whose sum leaves the int8 range cannot be represented; the check is bit-exactness on every voxel whose sum fits, and the cell prints the count outside the range, the correlation and the count that matches modulo 256.

import onnxruntime as ort
from onnx import helper

## The int8 features the kernel consumed: the graph's own QuantizeLinear, run in ORT.
quantize_node = next(node for node in model.graph.node if node.op_type == "QuantizeLinear")
constant_nodes = [node for node in model.graph.node if node.op_type == "Constant"]
feat_input = next(i for i in model.graph.input if i.name == "img_feat_with_depth")
quantize_graph = helper.make_graph(
    constant_nodes + [quantize_node],
    "quantize_only",
    [feat_input],
    [
        helper.make_tensor_value_info(
            quantize_node.output[0], onnx.TensorProto.INT8, list(img_feat.shape)
        )
    ],
)
quantize_session = ort.InferenceSession(
    helper.make_model(quantize_graph, opset_imports=model.opset_import).SerializeToString(),
    providers=["CPUExecutionProvider"],
)
img_feat_q = quantize_session.run(None, {"img_feat_with_depth": img_feat.astype(np.float32)})[0]

## The SDK sweep test's NumPy reference.
oracle_spec = importlib.util.spec_from_file_location(
    "voxel_pooling_oracle", example_dir / "voxel_pooling_oracle.py"
)
oracle = importlib.util.module_from_spec(oracle_spec)
oracle_spec.loader.exec_module(oracle)
expected = oracle.voxel_pooling_train_forward(
    geom_xyz, img_feat_q, tuple(int(v) for v in data["voxel_num"])
).astype(np.int64)

output_name = model.graph.output[0].name
iss_grid = np.asarray(iss_outputs[output_name]).reshape(expected.shape).astype(np.int64)
fits_int8 = (expected >= -128) & (expected <= 127)
exact = iss_grid == expected
corr = float(np.corrcoef(iss_grid.ravel(), expected.ravel())[0, 1])
wrapped_to_int8 = ((expected + 128) % 256) - 128

print(f"grid                          : {expected.shape}, {expected.size} voxels x channels")
print(f"exact match, all              : {exact.sum()}/{exact.size}")
print(f"exact match where sum fits i8 : {exact[fits_int8].sum()}/{fits_int8.sum()}")
print(f"sums outside int8 range       : {(~fits_int8).sum()}")
print(f"correlation ISS vs reference  : {corr:.6f}")
print(f"ISS == reference modulo 256   : {(iss_grid == wrapped_to_int8).sum()}/{exact.size}")

assert exact[
    fits_int8
].all(), "the kernel must be bit-exact wherever the int8 output can hold the sum"
print("accuracy proof PASSED")

grid                          : (1, 80, 128, 128), 1310720 voxels x channels
exact match, all              : 1310509/1310720
exact match where sum fits i8 : 1310509/1310509
sums outside int8 range       : 211
correlation ISS vs reference  : 0.925310
ISS == reference modulo 256   : 1310720/1310720
accuracy proof PASSED