NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/custom_op/ultraface_custom_op.ipynb.
import tvm.contrib.epu.graphutils as graphutils
import onnx
import os, pathlib
import logging
import onnxruntime
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
onnx_file = "./ultraface_quant_v2.onnx"
onnx_session = onnxruntime.InferenceSession(onnx_file)
model = onnx.load(onnx_file)
start
util = graphutils.CustomOpReplacer(model)
_, custom_model = util.replace_subgraph(
"concat_output_1",
[
"Transpose_48",
"Transpose_70",
"Transpose_88",
"Transpose_102",
"Transpose_53",
"Transpose_75",
"Transpose_93",
"Transpose_105",
],
"usercustom::ultrafaceBoundingBox",
False,
keep_constants=[
"317_scale",
"363_scale",
"403_scale",
"435_scale",
"331_scale",
"377_scale",
"417_scale",
"447_scale",
"467_quantized",
"467_scale",
"465_quantized",
"465_scale",
],
)
onnx_file_custom = os.path.basename(onnx_file.replace(".onnx", "-custom.onnx"))
print(onnx_file_custom)
onnx.save(custom_model, onnx_file_custom)
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: concat_output_0
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: scores_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 459_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 328_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 317_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 316
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 315_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 314
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 313_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 309_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 308_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 289_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 288
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 286_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 284_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 283
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 281_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 280
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 278_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 277
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 275_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 274
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 272_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 271
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 269_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 268
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 266_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 265
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 263_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 262
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 260_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 259
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 257_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 256
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 254_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 253
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 251_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 250
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 248_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 247
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 245_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 296_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 295
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 293_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 291_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 306_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 305
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 303_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 302
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 300_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 298_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 311_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 374_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 363_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 362
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 361_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 360
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 358_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 357
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 355_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 354
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 352_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 351
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 349_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 348
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 346_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 345
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 343_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 414_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 403_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 402
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 401_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 400
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 398_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 397
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 395_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 394
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 392_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 391
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 389_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 446_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 435_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 434
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 433_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 432
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 431_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 430
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 429_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: boxes_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 480
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 481
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 475_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 468_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 466_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 464_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 462
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 460_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 342_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 331_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 330
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 329_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 388_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 377_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 376
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 375_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 428_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 417_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 416
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 415_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 458_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 447_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 474_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 472
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 471_quantized
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 469
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 484
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 482
WARNING:root:ONNX node does not have a name, deriving temp name from its outputs. Node was named to: 485_quantized
ultraface_quant_v2-custom.onnx
end
cgc_job = ChimeraJob(model_p=onnx_file_custom, custom_ops="./ultraface_scores.hpp")
cgc_job.analyze_network()
2026-06-19 03:53 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 03:53 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 03:53 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 03:53 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
------------------------------------------ ----------------------------- ----------------------------- --------------------
CustomOp/usercustom::ultrafaceBoundingBox0 contrib.epu.quadric_custom_op (-32768.0, 32767.99998474121) 16
Analysis of ultraface_quant_v2-custom.onnx
2026-06-19 03:53 - INFO - epu - chimera_job - Function Prototypes for Custom OPs found
2026-06-19 03:53 - INFO - epu - chimera_job - Please see the file: ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp for a template example of custom op implementation
List of custom ops found in the network:
usercustom::ultrafaceBoundingBox
2026-06-19 03:53 - INFO - epu - chimera_job - Wrote custom op seed file to ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp.
'ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1_analyze_custom_op_template.hpp'
## implementation:
!cat ultraface_scores.hpp
#pragma once
#include <quadric/qil.h>
namespace chimera {
namespace usercustom {
/**
* @brief This handles computation of a single input for scores. This is called on for each of the 4 inputs.
* A minRoi is setup with shape (depth, core_dim, width). Tiles within minRoi are fetched along channels first and
* then width. MinRoi moves only in height within tensor. An output tile is generated after every 2 tiles since we
* already have enough data to compute a region of the output. This output tile is then
* written to OCM tensor via RAU.
*
* @tparam ScoresInputOcmTensor
* @tparam ScoresScaleFB
* @tparam ScoresOutputOcmTensor
* @param scoresInputOcm
* @param scoresScale
* @param scoresOutputOcm
* @param outWidthOffset
*/
template <typename ScoresInputOcmTensor, FracRepType ScoresScaleFB, typename ScoresOutputOcmTensor>
INLINE void ultrafaceBoundingBoxScoresSingleInput(ScoresInputOcmTensor& scoresInputOcm,
const FixedPoint32<ScoresScaleFB>& scoresScale,
ScoresOutputOcmTensor& scoresOutputOcm,
std::int32_t outWidthOffset) {
using inpElemType = typename ScoresInputOcmTensor::elemType;
using elemType = typename ScoresOutputOcmTensor::elemType;
constexpr std::int32_t depth = ScoresInputOcmTensor::NUM_CHN;
constexpr std::int32_t height = ScoresInputOcmTensor::NUM_ROWS;
constexpr std::int32_t width = ScoresInputOcmTensor::NUM_COLS;
constexpr std::int32_t numTiles = ScoresInputOcmTensor::NUM_TILES;
// Note: ROI, and input ocm tensor are the same shape for all inputs
using MinRoiTensor = OcmTensor<inpElemType, depth, core_array::coreDim, width>;
// MinRoiDesc describes how you iterate and process tiles within minRoi
using MinRoiDesc = MinRoiDescriptor<MinRoiTensor,
AxisGroup<Direction::Channel, Direction::Width, Direction::Height>,
Granularity::Square>;
// Access describes the overall access description including MinRoiDesc and movement of minRoi within ROI
using Access = TensorAccessor<MinRoiDesc, AxisGroup<Direction::Height, Direction::Channel, Direction::Width>>;
// readFlow is the final object that connects the Access to the ROI
ReadFlow<Access, ScoresInputOcmTensor> readFlow(scoresInputOcm);
rau::config(scoresOutputOcm);
constexpr std::int32_t numTilesInMinRoi = Access::MinRoiDesc::numTiles();
// loop over minRois is handled by heightOffset loop; minRois only move in height
for(std::int32_t heightOffset = 0; heightOffset < height; heightOffset += core_array::coreDim) {
container::NDArray<qVar_t<inpElemType>, numTilesInMinRoi> qInput = readFlow.read();
std::int32_t minRoiTileIdx = 0;
// widthOffset and ChannelOffset loop over tiles within minROI
for(std::int32_t widthOffset = 0; widthOffset < width; widthOffset += core_array::coreDim) {
// skip by two since we will process two channels at a time.
for(std::int32_t channelOffset = 0; channelOffset < depth; channelOffset += 2) {
qVar_t<std::uint32_t> qInWidth = widthOffset + qCol<>;
qVar_t<std::uint32_t> qInHeight = heightOffset + qRow<>;
qVar_t<std::uint32_t> qOutWidth =
outWidthOffset + channelOffset / 2 + (depth / 2) * qInWidth + (depth / 2) * width * qInHeight;
// calculate soft max
qVar_t<elemType> qX0 = static_cast<elemType>(qInput[minRoiTileIdx]) * scoresScale;
qVar_t<elemType> qX1 = static_cast<elemType>(qInput[minRoiTileIdx + 1]) * scoresScale;
qVar_t<elemType> qExp = (qX0 > qX1) ? math::exp(qX1 - qX0) : math::exp(qX0 - qX1);
qVar_t<elemType> qOutput0 = (qX0 > qX1) ? 1 / (1 + qExp) : qExp / (1 + qExp);
qVar_t<elemType> qOutput1 = 1 - qOutput0;
qVar_t<bool> qValid = (qInWidth < width) && (qInHeight < height);
// calculate rau store addresses to transfer to OCM
// depends on offsets
qVar_t<std::uint32_t> qAddress = rau::computeAddr<ScoresOutputOcmTensor>(0, 0, 0, qOutWidth);
if(qValid) {
rau::store::oneTile(qAddress, qOutput0, scoresOutputOcm);
qAddress = rau::computeAddr<ScoresOutputOcmTensor>(0, 0, 1, qOutWidth);
rau::store::oneTile(qAddress, qOutput1, scoresOutputOcm);
}
minRoiTileIdx += 2;
}
}
}
}
/**
* @brief This is a helper function for calculating score elements (this is independent from coordinates)
*
* @tparam ScoresInputDdr0Tensor
* @tparam ScoresScale0FB
* @tparam ScoresInputDdr1Tensor
* @tparam ScoresScale1FB
* @tparam ScoresInputDdr2Tensor
* @tparam ScoresScale2FB
* @tparam ScoresInputDdr3Tensor
* @tparam ScoresScale3FB
* @tparam ScoresOutputOcmTensor
* @tparam memAllocatorType
* @param scoresInputDdr0
* @param scoresScale0
* @param scoresInputDdr1
* @param scoresScale1
* @param scoresInputDdr2
* @param scoresScale2
* @param scoresInputDdr3
* @param scoresScale3
* @param scoresOutputOcm
* @param ocmMem
*/
template <typename ScoresInputDdr0Tensor,
FracRepType ScoresScale0FB,
typename ScoresInputDdr1Tensor,
FracRepType ScoresScale1FB,
typename ScoresInputDdr2Tensor,
FracRepType ScoresScale2FB,
typename ScoresInputDdr3Tensor,
FracRepType ScoresScale3FB,
typename ScoresOutputOcmTensor,
typename memAllocatorType>
INLINE void ultrafaceBoundingBoxScores(ScoresInputDdr0Tensor& scoresInputDdr0,
const FixedPoint32<ScoresScale0FB>& scoresScale0,
ScoresInputDdr1Tensor& scoresInputDdr1,
const FixedPoint32<ScoresScale1FB>& scoresScale1,
ScoresInputDdr2Tensor& scoresInputDdr2,
const FixedPoint32<ScoresScale2FB>& scoresScale2,
ScoresInputDdr3Tensor& scoresInputDdr3,
const FixedPoint32<ScoresScale3FB>& scoresScale3,
ScoresOutputOcmTensor& scoresOutputOcm,
memAllocatorType& ocmMem) {
using inpElemType = typename ScoresInputDdr0Tensor::elemType;
using elemType = typename ScoresOutputOcmTensor::elemType;
constexpr std::int32_t numBoxes = ScoresOutputOcmTensor::NUM_COLS;
using ScoresInputOcm0Tensor = OcmTensor<inpElemType, 1, 6, 30, 40>;
using ScoresInputOcm1Tensor = OcmTensor<inpElemType, 1, 4, 15, 20>;
using ScoresInputOcm2Tensor = OcmTensor<inpElemType, 1, 4, 8, 10>;
using ScoresInputOcm3Tensor = OcmTensor<inpElemType, 1, 6, 4, 5>;
// process scoresInputDdr0
std::int32_t widthOffset = 0;
ScoresInputOcm0Tensor scoresInputOcm0;
ocmMem.allocate(scoresInputOcm0);
memCpy(scoresInputDdr0, scoresInputOcm0);
ultrafaceBoundingBoxScoresSingleInput(scoresInputOcm0, scoresScale0, scoresOutputOcm, widthOffset);
ocmMem.free(scoresInputOcm0);
// process scoresInputDdr1
widthOffset +=
(ScoresInputOcm0Tensor::NUM_CHN / 2) * ScoresInputOcm0Tensor::NUM_ROWS * ScoresInputOcm0Tensor::NUM_COLS;
ScoresInputOcm1Tensor scoresInputOcm1;
ocmMem.allocate(scoresInputOcm1);
memCpy(scoresInputDdr1, scoresInputOcm1);
ultrafaceBoundingBoxScoresSingleInput(scoresInputOcm1, scoresScale1, scoresOutputOcm, widthOffset);
ocmMem.free(scoresInputOcm1);
// process scoresInputDdr2
widthOffset +=
(ScoresInputOcm1Tensor::NUM_CHN / 2) * ScoresInputOcm1Tensor::NUM_ROWS * ScoresInputOcm1Tensor::NUM_COLS;
ScoresInputOcm2Tensor scoresInputOcm2;
ocmMem.allocate(scoresInputOcm2);
memCpy(scoresInputDdr2, scoresInputOcm2);
ultrafaceBoundingBoxScoresSingleInput(scoresInputOcm2, scoresScale2, scoresOutputOcm, widthOffset);
ocmMem.free(scoresInputOcm2);
// process scoresInputDdr3
widthOffset +=
(ScoresInputOcm2Tensor::NUM_CHN / 2) * ScoresInputOcm2Tensor::NUM_ROWS * ScoresInputOcm2Tensor::NUM_COLS;
ScoresInputOcm3Tensor scoresInputOcm3;
ocmMem.allocate(scoresInputOcm3);
memCpy(scoresInputDdr3, scoresInputOcm3);
ultrafaceBoundingBoxScoresSingleInput(scoresInputOcm3, scoresScale3, scoresOutputOcm, widthOffset);
ocmMem.free(scoresInputOcm3);
}
/**
* @brief This handles computation of a single input for coordinates. This is called on for each of the 4 inputs.
* A minRoi is setup with shape (depth, core_dim, width). Tiles within minRoi are fetched along channels first and
* then width. MinRoi moves only in height within tensor. An output tile is generated after every 4 tiles since we
* already have enough data to compute a region of the output. This output tile is then
* written to OCM tensor via RAU. Weight tensors are accessed via RAU.
*
* @tparam CoordinatesInputOcmTensor
* @tparam CoordinatesScaleFB
* @tparam WeightsOcmTensor
* @tparam WeightsScaleAddFB
* @tparam WeightsScaleMulFB
* @tparam CoordinatesOutputOcmTensor
* @param coordinatesInputOcm
* @param coordScale
* @param weightsAddOcm
* @param weightsScaleAdd
* @param weightsMulOcm
* @param weightsScaleMul
* @param coordinatesOutputOcm
* @param outWidthOffset width offset to concatenate result in width. THis is used when calculating RAU address
*/
template <typename CoordinatesInputOcmTensor,
FracRepType CoordinatesScaleFB,
typename WeightsOcmTensor,
FracRepType WeightsScaleAddFB,
FracRepType WeightsScaleMulFB,
typename CoordinatesOutputOcmTensor>
INLINE void ultrafaceBoundingBoxCoordinatesSingleInput(CoordinatesInputOcmTensor& coordinatesInputOcm,
const FixedPoint32<CoordinatesScaleFB>& coordScale,
WeightsOcmTensor& weightsAddOcm,
const FixedPoint32<WeightsScaleAddFB>& weightsScaleAdd,
WeightsOcmTensor& weightsMulOcm,
const FixedPoint32<WeightsScaleMulFB>& weightsScaleMul,
CoordinatesOutputOcmTensor& coordinatesOutputOcm,
std::int32_t outWidthOffset) {
using elemType = typename CoordinatesOutputOcmTensor::elemType;
using inpElemType = typename CoordinatesInputOcmTensor::elemType;
constexpr std::int32_t depth = CoordinatesInputOcmTensor::NUM_CHN;
constexpr std::int32_t height = CoordinatesInputOcmTensor::NUM_ROWS;
constexpr std::int32_t width = CoordinatesInputOcmTensor::NUM_COLS;
constexpr std::int32_t numTiles = CoordinatesInputOcmTensor::NUM_TILES;
// Note: ROI, and input ocm tensor are the same shape for all inputs
using MinRoiTensor = OcmTensor<inpElemType, depth, core_array::coreDim, width>;
// MinRoiDesc describes how you iterate and process tiles within minRoi
using MinRoiDesc = MinRoiDescriptor<MinRoiTensor,
AxisGroup<Direction::Channel, Direction::Width, Direction::Height>,
Granularity::Square>;
// Access describes the overall access description including MinRoiDesc and movement of minRoi within ROI
using Access = TensorAccessor<MinRoiDesc, AxisGroup<Direction::Height, Direction::Channel, Direction::Width>>;
// readFlow is the final object that connects the Access to the ROI
ReadFlow<Access, CoordinatesInputOcmTensor> readFlow(coordinatesInputOcm);
// Initialize qAddress
constexpr std::int32_t numTilesInMinRoi = Access::MinRoiDesc::numTiles();
// loop over minRois is handled by heightOffset loop; minRois only move in height
for(std::int32_t heightOffset = 0; heightOffset < height; heightOffset += core_array::coreDim) {
container::NDArray<qVar_t<inpElemType>, numTilesInMinRoi> qInput = readFlow.read();
std::int32_t minRoiTileIdx = 0;
// widthOffset and ChannelOffset loop over tiles within minROI
for(std::int32_t widthOffset = 0; widthOffset < width; widthOffset += core_array::coreDim) {
// skip by 4 since we will process 4 channels at a time.
for(std::int32_t channelOffset = 0; channelOffset < depth; channelOffset += 4) {
// calculate soft max
qVar_t<elemType> qInput0 = static_cast<elemType>(qInput[minRoiTileIdx]) * coordScale;
qVar_t<elemType> qInput1 = static_cast<elemType>(qInput[minRoiTileIdx + 1]) * coordScale;
qVar_t<elemType> qInput2 = static_cast<elemType>(qInput[minRoiTileIdx + 2]) * coordScale;
qVar_t<elemType> qInput3 = static_cast<elemType>(qInput[minRoiTileIdx + 3]) * coordScale;
// calculate rau store addresses to transfer to OCM
// depends on offsets
qVar_t<std::uint32_t> qInWidth = widthOffset + qCol<>;
qVar_t<std::uint32_t> qInHeight = heightOffset + qRow<>;
qVar_t<std::uint32_t> qOutWidth =
outWidthOffset + channelOffset / 4 + (depth / 4) * qInWidth + (depth / 4) * width * qInHeight;
qVar_t<bool> qValid = (qInWidth < width) && (qInHeight < height);
// Note: shape of both weights is (1, 1, numBoxes, 2) while shape of output is (1, 1, 2, numBoxes)
// this is why the address offsets are transposed on weights relative to offsets for output tensor.
qVar_t<std::uint32_t> qWeightAddress0 = rau::computeAddr<WeightsOcmTensor>(0, 0, qOutWidth, 0);
qVar_t<std::uint32_t> qWeightAddress1 = rau::computeAddr<WeightsOcmTensor>(0, 0, qOutWidth, 1);
rau::config(weightsAddOcm);
qVar_t<inpElemType> qAdd0In = qValid ? rau::load::oneTile(qWeightAddress0, weightsAddOcm) : 0;
qVar_t<elemType> qAdd0 = static_cast<elemType>(qAdd0In) * weightsScaleAdd;
qVar_t<inpElemType> qAdd1In = qValid ? rau::load::oneTile(qWeightAddress1, weightsAddOcm) : 0;
qVar_t<elemType> qAdd1 = static_cast<elemType>(qAdd1In) * weightsScaleAdd;
rau::config(weightsMulOcm);
qVar_t<inpElemType> qMul0In = qValid ? rau::load::oneTile(qWeightAddress0, weightsMulOcm) : 0;
qVar_t<elemType> qMul0 = static_cast<elemType>(qMul0In) * weightsScaleMul;
qVar_t<inpElemType> qMul1In = qValid ? rau::load::oneTile(qWeightAddress1, weightsMulOcm) : 0;
qVar_t<elemType> qMul1 = static_cast<elemType>(qMul1In) * weightsScaleMul;
// calculation
qVar_t<elemType> qCenter0 = qMul0 * qInput0 / 10 + qAdd0;
qVar_t<elemType> qCenter1 = qMul1 * qInput1 / 10 + qAdd1;
qVar_t<elemType> qSide0 = qMul0 * math::exp(qInput2 / 5) / 2;
qVar_t<elemType> qSide1 = qMul1 * math::exp(qInput3 / 5) / 2;
qVar_t<elemType> qOutput0 = qCenter0 - qSide0;
qVar_t<elemType> qOutput1 = qCenter1 - qSide1;
qVar_t<elemType> qOutput2 = qCenter0 + qSide0;
qVar_t<elemType> qOutput3 = qCenter1 + qSide1;
rau::config(coordinatesOutputOcm);
qVar_t<std::uint32_t> qOutAddress = rau::computeAddr<CoordinatesOutputOcmTensor>(0, 0, 0, qOutWidth);
if(qValid) {
rau::store::oneTile(qOutAddress, qOutput0, coordinatesOutputOcm);
qOutAddress = rau::computeAddr<CoordinatesOutputOcmTensor>(0, 0, 1, qOutWidth);
rau::store::oneTile(qOutAddress, qOutput1, coordinatesOutputOcm);
qOutAddress = rau::computeAddr<CoordinatesOutputOcmTensor>(0, 0, 2, qOutWidth);
rau::store::oneTile(qOutAddress, qOutput2, coordinatesOutputOcm);
qOutAddress = rau::computeAddr<CoordinatesOutputOcmTensor>(0, 0, 3, qOutWidth);
rau::store::oneTile(qOutAddress, qOutput3, coordinatesOutputOcm);
}
// update indices
minRoiTileIdx += 4;
}
}
}
}
/**
* @brief This is a helper function for computing coordinates (independent calculation from scores)
*
* @tparam CoordinatesInputDdr0Tensor
* @tparam CoordinatesScale0FB
* @tparam CoordinatesInputDdr1Tensor
* @tparam CoordinatesScale1FB
* @tparam CoordinatesInputDdr2Tensor
* @tparam CoordinatesScale2FB
* @tparam CoordinatesInputDdr3Tensor
* @tparam CoordinatesScale3FB
* @tparam WeightsAddDdrTensor
* @tparam WeightsScaleAddFB
* @tparam WeightsMulDdrTensor
* @tparam WeightsScaleMulFB
* @tparam CoordinatesOutputOcmTensor
* @tparam memAllocatorType
* @param coordinatesInputDdr0
* @param coordScale0
* @param coordinatesInputDdr1
* @param coordScale1
* @param coordinatesInputDdr2
* @param coordScale2
* @param coordinatesInputDdr3
* @param coordScale3
* @param weightsAddDdr
* @param weightsScaleAdd
* @param weightsMulDdr
* @param weightsScaleMul
* @param coordinatesOutputOcm
* @param ocmMem
*/
template <typename CoordinatesInputDdr0Tensor,
FracRepType CoordinatesScale0FB,
typename CoordinatesInputDdr1Tensor,
FracRepType CoordinatesScale1FB,
typename CoordinatesInputDdr2Tensor,
FracRepType CoordinatesScale2FB,
typename CoordinatesInputDdr3Tensor,
FracRepType CoordinatesScale3FB,
typename WeightsAddDdrTensor,
FracRepType WeightsScaleAddFB,
typename WeightsMulDdrTensor,
FracRepType WeightsScaleMulFB,
typename CoordinatesOutputOcmTensor,
typename memAllocatorType>
INLINE void ultrafaceBoundingBoxCoordinates(CoordinatesInputDdr0Tensor& coordinatesInputDdr0,
const FixedPoint32<CoordinatesScale0FB>& coordScale0,
CoordinatesInputDdr1Tensor& coordinatesInputDdr1,
const FixedPoint32<CoordinatesScale1FB>& coordScale1,
CoordinatesInputDdr2Tensor& coordinatesInputDdr2,
const FixedPoint32<CoordinatesScale2FB>& coordScale2,
CoordinatesInputDdr3Tensor& coordinatesInputDdr3,
const FixedPoint32<CoordinatesScale3FB>& coordScale3,
WeightsAddDdrTensor& weightsAddDdr,
const FixedPoint32<WeightsScaleAddFB>& weightsScaleAdd,
WeightsMulDdrTensor& weightsMulDdr,
const FixedPoint32<WeightsScaleMulFB>& weightsScaleMul,
CoordinatesOutputOcmTensor& coordinatesOutputOcm,
memAllocatorType& ocmMem) {
using inpElemType = typename CoordinatesInputDdr0Tensor::elemType;
using elemType = typename CoordinatesOutputOcmTensor::elemType;
constexpr std::int32_t numBoxes = CoordinatesOutputOcmTensor::NUM_COLS;
using CoordinatesInputOcm0Tensor = OcmTensor<inpElemType, 1, 12, 30, 40>;
using CoordinatesInputOcm1Tensor = OcmTensor<inpElemType, 1, 8, 15, 20>;
using CoordinatesInputOcm2Tensor = OcmTensor<inpElemType, 1, 8, 8, 10>;
using CoordinatesInputOcm3Tensor = OcmTensor<inpElemType, 1, 12, 4, 5>;
using WeightsAddOcmTensor = OcmTensor<inpElemType, 1, 1, numBoxes, 2>;
using WeightsMulOcmTensor = OcmTensor<inpElemType, 1, 1, numBoxes, 2>;
WeightsAddOcmTensor weightsAddOcm;
ocmMem.allocate(weightsAddOcm);
memCpy(weightsAddDdr, weightsAddOcm);
WeightsMulOcmTensor weightsMulOcm;
ocmMem.allocate(weightsMulOcm);
memCpy(weightsMulDdr, weightsMulOcm);
// processing input 0
std::int32_t widthOffset = 0;
CoordinatesInputOcm0Tensor coordinatesInputOcm0;
ocmMem.allocate(coordinatesInputOcm0);
memCpy(coordinatesInputDdr0, coordinatesInputOcm0);
ultrafaceBoundingBoxCoordinatesSingleInput(coordinatesInputOcm0,
coordScale0,
weightsAddOcm,
weightsScaleAdd,
weightsMulOcm,
weightsScaleMul,
coordinatesOutputOcm,
widthOffset);
ocmMem.free(coordinatesInputOcm0);
// processing input 1
widthOffset += (CoordinatesInputOcm0Tensor::NUM_CHN / 4) * CoordinatesInputOcm0Tensor::NUM_ROWS *
CoordinatesInputOcm0Tensor::NUM_COLS;
CoordinatesInputOcm1Tensor coordinatesInputOcm1;
ocmMem.allocate(coordinatesInputOcm1);
memCpy(coordinatesInputDdr1, coordinatesInputOcm1);
ultrafaceBoundingBoxCoordinatesSingleInput(coordinatesInputOcm1,
coordScale1,
weightsAddOcm,
weightsScaleAdd,
weightsMulOcm,
weightsScaleMul,
coordinatesOutputOcm,
widthOffset);
ocmMem.free(coordinatesInputOcm1);
// processing input 2
widthOffset += (CoordinatesInputOcm1Tensor::NUM_CHN / 4) * CoordinatesInputOcm1Tensor::NUM_ROWS *
CoordinatesInputOcm1Tensor::NUM_COLS;
CoordinatesInputOcm2Tensor coordinatesInputOcm2;
ocmMem.allocate(coordinatesInputOcm2);
memCpy(coordinatesInputDdr2, coordinatesInputOcm2);
ultrafaceBoundingBoxCoordinatesSingleInput(coordinatesInputOcm2,
coordScale2,
weightsAddOcm,
weightsScaleAdd,
weightsMulOcm,
weightsScaleMul,
coordinatesOutputOcm,
widthOffset);
ocmMem.free(coordinatesInputOcm2);
// processing input 3
widthOffset += (CoordinatesInputOcm2Tensor::NUM_CHN / 4) * CoordinatesInputOcm2Tensor::NUM_ROWS *
CoordinatesInputOcm2Tensor::NUM_COLS;
CoordinatesInputOcm3Tensor coordinatesInputOcm3;
ocmMem.allocate(coordinatesInputOcm3);
memCpy(coordinatesInputDdr3, coordinatesInputOcm3);
ultrafaceBoundingBoxCoordinatesSingleInput(coordinatesInputOcm3,
coordScale3,
weightsAddOcm,
weightsScaleAdd,
weightsMulOcm,
weightsScaleMul,
coordinatesOutputOcm,
widthOffset);
ocmMem.free(coordinatesInputOcm3);
}
/**
* @brief This is the custom op at end of ultraface for creating scores and coordinates for bounding
* boxes.
*
* The general scheme involves processing scores and coordinates independently and then using a memcpy to do the
* final concatenation in height. Scores and coordinates are calculated in a similar fashion. Each of the input
* tensors can be processed one at a time. A input tensor is flown in with a square iterator such that channels
* sit on the same core. This enables doing all the required compute within core (no data movements needed). Then,
* the result is transferred to OCM via RAU. RAU takes care of the data transformations (reshape, concat, and
* transpose).
*
* @tparam ScoresInputDdr0Tensor score input with shape (1, 6, 30, 40) and elemType is fixedpoint
* @tparam ScoresInputDdr1Tensor score input with shape (1, 4, 15, 20) and elemType is fixedpoint
* @tparam ScoresInputDdr2Tensor score input with shape (1, 4, 8, 10) and elemType is fixedpoint
* @tparam ScoresInputDdr3Tensor score input with shape (1, 6, 4, 5) and elemType is fixedpoint
* @tparam CoordinatesInputDdr0Tensor coordinates input with shape (1, 12, 30, 40) and elemType is fixedpoint
* @tparam CoordinatesInputDdr1Tensor coordinates input with shape (1, 8, 15, 20) and elemType is fixedpoint
* @tparam CoordinatesInputDdr2Tensor coordinates input with shape (1, 8, 8, 10) and elemType is fixedpoint
* @tparam CoordinatesInputDdr3Tensor coordinates input with shape (1, 12, 4, 5) and elemType is fixedpoint
* @tparam WeightsMulDdrTensor multiplication op weights with shape (1, 1, 4420, 2) and elemType is fixedpoint
* @tparam WeightsAddDdrTensor addition op weights with shape (1, 1, 4420, 2) and elemType is fixedpoint
* @tparam OutputDdrTensor output with shape (1, 1, 6, 4420) and elemType is fixedpoint. Note the height stores 6
* elements ( = 2 scores + 4 coordinates)
* @tparam ScoresScale0FB number of fractional bits for score tensor 0
* @tparam ScoresScale1FB number of fractional bits for score tensor 1
* @tparam ScoresScale2FB number of fractional bits for score tensor 2
* @tparam ScoresScale3FB number of fractional bits for score tensor 3
* @tparam CoordinatesScale0FB number of fractional bits for coordinate tensor 0
* @tparam CoordinatesScale1FB number of fractional bits for coordinate tensor 1
* @tparam CoordinatesScale2FB number of fractional bits for coordinate tensor 2
* @tparam CoordinatesScale3FB number of fractional bits for coordinate tensor 3
* @tparam WeightsScaleAddFB number of fractional bits for bias add weight
* @tparam WeightsScaleMulFB number of fractional bits for bias mul weight
* @param scoresInputDdr0 score input 0
* @param scoresInputDdr1 score input 1
* @param scoresInputDdr2 score input 2
* @param scoresInputDdr3 score input 3
* @param coordinatesInputDdr0 coordinate input 0
* @param coordinatesInputDdr1 coordinate input 1
* @param coordinatesInputDdr2 coordinate input 2
* @param coordinatesInputDdr3 coordinate input 3
* @param scoresScale0 scaling factor for score input 0
* @param scoresScale1 scaling factor for score input 1
* @param scoresScale2 scaling factor for score input 2
* @param scoresScale3 scaling factor for score input 3
* @param coordScale0 scaling factor for coordinate input 0
* @param coordScale1 scaling factor for coordinate input 1
* @param coordScale2 scaling factor for coordinate input 2
* @param coordScale3 scaling factor for coordinate input 3
* @param weightsAddDdr addition op weights
* @param weightsScaleAdd scaling factor for add bias tensor
* @param weightsMulDdr multiplication op weights
* @param weightsScaleMul scaling factor for mul bias tensor
* @param outputDdr output
*
*/
template <typename ScoresInputDdr0Tensor,
typename ScoresInputDdr1Tensor,
typename ScoresInputDdr2Tensor,
typename ScoresInputDdr3Tensor,
typename CoordinatesInputDdr0Tensor,
typename CoordinatesInputDdr1Tensor,
typename CoordinatesInputDdr2Tensor,
typename CoordinatesInputDdr3Tensor,
typename WeightsAddDdrTensor,
typename WeightsMulDdrTensor,
typename OutputDdrTensor,
FracRepType ScoresScale0FB,
FracRepType ScoresScale1FB,
FracRepType ScoresScale2FB,
FracRepType ScoresScale3FB,
FracRepType CoordinatesScale0FB,
FracRepType CoordinatesScale1FB,
FracRepType CoordinatesScale2FB,
FracRepType CoordinatesScale3FB,
FracRepType WeightsScaleAddFB,
FracRepType WeightsScaleMulFB,
typename OcmAllocatorType>
INLINE void ultrafaceBoundingBox(ScoresInputDdr0Tensor& scoresInputDdr0,
ScoresInputDdr1Tensor& scoresInputDdr1,
ScoresInputDdr2Tensor& scoresInputDdr2,
ScoresInputDdr3Tensor& scoresInputDdr3,
CoordinatesInputDdr0Tensor& coordinatesInputDdr0,
CoordinatesInputDdr1Tensor& coordinatesInputDdr1,
CoordinatesInputDdr2Tensor& coordinatesInputDdr2,
CoordinatesInputDdr3Tensor& coordinatesInputDdr3,
const FixedPoint32<ScoresScale0FB>& scoresScale0,
const FixedPoint32<ScoresScale1FB>& scoresScale1,
const FixedPoint32<ScoresScale2FB>& scoresScale2,
const FixedPoint32<ScoresScale3FB>& scoresScale3,
const FixedPoint32<CoordinatesScale0FB>& coordScale0,
const FixedPoint32<CoordinatesScale1FB>& coordScale1,
const FixedPoint32<CoordinatesScale2FB>& coordScale2,
const FixedPoint32<CoordinatesScale3FB>& coordScale3,
WeightsAddDdrTensor& weightsAddDdr,
const FixedPoint32<WeightsScaleAddFB>& weightsScaleAdd,
WeightsMulDdrTensor& weightsMulDdr,
const FixedPoint32<WeightsScaleMulFB>& weightsScaleMul,
OutputDdrTensor& outputDdr,
OcmAllocatorType& ocmMem) {
using elemType = typename OutputDdrTensor::elemType;
constexpr std::int32_t numBoxes = OutputDdrTensor::NUM_COLS;
using ScoresOutputOcmTensor = OcmTensor<elemType, 1, 1, 2, numBoxes>;
ScoresOutputOcmTensor scoresOutputOcm;
ocmMem.allocate(scoresOutputOcm);
using CoordinatesOutputOcmTensor = OcmTensor<elemType, 1, 1, 4, numBoxes>;
CoordinatesOutputOcmTensor coordinatesOutputOcm;
ocmMem.allocate(coordinatesOutputOcm);
ultrafaceBoundingBoxScores(scoresInputDdr0,
scoresScale0,
scoresInputDdr1,
scoresScale1,
scoresInputDdr2,
scoresScale2,
scoresInputDdr3,
scoresScale3,
scoresOutputOcm,
ocmMem);
memCpy(scoresOutputOcm, outputDdr);
ultrafaceBoundingBoxCoordinates(coordinatesInputDdr0,
coordScale0,
coordinatesInputDdr1,
coordScale1,
coordinatesInputDdr2,
coordScale2,
coordinatesInputDdr3,
coordScale3,
weightsAddDdr,
weightsScaleAdd,
weightsMulDdr,
weightsScaleMul,
coordinatesOutputOcm,
ocmMem);
memCpy(coordinatesOutputOcm, outputDdr, 0, 0, 2, 0);
}
} // namespace usercustom
} // namespace chimera
Above you can see the custom op used to complete Ultraface on CGC. To learn more about this op please see Tutorial: Measuring Performance of a Custom Op.
cgc_job.compile()
2026-06-19 03:53 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 03:53 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 03:53 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 03:53 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
------------------------------------------ ----------------------------- ----------------------------- --------------------
CustomOp/usercustom::ultrafaceBoundingBox0 contrib.epu.quadric_custom_op (-32768.0, 32767.99998474121) 16
2026-06-19 03:53 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-06-19 03:53 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-06-19 03:53 - INFO - epu - codegen - START=================================relay_to_tir
2026-06-19 03:53 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-06-19 03:53 - INFO - epu - codegen - START==============================adapt_and_order
2026-06-19 03:53 - INFO - epu - mac_counter -
2026-06-19 03:53 - INFO - epu - mac_counter - ============================================================
2026-06-19 03:53 - INFO - epu - mac_counter - MAC Operation Count Summary
2026-06-19 03:53 - INFO - epu - mac_counter - ============================================================
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 16,588,800 ops (8,294,400 MACs) - 245_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 5,529,600 ops (2,764,800 MACs) - 248_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 19,660,800 ops (9,830,400 MACs) - 251_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 2,764,800 ops (1,382,400 MACs) - 254_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 9,830,400 ops (4,915,200 MACs) - 257_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 2,764,800 ops (1,382,400 MACs) - 260_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 9,830,400 ops (4,915,200 MACs) - 263_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 691,200 ops (345,600 MACs) - 266_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 4,915,200 ops (2,457,600 MACs) - 269_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,382,400 ops (691,200 MACs) - 272_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 9,830,400 ops (4,915,200 MACs) - 275_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,382,400 ops (691,200 MACs) - 278_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 9,830,400 ops (4,915,200 MACs) - 281_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,228,800 ops (614,400 MACs) - 284_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 2,764,800 ops (1,382,400 MACs) - 286_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 5,529,600 ops (2,764,800 MACs) - 289_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,228,800 ops (614,400 MACs) - 291_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 2,764,800 ops (1,382,400 MACs) - 293_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 5,529,600 ops (2,764,800 MACs) - 296_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,228,800 ops (614,400 MACs) - 298_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 2,073,600 ops (1,036,800 MACs) - 300_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 4,147,200 ops (2,073,600 MACs) - 303_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 5,529,600 ops (2,764,800 MACs) - 306_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 7,372,800 ops (3,686,400 MACs) - 309_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 9,830,400 ops (4,915,200 MACs) - 311_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,382,400 ops (691,200 MACs) - 315_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 921,600 ops (460,800 MACs) - 317_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 345,600 ops (172,800 MACs) - 343_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 4,915,200 ops (2,457,600 MACs) - 346_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 691,200 ops (345,600 MACs) - 349_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 9,830,400 ops (4,915,200 MACs) - 352_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 691,200 ops (345,600 MACs) - 355_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 9,830,400 ops (4,915,200 MACs) - 358_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 691,200 ops (345,600 MACs) - 361_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 307,200 ops (153,600 MACs) - 363_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 184,320 ops (92,160 MACs) - 389_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 5,242,880 ops (2,621,440 MACs) - 392_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 368,640 ops (184,320 MACs) - 395_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 10,485,760 ops (5,242,880 MACs) - 398_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 368,640 ops (184,320 MACs) - 401_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 163,840 ops (81,920 MACs) - 403_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 2,621,440 ops (1,310,720 MACs) - 429_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 23,040 ops (11,520 MACs) - 431_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 655,360 ops (327,680 MACs) - 433_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 552,960 ops (276,480 MACs) - 435_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,382,400 ops (691,200 MACs) - 329_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,843,200 ops (921,600 MACs) - 331_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 691,200 ops (345,600 MACs) - 375_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 614,400 ops (307,200 MACs) - 377_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 368,640 ops (184,320 MACs) - 415_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 327,680 ops (163,840 MACs) - 417_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - conv2d: 1,105,920 ops (552,960 MACs) - 447_quantized
2026-06-19 03:53 - INFO - epu - mac_counter - ------------------------------------------------------------
2026-06-19 03:53 - INFO - epu - mac_counter - Total: 200,837,120 ops (100,418,560 MACs)
2026-06-19 03:53 - INFO - epu - mac_counter - ============================================================
2026-06-19 03:53 - INFO - epu - mac_counter -
2026-06-19 03:53 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 03:53 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-06-19 03:54 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 03:54 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-06-19 03:54 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 03:54 - INFO - epu - codegen - START================================lrm_splitting
2026-06-19 03:55 - INFO - epu - fuse_weights - No optimization for node 306_quantized with properties { kernel_size: [3, 3], channels: 16, strides:[1, 1], padding:[5, 5, 5, 5], data_layout:NCHW, dilation:[5, 5], groups:1, kernel_layout:OIHW, out_dtype:int8, out_layout: }.Applying General Convolution algorithm. Performance for this node can be improved significantly. Please refer to the documentation or contact Quadric Support.
2026-06-19 03:55 - INFO - epu - fuse_weights - No optimization for node 289_quantized with properties { kernel_size: [3, 3], channels: 16, strides:[1, 1], padding:[2, 2, 2, 2], data_layout:NCHW, dilation:[2, 2], groups:1, kernel_layout:OIHW, out_dtype:int8, out_layout: }.Applying General Convolution algorithm. Performance for this node can be improved significantly. Please refer to the documentation or contact Quadric Support.
2026-06-19 03:55 - INFO - epu - fuse_weights - No optimization for node 296_quantized with properties { kernel_size: [3, 3], channels: 16, strides:[1, 1], padding:[3, 3, 3, 3], data_layout:NCHW, dilation:[3, 3], groups:1, kernel_layout:OIHW, out_dtype:int8, out_layout: }.Applying General Convolution algorithm. Performance for this node can be improved significantly. Please refer to the documentation or contact Quadric Support.
2026-06-19 03:55 - INFO - epu - codegen - START==============================ext_split_relay
2026-06-19 03:56 - INFO - epu - codegen - START====================================build_tir
2026-06-19 03:56 - INFO - epu - chimera_job - Compilation of ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful
%matplotlib inline
import numpy as np
from PIL import Image
import os
import yaml
import numpy as np
import pytest
from IPython.display import display
import matplotlib.pyplot as plt
from pathlib import Path
import sdk_cli.node_builtins.utils as node_utils
import sdk_cli.node_builtins.classical.nms as nms_node
from sdk_cli.node_builtins.classical.nms import (
execute_node,
execute_node_ccl,
box_utils,
)
from sdk_cli.visualizers import nms
from sdk_cli.utils import model_helpers
dataset_mean = [0.5, 0.5, 0.5]
dataset_std = [0.5, 0.5, 0.5]
dataset_input_size = (320, 240)
classes = [""] # providing no class label since there is only one class: "face"
mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
THREADS = 6
run_folder = r"../common/calibration/face"
allimages = mh.get_images(run_folder, THREADS * 2)
yaml_path = Path(nms_node.__file__).parent / "nms.yaml"
with Path(yaml_path).open("r") as stream:
try:
config = yaml.safe_load(stream)
except yaml.YAMLError as exc:
print(exc)
hw_config = node_utils.EXAMPLE_KWARGS["hardware_config"]
nmsVis = nms.NMSVisualizer()
def run_ultraface(image_p):
image = mh.load_image(Image.open(image_p))
output = onnx_session.run([], {"input": image})[0][0]
orig_img = Image.open(image_p)
net_input_img = mh.asym_crop_center(orig_img)
cgc_output = cgc_job.run_inference_harness(inputs={"input": image})
# Perform Chimera NMS and Display Using Output From CGC
config["inputs"]["input"]["value"] = cgc_output["concat_output_1"]
properties = {
key: value["value"] for key, value in config["properties"].items() if "value" in value
}
# Call the two functions and get their outputs
result_ccl = execute_node_ccl(
config["inputs"],
properties,
spec=config,
hardware_config=hw_config,
quiet=True,
)
# Perform CPU NMS and Display Using Output From Onnxruntime
config["inputs"]["input"]["value"] = output
# Call the two functions and get their outputs
result_cpu = execute_node(
config["inputs"],
properties,
spec=config,
hardware_config=hw_config,
)
cpu_boxes = result_cpu["boxes"]
ccl_boxes = result_ccl["boxes"]
num_boxes_ref = len(cpu_boxes)
iou_matrix = np.zeros((num_boxes_ref, num_boxes_ref))
# Calculate IoU for each pair of boxes and fill the matrix
for i in range(num_boxes_ref):
for j in range(num_boxes_ref):
iou = box_utils.iou_of(cpu_boxes[i], ccl_boxes[j])
iou_matrix[i][j] = iou
# Calc max across rows of matrix.
iou_scores = iou_matrix.max(axis=1)
# Determine mean IoU
mean_iou = np.mean(iou_scores)
return {
"result": result_cpu,
"result_ccl": result_ccl,
"image": image,
"mean_iou": mean_iou,
"class_names": classes,
}
def visualize(nn_output):
image = nn_output["image"]
result_cpu = nn_output["result"]
result_ccl = nn_output["result_ccl"]
class_names = nn_output["class_names"]
chimera_image = nmsVis.draw_boxes(result_ccl, image, class_names, line_width=2)
cpu_image = nmsVis.draw_boxes(result_cpu, image, class_names, line_width=2)
mean_iou_str = f"Mean (CPU vs Chimera) IoU: {nn_output['mean_iou']:.2f}"
plt.figure(figsize=(10, 5)) # Adjust as needed
plt.subplot(1, 2, 1) # Rows, columns, current index
plt.imshow(chimera_image)
plt.title("Chimera")
plt.axis("off")
plt.subplot(1, 2, 2)
plt.imshow(cpu_image)
plt.title("CPU (Onnxruntime)")
plt.axis("off")
# Add unified title
plt.suptitle(mean_iou_str)
plt.tight_layout() # To ensure there's no overlap in subplots
plt.show()
for image in allimages:
out = run_ultraface(image)
visualize(out)
2026-06-19 03:56 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143cabe5720>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:05<00:00, 2.81it/s]
2026-06-19 03:56 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143cabe5720>
2026-06-19 03:56 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:56 - INFO - epu - iss_testing - Done 0:00:00.091375
[SDK-CLI] : Compiling source
2026-06-19 03:56 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- The C compiler identification is Clang 8.0.1
-- The CXX compiler identification is Clang 8.0.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /quadric/llvm/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /quadric/llvm/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (0.3s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:56 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:57 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:57 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
2026-06-19 03:57 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143cabe5720>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:04<00:00, 3.24it/s]
2026-06-19 03:57 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bef9bb20>
2026-06-19 03:57 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:57 - INFO - epu - iss_testing - Done 0:00:00.074057
[SDK-CLI] : Compiling source
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:57 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:57 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
2026-06-19 03:57 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bf12d240>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:09<00:00, 1.51it/s]
2026-06-19 03:57 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bf12d240>
2026-06-19 03:57 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:57 - INFO - epu - iss_testing - Done 0:00:00.147782
[SDK-CLI] : Compiling source
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:57 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:57 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
2026-06-19 03:57 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bef9b640>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:05<00:00, 2.65it/s]
2026-06-19 03:57 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bef9b640>
2026-06-19 03:57 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:57 - INFO - epu - iss_testing - Done 0:00:00.055312
[SDK-CLI] : Compiling source
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:57 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:57 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:57 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bee6d540>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:05<00:00, 2.84it/s]
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143240ea2f0>
2026-06-19 03:58 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:58 - INFO - epu - iss_testing - Done 0:00:00.067669
[SDK-CLI] : Compiling source
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:58 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:58 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bf03e440>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:05<00:00, 2.87it/s]
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bf03e440>
2026-06-19 03:58 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:58 - INFO - epu - iss_testing - Done 0:00:00.070793
[SDK-CLI] : Compiling source
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:58 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:58 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x714324180d30>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:06<00:00, 2.46it/s]
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x71431c75d420>
2026-06-19 03:58 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:58 - INFO - epu - iss_testing - Done 0:00:00.056095
[SDK-CLI] : Compiling source
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:58 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:58 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7143bef3bd90>
FILM 15/15: 100%|███████████████████████████████████████████████████| 15/15 [00:03<00:00, 4.14it/s]
2026-06-19 03:58 - INFO - epu - iss_testing - No tranges found for input, use default float range: <tvm.contrib.epu.interval.Interval object at 0x7144f10b72e0>
2026-06-19 03:58 - INFO - epu - iss_testing - Started Executing Onnxruntime...
2026-06-19 03:58 - INFO - epu - iss_testing - Done 0:00:00.067125
[SDK-CLI] : Compiling source
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build -DNUM_GPNPUS=1 -DNUM_CORES=32 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=16384 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DHARDWARE_TARGET=OFF
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/build
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: make -j8
[ 25%] Building CXX object CMakeFiles/nms_epu_epu.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 50%] Building CXX object CMakeFiles/nms_epu_host.dir/quadric/sdk-cli/sdk_cli/node_builtins/classical/nms/nms_epu.cpp.o
[ 75%] Linking EPU executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_epu.s
[ 75%] Built target nms_epu_epu
[100%] Linking CXX executable /quadric/sdk-cli/examples/custom_op/nms_epu_QC-U_16MB_4KB_128GBps_128GBps/output/nms_epu_host
[100%] Built target nms_epu_host
[SDK-CLI] : Executing on QC-U simulator
2026-06-19 03:58 - DEBUG - sdk - cli - Executing command: ./nms_epu_host -c --ddrRdBwTotal 1048576.0 --ddrWrBwTotal 1048576.0 --ddrAxiWidth 128 --instMemDepth 1310720 --ocmSize 16777216 --cycleTimeNS 0.5882352941176471 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 1 --numClusters 1
2026-06-19 03:58 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-06-19 03:58 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.
[SDK-CLI] : Execution completed.
print(cgc_job)
cgc_job.plot_run_statistics()
╒═════════════════════╤══════════════════════════════════════════════════════════════════════════╕
│ Module Name │ ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ ultraface_quant_v2-custom.onnx │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Custom Ops │ /quadric/sdk-cli/examples/custom_op/ultraface_scores.hpp │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-U │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 16MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 1.107MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 0.506kB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.127MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 0.100 │
╘═════════════════════╧══════════════════════════════════════════════════════════════════════════╛
NOTE: CGC has used general convolution for some nodes, which may result in suboptimal performance. For performance-critical applications, please contact Quadric support to explore optimization strategies.
General convolution nodes: 306_quantized, 289_quantized, 296_quantized
For more details, see: https://app.quadric.ai/docs/latest/chimera-software-user-guide/chimera-graph-compiler-cgc/graph-optimizations-performed-by-cgc#general-convolution
╒════╤════════╤═════════════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪═════════════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input │ [1, 3, 240, 320] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼─────────────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ concat_output_1 │ [1, 6, 4420] │ tensor[FixedPoint32<16>] │ 0.000 │
╘════╧════════╧═════════════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 0.22 │
├──────────────────────────────────┼─────────┤
│ FPS │ 4649.69 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 1306.03 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 3560.17 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 1.31 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 0.13 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 5.96 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 0.58 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 1.68% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 365,616
[SDK-CLI] : Executions/second: 4,649.69
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 146.589K
data_array : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 64.52K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇ 36.563K
data_external: ▇▇▇▇ 13.3K
data_ocm : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 104.518K
for more information check run directory: /quadric/sdk-cli/examples/custom_op/ccl_build/ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_035845_2131c2
2026-06-19 03:58 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/custom_op/ccl_build/ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_035845_2131c2/data/ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/custom_op/ccl_build/ultraface_quant_v2_custom_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_035845_2131c2/data'
