NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/mediapipe/hand/MediapipeHand.ipynb.
TFLITE Mediapipe Hand Solution
MediaPipe Hands is a high-fidelity hand and finger tracking solution. It employs machine learning (ML) to infer 21 landmarks of a hand from just a single frame. Whereas current state-of-the-art approaches rely primarily on powerful desktop environments for inference, this method achieves real-time performance on a mobile phone, and even scales to multiple hands.
Hand Pipeline
MediaPipe Hands utilizes an ML pipeline consisting of multiple models working together: A palm detection model that operates on the full image and returns an oriented hand bounding box. A hand landmark model that operates on the cropped image region defined by the palm detector and returns 3D hand keypoints. This strategy is similar to that employed in MediaPipe Face solution, which uses a face detector together with a face landmark model.
Providing the accurately cropped hand image to the hand landmark model drastically reduces the need for data augmentation (e.g. rotations, translation and scale) and instead allows the network to dedicate most of its capacity towards coordinate prediction accuracy.
The following figure shows a hand solution example. When the palm detector model takes an original image in the left as an input, it detects a palm like the middle image.
The hand landmark model takes the detected palm as an input, and infers hand landmakrs as shown in the right.
Demo
Now, we are going to use actual Mediapipe models to infer palm positions and hand (finger) landmarks.
- Models
- Palm Detection Model
- In this notebook, palm_detection_lite is presented.
- Hand Landmark Model
- In this notebook, hand_landmark_lite is presented.
- Palm Detection Model
Convert TFLITE to ONNX
We are going to convert TFLITE files to ONNX files.
The following packages are installed for this conversion.
- tensorflow
- onnxconverter-common
- tensorflow-onnx
!pip3 install -r ../../../requirements.txt -q
[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[0m[33m
[0m
import os
import sys
from pathlib import Path
FILE_PATH = Path(os.path.abspath(""))
sys.path.append(f"{FILE_PATH.parent}")
from examples.models.zoo.zoo_utils import download_and_extract_zip
github_list = [
(
"https://github.com/microsoft/onnxconverter-common/archive/refs/tags/v%s.zip",
"1.15.0",
),
("https://github.com/onnx/tensorflow-onnx/archive/refs/tags/v%s.zip", "1.16.1"),
]
for url, version in github_list:
url = url % (version)
basename = url.split("/")[-5]
filename = Path(url).name
if not Path(basename).exists():
download_and_extract_zip(url, filename)
Path(f"{basename}-{version}").rename(basename)
Path(filename).unlink(missing_ok=True)
print(f"{basename} is setup.")
onnxconverter-common is setup.
tensorflow-onnx is setup.
The following TFLITE files are converted to ONNX files.
- palm_detection_lite.tflite
- hand_landmark_lite.tflite
from pathlib import Path
from mputils.convert_utils import create_onnx_from_tflite
tf_file_list = ["palm_detection_lite.tflite", "hand_landmark_lite.tflite"]
for tf_file in tf_file_list:
file = create_onnx_from_tflite(tf_file)
file = Path(file)
new_file = f"{file.stem}_float32{file.suffix}".replace("-convert", "")
os.rename(file, new_file)
print(f"onnx is renamed to {new_file}")
Generate onnx as palm_detection_lite-convert.onnx
onnx is renamed to palm_detection_lite_float32.onnx
Generate onnx as hand_landmark_lite-convert.onnx
onnx is renamed to hand_landmark_lite_float32.onnx
Compilation of each network
At first, those two models are compiled with ChimeraJob().
from pathlib import Path
import subprocess
import tvm.contrib.epu.chimera_job.core as core
from tvm.contrib.epu.chimera_job.hw_config import DEFAULT_8_ARRAY_SIZE
from tvm.contrib.epu.chimera_job.quantize import quadric_quantize
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from sdk_cli.utils import model_helpers
def get_cgc_job(
onnx_model, input_size, calibration_folder="images", hw_config=DEFAULT_8_ARRAY_SIZE
):
dataset_input_size = (input_size, input_size) # an (W, H) tuple
dataset_mean = [0, 0, 0]
dataset_std = [1, 1, 1]
# include quadric's cli helpers and instantiate a module to help
mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
print("Start quantizing %s" % (onnx_model))
quantize_result = quadric_quantize(
onnx_model, 100, mh, calibration_folder, asymmetric_activation=True
)
print("Define chimerajob with %s" % (quantize_result.qmodel_path))
cgc_job = ChimeraJob(
model_p=quantize_result.qmodel_path,
**hw_config.to_dict(),
)
print("Start network compilation")
cgc_job.compile(quiet=True)
return cgc_job, quantize_result.qmodel_path, mh
palm_detection_lite
To detect initial hand locations, a single-shot detector model optimized for mobile real-time is designed. Detecting hands is a decidedly complex task: lite model and full model have to work across a variety of hand sizes with a large scale span (~20x) relative to the image frame and be able to detect occluded and self-occluded hands. Whereas faces have high contrast patterns, e.g., in the eye and mouth region, the lack of such features in hands makes it comparatively difficult to detect them reliably from their visual features alone. Instead, providing additional context, like arm, body, or person features, aids accurate hand localization.
onnx_file = "palm_detection_lite_float32.onnx"
palm_detection_lite_cgc, qmodel_path, mh_detector_lite = get_cgc_job(
onnx_file,
input_size=192,
calibration_folder="../../../common/calibration/face",
)
print(palm_detection_lite_cgc)
/tmp/ipykernel_98286/3325241689.py:19: DeprecationWarning: Call to deprecated class ModelHelper. ('ModelHelper' class is being deprecated. Quadric APIs have been updated to use PyTorch datasets and transforms instead.) -- Deprecated since version 24.01.
mh = model_helpers.ModelHelper(dataset_input_size, dataset_mean, dataset_std)
Start quantizing palm_detection_lite_float32.onnx
2026-06-19 04:08 - INFO - epu - quantize - Collecting calibration data
2026-06-19 04:08 - INFO - epu - quantize - Optimized model to opset
2026-06-19 04:08 - INFO - epu - quantize - Saved optimized model to palm_detection_lite_float32_float32_opt.onnx
2026-06-19 04:08 - INFO - epu - quantize - Input shapes: [1, 3, 192, 192]. Input names: inputs0
2026-06-19 04:08 - INFO - epu - quantize - Output shapes: [[1, 2016, 18], [1, 2016, 1]]. Output names: ['outputs0', 'outputs1']
2026-06-19 04:08 - INFO - epu - quantize - applying calibration data to input: inputs0
2026-06-19 04:08 - INFO - epu - quantize - calibration set size: 8
2026-06-19 04:08 - INFO - epu - quantize - Running real quantization on this input: inputs0 with input shape: [1, 3, 192, 192]
2026-06-19 04:08 - DEBUG - epu - quantize - Full exclusion set for quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-06-19 04:08 - DEBUG - epu - quantize - excl_nodes []
2026-06-19 04:08 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-06-19 04:08 - INFO - epu - quantize - Quantization done succesfully!
2026-06-19 04:08 - INFO - epu - quantize - ONNX full precision model size: 3.72 MB
2026-06-19 04:08 - INFO - epu - quantize - ONNX quantized model size: 1.11 MB
2026-06-19 04:08 - INFO - epu - quantize - Saved quantized model to /quadric/sdk-cli/examples/models/mediapipe/hand/palm_detection_lite_opt_asym_int8_q.onnx
2026-06-19 04:08 - INFO - epu - quantize - Saved shape inferenced model to /quadric/sdk-cli/examples/models/mediapipe/hand/palm_detection_lite_opt_asym_int8_q.onnx
2026-06-19 04:08 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-06-19 04:08 - INFO - epu - quantize - Model still has FLOAT/FLOAT16 types. Creating ranges for floating point tensors using calibration data
2026-06-19 04:08 - INFO - epu - quantize - Saved tensor ranges to /quadric/sdk-cli/examples/models/mediapipe/hand/palm_detection_lite_opt_asym_int8_q.onnx.tranges
Define chimerajob with /quadric/sdk-cli/examples/models/mediapipe/hand/palm_detection_lite_opt_asym_int8_q.onnx
/tmp/ipykernel_98286/3325241689.py:27: DeprecationWarning: Specifying hardware configuration through individual parameters is deprecated. Please use hw_config parameter instead. Example: hw_cfg = HWConfig(product='QC-U', ocm_size='16MB'); ChimeraJob('model.onnx', hw_config=hw_cfg)
cgc_job = ChimeraJob(
Start network compilation
╒═════════════════════╤══════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ palm_detection_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/models/mediapipe/hand/palm_detection_lite_opt_asym_int8_q.onnx │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-N │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 0.728MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 1.500kB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 0.283 │
╘═════════════════════╧══════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤══════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪══════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ inputs0 │ [1, 3, 192, 192] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ outputs0 │ [1, 2016, 18] │ tensor[FixedPoint32<23>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ outputs1 │ [1, 2016, 1] │ tensor[FixedPoint32<26>] │ n/a │
╘════╧════════╧══════════╧══════════════════╧══════════════════════════╧═══════╛
As can be seen in the above figure, this model can run on less than 1MB L2M memory.
hand_landmark_lite
After the palm detection over the whole image, the subsequent hand landmark model performs precise keypoint localization of hand-knuckle coordinates inside the detected hand regions via regression, that is direct coordinate prediction. The model learns a consistent internal hand pose representation and is robust even to partially visible hands and self-occlusions.
onnx_file = "hand_landmark_lite_float32.onnx"
hand_landmark_lite_cgc, qmodel_path, mh_regressor_lite = get_cgc_job(
onnx_file,
input_size=224,
calibration_folder="../images/fingers",
)
print(hand_landmark_lite_cgc)
Start quantizing hand_landmark_lite_float32.onnx
2026-06-19 04:09 - INFO - epu - quantize - Collecting calibration data
2026-06-19 04:09 - INFO - epu - quantize - Optimized model to opset
2026-06-19 04:09 - INFO - epu - quantize - Saved optimized model to hand_landmark_lite_float32_float32_opt.onnx
2026-06-19 04:09 - INFO - epu - quantize - Input shapes: [1, 3, 224, 224]. Input names: inputs0
2026-06-19 04:09 - INFO - epu - quantize - Output shapes: [[1, 63], [1, 1], [1, 1], [1, 63]]. Output names: ['outputs0', 'outputs1', 'outputs2', 'outputs3']
2026-06-19 04:09 - INFO - epu - quantize - applying calibration data to input: inputs0
2026-06-19 04:09 - INFO - epu - quantize - calibration set size: 11
2026-06-19 04:09 - INFO - epu - quantize - Running real quantization on this input: inputs0 with input shape: [1, 3, 224, 224]
2026-06-19 04:09 - INFO - epu - quantize - Quantization started...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-06-19 04:09 - INFO - epu - quantize - Quantization done succesfully!
2026-06-19 04:09 - INFO - epu - quantize - ONNX full precision model size: 3.91 MB
2026-06-19 04:09 - INFO - epu - quantize - ONNX quantized model size: 1.11 MB
2026-06-19 04:09 - INFO - epu - quantize - Saved quantized model to /quadric/sdk-cli/examples/models/mediapipe/hand/hand_landmark_lite_opt_asym_int8_q.onnx
2026-06-19 04:09 - INFO - epu - quantize - Saved shape inferenced model to /quadric/sdk-cli/examples/models/mediapipe/hand/hand_landmark_lite_opt_asym_int8_q.onnx
2026-06-19 04:09 - INFO - epu - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-06-19 04:09 - INFO - epu - quantize - Model still has FLOAT/FLOAT16 types. Creating ranges for floating point tensors using calibration data
2026-06-19 04:09 - INFO - epu - quantize - Saved tensor ranges to /quadric/sdk-cli/examples/models/mediapipe/hand/hand_landmark_lite_opt_asym_int8_q.onnx.tranges
Define chimerajob with /quadric/sdk-cli/examples/models/mediapipe/hand/hand_landmark_lite_opt_asym_int8_q.onnx
/tmp/ipykernel_98286/3325241689.py:27: DeprecationWarning: Specifying hardware configuration through individual parameters is deprecated. Please use hw_config parameter instead. Example: hw_cfg = HWConfig(product='QC-U', ocm_size='16MB'); ChimeraJob('model.onnx', hw_config=hw_cfg)
cgc_job = ChimeraJob(
Start network compilation
╒═════════════════════╤═════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ hand_landmark_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/models/mediapipe/hand/hand_landmark_lite_opt_asym_int8_q.onnx │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-N │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 1.160MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 1.875kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 0.146 │
╘═════════════════════╧═════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤══════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪══════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ inputs0 │ [1, 3, 224, 224] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ outputs0 │ [1, 63] │ tensor[FixedPoint32<23>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ outputs1 │ [1, 1] │ tensor[FixedPoint32<31>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 3 │ Output │ outputs2 │ [1, 1] │ tensor[FixedPoint32<31>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 4 │ Output │ outputs3 │ [1, 63] │ tensor[FixedPoint32<31>] │ n/a │
╘════╧════════╧══════════╧══════════════════╧══════════════════════════╧═══════╛
As can be seen in the above figure, this model can run less than 2MB L2M memory.
Inference
Palm Detection
By using the palm detection model (palm_detection_lite), palm positions are detected from input images with the following patterns.
- Int8 Quantized ONNX: palm_detection_lite_cgc
- ISS: palm_detection_lite_cgc
import os
import sys
from pathlib import Path
from typing import Tuple
import onnx
import matplotlib.pyplot as plt
import numpy as np
import torch
import cv2
import glob
from sdk_cli.node_builtins.classical.normalize import normalize
from sdk_cli.node_builtins.classical.object_detector_postprocessing import (
MEDIAPIPE_FULL_RANGE_HAND_DETECTOR_PARAMETERS,
ObjectDetectorParameters,
object_detector_postprocessing,
)
from sdk_cli.node_builtins.classical.resize_pad import resize_pad
from sdk_cli.node_builtins.inputs.image import load_image
FILE_PATH = Path(os.path.abspath(""))
sys.path.append(f"{FILE_PATH.parent}")
from sdk_cli.lib.inference import InferenceEngine
all_images = ["../../../common/calibration/face/nigel_daniel.jpg"]
NUM_IMAGES = len(all_images)
THREADS = min(NUM_IMAGES, 6)
hand_detector_parameters: ObjectDetectorParameters = MEDIAPIPE_FULL_RANGE_HAND_DETECTOR_PARAMETERS
engines = {
InferenceEngine.CHIMERA_ORT_INT8: palm_detection_lite_cgc,
InferenceEngine.CHIMERA_ISS_INT8: palm_detection_lite_cgc,
}
def preprocess_detector(image_path: str, input_image_size: Tuple[int, int]):
"""Preprocess stuff."""
original_image = load_image(image_path)
input_image, scale, pad = resize_pad(original_image, input_image_size)
# input_image is channels first, usually channels last here
input_image = normalize(input_image, np.array([0.0, 0.0, 0.0]), np.array([1.0, 1.0, 1.0]))
# input_image is channels first and has a batch dimension
assert input_image.shape[1] == 3, "Assertion failed"
assert input_image.shape[2] == input_image_size[1], "Assertion failed"
assert input_image.shape[3] == input_image_size[0], "Assertion failed"
return original_image, input_image.astype(np.float32), scale, pad
def postprocess_detector(
original_image: np.ndarray,
out0: np.ndarray,
out1: np.ndarray,
scale: float,
pad: Tuple[int, int],
object_detector_parametrs: ObjectDetectorParameters,
) -> Tuple[np.ndarray, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
"""Postprocess stuff."""
copy_of_original_image = original_image.copy()
img, affine, box, obj_detections = object_detector_postprocessing(
copy_of_original_image, out0, out1, scale, pad, object_detector_parametrs
)
return (
np.moveaxis(copy_of_original_image, 0, -1),
torch.from_numpy(img),
torch.from_numpy(affine),
torch.from_numpy(box),
torch.from_numpy(obj_detections),
)
outputs_per_inference_engine = dict()
for inference_engine in engines.keys():
all_preprocessed_inputs = []
outputs_per_inference_engine[inference_engine] = []
for index, image in enumerate(all_images):
original_image, transformed_image, scale, pad = preprocess_detector(
image, hand_detector_parameters.input_image_size
)
if inference_engine == InferenceEngine.ONNXRUNTIME_FP32:
input_name = engines[inference_engine].get_inputs()[0].name
inference_outputs = engines[inference_engine].run(None, {input_name: transformed_image})
outputs = postprocess_detector(
original_image,
*inference_outputs,
scale=scale,
pad=pad,
object_detector_parametrs=hand_detector_parameters,
)
outputs_per_inference_engine[inference_engine].append(outputs)
else: # if inference_engine != InferenceEngine.ONNXRUNTIME_FP32:
all_preprocessed_inputs.append((original_image, transformed_image, scale, pad))
if inference_engine != InferenceEngine.ONNXRUNTIME_FP32:
all_model_inputs = []
for preprocessed_input in all_preprocessed_inputs:
all_model_inputs.append({"inputs0": preprocessed_input[1]})
if inference_engine == InferenceEngine.CHIMERA_ORT_INT8:
all_inference_outputs = engines[inference_engine].run_batch_ort_harness(
inputs=all_model_inputs, threads=THREADS
)
elif inference_engine == InferenceEngine.CHIMERA_ISS_INT8:
all_inference_outputs = engines[inference_engine].run_batch_inference_harness(
inputs=all_model_inputs, threads=THREADS
)
else:
raise ValueError(f"{inference_engine} is not implemented.")
assert len(all_inference_outputs) == len(
all_model_inputs
), f"Length mismatch: {len(all_inference_outputs)} vs {len(all_model_inputs)}."
for index, inference_outputs in enumerate(all_inference_outputs):
original_image = all_preprocessed_inputs[index][0]
scale = all_preprocessed_inputs[index][2]
pad = all_preprocessed_inputs[index][3]
outputs = postprocess_detector(
original_image,
inference_outputs["outputs0"],
inference_outputs["outputs1"],
scale=scale,
pad=pad,
object_detector_parametrs=hand_detector_parameters,
)
outputs_per_inference_engine[inference_engine].append(outputs)
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 22.90it/s]
0%| | 0/1 [00:00<?, ?it/s]
0%| | 0/27 [00:00<?, ?it/s]
FILM 1/27: 4%|█▉ | 1/27 [00:00<00:00, 4485.89it/s]
FILM 1/27: 4%|█▉ | 1/27 [00:00<00:00, 1309.90it/s]
FILM 1/27: 7%|███▉ | 2/27 [00:00<00:09, 2.53it/s]
FILM 2/27: 7%|███▉ | 2/27 [00:00<00:09, 2.53it/s]
FILM 2/27: 7%|███▉ | 2/27 [00:00<00:09, 2.53it/s]
FILM 2/27: 11%|█████▉ | 3/27 [00:02<00:18, 1.31it/s]
FILM 3/27: 11%|█████▉ | 3/27 [00:02<00:18, 1.31it/s]
FILM 3/27: 11%|█████▉ | 3/27 [00:02<00:18, 1.31it/s]
FILM 3/27: 15%|███████▊ | 4/27 [00:03<00:22, 1.04it/s]
FILM 4/27: 15%|███████▊ | 4/27 [00:03<00:22, 1.04it/s]
FILM 4/27: 15%|███████▊ | 4/27 [00:03<00:22, 1.04it/s]
FILM 4/27: 19%|█████████▊ | 5/27 [00:04<00:24, 1.10s/it]
FILM 5/27: 19%|█████████▊ | 5/27 [00:04<00:24, 1.10s/it]
FILM 5/27: 19%|█████████▊ | 5/27 [00:04<00:24, 1.10s/it]
FILM 5/27: 22%|███████████▊ | 6/27 [00:05<00:17, 1.17it/s]
FILM 6/27: 22%|███████████▊ | 6/27 [00:05<00:17, 1.17it/s]
FILM 6/27: 22%|███████████▊ | 6/27 [00:05<00:17, 1.17it/s]
FILM 6/27: 26%|█████████████▋ | 7/27 [00:05<00:17, 1.14it/s]
FILM 7/27: 26%|█████████████▋ | 7/27 [00:05<00:17, 1.14it/s]
FILM 7/27: 26%|█████████████▋ | 7/27 [00:05<00:17, 1.14it/s]
FILM 7/27: 30%|███████████████▋ | 8/27 [00:06<00:15, 1.24it/s]
FILM 8/27: 30%|███████████████▋ | 8/27 [00:06<00:15, 1.24it/s]
FILM 8/27: 30%|███████████████▋ | 8/27 [00:06<00:15, 1.24it/s]
FILM 8/27: 33%|█████████████████▋ | 9/27 [00:07<00:13, 1.31it/s]
FILM 9/27: 33%|█████████████████▋ | 9/27 [00:07<00:13, 1.31it/s]
FILM 9/27: 33%|█████████████████▋ | 9/27 [00:07<00:13, 1.31it/s]
FILM 9/27: 37%|███████████████████▎ | 10/27 [00:07<00:09, 1.70it/s]
FILM 10/27: 37%|██████████████████▉ | 10/27 [00:07<00:09, 1.70it/s]
FILM 10/27: 37%|██████████████████▉ | 10/27 [00:07<00:09, 1.70it/s]
FILM 10/27: 41%|████████████████████▊ | 11/27 [00:07<00:08, 1.80it/s]
FILM 11/27: 41%|████████████████████▊ | 11/27 [00:07<00:08, 1.80it/s]
FILM 11/27: 41%|████████████████████▊ | 11/27 [00:07<00:08, 1.80it/s]
FILM 11/27: 44%|██████████████████████▋ | 12/27 [00:08<00:07, 2.02it/s]
FILM 12/27: 44%|██████████████████████▋ | 12/27 [00:08<00:07, 2.02it/s]
FILM 12/27: 44%|██████████████████████▋ | 12/27 [00:08<00:07, 2.02it/s]
FILM 12/27: 48%|████████████████████████▌ | 13/27 [00:08<00:06, 2.19it/s]
FILM 13/27: 48%|████████████████████████▌ | 13/27 [00:08<00:06, 2.19it/s]
FILM 13/27: 48%|████████████████████████▌ | 13/27 [00:08<00:06, 2.19it/s]
FILM 13/27: 52%|██████████████████████████▍ | 14/27 [00:08<00:04, 2.68it/s]
FILM 14/27: 52%|██████████████████████████▍ | 14/27 [00:08<00:04, 2.68it/s]
FILM 14/27: 52%|██████████████████████████▍ | 14/27 [00:08<00:04, 2.68it/s]
FILM 14/27: 56%|████████████████████████████▎ | 15/27 [00:09<00:04, 2.52it/s]
FILM 15/27: 56%|████████████████████████████▎ | 15/27 [00:09<00:04, 2.52it/s]
FILM 15/27: 56%|████████████████████████████▎ | 15/27 [00:09<00:04, 2.52it/s]
FILM 15/27: 59%|██████████████████████████████▏ | 16/27 [00:09<00:04, 2.67it/s]
FILM 16/27: 59%|██████████████████████████████▏ | 16/27 [00:09<00:04, 2.67it/s]
FILM 16/27: 59%|██████████████████████████████▏ | 16/27 [00:09<00:04, 2.67it/s]
FILM 16/27: 63%|████████████████████████████████ | 17/27 [00:10<00:03, 2.73it/s]
FILM 17/27: 63%|████████████████████████████████ | 17/27 [00:10<00:03, 2.73it/s]
FILM 17/27: 63%|████████████████████████████████ | 17/27 [00:10<00:03, 2.73it/s]
FILM 18/27: 67%|██████████████████████████████████ | 18/27 [00:10<00:03, 2.73it/s]
FILM 18/27: 67%|██████████████████████████████████ | 18/27 [00:10<00:03, 2.73it/s]
FILM 18/27: 70%|███████████████████████████████████▉ | 19/27 [00:10<00:02, 3.31it/s]
FILM 19/27: 70%|███████████████████████████████████▉ | 19/27 [00:10<00:02, 3.31it/s]
FILM 19/27: 70%|███████████████████████████████████▉ | 19/27 [00:10<00:02, 3.31it/s]
FILM 19/27: 74%|█████████████████████████████████████▊ | 20/27 [00:11<00:02, 2.70it/s]
FILM 20/27: 74%|█████████████████████████████████████▊ | 20/27 [00:11<00:02, 2.70it/s]
FILM 20/27: 74%|█████████████████████████████████████▊ | 20/27 [00:11<00:02, 2.70it/s]
FILM 20/27: 78%|███████████████████████████████████████▋ | 21/27 [00:11<00:02, 2.68it/s]
FILM 21/27: 78%|███████████████████████████████████████▋ | 21/27 [00:11<00:02, 2.68it/s]
FILM 21/27: 78%|███████████████████████████████████████▋ | 21/27 [00:11<00:02, 2.68it/s]
FILM 21/27: 81%|█████████████████████████████████████████▌ | 22/27 [00:11<00:02, 2.39it/s]
FILM 22/27: 81%|█████████████████████████████████████████▌ | 22/27 [00:11<00:02, 2.39it/s]
FILM 22/27: 81%|█████████████████████████████████████████▌ | 22/27 [00:11<00:02, 2.39it/s]
FILM 22/27: 85%|███████████████████████████████████████████▍ | 23/27 [00:12<00:01, 2.02it/s]
FILM 23/27: 85%|███████████████████████████████████████████▍ | 23/27 [00:12<00:01, 2.02it/s]
FILM 23/27: 85%|███████████████████████████████████████████▍ | 23/27 [00:12<00:01, 2.02it/s]
FILM 23/27: 89%|█████████████████████████████████████████████▎ | 24/27 [00:13<00:01, 2.04it/s]
FILM 24/27: 89%|█████████████████████████████████████████████▎ | 24/27 [00:13<00:01, 2.04it/s]
FILM 24/27: 89%|█████████████████████████████████████████████▎ | 24/27 [00:13<00:01, 2.04it/s]
FILM 24/27: 93%|███████████████████████████████████████████████▏ | 25/27 [00:13<00:00, 2.48it/s]
FILM 25/27: 93%|███████████████████████████████████████████████▏ | 25/27 [00:13<00:00, 2.48it/s]
FILM 25/27: 93%|███████████████████████████████████████████████▏ | 25/27 [00:13<00:00, 2.48it/s]
FILM 26/27: 96%|█████████████████████████████████████████████████ | 26/27 [00:13<00:00, 2.48it/s]
FILM 26/27: 96%|█████████████████████████████████████████████████ | 26/27 [00:13<00:00, 2.48it/s]
FILM 27/27: 100%|███████████████████████████████████████████████████| 27/27 [00:13<00:00, 2.48it/s]
FILM 27/27: 100%|███████████████████████████████████████████████████| 27/27 [00:13<00:00, 2.02it/s]
100%|████████████████████████████████████████████| 1/1 [00:13<00:00, 13.69s/it]
print(palm_detection_lite_cgc)
palm_detection_lite_cgc.plot_run_statistics()
╒═════════════════════╤══════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ palm_detection_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/models/mediapipe/hand/palm_detection_lite_opt_asym_int8_q.onnx │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-N │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 0.728MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 1.500kB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼──────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 0.283 │
╘═════════════════════╧══════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤══════════╤══════════════════╤══════════════════════════╤════════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪══════════╪══════════════════╪══════════════════════════╪════════╡
│ 0 │ Input │ inputs0 │ [1, 3, 192, 192] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼────────┤
│ 1 │ Output │ outputs0 │ [1, 2016, 18] │ tensor[FixedPoint32<23>] │ 12.437 │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼────────┤
│ 2 │ Output │ outputs1 │ [1, 2016, 1] │ tensor[FixedPoint32<26>] │ 0.866 │
╘════╧════════╧══════════╧══════════════════╧══════════════════════════╧════════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 2.52 │
├──────────────────────────────────┼─────────┤
│ FPS │ 396.95 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 219.00 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 1812.56 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 1.76 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 0.15 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 0.68 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 0.06 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 6.46% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 4,282,654
[SDK-CLI] : Executions/second: 396.95
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 2.578M
data_array : ▇▇▇▇▇▇▇▇▇▇▇▇▇ 697.334K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 836.316K
data_external: ▏ 14.166K
data_ocm : ▇▇ 151.505K
for more information check run directory: /quadric/sdk-cli/examples/models/mediapipe/hand/ccl_build/palm_detection_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_041054_cf7af2
2026-06-19 04:11 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/models/mediapipe/hand/ccl_build/palm_detection_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_041054_cf7af2/data/palm_detection_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/models/mediapipe/hand/ccl_build/palm_detection_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_041054_cf7af2/data'
from sdk_cli.node_builtins.outputs.bounding_box_visualizer import (
draw_detections,
draw_roi,
)
def draw_detected_hands(
image: torch.Tensor, boxes: torch.Tensor, detections: torch.Tensor
) -> np.ndarray:
"""Draw detected bounding boxes onto original image."""
image_with_drawings = np.array(image.copy())
draw_roi(image=image_with_drawings, regions_of_interest=boxes)
draw_detections(image=image_with_drawings, detections=detections)
return image_with_drawings
all_opts = dict()
for inference_engine in outputs_per_inference_engine.keys():
all_opts[inference_engine] = []
for outputs in zip(
*[
outputs_per_inference_engine[inference_engine]
for inference_engine in outputs_per_inference_engine.keys()
]
):
ax, idx = dict(), 0
fig = plt.figure(figsize=(4 * len(outputs_per_inference_engine.keys()), 4), tight_layout=True)
for output, inference_engine in zip(outputs, outputs_per_inference_engine.keys()):
idx += 1
ax[idx] = fig.add_subplot(int("1%s%s" % (len(outputs_per_inference_engine.keys()), idx)))
all_opts[inference_engine].append(output)
ax[idx].imshow(draw_detected_hands(output[0].astype(np.uint8), output[3], output[4]))
ax[idx].set_title(f"Detected Palms: {str(inference_engine).upper()}")
ax[idx].axis("off")
fig.show()
Hand Landmark
By using the detected palms above, hand landmakrs are detected with hand_landmark_lite. Like the palm detection, the following patterns are experimented.
- Int8 Quantized ONNX: hand_landmark_lite_cgc
- ISS: hand_landmark_lite_cgc
Then, by using the drawing function, those landmarks are drawn on the original images.
from typing import Optional
from PIL import Image
from sdk_cli.node_builtins.classical.denormalize_landmarks import (
denormalize_landmarks,
)
from sdk_cli.node_builtins.classical.object_detector_postprocessing import (
MEDIAPIPE_HAND_DETECTOR_BOUNDING_BOX_REGRESSION_PARAMETERS,
ObjectDetectorParameters,
)
from sdk_cli.node_builtins.classical.resize.resize import one_step_resize
def preprocess_landmark(image_path: str, input_image_size: Tuple[int, int]):
"""Preprocess some stuff."""
original_image = load_image(image_path) # (224, 224)
transformed_image = one_step_resize(np.moveaxis(original_image, 0, -1), (192, 192, 3)).astype(
np.float32
)
transformed_image = normalize(
np.moveaxis(transformed_image, -1, 0),
np.array([0.0, 0.0, 0.0]),
np.array([1.0, 1.0, 1.0]),
)
return original_image, transformed_image
hand_regressor_parameters: ObjectDetectorParameters = (
MEDIAPIPE_HAND_DETECTOR_BOUNDING_BOX_REGRESSION_PARAMETERS
)
engines = {
InferenceEngine.CHIMERA_ORT_INT8: hand_landmark_lite_cgc,
InferenceEngine.CHIMERA_ISS_INT8: hand_landmark_lite_cgc,
}
detector_outputs_for_all_ainference_engines = []
for index in range(len(all_images)):
detector_outputs_for_all_ainference_engines.append(
{inference_engine: all_opts[inference_engine][index] for inference_engine in engines.keys()}
)
landmark_outputs_per_inference_engine = dict()
for inference_engine in engines.keys():
all_preprocessed_inputs = []
landmark_outputs_per_inference_engine[inference_engine] = []
for detector_output in [
detector_output[inference_engine]
for detector_output in detector_outputs_for_all_ainference_engines
]:
original_image, detector_outputs, affines, boxes, detections = detector_output
landmark_model_inputs = np.expand_dims(
np.moveaxis(
cv2.resize(
np.moveaxis(detector_outputs.numpy()[0], 0, -1),
hand_regressor_parameters.input_image_size,
),
-1,
0,
),
axis=0,
)
landmark_outputs, flag_outputs = [], []
if inference_engine == InferenceEngine.ONNXRUNTIME_FP32:
for landmark_model_input, affine in zip(landmark_model_inputs, affines):
input_name = engines[inference_engine].get_inputs()[0].name
inference_outputs = engines[inference_engine].run(
None, {input_name: np.expand_dims(landmark_model_input, axis=0)}
)
outputs = denormalize_landmarks(
original_image,
inference_outputs[0],
hand_regressor_parameters.input_image_size,
row_size=hand_regressor_parameters.row_size,
affines=affine,
)
landmark_outputs.append(outputs[0])
flag_outputs.append(inference_outputs[1][0])
else: # if inference_engine != InferenceEngine.ONNXRUNTIME_FP32:
all_model_inputs = []
for detector_output in detector_outputs.numpy():
landmark_model_input = np.expand_dims(
np.moveaxis(
cv2.resize(
np.moveaxis(detector_output, 0, -1),
hand_regressor_parameters.input_image_size,
),
-1,
0,
),
axis=0,
)
all_model_inputs.append({"inputs0": landmark_model_input})
if inference_engine == InferenceEngine.CHIMERA_ORT_INT8:
all_inference_outputs = engines[inference_engine].run_batch_ort_harness(
inputs=all_model_inputs, threads=min(len(all_model_inputs), 6)
)
elif inference_engine == InferenceEngine.CHIMERA_ISS_INT8:
all_inference_outputs = engines[inference_engine].run_batch_inference_harness(
inputs=all_model_inputs, threads=min(len(all_model_inputs), 6)
)
else:
raise ValueError(f"{inference_engine} is not implemented.")
for inference_outputs, affine in zip(all_inference_outputs, affines):
outputs = denormalize_landmarks(
original_image,
inference_outputs["outputs0"],
hand_regressor_parameters.input_image_size,
row_size=hand_regressor_parameters.row_size,
affines=affine,
)
landmark_outputs.append(outputs[0])
flag_outputs.append(inference_outputs["outputs1"][0])
landmark_outputs_per_inference_engine[inference_engine].append(
(
original_image,
np.stack(landmark_outputs),
np.stack(flag_outputs),
boxes,
detections,
)
)
100%|████████████████████████████████████████████| 1/1 [00:00<00:00, 33.18it/s]
0%| | 0/1 [00:00<?, ?it/s]
0%| | 0/16 [00:00<?, ?it/s]
FILM 1/16: 6%|███▏ | 1/16 [00:00<00:00, 5433.04it/s]
FILM 1/16: 6%|███▏ | 1/16 [00:00<00:00, 1582.76it/s]
FILM 1/16: 12%|██████▋ | 2/16 [00:02<00:14, 1.05s/it]
FILM 2/16: 12%|██████▋ | 2/16 [00:02<00:14, 1.05s/it]
FILM 2/16: 12%|██████▋ | 2/16 [00:02<00:14, 1.05s/it]
FILM 2/16: 19%|█████████▉ | 3/16 [00:02<00:11, 1.11it/s]
FILM 3/16: 19%|█████████▉ | 3/16 [00:02<00:11, 1.11it/s]
FILM 3/16: 19%|█████████▉ | 3/16 [00:02<00:11, 1.11it/s]
FILM 3/16: 25%|█████████████▎ | 4/16 [00:04<00:13, 1.09s/it]
FILM 4/16: 25%|█████████████▎ | 4/16 [00:04<00:13, 1.09s/it]
FILM 4/16: 25%|█████████████▎ | 4/16 [00:04<00:13, 1.09s/it]
FILM 4/16: 31%|████████████████▌ | 5/16 [00:04<00:09, 1.15it/s]
FILM 5/16: 31%|████████████████▌ | 5/16 [00:04<00:09, 1.15it/s]
FILM 5/16: 31%|████████████████▌ | 5/16 [00:04<00:09, 1.15it/s]
FILM 5/16: 38%|███████████████████▉ | 6/16 [00:05<00:07, 1.33it/s]
FILM 6/16: 38%|███████████████████▉ | 6/16 [00:05<00:07, 1.33it/s]
FILM 6/16: 38%|███████████████████▉ | 6/16 [00:05<00:07, 1.33it/s]
FILM 6/16: 44%|███████████████████████▏ | 7/16 [00:05<00:05, 1.54it/s]
FILM 7/16: 44%|███████████████████████▏ | 7/16 [00:05<00:05, 1.54it/s]
FILM 7/16: 44%|███████████████████████▏ | 7/16 [00:05<00:05, 1.54it/s]
FILM 7/16: 50%|██████████████████████████▌ | 8/16 [00:05<00:04, 1.92it/s]
FILM 8/16: 50%|██████████████████████████▌ | 8/16 [00:05<00:04, 1.92it/s]
FILM 8/16: 50%|██████████████████████████▌ | 8/16 [00:05<00:04, 1.92it/s]
FILM 8/16: 56%|█████████████████████████████▊ | 9/16 [00:06<00:03, 2.21it/s]
FILM 9/16: 56%|█████████████████████████████▊ | 9/16 [00:06<00:03, 2.21it/s]
FILM 9/16: 56%|█████████████████████████████▊ | 9/16 [00:06<00:03, 2.21it/s]
FILM 9/16: 62%|████████████████████████████████▌ | 10/16 [00:06<00:02, 2.35it/s]
FILM 10/16: 62%|███████████████████████████████▉ | 10/16 [00:06<00:02, 2.35it/s]
FILM 10/16: 62%|███████████████████████████████▉ | 10/16 [00:06<00:02, 2.35it/s]
FILM 10/16: 69%|███████████████████████████████████ | 11/16 [00:06<00:02, 2.43it/s]
FILM 11/16: 69%|███████████████████████████████████ | 11/16 [00:06<00:02, 2.43it/s]
FILM 11/16: 69%|███████████████████████████████████ | 11/16 [00:06<00:02, 2.43it/s]
FILM 11/16: 75%|██████████████████████████████████████▎ | 12/16 [00:07<00:02, 1.82it/s]
FILM 12/16: 75%|██████████████████████████████████████▎ | 12/16 [00:07<00:02, 1.82it/s]
FILM 12/16: 75%|██████████████████████████████████████▎ | 12/16 [00:07<00:02, 1.82it/s]
FILM 13/16: 81%|█████████████████████████████████████████▍ | 13/16 [00:07<00:01, 1.82it/s]
FILM 13/16: 81%|█████████████████████████████████████████▍ | 13/16 [00:07<00:01, 1.82it/s]
FILM 14/16: 88%|████████████████████████████████████████████▋ | 14/16 [00:07<00:01, 1.82it/s]
FILM 14/16: 88%|████████████████████████████████████████████▋ | 14/16 [00:07<00:01, 1.82it/s]
FILM 15/16: 94%|███████████████████████████████████████████████▊ | 15/16 [00:07<00:00, 1.82it/s]
FILM 15/16: 94%|███████████████████████████████████████████████▊ | 15/16 [00:07<00:00, 1.82it/s]
FILM 16/16: 100%|███████████████████████████████████████████████████| 16/16 [00:07<00:00, 1.82it/s]
FILM 16/16: 100%|███████████████████████████████████████████████████| 16/16 [00:07<00:00, 2.06it/s]
100%|████████████████████████████████████████████| 1/1 [00:08<00:00, 8.13s/it]
from typing import List, Optional
from sdk_cli.node_builtins.outputs.bounding_box_visualizer import (
draw_detections,
draw_landmarks,
draw_roi,
)
def draw_detected_hand_landmarks(
original_image: torch.Tensor,
all_landmarks_per_image: torch.Tensor,
all_flags_per_image: torch.Tensor,
connections: List[Tuple[int, int]],
draw_threshold: float,
boxes: Optional[torch.Tensor] = None,
detections: Optional[torch.Tensor] = None,
landmark_cut: Optional[int] = None,
) -> np.ndarray:
"""Draw for fun."""
image_with_drawings = np.array(original_image.copy())
for all_landmarks_per_detection, all_flags_per_detection in zip(
all_landmarks_per_image, all_flags_per_image
):
for landmark, flag in zip(
np.expand_dims(all_landmarks_per_detection, 0), all_flags_per_detection
):
if flag > draw_threshold:
landmark_to_draw = (
all_landmarks_per_detection[:landmark_cut, :]
if landmark_cut
else all_landmarks_per_detection
)
draw_landmarks(
image=image_with_drawings,
points=landmark_to_draw[:, :2],
connections=connections,
)
if boxes is not None:
draw_roi(image=image_with_drawings, regions_of_interest=boxes)
if detections is not None:
draw_detections(image=image_with_drawings, detections=detections)
return image_with_drawings
for landmark_outputs in zip(
*[
landmark_outputs_per_inference_engine[inference_engine]
for inference_engine in landmark_outputs_per_inference_engine.keys()
]
):
ax, idx = dict(), 0
fig = plt.figure(
figsize=(4 * len(landmark_outputs_per_inference_engine.keys()), 4),
tight_layout=True,
)
for output, inference_engine in zip(
landmark_outputs, landmark_outputs_per_inference_engine.keys()
):
idx += 1
ax[idx] = fig.add_subplot(
int("1%s%s" % (len(landmark_outputs_per_inference_engine.keys()), idx))
)
ax[idx].imshow(
draw_detected_hand_landmarks(
output[0].astype(np.uint8),
output[1],
output[2],
hand_regressor_parameters.connections,
hand_regressor_parameters.draw_threshold,
boxes=output[3],
detections=output[4],
)
)
ax[idx].set_title(f"Hand Landmarks: {str(inference_engine).upper()}")
ax[idx].axis("off")
fig.show()
print(hand_landmark_lite_cgc)
hand_landmark_lite_cgc.plot_run_statistics()
╒═════════════════════╤═════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name │ hand_landmark_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File │ /quadric/sdk-cli/examples/models/mediapipe/hand/hand_landmark_lite_opt_asym_int8_q.onnx │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-N │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 8MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 1.160MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 1.875kB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 0.000MB │
├─────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 0.146 │
╘═════════════════════╧═════════════════════════════════════════════════════════════════════════════════════════╛
╒════╤════════╤══════════╤══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞════╪════════╪══════════╪══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ inputs0 │ [1, 3, 224, 224] │ tensor[FixedPoint32<27>] │ n/a │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 1 │ Output │ outputs0 │ [1, 63] │ tensor[FixedPoint32<23>] │ 0.842 │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 2 │ Output │ outputs1 │ [1, 1] │ tensor[FixedPoint32<31>] │ 0.000 │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 3 │ Output │ outputs2 │ [1, 1] │ tensor[FixedPoint32<31>] │ 0.000 │
├────┼────────┼──────────┼──────────────────┼──────────────────────────┼───────┤
│ 4 │ Output │ outputs3 │ [1, 63] │ tensor[FixedPoint32<31>] │ 0.000 │
╘════╧════════╧══════════╧══════════════════╧══════════════════════════╧═══════╛
Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation:
╒══════════════════════════════════╤═════════╕
│ Latency (ms) │ 1.49 │
├──────────────────────────────────┼─────────┤
│ FPS │ 671.33 │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW) │ 236.89 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 2833.95 │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB) │ 2.24 │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB) │ 0.00 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps) │ 1.47 │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps) │ 0.00 │
├──────────────────────────────────┼─────────┤
│ MAC Utilization │ 5.62% │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm
[SDK-CLI] : TotalCycles: 2,532,271
[SDK-CLI] : Executions/second: 671.33
compute : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1.272M
data_array : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 643.748K
mac : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 551.044K
data_external: ▏ 18.945K
data_ocm : ▇ 42.743K
for more information check run directory: /quadric/sdk-cli/examples/models/mediapipe/hand/ccl_build/hand_landmark_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_041108_ecb231
2026-06-19 04:11 - INFO - epu - chimera_job - Combined plots generated and saved to:
/quadric/sdk-cli/examples/models/mediapipe/hand/ccl_build/hand_landmark_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_041108_ecb231/data/hand_landmark_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png
'/quadric/sdk-cli/examples/models/mediapipe/hand/ccl_build/hand_landmark_lite_opt_asym_int8_q_QC_N_1d7_8MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260619_041108_ecb231/data'
