Model Demo: MaskRCNN Detection and Segmentation

NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn.ipynb.

Mask R-CNN for Object Detection and Segmentation

Abstract

The model generates bounding boxes and segmentation masks for each instance of an object in the image. It's based on Feature Pyramid Network (FPN) and a ResNet101 backbone.

These methods are implemented in Mask R-CNN.

Mask R-CNN models can be downloaded from torchvision. In this implementation, we are going to use the torchvision version.

Please refer papers for details.

• Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick, "Mask R-CNN"

Object Detection and Segmentation

Mask R-CNN model can predict both bounding boxes and masks of objects in one run.

Notebook Outline

In this notebook, the following steps are taken to experiment the model.

• ONNX Export
• Quantization and Compilation
• Inference
• Display Inference Results

import matplotlib.pyplot as plt
import numpy as np
import onnx
import os
from pathlib import Path
from PIL import Image
import torch, torchvision
from torch.utils.data import Subset
from torchvision.transforms import Compose, Normalize, ToTensor
import warnings

from examples.models.zoo.zoo_utils import onnx_check_and_simplify
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
import tvm.contrib.epu.chimera_job.constants as sdk_constants
from sdk_cli.lib.inference import InferenceEngine, batch_inference
from sdk_cli.lib.quantize import (
    QuantizedONNXModel,
    quantize_onnx_model,
)
from sdk_cli.utils.models.rcnn import RCNNModelVariant
from sdk_cli.node_builtins.classical.rcnn_postprocessing import (
    get_rcnn_parameters,
    get_rcnn_postprocessor,
    rcnn_postprocessing,
)
from sdk_cli.node_builtins.outputs.bbox_label_visualizer import draw_bbox
from sdk_cli.node_builtins.outputs.segmentation_visualizer import (
    SegmentationVariant,
    draw_segmentation,
)
from sdk_cli.utils.transforms import ResizePad
from sdk_cli.utils.datasets import QuadricCalibration
from sdk_cli.utils.datasets.COCO import COCO91CLASSES
import tvm.contrib.epu.graphutils as gutils


warnings.filterwarnings("ignore")

Model Selection

In this notebook, the following models can be experimented. We are going to use mask_rcnn_800x800 here.

• mask_rcnn_800x800
• mask_rcnn_800x1344

MODEL_NAME = RCNNModelVariant.mask_rcnn_800x800

rcnn_parameters = get_rcnn_parameters(MODEL_NAME)
model_input_size = rcnn_parameters.model_input_size

ONNX Export

Load Model and Export Full ONNX

model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)

x = torch.rand(1, 3, *model_input_size[::-1])
onnx_file = MODEL_NAME + ".onnx"

torch.onnx.export(
    model,
    x,  # ONNX requires fixed input size
    onnx_file,
    do_constant_folding=True,
    dynamic_axes={
        "images_tensors": [0, 1, 2, 3],
        "boxes": [0, 1],
        "labels": [0],
        "scores": [0],
        "masks": [0, 1, 2, 3],
    },
    input_names=["input_image"],
    output_names=["boxes", "labels", "scores", "masks"],
    opset_version=sdk_constants.DEFAULT_ONNX_OPSET,
)

## load your predefined ONNX model
model = onnx_check_and_simplify(onnx.load(onnx_file))

onnx.save(model, onnx_file)
print(f"Simplified {MODEL_NAME} onnx is saved as {onnx_file}")

Simplified mask_rcnn_800x800 onnx is saved as mask_rcnn_800x800.onnx

Extract Backbone ONNX

## load the ONNX model
model = onnx.load(onnx_file)
backbone_onnx_file = MODEL_NAME + "-backbone.onnx"

util = gutils.CustomOpReplacer(model)
sub_graph, _ = util.extract_subgraph_by_name_matching(["/backbone/.*"])
onnx.save(sub_graph, backbone_onnx_file)
print(f"{MODEL_NAME}'s backbone onnx is saved as {backbone_onnx_file}")

mask_rcnn_800x800's backbone onnx is saved as mask_rcnn_800x800-backbone.onnx

Quantization and Compilation

Quantization

dataset_mean, dataset_std = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]
transforms = Compose(
    [
        ResizePad(model_input_size),
        ToTensor(),
        Normalize(dataset_mean, dataset_std),
    ]
)

## Path to directory containing data to use for for numerical range calibration during quantization
## Data is used also used to compare accuracy of fp32 and int8 models
dataset = QuadricCalibration.Dataset(transform=transforms)

## NOTE: `coco-like` is the 0th index target for `QuadricCalibration.Dataset`
coco_like_data_indices = [index for index, target in enumerate(dataset.targets) if target == 0]
coco_like_subset_of_dataset = Subset(dataset, coco_like_data_indices)

quantized_onnx_model: QuantizedONNXModel = quantize_onnx_model(
    backbone_onnx_file,
    coco_like_subset_of_dataset,
    asymmetric_activation=True,
)
print(f"{backbone_onnx_file}'s quantized onnx is saved as {quantized_onnx_model.model_path}")

2026-09-22 02:53 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-09-22 02:53 - DEBUG - sdk - quantize - Forcing node types: []
2026-09-22 02:53 - DEBUG - sdk - quantize - ONNX Node types excluded from quantization: ['Softmax', 'Sigmoid', 'QuadricCustomOp']
2026-09-22 02:53 - DEBUG - sdk - quantize - ONNX Node names excluded from quantization: []
2026-09-22 02:53 - INFO - sdk - quantize - Starting quantization...
WARNING:root:Please use QuantFormat.QDQ for activation type QInt8 and weight type QInt8. Or it will lead to bad performance on x64.
2026-09-22 02:54 - INFO - sdk - quantize - Quantization completed! Quantized model saved to /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q.onnx
2026-09-22 02:54 - INFO - sdk - quantize - ONNX full precision model size: 102.51MB
2026-09-22 02:54 - INFO - sdk - quantize - ONNX quantized model size: 25.79MB
2026-09-22 02:54 - INFO - sdk - quantize - ONNX model shapes inferred.
2026-09-22 02:54 - INFO - sdk - quantize - ONNX Model with well-defined shapes has been saved at `/quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx`.
2026-09-22 02:54 - DEBUG - sdk - quantize - Checking for FLOAT/FLOAT16 types...
2026-09-22 02:54 - INFO - sdk - quantize - Checking for remaining FLOAT/FLOAT16 types.
2026-09-22 02:54 - INFO - sdk - quantize - Model still has FLOAT/FLOAT16 types after quantization. Creating ranges for floating point tensors using calibration data...
2026-09-22 02:54 - INFO - sdk - quantize - Saved computed tensor ranges to /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges.
2026-09-22 02:54 - INFO - sdk - quantize - 
╒═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╤══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Quantized ONNX Model                                                                                                      │ Tensor Ranges File                                                                                                           │
╞═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx │ /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q_shaped.tranges │
╘═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╧══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╛


mask_rcnn_800x800-backbone.onnx's quantized onnx is saved as /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx

Compilation

cgc_job = ChimeraJob(
    model_p=str(quantized_onnx_model.model_path),
    trange_file=str(quantized_onnx_model.tensor_ranges_path),
)
cgc_job.compile(quiet=True)
print(cgc_job)

╒═════════════════════╤═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name         │ mask_rcnn_800x800_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1            │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File           │ /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target      │ QC-U                                                                                                                      │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores     │ 1                                                                                                                         │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700                                                                                                                     │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size            │ 16MB                                                                                                                      │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size            │ 4kB                                                                                                                       │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW    │ 128GBps                                                                                                                   │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW   │ 128GBps                                                                                                                   │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE         │ 16                                                                                                                        │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M             │ 15.906MB                                                                                                                  │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM             │ 3.000kB                                                                                                                   │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes  │ 36.621MB                                                                                                                  │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs       │ 88.381                                                                                                                    │
╘═════════════════════╧═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╛

╒════╤════════╤═════════════════════════════════════════════════════════════╤════════════════════╤══════════════════════════╤═══════╕
│    │ Type   │ Name                                                        │ shape              │ type                     │ mse   │
╞════╪════════╪═════════════════════════════════════════════════════════════╪════════════════════╪══════════════════════════╪═══════╡
│  0 │ Input  │ /transform/Unsqueeze_12_output_0                            │ [1, 3, 800, 800]   │ tensor[FixedPoint32<29>] │ n/a   │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  1 │ Output │ /backbone/fpn/layer_blocks.0/layer_blocks.0.0/Conv_output_0 │ [1, 256, 200, 200] │ tensor[FixedPoint32<27>] │ n/a   │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  2 │ Output │ /backbone/fpn/layer_blocks.1/layer_blocks.1.0/Conv_output_0 │ [1, 256, 100, 100] │ tensor[FixedPoint32<27>] │ n/a   │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  3 │ Output │ /backbone/fpn/layer_blocks.2/layer_blocks.2.0/Conv_output_0 │ [1, 256, 50, 50]   │ tensor[FixedPoint32<27>] │ n/a   │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  4 │ Output │ /backbone/fpn/layer_blocks.3/layer_blocks.3.0/Conv_output_0 │ [1, 256, 25, 25]   │ tensor[FixedPoint32<27>] │ n/a   │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  5 │ Output │ /backbone/fpn/extra_blocks/MaxPool_output_0                 │ [1, 256, 13, 13]   │ tensor[FixedPoint32<27>] │ n/a   │
╘════╧════════╧═════════════════════════════════════════════════════════════╧════════════════════╧══════════════════════════╧═══════╛

Demo

Inference as Batch<a id='Inference-as-Batch'></a>

ChimeraJob supports batch execution for both ort and iss. This invokes specified number of threads to execute inference in parallel.

Here, we invoke multiple threads for images so that the inference on the target images finishes faster.

all_image_paths = [
    "../../../common/calibration/face/daniel_maverick.png",
    "../../../common/calibration/face/sales_squad_jani.jpg",
    "../../../common/calibration/coco-like/33823288584_1d21cf0a26_k.jpeg",
]

all_images = []
for image_path in all_image_paths:
    all_images.append(np.expand_dims(transforms(Image.open(image_path)), axis=0))
NUM_IMAGES = len(all_images)

engines = {
    InferenceEngine.CHIMERA_ORT_INT8: cgc_job,
    InferenceEngine.CHIMERA_ISS_INT8: cgc_job,
}

outputs_per_inference_engine = {}
THREADS = min(NUM_IMAGES, 6)
for inference_engine, engine in engines.items():
    all_backbone_outputs = batch_inference(
        inference_engine,
        engine,
        all_images,
        threads=THREADS,
    )
    outputs_per_inference_engine[inference_engine] = all_backbone_outputs

2026-09-22 02:57 - WARNING - epu - chimera_job - ORT is not threadsafe -- forcing single threaded batch execution
100%|████████████████████████████████████████████| 3/3 [00:03<00:00,  1.09s/it]
Processing: 100%|███████████████████████████████| 3/3 [05:41<00:00, 113.78s/it]

Run Statistics for Mask RCNN Backbone

print(cgc_job)
cgc_job.plot_run_statistics()

╒═════════════════════╤═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╕
│ Module Name         │ mask_rcnn_800x800_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1            │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ONNX File           │ /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/mask_rcnn_800x800-backbone_OpSet16_optimized_asym_int8_q_shaped.onnx │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Product Target      │ QC-U                                                                                                                      │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Number of Cores     │ 1                                                                                                                         │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700                                                                                                                     │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ L2M Size            │ 16MB                                                                                                                      │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ LRM Size            │ 4kB                                                                                                                       │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Read BW    │ 128GBps                                                                                                                   │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ External Write BW   │ 128GBps                                                                                                                   │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ MACS per PE         │ 16                                                                                                                        │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max L2M             │ 15.906MB                                                                                                                  │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max LRM             │ 3.000kB                                                                                                                   │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes  │ 36.621MB                                                                                                                  │
├─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ Network GMACs       │ 88.381                                                                                                                    │
╘═════════════════════╧═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╛

╒════╤════════╤═════════════════════════════════════════════════════════════╤════════════════════╤══════════════════════════╤═══════╕
│    │ Type   │ Name                                                        │ shape              │ type                     │ mse   │
╞════╪════════╪═════════════════════════════════════════════════════════════╪════════════════════╪══════════════════════════╪═══════╡
│  0 │ Input  │ /transform/Unsqueeze_12_output_0                            │ [1, 3, 800, 800]   │ tensor[FixedPoint32<29>] │ n/a   │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  1 │ Output │ /backbone/fpn/layer_blocks.0/layer_blocks.0.0/Conv_output_0 │ [1, 256, 200, 200] │ tensor[FixedPoint32<27>] │ 0.002 │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  2 │ Output │ /backbone/fpn/layer_blocks.1/layer_blocks.1.0/Conv_output_0 │ [1, 256, 100, 100] │ tensor[FixedPoint32<27>] │ 0.002 │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  3 │ Output │ /backbone/fpn/layer_blocks.2/layer_blocks.2.0/Conv_output_0 │ [1, 256, 50, 50]   │ tensor[FixedPoint32<27>] │ 0.002 │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  4 │ Output │ /backbone/fpn/layer_blocks.3/layer_blocks.3.0/Conv_output_0 │ [1, 256, 25, 25]   │ tensor[FixedPoint32<27>] │ 0.001 │
├────┼────────┼─────────────────────────────────────────────────────────────┼────────────────────┼──────────────────────────┼───────┤
│  5 │ Output │ /backbone/fpn/extra_blocks/MaxPool_output_0                 │ [1, 256, 13, 13]   │ tensor[FixedPoint32<27>] │ 0.001 │
╘════╧════════╧═════════════════════════════════════════════════════════════╧════════════════════╧══════════════════════════╧═══════╛

Post-ISS Report 1.7 GHz ***
Fully placed-and-routed gate simulation: 
╒══════════════════════════════════╤═════════╕
│ Latency (ms)                     │ 11.33   │
├──────────────────────────────────┼─────────┤
│ FPS                              │ 88.29   │
├──────────────────────────────────┼─────────┤
│ Average Power @ 3nm SSGNP (mW)   │ 3958.78 │
├──────────────────────────────────┼─────────┤
│ FPS per Watt @ 3nm SSGNP (FPS/W) │ 22.30   │
├──────────────────────────────────┼─────────┤
│ Ext Rd Bytes (MB)                │ 130.50  │
├──────────────────────────────────┼─────────┤
│ Ext Wr Bytes (MB)                │ 139.48  │
├──────────────────────────────────┼─────────┤
│ Avg Ext Rd BW (GBps)             │ 11.25   │
├──────────────────────────────────┼─────────┤
│ Avg Ext Wr BW (GBps)             │ 12.03   │
├──────────────────────────────────┼─────────┤
│ MAC Utilization                  │ 28.02%  │
╘══════════════════════════════════╧═════════╛
*** Data generated using 7nm SSGNP gatesim and scaled to 3nm

[SDK-CLI] : TotalCycles: 19,254,213
[SDK-CLI] : Executions/second: 88.29

compute      : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 6.08M
data_array   : ▇▇▇▇ 715.195K
mac          : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 8.673M
data_external: ▇▇▇▇▇▇▇▇▇▇▇ 1.942M
data_ocm     : ▇▇▇▇▇▇▇▇▇ 1.6M

for more information check run directory: /quadric/sdk-cli/examples/models/mask_rcnn/segmentor/ccl_build/mask_rcnn_800x800_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_025747_aea971


2026-09-22 03:03 - INFO - epu - chimera_job - Combined plots generated and saved to: 
/quadric/sdk-cli/examples/models/mask_rcnn/segmentor/ccl_build/mask_rcnn_800x800_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_025747_aea971/data/mask_rcnn_800x800_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1.combined.png





'/quadric/sdk-cli/examples/models/mask_rcnn/segmentor/ccl_build/mask_rcnn_800x800_backbone_OpSet16_optimized_asym_int8_q_shaped_QC_U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1/run/20260922_025747_aea971/data'

Display Inference Results

Postprocessing

At first, post-processing for inference results is done here to retrieve bounding boxes and segmentation masks.

maskrcnn_postprocessor = get_rcnn_postprocessor(rcnn_parameters.postprocess_model)

detections_per_inference_engine = {}
masks_per_inference_engine = {}

for inference_engine, all_outputs in outputs_per_inference_engine.items():
    all_detections = []
    all_masks = []
    for outputs, image in zip(all_outputs, all_image_paths):
        detections, masks, _, _ = rcnn_postprocessing(
            outputs,
            model_input_size,
            Image.open(image).size,
            postprocessor=maskrcnn_postprocessor,
            score_threshold=rcnn_parameters.score_threshold,
        )
        all_detections.append(detections)
        all_masks.append(masks)

    detections_per_inference_engine[inference_engine] = all_detections
    masks_per_inference_engine[inference_engine] = all_masks

Display Detected Objects and Segmentation

Both bounding boxes and segmentation are drawn together.

%matplotlib inline


def trasnsform_masks(masks, model_input_size):
    width, height = model_input_size
    masks = masks[:, :height, :width]
    return masks


classname_to_classid = {classname: classid for classid, classname in COCO91CLASSES.items()}
table_id = classname_to_classid["diningtable"]

pic_len = len(engines) + 1

for i in range(len(all_image_paths)):
    ax, idx = {}, 1
    fig = plt.figure(figsize=(6 * pic_len, 6), tight_layout=True)

    ax[idx] = fig.add_subplot(int("1%s%s" % (pic_len, idx)))
    ax[idx].imshow(Image.open(all_image_paths[i]))
    ax[idx].set_title("Original Image")
    ax[idx].axis("off")

    for (inference_engine, bboxes), (_, masks) in zip(
        detections_per_inference_engine.items(), masks_per_inference_engine.items()
    ):
        idx += 1
        ax[idx] = fig.add_subplot(int("1%s%s" % (pic_len, idx)))
        # If there aredining tables, they are segmented at first.
        detected_frame = draw_segmentation(
            SegmentationVariant.InstanceSegmentationFullScale,
            np.array(Image.open(all_image_paths[i])),
            trasnsform_masks(masks[i], model_input_size),
            classes=COCO91CLASSES,
            bboxes=bboxes[i].numpy(),
            vis_class=[table_id],
            alpha=0.7,
            random_seed=65,
        )
        # All objects other than dining tables are segmented.
        detected_frame = draw_segmentation(
            SegmentationVariant.InstanceSegmentationFullScale,
            detected_frame,
            trasnsform_masks(masks[i], model_input_size),
            classes=COCO91CLASSES,
            bboxes=bboxes[i].numpy(),
            no_vis_class=[table_id],
            alpha=0.7,
            random_seed=65,
        )
        # Bounding boxes are drawn.
        detected_frame = draw_bbox(
            detected_frame, bboxes[i], classes=COCO91CLASSES, show_mask=False
        )
        ax[idx].imshow(detected_frame)
        ax[idx].set_title(f"{MODEL_NAME}: {str(inference_engine).upper()}")
        ax[idx].axis("off")

    fig.show()

Citation

@misc{matterport_maskrcnn_2017,
  title={Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow},
  author={Waleed Abdulla},
  year={2017},
  publisher={Github},
  journal={GitHub repository},
  howpublished={\url{https://github.com/matterport/Mask_RCNN}},
}