Model Demo: SegFormer-B0 Landscape Segmentation (Hand-Written Kernels)

NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/segformer_b0_ade_cityscapes_landscape/segformer_b0_landscape.ipynb.

SegFormer-B0 Landscape Segmentation with Hand-Written Kernels on Chimera GPNPU

This notebook takes two landscape variants of SegFormer-B0 from a floating-point ONNX graph to a single program running on the Chimera GPNPU, and it differs from the other model demos in one respect: the kernels are hand-written and hand-optimized rather than left as compiler output. The graph compiler is still what produces every kernel and every constant blob; the demo then shows which kernels were edited by hand, what each edit does, and what it is worth in cycles.

SegFormer pairs a hierarchical Mix Transformer (MiT) backbone with a lightweight all-MLP decode head to turn an image into a dense per-pixel label map.

Why hand-write the kernels?

A transformer segmentation model has two very different kinds of work: dense convolution in the stem and decode head, and attention in between. Editing the generated kernels lets each part be expressed the way the hardware runs it best -- space-to-depth feeding a convolution for the stem, tile-block flows for attention, and a decode head that reduces the class dimension before its output leaves the chip. Composing all seventeen kernels behind a single entry point means the host issues one call per frame and every intermediate stays in device memory.

Target

Every figure in this notebook is measured on QC-U at 1.7 GHz: a 32x32 PE array, 16 MACs per PE, 16 MB OCM, 4 kB LRM, 128 GB/s in each direction, one core and one cluster -- QC-U_1d7_16MB_4kB_128GBps_128GBps_16_OFF_x1_x1. Both variants sustain over 350 frames per second at 256x512 with a full-resolution label map.

Pipeline

Model: SegFormer-B0 (MiT-B0 backbone, 256x512 input, 3.8 M parameters, 8.4 GMACs) in two variants -- Cityscapes (19 classes) and ADE20K (150 classes).

Related demo

examples/models/segformer/segformer.ipynb -- published as SegFormer Semantic Segmentation -- compiles a Cityscapes-trained SegFormer-B0 end to end with the Chimera Graph Compiler and hands back a single small CCL requantization kernel. That notebook is the compiler baseline; this one starts from the same compiler output and shows what hand-writing the kernels buys on top of it, across two decode heads rather than one. The two also ship different checkpoints under different licences -- see the License Notice below -- so they stay separate rather than merging.

License Notice

This example ships two variants whose weights come from different upstreams and are under different licences.

Cityscapes-19 is NVIDIA's own checkpoint, nvidia/segformer-b0-finetuned-cityscapes-512-1024, under the NVIDIA Source Code License, which permits use "non-commercially … for research or evaluation purposes only" (§3.3). A complete copy ships beside this notebook as LICENSE-nvidia-segformer.txt. Its weights are trained on Cityscapes, whose licence states that you may not use the dataset or any derivative work for commercial purposes: https://www.cityscapes-dataset.com/license/

ADE-150 is OpenMMLab's MMSegmentation ADE20K checkpoint, released under the Apache License 2.0. Its weights are trained on ADE20K, which is for non-commercial research and educational use: https://ade20k.csail.mit.edu/terms/

The SegFormer architecture is by NVIDIA Research. The evaluation frames are U.S. Government Works, not dataset validation images. Full detail: ATTRIBUTION.md.

What runs where. The model itself runs end to end on the Chimera GPNPU: all 17 kernels, in one entry point, with no host work between them. Two things around it run on the host CPU. The Cityscapes input is quantized to int8 before it is sent, which is what makes its input a quarter the size of ADE's — the mapping is exact integer arithmetic and reproduces the on-device quantizer bit for bit. And the reference comparison and every figure below are host-side: ONNX Runtime computes the float32 reference, and matplotlib draws the panels. Neither is part of the measured cycle counts, which come from the simulator's own per-region profile.

1. Setup

This demo requires SDK 26.08 or newer and carries its own copy of the attention head, built against the 26.08 tensor-slice API.

import hashlib
import json
import re
from pathlib import Path

import numpy as np

import segformer_landscape_helpers as helpers

%matplotlib inline

HERE = Path("").resolve()
WORK = HERE / "_run"
WORK.mkdir(exist_ok=True)
print(f"{len(helpers.KERNELS)} kernels, composed into one program with a single entry point")
print(f"variants: {', '.join(m['label'] for m in helpers.MODELS.values())}")

17 kernels, composed into one program with a single entry point
variants: Cityscapes-19, ADE-150

2. Quantization

The two float32 graphs and their calibration ranges are downloaded on first use and cached under _run/, so they are fetched once per checkout rather than committed here.

Both variants are quantized to symmetric per-tensor int8: calibration produces one range per activation tensor, and the scale follows directly from it as scale = max(|lo|, |hi|) / 127. Weights and activations are both int8. The same scales drive the graph compiler and the fixed-point shifts written into the kernels, which is what keeps the two consistent.

The attention softmaxes are held out of int8 — eight of them, excluded by node type at quantization — and run in fixed point on device with the exponential evaluated at 12 fractional bits. Everything else is int8.

The calibration sets differ between the variants: the Cityscapes ranges come from 12 Cityscapes training images, the ADE ranges from three generic photographs.

for variant, spec in helpers.MODELS.items():
    _, ranges_path = helpers.artifact_paths(variant)
    stats = helpers.calibration_summary(ranges_path)
    scales = f"min {stats['min']:.2e}   median {stats['median']:.2e}   max {stats['max']:.2e}"
    print(f"{spec['label']:<14} {stats['count']} calibrated tensors")
    print(f"{'':<14} scale  {scales}")
    print(f"{'':<14} calibrated on {helpers.CALIBRATION[variant]}")
onnx_path, _ = helpers.artifact_paths("cityscapes")
held = ", ".join(f"{op} x{n}" for op, n in helpers.unquantized_nodes(onnx_path).items())
print(f"\nheld out of int8: {held} -- the attention softmaxes")

Cityscapes-19  597 calibrated tensors
               scale  min 1.67e-04   median 3.12e-02   max 5.12e+01
               calibrated on 12 Cityscapes training images
ADE-150        601 calibrated tensors
               scale  min 1.76e-04   median 3.29e-02   max 6.56e+01
               calibrated on 3 generic photographs, not ADE20K scenes

held out of int8: Softmax x8 -- the attention softmaxes

The per-tensor scheme has to cover a wide dynamic range: the largest scale above is roughly five orders of magnitude above the smallest, which is why a single scale for the whole graph would not work. What int8 costs in accuracy is measured in section 9, against the float graph's own output.

The two variants take different input types, and that difference runs through to the Summary:

Quantizing on the host is worth a quarter of the input traffic for the same labels, and the mapping below reproduces the on-device quantizer exactly rather than approximating it.

sample = np.array([-(1 << 28), -1000, 0, 1000, (1 << 28)], dtype=np.int32)
print("fixed-point in :", sample.tolist())
print("int8 out       :", helpers.quantize_input(sample).tolist())
print(f"bytes per frame: {1 * 3 * 256 * 512:,} int8  (vs {4 * 3 * 256 * 512:,} as 32-bit)")

fixed-point in : [-268435456, -1000, 0, 1000, 268435456]
int8 out       : [-30, 0, 0, 0, 30]
bytes per frame: 393,216 int8  (vs 1,572,864 as 32-bit)

3. Splitting the graph into kernels

The graph is cut at 17 tensor boundaries, and each fragment becomes one kernel. The subgraphs were extracted ahead of time with onnx.utils.extract_model; the boundaries it was given are recorded below.

boundaries = json.loads((HERE / "kernel_boundaries.json").read_text())
print(f"{len(boundaries)} kernel boundaries\n")
for name in ("pc00", "pc01", "pc16"):
    spec = boundaries[name]
    print(f"  {name}: {len(spec['inputs'])} input(s) -> {len(spec['outputs'])} output(s)")
    print(f"        out: {spec['outputs'][0][-46:]}")

17 kernel boundaries

  pc00: 1 input(s) -> 1 output(s)
        out: /backbone/layers.0.0/norm/Add_1_output_0
  pc01: 1 input(s) -> 1 output(s)
        out: yers.0.1.0/attn/Transpose_4_output_0_quantized
  pc16: 5 input(s) -> 1 output(s)
        out: logits

4. The kernels: what the compiler emits, and what was changed

The graph compiler compiles all 17 subgraphs. For each one it emits the CCL kernel source and a packed constant blob, and the blobs ship here exactly as it emitted them. The kernel sources ship edited: the cell below lists every hand-written change in the file, by kernel.

Every edit is to data movement only -- when a tensor is fetched, which memory it lands in, how it is walked -- so each one is bit-exact. Four kernels needed no changes at all and ship exactly as generated.

for variant, spec in helpers.MODELS.items():
    src = (HERE / spec["kernels"]).read_text()
    marked = helpers.edits_by_kernel(src, helpers.KERNEL_EDITS)
    total = len(helpers.KERNELS)
    print(f"{spec['label']}: hand-written changes in {len(marked)} of {total} kernels")
    for kernel, notes in marked.items():
        print(f"    {kernel}  {'; '.join(notes)}")

Cityscapes-19: hand-written changes in 5 of 17 kernels
    pc00  int8 input, no quantize pass; tile flow read in a single walk
    pc05  activation staged for the next kernel
    pc13  next kernel's constants prefetched
    pc15  next kernel's constants prefetched
    pc16  class reduction fused after the upsample; logits kept int8 through the upsample; view lifetime corrected
ADE-150: hand-written changes in 4 of 17 kernels
    pc05  activation staged for the next kernel
    pc13  activation staged for the next kernel
    pc15  activation staged for the next kernel; branches read directly, no fusion buffer
    pc16  branches read directly, no fusion buffer

5. Optimization

Optimization came in two phases, and they are different in kind.

Phase one moves data; it does not change arithmetic. Each transformation below moves a read earlier, changes where a tensor lands, or removes a redundant copy, so every one is bit-exact and the label output is unchanged -- which is what makes them safe to stack.

The next cell gives measured cycles for transformations 4 to 8, each taken one at a time against a composition that already carries 1 to 3. A dash means the transformation does not apply to that variant: the stem row is Cityscapes-only, because the ADE stem is the plain convolution with no surplus walk to remove, and the fusion-buffer row is ADE-only.

Transformation 8 is worth noting for a reason other than its size. Merging entry points removes the transfer drain that happens between them, and the kernels rely on that drain: the fix is to place one explicitly ahead of each kernel. A drain is a fence rather than a cost, and adding all of them back is what makes the single-entry program land slightly ahead of the six-entry one rather than behind it.

Phase two changes arithmetic, so it is gated differently. Each of its eleven rows is measured for cycles and for per-pixel agreement against the host reference, and a row only lands if it holds its accuracy. Five of the eleven leave the label map unchanged to the byte, two replace an approximation with a lookup that is exact over its quantized input domain, and one replaces an iterative square root with an identity that resolves more digits than it did -- so the agreement at the end of the table is higher than at the start on both variants: Cityscapes 97.866% to 98.183%, ADE 98.248% to 98.267%.

Those two pairs were measured during development on dataset validation frames, which the datasets' terms do not allow this repository to redistribute; the frame section 9 ships and scores is a different, licence-clean image, so its agreement is not comparable with them. What the pairs record is the direction each transformation moved accuracy, which is what gated it. The two worth reading about:

• The softmax exponential becomes a table lookup. Its input is an int8 score and the argument is exp((s_i - s_max) * scale), so the integer difference runs over -255..0 and a 256-value table covers every input the kernel can ever see. That makes the lookup exact, not an approximation, and it replaces a ~20-cycle iterative CORDIC with an indexed local-memory read.

• The decode head fuses before it upsamples, not after. It used to lift four backbone stages to 256 channels, upsample all four to 64x128, and only then reduce them to 19 -- so the fusion convolution read 8.4 MB out of on-chip memory and the resizes carried 29,184 pixels of 256 channels. That convolution is a 1x1 and therefore linear, so it commutes with the upsample: each branch is convolved at its own resolution and the running sum climbs with the cascade, for 10,752 upsampled pixels and one branch read at full resolution instead of four. The four per-branch weight matrices are the four column bands of the existing fusion weights -- the same numbers, regrouped, not retrained. conv_seg does not fold in with it, because there is a ReLU in between; it stays exactly where it was.

  The two variants need different numbers here, and the difference is instructive. The quantizer requantizes a different pair of branches in each graph on its way into the concat -- 1 and 2 for Cityscapes, 1 and 3 for ADE -- and ADE's running sum spans a range that saturates int8 at the fusion output scale, where Cityscapes' does not. So ADE folds each branch's requant into its level scale instead of applying it to the branch (W.(r.b) = r.(W.b), which removes an int8 round trip) and carries the accumulator at twice the output scale. Both choices were swept on each variant's own calibration images and scored against the real-valued decode head, never against the evaluation frames. Copying one variant's constants to the other would have cost about half a point.

• The ADE stem stops gathering its 7x7 window one tap at a time. generalLocalDataMovement issues numChIn x kernelY x kernelX = 3 x 7 x 7 = 147 separate tile-loads per output pixel, and they were 322,818 of that region's 405,634 cycles -- with zero external bytes moved, so it is transaction count, not bandwidth. Padding by 4, taking a 4x space-to-depth and then convolving Cin=48 at stride 1 with no padding is the same arithmetic, and stride-1-unpadded is the precondition for the structured-flow path that fetches a receptive field with a handful of flows instead of 147 loads. pc00: 485,073 -> 169,991, and the label map is unchanged to the byte. The Cityscapes stem has been built this way all along; ADE's had not.

• The stage-0 attention's spatial-reduction convolution stops gathering too. Same disease as the stem, one layer in: an 8x8 stride-8 convolution over 32 channels gathered 32 x 8 x 8 = 2,048 tile-loads per output pixel, 119,742 of that region's 146,014 cycles, twice over. The convolution was already the 1x1 form over a 2,048-input LRM array, so nothing about it changes -- only how those 2,048 bytes arrive. A packed read cannot take an 8-wide block, but the cap is on the column count alone, and an 8x8 block is a 4x4 then a 2x2, each inside it. The second pass's read is the convolution's input, so one intermediate tensor covers it. Stall[LOAD] 119,742 -> 43,917 in each of two regions, in both variants, and both label maps are unchanged to the byte.

• LayerNorm takes 1/sigma from exp(-ln(v)/2). The GPNPU has no square-root instruction, so the previous code ran a 16-iteration restoring root and then divided by it per element. It does have single-instruction log and exp, and nothing downstream needs sigma itself. The result is also more accurate: the restoring root resolved sigma to about 1e-3, where the logarithm carries 26 fractional bits.

base = {"cityscapes": 6_933_597, "ade": 7_203_688}
VARIANTS = (("cityscapes", "Cityscapes-19"), ("ade", "ADE-150"))
print(f"{'transformation':<48}{'Cityscapes':>12}{'ADE':>12}")
for name, city, ade in helpers.OPTIMIZATIONS:
    print(f"{name:<48}{f'{city:+,}' if city else '-':>12}{f'{ade:+,}' if ade else '-':>12}")
## The column index doubles as the position of that variant's delta in each OPTIMIZATIONS row.
for column_index, (key, label) in enumerate(VARIANTS, start=1):
    final = helpers.MODELS[key]["cycles_composed"]
    delta = final - base[key]
    column = sum(row[column_index] for row in helpers.OPTIMIZATIONS)
    share = 100 * delta / base[key]
    print(f"\n  {label:<15}{base[key]:>10,} -> {final:,}   {delta:+,} ({share:+.2f}%)")
    if column != delta:
        print(f"  {'':<15}column sums to {column:+,}: each row was measured on its own, so they")
        print(f"  {'':<15}overlap slightly rather than composing exactly")

transformation                                    Cityscapes         ADE
Stem kernel reads its tile flow in a single walk    -213,238           -
Stage-1 activation staged ahead of its consumer      -33,110     -31,379
Stage-2 activation staged ahead of its consumer      -15,612     -15,553
Decode-head weights prefetched                       -10,492     -11,157
Stage-4 weights prefetched                           -10,238     -10,996
Decode-head branches read in place                         -    -450,596
Composed behind a single entry point                  -4,656      -4,425

  Cityscapes-19   6,933,597 -> 6,553,649   -379,948 (-5.48%)
                 column sums to -287,346: each row was measured on its own, so they
                 overlap slightly rather than composing exactly

  ADE-150         7,203,688 -> 6,586,818   -616,870 (-8.56%)
                 column sums to -524,106: each row was measured on its own, so they
                 overlap slightly rather than composing exactly

Phase two: the kernel-arithmetic transformations

Same format, but closing against the figure the notebook measured above. Each row was taken on its own against the row before it, and each was gated on agreement as well as cycles.

base2 = {key: helpers.MODELS[key]["cycles_composed"] for key, _ in VARIANTS}
print(f"{'transformation':<58}{'Cityscapes':>12}{'ADE':>12}")
for name, city, ade in helpers.KERNEL_OPTIMIZATIONS:
    print(f"{name:<58}{f'{city:+,}' if city else '-':>12}{f'{ade:+,}' if ade else '-':>12}")
for column_index, (key, label) in enumerate(VARIANTS, start=1):
    final = helpers.MODELS[key]["cycles_per_frame"]
    delta = final - base2[key]
    column = sum(row[column_index] for row in helpers.KERNEL_OPTIMIZATIONS)
    share = 100 * delta / base2[key]
    print(f"\n  {label:<15}{base2[key]:>10,} -> {final:,}   {delta:+,} ({share:+.2f}%)")
    if column != delta:
        print(f"  {'':<15}column sums to {column:+,}: each row was measured on its own, so they")
        print(f"  {'':<15}overlap slightly rather than composing exactly")

transformation                                              Cityscapes         ADE
4x bilinear resize split into two 2x passes                   -324,341    -250,253
Stage-0 LayerNorm output shared in OCM, prefetch dropped       -34,708           -
Softmax exp -> table lookup, multi-MROI attention             -116,608    -118,509
Softmax exp -> table lookup, single-MROI attention            -124,801    -124,703
LayerNorm 1/sigma from exp(-ln(v)/2), no square root          -446,096    -447,045
LayerNorm mean in int32, one less 64-bit cross-PE sum         -250,020    -248,837
Stem space-to-depth as a packed flow, not 384 RAU stores      -122,922           -
Decode head: fuse before upsampling, not after                -238,573    -245,866
Stem 7x7/4 gather -> space-to-depth + stride-1 conv                  -    -314,472
Stage-0 SR conv 8x8/8 gather -> two packed space-to-depth passes    -133,331    -133,250
Stage-0 LayerNorm output shared in OCM (ADE)                         -     -30,629

  Cityscapes-19   6,553,649 -> 4,762,249   -1,791,400 (-27.33%)

  ADE-150         6,586,818 -> 4,673,254   -1,913,564 (-29.05%)

6. Where the kernels and constants come from

Both halves of what section 4 describes come from the graph compiler. The 17 subgraphs at the boundaries listed in section 3 were extracted and compiled ahead of time; for each one the compiler emitted a CCL kernel source file and a packed constant blob, with the whole tensor-range file passed every time and the compiler selecting the ranges it needs.

The constant blobs ship here exactly as it emitted them, concatenated into one blob per model. That was checked by recompiling a kernel and comparing bytes against the committed blob: identical. The kernels and constants ship as the compiler emitted them, so the compile is not repeated here.

The kernel sources are the starting point for the hand-written changes in section 4, so the build below starts from those edited sources rather than from a fresh compile.

7. Compose into one program

The 17 kernels are composed into a single program with one entry point, so the host issues one call per frame. Composition concatenates the kernels' constant blobs into one and pairs them with the kernel sources; nothing is rewritten, the blobs are used exactly as the compiler emitted them, and the byte counts are asserted against each kernel's own signature.

Putting every kernel behind one entry point is constrained by local memory rather than by anything about the graph. Each entry point gets a 4 kB local-memory budget, and the kernels that allocate inside it must be callable rather than forced inline for the merged entry to fit. Two attention kernels hold their allocation, so they are split at a point where nothing is live and only the part ahead of the attention is converted. That is enough: the merged entry lands inside the budget without a single attention-adjacent block being touched.

The result is bit-identical to the same kernels composed behind six entry points, which is the control that makes the change safe to keep.

entries = len(helpers.KERNEL_GROUPS)
for index, group in enumerate(helpers.KERNEL_GROUPS):
    print(f"  entry {index}: {len(group)} kernels  {group[0]} ... {group[-1]}")
print(f"\n  {len(helpers.KERNELS)} kernels -> {entries} entry point, one program")
print("  one runKernel call per frame; intermediates stay in device memory")
print(f"  constants: one blob of {(HERE / 'constants/cityscapes/c_all.bin').stat().st_size:,} B")

  entry 0: 17 kernels  pc00 ... pc16

  17 kernels -> 1 entry point, one program
  one runKernel call per frame; intermediates stay in device memory
  constants: one blob of 4,430,384 B

8. Build and run

The composed program is built and simulated. One frame is enough: cycle counts and label output are both per-frame quantities.

results = {}
for variant, spec in helpers.MODELS.items():
    fixed_input, reference = helpers.load_evaluation_frame(variant, HERE)
    build = WORK / variant / spec["module"] / "build"
    build.mkdir(parents=True, exist_ok=True)
    written = helpers.write_fixed_point_input(variant, fixed_input, build / "x.bin")
    cycles, labels, profile, entries = helpers.stage_and_run(variant, HERE, WORK, frames=1)
    results[variant] = {"cycles": cycles, "labels": labels, "profile": profile, "entries": entries}
    results[variant].update(reference=reference, rgb=helpers.preview_rgb(fixed_input))
    digest = hashlib.sha256(labels.tobytes()).hexdigest()[:16]
    reference_run = digest == spec["reference_digest"]
    stats = f"{written:>10,} B in ({spec['input_dtype']:<7}){cycles:>12,} cycles/frame"
    # The digest identifies this frame's label map exactly, which makes a re-run easy to
    # compare against. It is a property of the frame AND of the toolchain that produced
    # it: a release that rounds differently gives a different digest for the same input,
    # so it is reported rather than required to match. The agreement figure below is what
    # says whether the output is right.
    seen = (
        "matches the reference run"
        if reference_run
        else f"differs from the reference run ({spec['reference_digest']})"
    )
    print(f"{spec['label']:<16}{stats}   {digest}  {seen}")

  toolchain: attention helpers taken from segformer_attention_helpers.hpp
Cityscapes-19      393,216 B in (int8   )   4,725,194 cycles/frame   3eb26c979ddd60dc  matches the reference run
  toolchain: attention helpers taken from segformer_attention_helpers.hpp
ADE-150          1,572,864 B in (fixed32)   4,633,283 cycles/frame   17f788c1d123045c  matches the reference run

9. Results

Device labels are compared against the fp32 ONNX graph's own output, computed with ONNX Runtime and shipped beside this notebook. Agreement is reported per variant: Cityscapes-19 resolves 19 classes and ADE-150 resolves 150, so a 150-way decision over the same feature map is the harder one. The two decode heads also place their class reduction differently, which is visible at object boundaries: the Cityscapes head upsamples first and reduces on the finer grid, while the ADE head reduces first and replicates the labels.

Each variant ships one evaluation frame, built from a U.S. Government Work photograph rather than a dataset validation image -- neither Cityscapes' nor ADE20K's terms allow redistributing one. Both are generic scenes rather than the street and indoor views these heads were trained on. evaluation/PROVENANCE.json records the source image, the graph, the preprocessing and the ONNX Runtime settings for each, and evaluation/make_evaluation_frame.py rebuilds both from those records.

The next cell checks both results rather than only reporting them. Each variant must agree with the ONNX Runtime reference on at least 90% of pixels, and the composed program must have exactly one entry point. The floor is segformer_landscape_helpers.AGREEMENT_GATE, and it is a floor: it is set where a broken build fails and a licence-clean frame does not, rather than at the measurement itself, which the cell prints.

PALETTES = {"cityscapes": helpers.CITYSCAPES_PALETTE, "ade": helpers.ade_palette()}

for variant, spec in helpers.MODELS.items():
    entry = results[variant]
    agreement = helpers.compare_segmentations(
        entry["rgb"], entry["reference"], entry["labels"], PALETTES[variant], spec["label"]
    )
    entry["agreement"] = agreement
    covered = helpers.top_classes(entry["labels"], helpers.CLASS_NAMES[variant])
    print(f"{spec['label']}: {agreement:.3f}% of pixels agree with the reference")
    print(f"    frame covered by: {covered}")

## The gate, and the structural claim, checked rather than described. GATE is a floor on a
## measurement that moves by a fraction of a percent between toolchain releases, so it sits
## well below the figures above; a break is not subtle, since kernels wired up wrongly agree
## in the tens of percent. The entry-point count is a property of how the program is composed
## and holds whatever the toolchain does, so it is checked exactly.
for variant, spec in helpers.MODELS.items():
    entry = results[variant]
    assert entry["agreement"] >= helpers.AGREEMENT_GATE[variant], (
        f"{spec['label']}: {entry['agreement']:.3f}% agreement, "
        f"below the {helpers.AGREEMENT_GATE[variant]}% gate"
    )
    assert entry["entries"] == 1, f"{spec['label']}: {entry['entries']} entry points, expected 1"
print("\nPASS: both variants cleared their agreement gate, each behind one entry point")

Cityscapes-19: 94.784% of pixels agree with the reference
    frame covered by: person 25%, rider 17%, vegetation 16%, bicycle 16%, sky 14%

ADE-150: 93.815% of pixels agree with the reference
    frame covered by: tree 36%, road 29%, sky 20%, signboard 10%, sidewalk 1%

PASS: both variants cleared their agreement gate, each behind one entry point

10. Performance profile

Cycle counts come from the simulator's own region profile, for the target named at the top: QC-U at 1.7 GHz -- a 32x32 PE array, 16 MACs per PE, 16 MB OCM, 4 kB LRM, 128 GB/s each way.

print(f"  target {helpers.TARGET}")
## All seventeen kernels sit behind one entry point, so the host makes a single call per
## frame. That is a property of how the program is composed, so it is stated exactly;
## the timings below are a measurement on this toolchain and this device configuration.
for variant, spec in helpers.MODELS.items():
    print(f"  {spec['label']:<16}{results[variant]['entries']} entry point(s)")
print()
print(f"{'variant':<16}{'cycles/frame':>14}{'FPS':>9}{'ms':>8}")
for variant, spec in helpers.MODELS.items():
    cycles = results[variant]["cycles"]
    print(
        f"{spec['label']:<16}{cycles:>14,}"
        f"{helpers.CLOCK_HZ / cycles:>9.1f}{1e3 * cycles / helpers.CLOCK_HZ:>8.2f}"
    )

categories = helpers.cycle_categories(results["cityscapes"]["profile"])
helpers.plot_cycle_categories(
    categories,
    results["cityscapes"]["cycles"],
    f"Cityscapes-19 execution cycles by category  ({helpers.TARGET})",
)

  target QC-U, 1.7 GHz, 32x32 PEs, 16 MACs/PE, 16 MB OCM, 4 kB LRM, 128 GB/s each way
  Cityscapes-19   1 entry point(s)
  ADE-150         1 entry point(s)

variant           cycles/frame      FPS      ms
Cityscapes-19        4,725,194    359.8    2.78
ADE-150              4,633,283    366.9    2.73

Summary

Measured performance

Agreement is measured on the frame each variant ships, a redistributable photograph rather than a dataset validation image; section 9 says what it is being compared against. The cycle counts are independent of the frame.

Measured on a development build after SDK 26.08, QC-U at 1.7 GHz with 16 MB of on-chip memory, at 256x512. SDK 26.08 is the minimum: the demo carries its own copy of the attention head, built against the 26.08 tensor-slice API. A cycle count moves a little between releases -- the same quantized arithmetic, rounded differently -- so the figures the cells above print are the ones that describe your toolchain, and they are the ones to quote.

For reference, the same sources before the phase-two kernel work measured 6,553,649 and 6,586,818 on this toolchain: 27.3% and 29.1% faster now. Accuracy went up rather than down over that work -- 97.866% to 98.183% on Cityscapes and 98.248% to 98.267% on ADE -- though that pair was measured during development on dataset validation frames this repository cannot redistribute, so it is not comparable with the agreement column above.

The composition is what this demo fixes -- all seventeen kernels behind a single entry point, one host call per frame -- and it is unchanged by any of the optimization work. The notebook prints the entry-point count each run so the property is visible rather than asserted in prose.

Key takeaways

1. Both variants run end to end on the Chimera GPNPU at over 350 frames per second on QC-U at 1.7 GHz, at 256x512 with a full-resolution label map and a single program launch per frame, agreeing with the float reference on roughly 94% of the pixels of the frames they ship.
2. Hand-writing the kernels is what buys the layout: a space-to-depth stem, tile-block attention flows, and a decode head that reduces the class dimension before its output leaves the chip, so only labels are written out.
3. The two optimization phases are gated differently, and both are shown. The eight composition transformations are bit-exact -- they move data and change nothing about the arithmetic. Of the eleven kernel transformations, most do change arithmetic, so each is measured for agreement as well as cycles; five leave the label map unchanged to the byte and two are exact over their quantized input domain rather than approximate, and accuracy did not drop on either variant over that work.

Citation

@inproceedings{xie2021segformer,
  title     = {SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers},
  author    = {Xie, Enze and Wang, Wenhai and Yu, Zhiding and Anandkumar, Anima and
               Alvarez, Jose M and Luo, Ping},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2021}
}