Unlock this feature

This feature isn’t part of your plan yet

Contact sales to get upgraded to the full DevStudio experience.

Unlock this feature

This feature isn't part of your plan yet.

Model Demo: Qwen3-8B Decode on Chimera GPNPU


NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:

$ quadric sdk notebook

From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/qwen/qwen3_8b/qwen3_8b.ipynb.


Qwen3-8B Decode on Chimera GPNPU

This notebook compiles Qwen3-8B with CGC (Chimera Graph Compiler) and runs the autoregressive decode loop on the ISS (Instruction Set Simulator) in a 4-core QC-P configuration: one token in, one token out, the KV cache carried on-chip between steps. Roughly 18–19 tokens/s average (20–21 peak) at 1 GHz.

Its companion, qwen3_8b_prefill.ipynb, covers the other half — prefill, the whole 1024-token prompt through all 36 layers in one pass, which fills the KV cache this loop then consumes. Both are cut from the same W4A8 SmoothQuant export and share its tensor ranges and custom operators; only the shape-fixing step and the host runner differ. They are kept as two notebooks so that each phase runs in its own kernel.

Pipeline

Qwen3-8B Pipeline

Model: Qwen3-8B (8.2B parameters, 36 decoder layers, hidden size 4096, 32 query / 8 KV heads, 151,936-token vocabulary), W4A8 SmoothQuant (α = 0.36), 1024-token context

PhaseGraph shapeOn the ISSWhere
Decode1 token in, 1023 cachedThe full autoregressive loop on 4× QC-P, checked against ONNX Runtimesections 3–9 below
Prefill1024 tokens in, empty cacheOne decoder layer validated against ONNX Runtime; all 36 compiledqwen3_8b_prefill.ipynb

1. Setup

Imports, the model's architecture constants and the hardware target. The QC-P configuration below is shared by every compile and ISS run in this notebook: 4 cores, 2 MB OCM (On-Chip Memory) per core, 16 MACs per PE, 1 GHz, 24 GB/s of DDR bandwidth per core over a 256-bit AXI interface. CGC needs a deep stack to schedule an 8-billion-parameter graph, so the limit is raised once here.

import re
from pathlib import Path

import numpy as np
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from tvm.contrib.epu.chimera_job.hw_config import HWConfig

from examples.models.qwen.qwen_helpers import (
    compare_decode_with_ort,
    kv_cache_io_names,
    plot_decode_profile,
    raise_stack_limit,
    report_decode_throughput,
    stage_const_tensor,
)
from custom_op_match import qwen3_custom_op_replacer
from fix_shapes import fix_shapes
from qwen3_8b_helpers import download_decode_model

%matplotlib inline
NUM_DECODERS = 36
EMBED_DIM = 4096
NUM_KV_HEADS = 8  # 32 query heads share 8 key/value heads (grouped-query attention)
SEQ_LEN = 1024  # context window: prompt + generated tokens
NEW_TOKENS = 1

HW_CONFIG = HWConfig(product="QC-P", ocm_size="2MB", macs_per_pe=16)

raise_stack_limit()
Stack limit raised to 32 MB for CGC compilation

2. Download the W4A8 Checkpoint

Qwen3-8B was quantized offline to W4A8 — 4-bit per-channel weights, 8-bit activations — with SmoothQuant (α = 0.36) migrating activation outliers into the weights before quantization. The export also lifts the RoPE sin/cos tables out as graph inputs, so the runner supplies the rotary angle for each position, and carries an attention mask shaped for both the prefill and the decode stage. Three files make up the decode graph:

FileContents
qwen_0_36.onnxThe quantized graph with dynamic batch and sequence dimensions
9b9822ba-….dataExternal weight data (~6 GB)
qwen_0_36.onnx.trangesPer-tensor value ranges that parameterize the fixed-point custom ops
decode_onnx, _decode_weights, tranges_path = map(str, download_decode_model())
Downloading qwen_0_36.onnx...
  Downloaded qwen_0_36.onnx (0.02 GB)
Downloading 9b9822ba-a73f-11f0-9995-10ffe060dbe7.data...
  Downloaded 9b9822ba-a73f-11f0-9995-10ffe060dbe7.data (6.27 GB)
Downloading qwen_0_36.onnx.tranges...
  Downloaded qwen_0_36.onnx.tranges (0.00 GB)

3. Fix Shapes for Autoregressive Decode

CGC compiles static graphs. The downloaded model still has symbolic batch_size, sequence_length and past_sequence_length dimensions, so fix_shapes pins them for the decode step — one new token attending to a 1023-entry KV cache — and runs ONNX Runtime's quantization pre-processing (shape inference and constant folding) so every tensor has a known shape before custom-op matching. The same helper produces the prefill layout with mode="prefill" (1024 tokens in, empty cache); section 10 downloads that graph pre-built.

Runtime: ~1 minute

decode_seq_onnx = "qwen3_seq1024.onnx"
fix_shapes(decode_onnx, decode_seq_onnx, seq_len=SEQ_LEN, mode="autoregressive")
Loading model from qwen_0_36.onnx...
Fixing shapes for autoregressive mode: seq_len=1024
Saving intermediate fixed model to fixed.onnx...
Running quantization pre-processing...
Shape fixing complete! Output saved to qwen3_seq1024.onnx

4. Replace Attention, Projections and the LM Head with Custom Ops

CGC compiles most of the graph on its own. Three patterns are swapped for CCL (Chimera Compute Language) kernels from the SDK's neural-network block library, because a fused kernel keeps the whole operation inside OCM and reads each weight exactly once per token:

ONNX subgraphCustom opWhat it buys
Q/K/V projections → QK-RMSNorm → RoPE → grouped-query attention over the KV cachenn::qwen3AttentionOne kernel per layer appends to the cache and computes attention in OCM
Gate and up projections + SiLUnn::gateProjThe SwiGLU pair in a single pass over the hidden state
Every INT4-weight MatMul, including the 4096 × 151,936 LM headlinalg::channelwiseQuantMatMulINT8 activations against INT4 weights packed eight per 32-bit word (v8i4), per-channel scales

qwen3_custom_op_replacer pattern-matches these subgraphs across all 36 layers, turns the tensor ranges into the fixed-point fractional bits each kernel expects as template arguments, and packs the INT4 weights — the packing is what takes the time.

Runtime: ~30–45 minutes

decode_custom_onnx = "qwen3_custom_ops_seq1024.onnx"
qwen3_custom_op_replacer(
    decode_seq_onnx,
    decode_custom_onnx,
    tranges_path,
    num_heads=NUM_KV_HEADS,
    embed_dim=EMBED_DIM,
    seq_length=SEQ_LEN,
    num_decoders=NUM_DECODERS,
)
Successfully saved modified model to qwen3_custom_ops_seq1024.onnx

The weight packing is the one piece worth seeing in full. Eight signed 4-bit values are stacked along the input dimension into one int32, so a [K, N] weight becomes [K/8, N] words that the kernel unpacks on the fly — the same layout the v8i4 vector type uses on the GPNPU:

source = Path("custom_op_match.py").read_text()
print(re.search(r"def pack_row_v8i4.*?\n    return output\n", source, re.DOTALL).group(0))
def pack_row_v8i4(input_array):
    """Pack 8 rows of 4-bit values into 32-bit integers.

    Parameters
    ----------
    input_array : numpy.ndarray
        The input 2D array with shape (height, width) to pack.

    Returns
    -------
    numpy.ndarray
        The packed array with shape (ceil(height/8), width).
    """
    pack_factor = 8
    height, width = input_array.shape
    output_height = math.ceil(height / pack_factor)
    shift_granularity = 32 // pack_factor
    mask = (1 << shift_granularity) - 1

    # Pad input if height is not divisible by pack_factor
    if height % pack_factor != 0:
        padding = pack_factor - (height % pack_factor)
        input_array = np.pad(
            input_array, ((0, padding), (0, 0)), mode="constant", constant_values=0
        )

    # Reshape to group rows by pack_factor
    reshaped = input_array.reshape(output_height, pack_factor, width)

    # Convert to int32 and mask to get unsigned values
    unsigned_vals = reshaped.astype(np.int32) & mask

    # Create shift amounts for each position in the pack
    shifts = np.arange(pack_factor) * shift_granularity
    shifts = shifts.reshape(pack_factor, 1)

    # Apply shifts to each group
    shifted = unsigned_vals * (1 << shifts)

    # Combine all values in each group using bitwise OR (sum works since non-overlapping bits)
    output = shifted.sum(axis=1, dtype=np.int32)

    return output

5. Compile the Decode Graph with CGC

CGC turns the custom-op graph into C++ for the QC-P target: it schedules the 36 layers, tiles every tensor to fit the 2 MB OCM, and emits the kernel plus a const_tensor_data.bin holding the packed weights. The KV cache tensors and the attention mask are listed in io_to_ignore — the autoregressive runner in section 7 owns them across token steps, so CGC leaves them out of the kernel's I/O.

Memory: ~55 GB peak · Runtime: ~25 minutes

decode_job = ChimeraJob(
    decode_custom_onnx,
    hw_config=HW_CONFIG,
    trange_file=tranges_path,
    target_lang="QIL",  # Quadric Intermediate Language
    io_to_ignore=kv_cache_io_names(NUM_DECODERS),
)
decode_job.compile()
print(decode_job)
2026-10-23 01:03 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-10-23 01:03 - INFO - epu - chimera_job - Numerical ranges provided
2026-10-23 01:04 - INFO - epu - util - START===============================optimize_relay
2026-10-23 01:04 - INFO - epu - util - START====================quantize_to_cpu_runnable_fx
2026-10-23 01:04 - INFO - epu - fx - 

Source name                                          Op                             Output 0 Range             Output 0 Frac Bits
---------------------------------------------------  -----------------------------  -----------------------  --------------------
/model/embed_tokens/Gather                           contrib.epu.embedding          [-0.628906f, 0.8125f]                      31
/model/layers.0/input_layernorm/Mul_1                contrib.epu.rms_norm           [-0.359118f, 0.37561f]                     31
/model/layers.0/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-0.287793f, 0.307279f]                    31
/model/layers.0/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-0.247679f, 0.223768f]                    31
/model/layers.0/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.100749f, 0.107016f]                    31
CustomOp/linalg::channelwiseQuantMatMul<43>3         contrib.epu.quadric_custom_op  [-2.25614f, 3.85482f]                      29
/model/layers.0/Add                                  add                            [-2.30253f, 4.25466f]                      28
/model/layers.0/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-1.23024f, 1.59211f]                      29
/model/layers.0/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-0.682836f, 0.548619f]                    28
CustomOp/nn::gateProj<41, 29>1                       contrib.epu.quadric_custom_op  [-0.278465f, 3.85478f]                     29
/model/layers.0/mlp/up_proj/MatMul_smooth_mul        multiply                       [-0.421256f, 0.474667f]                    29
CustomOp/linalg::channelwiseQuantMatMul<42>0         contrib.epu.quadric_custom_op  [-3.34433f, 3.38188f]                      29
/model/layers.0/mlp/Mul                              multiply                       [-11.3828f, 9.95375f]                      27
/model/layers.0/mlp/down_proj/MatMul_smooth_mul      multiply                       [-2.19349f, 2.29969f]                      26
CustomOp/linalg::channelwiseQuantMatMul<38>2         contrib.epu.quadric_custom_op  [-5.18762f, 16.7897f]                      26
/model/layers.0/Add_1                                add                            [-6.67269f, 19.2419f]                      26
/model/layers.1/input_layernorm/Mul_1                contrib.epu.rms_norm           [-1.30106f, 1.14032f]                      30
/model/layers.1/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-0.549389f, 0.618498f]                    28
/model/layers.1/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-0.547089f, 0.290293f]                    28
/model/layers.1/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.18635f, 0.285787f]                     28
CustomOp/linalg::channelwiseQuantMatMul<43>7         contrib.epu.quadric_custom_op  [-1.41003f, 1.53298f]                      30
/model/layers.1/Add                                  add                            [-6.78994f, 19.2634f]                      26
/model/layers.1/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-51.933f, 47.6116f]                       25
/model/layers.1/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-3.17403f, 3.52375f]                      22
CustomOp/nn::gateProj<38, 27>5                       contrib.epu.quadric_custom_op  [-0.278465f, 7.77009f]                     28
/model/layers.1/mlp/up_proj/MatMul_smooth_mul        multiply                       [-2.03129f, 2.22485f]                      22
CustomOp/linalg::channelwiseQuantMatMul<39>4         contrib.epu.quadric_custom_op  [-13.6112f, 16.4607f]                      26
/model/layers.1/mlp/Mul                              multiply                       [-22.4947f, 19.6069f]                      23
/model/layers.1/mlp/down_proj/MatMul_smooth_mul      multiply                       [-3.95853f, 6.61841f]                      24
CustomOp/linalg::channelwiseQuantMatMul<37>6         contrib.epu.quadric_custom_op  [-12.4679f, 38.1379f]                      25
/model/layers.1/Add_1                                add                            [-18.1108f, 57.4013f]                      25
/model/layers.2/input_layernorm/Mul_1                contrib.epu.rms_norm           [-0.862898f, 1.28555f]                     29
/model/layers.2/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-0.369643f, 0.667969f]                    28
/model/layers.2/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-0.560969f, 0.593967f]                    28
/model/layers.2/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.209467f, 0.313265f]                    28
CustomOp/linalg::channelwiseQuantMatMul<42>11        contrib.epu.quadric_custom_op  [-1.03252f, 1.31911f]                      30
/model/layers.2/Add                                  add                            [-18.1085f, 57.4384f]                      25
/model/layers.2/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-82.5232f, 81.3135f]                      24
/model/layers.2/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-12.8995f, 14.1477f]                      25
CustomOp/nn::gateProj<35, 24>9                       contrib.epu.quadric_custom_op  [-0.278465f, 7.80494f]                     28
/model/layers.2/mlp/up_proj/MatMul_smooth_mul        multiply                       [-12.5991f, 14.1477f]                      25
CustomOp/linalg::channelwiseQuantMatMul<35>8         contrib.epu.quadric_custom_op  [-122.194f, 103.963f]                      24
/model/layers.2/mlp/Mul                              multiply                       [-26.306f, 24.5606f]                       21
/model/layers.2/mlp/down_proj/MatMul_smooth_mul      multiply                       [-5.16373f, 2.29959f]                      22
CustomOp/linalg::channelwiseQuantMatMul<37>10        contrib.epu.quadric_custom_op  [-10.8904f, 26.5761f]                      26
/model/layers.2/Add_1                                add                            [-23.2706f, 81.9635f]                      24
/model/layers.3/input_layernorm/Mul_1                contrib.epu.rms_norm           [-1.24747f, 3.00854f]                      28
/model/layers.3/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-0.43238f, 0.730476f]                     27
/model/layers.3/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-0.371823f, 1.19244f]                     27
/model/layers.3/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.246321f, 0.362407f]                    27
CustomOp/linalg::channelwiseQuantMatMul<42>15        contrib.epu.quadric_custom_op  [-1.84038f, 1.97427f]                      30
/model/layers.3/Add                                  add                            [-23.1594f, 81.7177f]                      24
/model/layers.3/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-53.3995f, 62.6125f]                      25
/model/layers.3/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-3.54248f, 4.5413f]                       22
CustomOp/nn::gateProj<37, 26>13                      contrib.epu.quadric_custom_op  [-0.278465f, 8.11204f]                     27
/model/layers.3/mlp/up_proj/MatMul_smooth_mul        multiply                       [-2.58039f, 4.31592f]                      22
CustomOp/linalg::channelwiseQuantMatMul<37>12        contrib.epu.quadric_custom_op  [-17.4614f, 10.6391f]                      26
/model/layers.3/mlp/Mul                              multiply                       [-23.0629f, 23.7062f]                      23
/model/layers.3/mlp/down_proj/MatMul_smooth_mul      multiply                       [-3.35231f, 3.48879f]                      25
CustomOp/linalg::channelwiseQuantMatMul<39>14        contrib.epu.quadric_custom_op  [-7.28584f, 7.4731f]                       28
/model/layers.3/Add_1                                add                            [-23.3718f, 85.3067f]                      24
/model/layers.4/input_layernorm/Mul_1                contrib.epu.rms_norm           [-1.80589f, 2.00268f]                      29
/model/layers.4/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-0.898383f, 0.659137f]                    28
/model/layers.4/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-0.707868f, 1.08651f]                     28
/model/layers.4/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.439939f, 0.234896f]                    28
CustomOp/linalg::channelwiseQuantMatMul<42>19        contrib.epu.quadric_custom_op  [-3.53964f, 2.62418f]                      29
/model/layers.4/Add                                  add                            [-23.2038f, 85.5677f]                      24
/model/layers.4/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-32.953f, 41.0647f]                       25
/model/layers.4/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-3.60791f, 4.06511f]                      25
CustomOp/nn::gateProj<37, 27>17                      contrib.epu.quadric_custom_op  [-0.278465f, 9.32974f]                     27
/model/layers.4/mlp/up_proj/MatMul_smooth_mul        multiply                       [-2.50043f, 4.23358f]                      25
CustomOp/linalg::channelwiseQuantMatMul<37>16        contrib.epu.quadric_custom_op  [-8.54877f, 10.1004f]                      27
/model/layers.4/mlp/Mul                              multiply                       [-20.2387f, 20.039f]                       24
/model/layers.4/mlp/down_proj/MatMul_smooth_mul      multiply                       [-2.69153f, 2.8874f]                       26
CustomOp/linalg::channelwiseQuantMatMul<39>18        contrib.epu.quadric_custom_op  [-6.69227f, 12.1684f]                      27
/model/layers.4/Add_1                                add                            [-22.4392f, 87.0669f]                      24
/model/layers.5/input_layernorm/Mul_1                contrib.epu.rms_norm           [-2.14681f, 3.19879f]                      28
/model/layers.5/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-1.45607f, 0.922822f]                     28
/model/layers.5/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-1.13808f, 1.1609f]                       28
/model/layers.5/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.502765f, 0.449179f]                    28
CustomOp/linalg::channelwiseQuantMatMul<42>23        contrib.epu.quadric_custom_op  [-6.97809f, 4.76181f]                      28
/model/layers.5/Add                                  add                            [-21.9713f, 84.54f]                        24
/model/layers.5/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-24.6416f, 19.9332f]                      26
/model/layers.5/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-3.30293f, 4.25595f]                      25
CustomOp/nn::gateProj<37, 27>21                      contrib.epu.quadric_custom_op  [-0.278465f, 7.52237f]                     28
/model/layers.5/mlp/up_proj/MatMul_smooth_mul        multiply                       [-1.51029f, 2.94375f]                      25
CustomOp/linalg::channelwiseQuantMatMul<38>20        contrib.epu.quadric_custom_op  [-6.38602f, 7.07606f]                      28
/model/layers.5/mlp/Mul                              multiply                       [-18.2442f, 18.1234f]                      25
/model/layers.5/mlp/down_proj/MatMul_smooth_mul      multiply                       [-3.21112f, 2.15665f]                      26
CustomOp/linalg::channelwiseQuantMatMul<38>22        contrib.epu.quadric_custom_op  [-37.2974f, 18.5171f]                      25
/model/layers.5/Add_1                                add                            [-18.6574f, 63.1892f]                      24
/model/layers.6/input_layernorm/Mul_1                contrib.epu.rms_norm           [-1.99346f, 7.23117f]                      27
/model/layers.6/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-0.737696f, 4.47784f]                     27
/model/layers.6/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-0.58721f, 1.42655f]                      27
/model/layers.6/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.422668f, 1.02092f]                     27
CustomOp/linalg::channelwiseQuantMatMul<40>27        contrib.epu.quadric_custom_op  [-31.7815f, 8.99221f]                      26
/model/layers.6/Add                                  add                            [-12.6824f, 38.3116f]                      24
/model/layers.6/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-9.84406f, 199.177f]                      23
/model/layers.6/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-0.969524f, 14.0399f]                     22
CustomOp/nn::gateProj<34, 24>25                      contrib.epu.quadric_custom_op  [-0.278465f, 65.6915f]                     24
/model/layers.6/mlp/up_proj/MatMul_smooth_mul        multiply                       [-0.875245f, 14.5852f]                     22
CustomOp/linalg::channelwiseQuantMatMul<34>24        contrib.epu.quadric_custom_op  [-41.8197f, 74.1344f]                      24
/model/layers.6/mlp/Mul                              multiply                       [-1282.21f, 4841.01f]                      18
/model/layers.6/mlp/down_proj/MatMul_smooth_mul      multiply                       [-112.681f, 254.839f]                      18
CustomOp/linalg::channelwiseQuantMatMul<27>26        contrib.epu.quadric_custom_op  [-2118.69f, 9638.07f]                      17
/model/layers.6/Add_1                                add                            [-2121.44f, 9654.1f]                       17
/model/layers.7/input_layernorm/Mul_1                contrib.epu.rms_norm           [-5.1855f, 10.441f]                        27
/model/layers.7/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-1.18851f, 2.07018f]                      27
/model/layers.7/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-1.36347f, 1.4916f]                       26
/model/layers.7/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.655511f, 1.0774f]                      27
CustomOp/linalg::channelwiseQuantMatMul<41>31        contrib.epu.quadric_custom_op  [-4.27446f, 3.6406f]                       28
/model/layers.7/Add                                  add                            [-2121.23f, 9654.21f]                      17
/model/layers.7/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-6.89828f, 54.4274f]                      25
/model/layers.7/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-1.00739f, 4.84292f]                      24
CustomOp/nn::gateProj<38, 27>29                      contrib.epu.quadric_custom_op  [-0.278465f, 11.2674f]                     27
/model/layers.7/mlp/up_proj/MatMul_smooth_mul        multiply                       [-1.13376f, 1.88627f]                      24
CustomOp/linalg::channelwiseQuantMatMul<40>28        contrib.epu.quadric_custom_op  [-13.397f, 10.9477f]                       27
/model/layers.7/mlp/Mul                              multiply                       [-16.9021f, 17.2346f]                      23
/model/layers.7/mlp/down_proj/MatMul_smooth_mul      multiply                       [-2.6115f, 2.85751f]                       26
CustomOp/linalg::channelwiseQuantMatMul<37>30        contrib.epu.quadric_custom_op  [-6.76926f, 12.1386f]                      27
/model/layers.7/Add_1                                add                            [-2121.18f, 9654.34f]                      17
/model/layers.8/input_layernorm/Mul_1                contrib.epu.rms_norm           [-4.59068f, 9.9222f]                       27
/model/layers.8/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-1.28958f, 1.87046f]                      27
/model/layers.8/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-0.814643f, 1.79062f]                     27
/model/layers.8/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.422026f, 0.896034f]                    27
CustomOp/linalg::channelwiseQuantMatMul<41>35        contrib.epu.quadric_custom_op  [-3.77026f, 7.3769f]                       28
/model/layers.8/Add                                  add                            [-2120.87f, 9653.89f]                      17
/model/layers.8/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-7.8127f, 6.98587f]                       28
/model/layers.8/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-1.26424f, 1.58606f]                      27
CustomOp/nn::gateProj<39, 27>33                      contrib.epu.quadric_custom_op  [-0.278465f, 6.3098f]                      28
/model/layers.8/mlp/up_proj/MatMul_smooth_mul        multiply                       [-1.4333f, 1.11036f]                       27
CustomOp/linalg::channelwiseQuantMatMul<40>32        contrib.epu.quadric_custom_op  [-6.76552f, 5.97511f]                      28
/model/layers.8/mlp/Mul                              multiply                       [-15.8356f, 18.0968f]                      25
/model/layers.8/mlp/down_proj/MatMul_smooth_mul      multiply                       [-2.78291f, 3.87402f]                      27
CustomOp/linalg::channelwiseQuantMatMul<38>34        contrib.epu.quadric_custom_op  [-10.3458f, 12.5126f]                      27
/model/layers.8/Add_1                                add                            [-2120.62f, 9654.73f]                      17
/model/layers.9/input_layernorm/Mul_1                contrib.epu.rms_norm           [-4.52299f, 10.3721f]                      27
/model/layers.9/self_attn/q_proj/MatMul_smooth_mul   multiply                       [-1.02019f, 1.58785f]                      26
/model/layers.9/self_attn/k_proj/MatMul_smooth_mul   multiply                       [-1.23027f, 1.70964f]                      26
/model/layers.9/self_attn/v_proj/MatMul_smooth_mul   multiply                       [-0.594754f, 0.902843f]                    26
CustomOp/linalg::channelwiseQuantMatMul<39>39        contrib.epu.quadric_custom_op  [-7.31632f, 7.46158f]                      28
/model/layers.9/Add                                  add                            [-2120.34f, 9654.1f]                       17
/model/layers.9/post_attention_layernorm/Mul_1       contrib.epu.rms_norm           [-8.45564f, 8.75462f]                      27
/model/layers.9/mlp/gate_proj/MatMul_smooth_mul      multiply                       [-1.24338f, 1.8245f]                       27
CustomOp/nn::gateProj<39, 27>37                      contrib.epu.quadric_custom_op  [-0.278465f, 9.10514f]                     27
/model/layers.9/mlp/up_proj/MatMul_smooth_mul        multiply                       [-1.7809f, 1.67158f]                       27
CustomOp/linalg::channelwiseQuantMatMul<39>36        contrib.epu.quadric_custom_op  [-7.51536f, 6.51499f]                      28
/model/layers.9/mlp/Mul                              multiply                       [-15.1261f, 14.7165f]                      24
/model/layers.9/mlp/down_proj/MatMul_smooth_mul      multiply                       [-2.19853f, 3.02141f]                      27
CustomOp/linalg::channelwiseQuantMatMul<38>38        contrib.epu.quadric_custom_op  [-9.86182f, 8.2895f]                       27
/model/layers.9/Add_1                                add                            [-2120.34f, 9656.53f]                      17
/model/layers.10/input_layernorm/Mul_1               contrib.epu.rms_norm           [-7.60531f, 17.5566f]                      26
/model/layers.10/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.31638f, 2.45222f]                      26
/model/layers.10/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.14123f, 2.85072f]                      26
/model/layers.10/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.635305f, 0.964866f]                    26
CustomOp/linalg::channelwiseQuantMatMul<40>43        contrib.epu.quadric_custom_op  [-7.399f, 5.84284f]                        28
/model/layers.10/Add                                 add                            [-2119.36f, 9655.58f]                      17
/model/layers.10/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-8.33987f, 9.41487f]                      27
/model/layers.10/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.41946f, 1.68869f]                      26
CustomOp/nn::gateProj<39, 27>41                      contrib.epu.quadric_custom_op  [-0.278465f, 8.82315f]                     27
/model/layers.10/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.41946f, 1.19373f]                      27
CustomOp/linalg::channelwiseQuantMatMul<40>40        contrib.epu.quadric_custom_op  [-5.76626f, 8.25113f]                      27
/model/layers.10/mlp/Mul                             multiply                       [-15.7846f, 20.8058f]                      24
/model/layers.10/mlp/down_proj/MatMul_smooth_mul     multiply                       [-5.70319f, 2.03262f]                      26
CustomOp/linalg::channelwiseQuantMatMul<37>42        contrib.epu.quadric_custom_op  [-5.03255f, 18.6078f]                      26
/model/layers.10/Add_1                               add                            [-2119.79f, 9656.97f]                      17
/model/layers.11/input_layernorm/Mul_1               contrib.epu.rms_norm           [-5.77417f, 10.4802f]                      27
/model/layers.11/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.24383f, 1.49914f]                      28
/model/layers.11/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-0.911936f, 1.83978f]                     28
/model/layers.11/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.494405f, 0.873254f]                    28
CustomOp/linalg::channelwiseQuantMatMul<40>47        contrib.epu.quadric_custom_op  [-7.13255f, 7.60828f]                      28
/model/layers.11/Add                                 add                            [-2118.77f, 9655.72f]                      17
/model/layers.11/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-9.30534f, 7.62255f]                      27
/model/layers.11/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.12876f, 1.54745f]                      26
CustomOp/nn::gateProj<39, 27>45                      contrib.epu.quadric_custom_op  [-0.278465f, 12.7414f]                     27
/model/layers.11/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.37915f, 0.865526f]                     26
CustomOp/linalg::channelwiseQuantMatMul<40>44        contrib.epu.quadric_custom_op  [-8.25134f, 6.12666f]                      27
/model/layers.11/mlp/Mul                             multiply                       [-12.4377f, 12.5137f]                      24
/model/layers.11/mlp/down_proj/MatMul_smooth_mul     multiply                       [-3.31324f, 2.34436f]                      27
CustomOp/linalg::channelwiseQuantMatMul<38>46        contrib.epu.quadric_custom_op  [-7.59268f, 11.5153f]                      27
/model/layers.11/Add_1                               add                            [-2118.11f, 9657.75f]                      17
/model/layers.12/input_layernorm/Mul_1               contrib.epu.rms_norm           [-6.76928f, 11.1257f]                      27
/model/layers.12/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.31845f, 1.85694f]                      28
/model/layers.12/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.1593f, 1.7855f]                        27
/model/layers.12/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.844806f, 0.934658f]                    28
CustomOp/linalg::channelwiseQuantMatMul<40>51        contrib.epu.quadric_custom_op  [-7.8656f, 13.7375f]                       27
/model/layers.12/Add                                 add                            [-2117.66f, 9657.01f]                      17
/model/layers.12/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-10.1594f, 7.15519f]                      27
/model/layers.12/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.37687f, 1.39484f]                      26
CustomOp/nn::gateProj<40, 27>49                      contrib.epu.quadric_custom_op  [-0.278465f, 7.39754f]                     28
/model/layers.12/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.92166f, 1.10387f]                      26
CustomOp/linalg::channelwiseQuantMatMul<39>48        contrib.epu.quadric_custom_op  [-9.58119f, 5.33228f]                      27
/model/layers.12/mlp/Mul                             multiply                       [-60.1034f, 12.2375f]                      24
/model/layers.12/mlp/down_proj/MatMul_smooth_mul     multiply                       [-3.8142f, 3.11107f]                       25
CustomOp/linalg::channelwiseQuantMatMul<37>50        contrib.epu.quadric_custom_op  [-12.1023f, 17.851f]                       26
/model/layers.12/Add_1                               add                            [-2118.72f, 9659.33f]                      17
/model/layers.13/input_layernorm/Mul_1               contrib.epu.rms_norm           [-5.04587f, 9.31552f]                      27
/model/layers.13/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.00549f, 1.73688f]                      28
/model/layers.13/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.76774f, 1.88082f]                      28
/model/layers.13/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.604113f, 0.910164f]                    28
CustomOp/linalg::channelwiseQuantMatMul<40>55        contrib.epu.quadric_custom_op  [-5.54142f, 8.70166f]                      27
/model/layers.13/Add                                 add                            [-2117.86f, 9659.14f]                      17
/model/layers.13/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-11.9317f, 6.23108f]                      27
/model/layers.13/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.47657f, 1.26196f]                      26
CustomOp/nn::gateProj<40, 26>53                      contrib.epu.quadric_custom_op  [-0.278465f, 17.2259f]                     26
/model/layers.13/mlp/up_proj/MatMul_smooth_mul       multiply                       [-2.24165f, 1.21306f]                      26
CustomOp/linalg::channelwiseQuantMatMul<39>52        contrib.epu.quadric_custom_op  [-30.4806f, 32.1002f]                      25
/model/layers.13/mlp/Mul                             multiply                       [-16.9783f, 33.7621f]                      21
/model/layers.13/mlp/down_proj/MatMul_smooth_mul     multiply                       [-2.68679f, 2.99732f]                      25
CustomOp/linalg::channelwiseQuantMatMul<38>54        contrib.epu.quadric_custom_op  [-11.2142f, 11.9456f]                      27
/model/layers.13/Add_1                               add                            [-2120.76f, 9664.39f]                      17
/model/layers.14/input_layernorm/Mul_1               contrib.epu.rms_norm           [-6.68111f, 13.0119f]                      27
/model/layers.14/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.40054f, 1.87548f]                      27
/model/layers.14/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.56909f, 1.68693f]                      28
/model/layers.14/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.623238f, 0.820612f]                    28
CustomOp/linalg::channelwiseQuantMatMul<40>59        contrib.epu.quadric_custom_op  [-8.29877f, 19.2297f]                      26
/model/layers.14/Add                                 add                            [-2120.07f, 9663.34f]                      17
/model/layers.14/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-14.412f, 6.25216f]                       27
/model/layers.14/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.47538f, 1.42171f]                      26
CustomOp/nn::gateProj<39, 26>57                      contrib.epu.quadric_custom_op  [-0.278465f, 17.3895f]                     26
/model/layers.14/mlp/up_proj/MatMul_smooth_mul       multiply                       [-2.12944f, 0.916271f]                     26
CustomOp/linalg::channelwiseQuantMatMul<39>56        contrib.epu.quadric_custom_op  [-24.6362f, 32.0892f]                      25
/model/layers.14/mlp/Mul                             multiply                       [-84.1848f, 13.7615f]                      21
/model/layers.14/mlp/down_proj/MatMul_smooth_mul     multiply                       [-4.83579f, 2.83515f]                      24
CustomOp/linalg::channelwiseQuantMatMul<36>58        contrib.epu.quadric_custom_op  [-18.0127f, 15.0041f]                      26
/model/layers.14/Add_1                               add                            [-2123.29f, 9666.16f]                      17
/model/layers.15/input_layernorm/Mul_1               contrib.epu.rms_norm           [-7.40155f, 13.516f]                       27
/model/layers.15/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-2.20786f, 2.05356f]                      27
/model/layers.15/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.312f, 1.55167f]                        27
/model/layers.15/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.944417f, 1.36199f]                     28
CustomOp/linalg::channelwiseQuantMatMul<39>63        contrib.epu.quadric_custom_op  [-6.83366f, 13.2874f]                      27
/model/layers.15/Add                                 add                            [-2122.7f, 9664.69f]                       17
/model/layers.15/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-16.0488f, 6.0668f]                       26
/model/layers.15/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.4562f, 1.5061f]                        26
CustomOp/nn::gateProj<39, 26>61                      contrib.epu.quadric_custom_op  [-0.278465f, 19.5172f]                     26
/model/layers.15/mlp/up_proj/MatMul_smooth_mul       multiply                       [-3.49456f, 0.898868f]                     26
CustomOp/linalg::channelwiseQuantMatMul<38>60        contrib.epu.quadric_custom_op  [-16.5025f, 59.7275f]                      25
/model/layers.15/mlp/Mul                             multiply                       [-27.8356f, 48.4799f]                      20
/model/layers.15/mlp/down_proj/MatMul_smooth_mul     multiply                       [-3.85041f, 4.73925f]                      25
CustomOp/linalg::channelwiseQuantMatMul<36>62        contrib.epu.quadric_custom_op  [-16.8693f, 20.4889f]                      26
/model/layers.15/Add_1                               add                            [-2123.21f, 9668.25f]                      17
/model/layers.16/input_layernorm/Mul_1               contrib.epu.rms_norm           [-8.15026f, 15.6546f]                      26
/model/layers.16/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.3261f, 2.04648f]                       27
/model/layers.16/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.65707f, 1.55809f]                      27
/model/layers.16/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.734148f, 0.882209f]                    28
CustomOp/linalg::channelwiseQuantMatMul<38>67        contrib.epu.quadric_custom_op  [-39.5976f, 22.6377f]                      25
/model/layers.16/Add                                 add                            [-2122.92f, 9666.15f]                      17
/model/layers.16/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-36.8359f, 6.53895f]                      25
/model/layers.16/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-4.24899f, 2.25932f]                      25
CustomOp/nn::gateProj<37, 23>65                      contrib.epu.quadric_custom_op  [-0.278465f, 78.4472f]                     24
/model/layers.16/mlp/up_proj/MatMul_smooth_mul       multiply                       [-4.45054f, 1.66161f]                      25
CustomOp/linalg::channelwiseQuantMatMul<37>64        contrib.epu.quadric_custom_op  [-39.9357f, 84.0414f]                      24
/model/layers.16/mlp/Mul                             multiply                       [-191.33f, 4661.68f]                       18
/model/layers.16/mlp/down_proj/MatMul_smooth_mul     multiply                       [-40.6463f, 352.551f]                      18
CustomOp/linalg::channelwiseQuantMatMul<27>66        contrib.epu.quadric_custom_op  [-1202.39f, 13581.2f]                      17
/model/layers.16/Add_1                               add                            [-2130f, 13602.6f]                         16
/model/layers.17/input_layernorm/Mul_1               contrib.epu.rms_norm           [-7.88683f, 13.0965f]                      27
/model/layers.17/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.33071f, 1.92274f]                      27
/model/layers.17/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.34317f, 1.59226f]                      27
/model/layers.17/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.929569f, 1.09737f]                     28
CustomOp/linalg::channelwiseQuantMatMul<37>71        contrib.epu.quadric_custom_op  [-14.5229f, 61.5807f]                      25
/model/layers.17/Add                                 add                            [-2129.24f, 13637f]                        16
/model/layers.17/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-12.2827f, 6.15562f]                      27
/model/layers.17/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.52264f, 1.37121f]                      26
CustomOp/nn::gateProj<39, 28>69                      contrib.epu.quadric_custom_op  [-0.278465f, 7.73579f]                     28
/model/layers.17/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.52896f, 0.967658f]                     26
CustomOp/linalg::channelwiseQuantMatMul<40>68        contrib.epu.quadric_custom_op  [-7.26674f, 35.3441f]                      25
/model/layers.17/mlp/Mul                             multiply                       [-13.8208f, 9.58101f]                      22
/model/layers.17/mlp/down_proj/MatMul_smooth_mul     multiply                       [-3.27512f, 3.71965f]                      27
CustomOp/linalg::channelwiseQuantMatMul<38>70        contrib.epu.quadric_custom_op  [-21.4484f, 22.413f]                       26
/model/layers.17/Add_1                               add                            [-2128.21f, 13636.9f]                      16
/model/layers.18/input_layernorm/Mul_1               contrib.epu.rms_norm           [-10.6242f, 15.4063f]                      26
/model/layers.18/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.42803f, 2.10296f]                      27
/model/layers.18/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.48392f, 1.69128f]                      27
/model/layers.18/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.998706f, 1.36009f]                     28
CustomOp/linalg::channelwiseQuantMatMul<40>75        contrib.epu.quadric_custom_op  [-11.2483f, 15.3732f]                      27
/model/layers.18/Add                                 add                            [-2127.25f, 13638.1f]                      16
/model/layers.18/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-11.7031f, 6.12549f]                      27
/model/layers.18/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.18675f, 1.43604f]                      26
CustomOp/nn::gateProj<39, 27>73                      contrib.epu.quadric_custom_op  [-0.278465f, 8.13071f]                     27
/model/layers.18/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.42666f, 1.13167f]                      26
CustomOp/linalg::channelwiseQuantMatMul<39>72        contrib.epu.quadric_custom_op  [-6.42202f, 15.0935f]                      27
/model/layers.18/mlp/Mul                             multiply                       [-9.16472f, 64.1296f]                      24
/model/layers.18/mlp/down_proj/MatMul_smooth_mul     multiply                       [-3.79416f, 8.76246f]                      25
CustomOp/linalg::channelwiseQuantMatMul<35>74        contrib.epu.quadric_custom_op  [-27.1561f, 62.6349f]                      25
/model/layers.18/Add_1                               add                            [-2127.74f, 13638.1f]                      16
/model/layers.19/input_layernorm/Mul_1               contrib.epu.rms_norm           [-16.3269f, 23.5058f]                      26
/model/layers.19/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.87381f, 2.36709f]                      27
/model/layers.19/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-1.99903f, 2.0809f]                       27
/model/layers.19/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.96973f, 1.21001f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>79        contrib.epu.quadric_custom_op  [-9.69087f, 19.3656f]                      26
/model/layers.19/Add                                 add                            [-2127.19f, 13637.7f]                      16
/model/layers.19/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-11.4451f, 6.57414f]                      27
/model/layers.19/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.50948f, 1.27079f]                      27
CustomOp/nn::gateProj<40, 27>77                      contrib.epu.quadric_custom_op  [-0.278465f, 8.66148f]                     27
/model/layers.19/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.77521f, 1.54593f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>76        contrib.epu.quadric_custom_op  [-6.79348f, 5.92249f]                      28
/model/layers.19/mlp/Mul                             multiply                       [-10.5208f, 11.7491f]                      25
/model/layers.19/mlp/down_proj/MatMul_smooth_mul     multiply                       [-3.32359f, 3.64369f]                      27
CustomOp/linalg::channelwiseQuantMatMul<36>78        contrib.epu.quadric_custom_op  [-28.7991f, 32.1844f]                      25
/model/layers.19/Add_1                               add                            [-2127.2f, 13637.6f]                       16
/model/layers.20/input_layernorm/Mul_1               contrib.epu.rms_norm           [-17.2382f, 20.6114f]                      26
/model/layers.20/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-1.79777f, 2.35375f]                      27
/model/layers.20/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-2.07954f, 1.97869f]                      27
/model/layers.20/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-0.97567f, 1.51478f]                      27
CustomOp/linalg::channelwiseQuantMatMul<40>83        contrib.epu.quadric_custom_op  [-11.7166f, 31.5278f]                      26
/model/layers.20/Add                                 add                            [-2126.62f, 13634.6f]                      16
/model/layers.20/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-12.2707f, 7.12023f]                      27
/model/layers.20/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.27321f, 1.48049f]                      28
CustomOp/nn::gateProj<40, 27>81                      contrib.epu.quadric_custom_op  [-0.278465f, 8.94129f]                     27
/model/layers.20/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.25212f, 1.368f]                        27
CustomOp/linalg::channelwiseQuantMatMul<40>80        contrib.epu.quadric_custom_op  [-6.33998f, 8.02105f]                      27
/model/layers.20/mlp/Mul                             multiply                       [-18.5987f, 19.8616f]                      24
/model/layers.20/mlp/down_proj/MatMul_smooth_mul     multiply                       [-4.02727f, 2.56306f]                      26
CustomOp/linalg::channelwiseQuantMatMul<38>82        contrib.epu.quadric_custom_op  [-16.8789f, 16.8205f]                      26
/model/layers.20/Add_1                               add                            [-2125.87f, 13634.7f]                      16
/model/layers.21/input_layernorm/Mul_1               contrib.epu.rms_norm           [-23.1988f, 24.3195f]                      26
/model/layers.21/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-2.32063f, 2.84108f]                      27
/model/layers.21/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-2.10805f, 2.21897f]                      27
/model/layers.21/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-1.10305f, 1.53724f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>87        contrib.epu.quadric_custom_op  [-8.88642f, 21.276f]                       26
/model/layers.21/Add                                 add                            [-2124.82f, 13636.5f]                      16
/model/layers.21/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-10.9728f, 7.28116f]                      27
/model/layers.21/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.61932f, 1.36054f]                      26
CustomOp/nn::gateProj<39, 27>85                      contrib.epu.quadric_custom_op  [-0.278465f, 9.86578f]                     27
/model/layers.21/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.27146f, 1.53064f]                      26
CustomOp/linalg::channelwiseQuantMatMul<39>84        contrib.epu.quadric_custom_op  [-10.277f, 10.0276f]                       27
/model/layers.21/mlp/Mul                             multiply                       [-41.594f, 53.735f]                        24
/model/layers.21/mlp/down_proj/MatMul_smooth_mul     multiply                       [-5.86662f, 8.1078f]                       25
CustomOp/linalg::channelwiseQuantMatMul<37>86        contrib.epu.quadric_custom_op  [-32.024f, 16.017f]                        25
/model/layers.21/Add_1                               add                            [-2124.91f, 13636.6f]                      16
/model/layers.22/input_layernorm/Mul_1               contrib.epu.rms_norm           [-30.8369f, 32.7385f]                      25
/model/layers.22/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-2.71083f, 3.39645f]                      27
/model/layers.22/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-3.44054f, 3.20511f]                      26
/model/layers.22/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-1.28456f, 1.37292f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>91        contrib.epu.quadric_custom_op  [-12.5842f, 31.0796f]                      26
/model/layers.22/Add                                 add                            [-2124.1f, 13635.5f]                       16
/model/layers.22/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-10.0459f, 8.97793f]                      27
/model/layers.22/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.56994f, 1.54569f]                      27
CustomOp/nn::gateProj<39, 26>89                      contrib.epu.quadric_custom_op  [-0.278465f, 16.0594f]                     26
/model/layers.22/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.54481f, 1.40509f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>88        contrib.epu.quadric_custom_op  [-8.84424f, 9.51616f]                      27
/model/layers.22/mlp/Mul                             multiply                       [-42.2895f, 37.618f]                       23
/model/layers.22/mlp/down_proj/MatMul_smooth_mul     multiply                       [-5.37248f, 5.52488f]                      26
CustomOp/linalg::channelwiseQuantMatMul<37>90        contrib.epu.quadric_custom_op  [-24.8563f, 17.5565f]                      26
/model/layers.22/Add_1                               add                            [-2123.61f, 13635.4f]                      16
/model/layers.23/input_layernorm/Mul_1               contrib.epu.rms_norm           [-32.5652f, 31.0209f]                      25
/model/layers.23/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-3.54566f, 3.57557f]                      27
/model/layers.23/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-2.95217f, 3.04757f]                      27
/model/layers.23/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-1.23035f, 1.36422f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>95        contrib.epu.quadric_custom_op  [-8.13639f, 35.0445f]                      25
/model/layers.23/Add                                 add                            [-2123.06f, 13632.6f]                      16
/model/layers.23/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-10.1758f, 10.4433f]                      27
/model/layers.23/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.73753f, 1.58375f]                      27
CustomOp/nn::gateProj<39, 26>93                      contrib.epu.quadric_custom_op  [-0.278465f, 15.0449f]                     27
/model/layers.23/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.42182f, 1.61803f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>92        contrib.epu.quadric_custom_op  [-12.3414f, 10.3189f]                      27
/model/layers.23/mlp/Mul                             multiply                       [-64.6526f, 60.3269f]                      23
/model/layers.23/mlp/down_proj/MatMul_smooth_mul     multiply                       [-5.1301f, 6.31523f]                       25
CustomOp/linalg::channelwiseQuantMatMul<37>94        contrib.epu.quadric_custom_op  [-24.6908f, 31.6436f]                      26
/model/layers.23/Add_1                               add                            [-2122.79f, 13632.6f]                      16
/model/layers.24/input_layernorm/Mul_1               contrib.epu.rms_norm           [-50.8646f, 39.9522f]                      25
/model/layers.24/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-4.28403f, 3.79681f]                      26
/model/layers.24/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-3.08274f, 3.52826f]                      27
/model/layers.24/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-2.11367f, 2.06183f]                      27
CustomOp/linalg::channelwiseQuantMatMul<38>99        contrib.epu.quadric_custom_op  [-16.7965f, 31.3655f]                      26
/model/layers.24/Add                                 add                            [-2123.99f, 13635.1f]                      16
/model/layers.24/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-10.2601f, 12.5924f]                      27
/model/layers.24/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.79581f, 1.95177f]                      26
CustomOp/nn::gateProj<39, 26>97                      contrib.epu.quadric_custom_op  [-0.278465f, 17.9184f]                     26
/model/layers.24/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.58951f, 1.69809f]                      26
CustomOp/linalg::channelwiseQuantMatMul<39>96        contrib.epu.quadric_custom_op  [-11.0467f, 11.354f]                       27
/model/layers.24/mlp/Mul                             multiply                       [-107.554f, 90.9104f]                      23
/model/layers.24/mlp/down_proj/MatMul_smooth_mul     multiply                       [-6.40439f, 6.75044f]                      25
CustomOp/linalg::channelwiseQuantMatMul<36>98        contrib.epu.quadric_custom_op  [-22.1531f, 38.053f]                       25
/model/layers.24/Add_1                               add                            [-2123.99f, 13635.3f]                      16
/model/layers.25/input_layernorm/Mul_1               contrib.epu.rms_norm           [-40.2525f, 33.3074f]                      25
/model/layers.25/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-3.55716f, 3.96355f]                      26
/model/layers.25/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-2.68001f, 3.21491f]                      27
/model/layers.25/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-1.83399f, 1.64135f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>103       contrib.epu.quadric_custom_op  [-10.4602f, 17.5029f]                      26
/model/layers.25/Add                                 add                            [-2123.75f, 13635.5f]                      16
/model/layers.25/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-11.309f, 12.7584f]                       27
/model/layers.25/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.95923f, 2.00956f]                      26
CustomOp/nn::gateProj<39, 26>101                     contrib.epu.quadric_custom_op  [-0.278465f, 20.8095f]                     26
/model/layers.25/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.65599f, 1.83448f]                      26
CustomOp/linalg::channelwiseQuantMatMul<39>100       contrib.epu.quadric_custom_op  [-16.033f, 12.6045f]                       26
/model/layers.25/mlp/Mul                             multiply                       [-117.555f, 109.813f]                      22
/model/layers.25/mlp/down_proj/MatMul_smooth_mul     multiply                       [-7.04222f, 8.54142f]                      24
CustomOp/linalg::channelwiseQuantMatMul<36>102       contrib.epu.quadric_custom_op  [-19.0471f, 36.3233f]                      25
/model/layers.25/Add_1                               add                            [-2123.73f, 13635.5f]                      16
/model/layers.26/input_layernorm/Mul_1               contrib.epu.rms_norm           [-50.6021f, 38.8768f]                      25
/model/layers.26/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-4.46383f, 4.41033f]                      26
/model/layers.26/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-3.09661f, 3.23338f]                      27
/model/layers.26/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-1.95177f, 1.62671f]                      27
CustomOp/linalg::channelwiseQuantMatMul<40>107       contrib.epu.quadric_custom_op  [-8.51294f, 19.8805f]                      26
/model/layers.26/Add                                 add                            [-2124.04f, 13636.4f]                      16
/model/layers.26/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-12.5561f, 15.4058f]                      26
/model/layers.26/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-2.00442f, 1.59836f]                      26
CustomOp/nn::gateProj<39, 26>105                     contrib.epu.quadric_custom_op  [-0.278465f, 22.6042f]                     26
/model/layers.26/mlp/up_proj/MatMul_smooth_mul       multiply                       [-1.56655f, 2.01047f]                      26
CustomOp/linalg::channelwiseQuantMatMul<38>104       contrib.epu.quadric_custom_op  [-12.8656f, 13.9187f]                      27
/model/layers.26/mlp/Mul                             multiply                       [-105.01f, 132.743f]                       22
/model/layers.26/mlp/down_proj/MatMul_smooth_mul     multiply                       [-6.63407f, 7.95295f]                      24
CustomOp/linalg::channelwiseQuantMatMul<36>106       contrib.epu.quadric_custom_op  [-21.3917f, 49.6387f]                      25
/model/layers.26/Add_1                               add                            [-2124.03f, 13636.6f]                      16
/model/layers.27/input_layernorm/Mul_1               contrib.epu.rms_norm           [-61.2838f, 48.1027f]                      25
/model/layers.27/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-4.33818f, 4.62879f]                      26
/model/layers.27/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-2.61422f, 3.56265f]                      26
/model/layers.27/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-2.02177f, 2.13091f]                      27
CustomOp/linalg::channelwiseQuantMatMul<39>111       contrib.epu.quadric_custom_op  [-8.8824f, 19.0919f]                       26
/model/layers.27/Add                                 add                            [-2124.55f, 13638.8f]                      16
/model/layers.27/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-14.5542f, 17.2044f]                      26
/model/layers.27/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.75765f, 1.86848f]                      26
CustomOp/nn::gateProj<39, 26>109                     contrib.epu.quadric_custom_op  [-0.278465f, 19.2789f]                     26
/model/layers.27/mlp/up_proj/MatMul_smooth_mul       multiply                       [-3.00212f, 1.82197f]                      26
CustomOp/linalg::channelwiseQuantMatMul<38>108       contrib.epu.quadric_custom_op  [-13.1627f, 12.5415f]                      27
/model/layers.27/mlp/Mul                             multiply                       [-137.571f, 122.284f]                      23
/model/layers.27/mlp/down_proj/MatMul_smooth_mul     multiply                       [-8.10809f, 8.06381f]                      24
CustomOp/linalg::channelwiseQuantMatMul<36>110       contrib.epu.quadric_custom_op  [-29.7459f, 42.6272f]                      25
/model/layers.27/Add_1                               add                            [-2124.49f, 13639.2f]                      16
/model/layers.28/input_layernorm/Mul_1               contrib.epu.rms_norm           [-65.7556f, 56.4749f]                      24
/model/layers.28/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-4.29473f, 4.18318f]                      26
/model/layers.28/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-3.32743f, 3.96202f]                      27
/model/layers.28/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-2.12739f, 2.4797f]                       27
CustomOp/linalg::channelwiseQuantMatMul<39>115       contrib.epu.quadric_custom_op  [-16.1447f, 22.5356f]                      26
/model/layers.28/Add                                 add                            [-2125.41f, 13642.1f]                      16
/model/layers.28/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-13.3559f, 17.9234f]                      26
/model/layers.28/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-2.34241f, 2.10349f]                      23
CustomOp/nn::gateProj<38, 26>113                     contrib.epu.quadric_custom_op  [-0.278465f, 23.2141f]                     26
/model/layers.28/mlp/up_proj/MatMul_smooth_mul       multiply                       [-2.12235f, 1.94853f]                      23
CustomOp/linalg::channelwiseQuantMatMul<38>112       contrib.epu.quadric_custom_op  [-18.7225f, 14.9992f]                      26
/model/layers.28/mlp/Mul                             multiply                       [-132.766f, 140.229f]                      22
/model/layers.28/mlp/down_proj/MatMul_smooth_mul     multiply                       [-14.6529f, 10.7982f]                      24
CustomOp/linalg::channelwiseQuantMatMul<35>114       contrib.epu.quadric_custom_op  [-38.7672f, 58.2323f]                      25
/model/layers.28/Add_1                               add                            [-2125.36f, 13642.5f]                      16
/model/layers.29/input_layernorm/Mul_1               contrib.epu.rms_norm           [-87.7913f, 63.9043f]                      24
/model/layers.29/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-5.64585f, 4.53467f]                      26
/model/layers.29/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-4.66197f, 4.02658f]                      26
/model/layers.29/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-3.72409f, 3.62137f]                      26
CustomOp/linalg::channelwiseQuantMatMul<39>119       contrib.epu.quadric_custom_op  [-15.1006f, 23.0286f]                      26
/model/layers.29/Add                                 add                            [-2126.23f, 13643.9f]                      16
/model/layers.29/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-14.5663f, 19.9309f]                      26
/model/layers.29/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-1.95651f, 1.86312f]                      25
CustomOp/nn::gateProj<39, 26>117                     contrib.epu.quadric_custom_op  [-0.278465f, 19.1744f]                     26
/model/layers.29/mlp/up_proj/MatMul_smooth_mul       multiply                       [-2.62913f, 2.32513f]                      25
CustomOp/linalg::channelwiseQuantMatMul<38>116       contrib.epu.quadric_custom_op  [-16.4508f, 15.0669f]                      26
/model/layers.29/mlp/Mul                             multiply                       [-150.54f, 123.012f]                       22
/model/layers.29/mlp/down_proj/MatMul_smooth_mul     multiply                       [-11.6399f, 11.8174f]                      24
CustomOp/linalg::channelwiseQuantMatMul<35>118       contrib.epu.quadric_custom_op  [-49.9533f, 76.5029f]                      24
/model/layers.29/Add_1                               add                            [-2126.19f, 13644f]                        16
/model/layers.30/input_layernorm/Mul_1               contrib.epu.rms_norm           [-97.8058f, 70.9147f]                      24
/model/layers.30/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-8.32574f, 6.67203f]                      26
/model/layers.30/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-5.20679f, 4.87224f]                      26
/model/layers.30/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-3.51157f, 3.44255f]                      26
CustomOp/linalg::channelwiseQuantMatMul<38>123       contrib.epu.quadric_custom_op  [-26.3882f, 42.82f]                        25
/model/layers.30/Add                                 add                            [-2126.46f, 13648.6f]                      16
/model/layers.30/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-16.1268f, 19.8625f]                      26
/model/layers.30/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-2.32769f, 1.89138f]                      27
CustomOp/nn::gateProj<38, 26>121                     contrib.epu.quadric_custom_op  [-0.278465f, 20.8345f]                     26
/model/layers.30/mlp/up_proj/MatMul_smooth_mul       multiply                       [-2.93709f, 3.14719f]                      27
CustomOp/linalg::channelwiseQuantMatMul<38>120       contrib.epu.quadric_custom_op  [-22.3626f, 17.1751f]                      26
/model/layers.30/mlp/Mul                             multiply                       [-147.928f, 141.815f]                      22
/model/layers.30/mlp/down_proj/MatMul_smooth_mul     multiply                       [-15.4673f, 11.2169f]                      24
CustomOp/linalg::channelwiseQuantMatMul<35>122       contrib.epu.quadric_custom_op  [-53.484f, 106.991f]                       24
/model/layers.30/Add_1                               add                            [-2126.44f, 13648.7f]                      16
/model/layers.31/input_layernorm/Mul_1               contrib.epu.rms_norm           [-119.533f, 93.6401f]                      24
/model/layers.31/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-7.37432f, 6.46369f]                      26
/model/layers.31/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-5.50964f, 5.44922f]                      25
/model/layers.31/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-3.79376f, 3.57184f]                      26
CustomOp/linalg::channelwiseQuantMatMul<38>127       contrib.epu.quadric_custom_op  [-17.2974f, 50.1237f]                      25
/model/layers.31/Add                                 add                            [-2126.83f, 13653.5f]                      16
/model/layers.31/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-17.2516f, 18.0006f]                      26
/model/layers.31/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-2.36857f, 2.28309f]                      27
CustomOp/nn::gateProj<38, 26>125                     contrib.epu.quadric_custom_op  [-0.278465f, 21.2043f]                     26
/model/layers.31/mlp/up_proj/MatMul_smooth_mul       multiply                       [-2.60498f, 3.19835f]                      27
CustomOp/linalg::channelwiseQuantMatMul<37>124       contrib.epu.quadric_custom_op  [-18.9059f, 19.1897f]                      26
/model/layers.31/mlp/Mul                             multiply                       [-147.492f, 171.108f]                      22
/model/layers.31/mlp/down_proj/MatMul_smooth_mul     multiply                       [-18.9471f, 14.1054f]                      24
CustomOp/linalg::channelwiseQuantMatMul<34>126       contrib.epu.quadric_custom_op  [-42.7217f, 101.298f]                      24
/model/layers.31/Add_1                               add                            [-2126.65f, 13653.4f]                      16
/model/layers.32/input_layernorm/Mul_1               contrib.epu.rms_norm           [-129.345f, 108.642f]                      23
/model/layers.32/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-7.50556f, 7.79626f]                      26
/model/layers.32/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-6.04744f, 5.6172f]                       26
/model/layers.32/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-5.33153f, 4.74853f]                      26
CustomOp/linalg::channelwiseQuantMatMul<37>131       contrib.epu.quadric_custom_op  [-26.9792f, 52.8894f]                      25
/model/layers.32/Add                                 add                            [-2125.55f, 13676.5f]                      16
/model/layers.32/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-17.899f, 23.7326f]                       26
/model/layers.32/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-2.44445f, 3.21304f]                      26
CustomOp/nn::gateProj<38, 26>129                     contrib.epu.quadric_custom_op  [-0.278465f, 25.1264f]                     26
/model/layers.32/mlp/up_proj/MatMul_smooth_mul       multiply                       [-4.51652f, 3.91687f]                      26
CustomOp/linalg::channelwiseQuantMatMul<37>128       contrib.epu.quadric_custom_op  [-18.7255f, 22.3183f]                      26
/model/layers.32/mlp/Mul                             multiply                       [-190.096f, 285.781f]                      21
/model/layers.32/mlp/down_proj/MatMul_smooth_mul     multiply                       [-16.2074f, 13.687f]                       24
CustomOp/linalg::channelwiseQuantMatMul<34>130       contrib.epu.quadric_custom_op  [-67.0631f, 111.421f]                      24
/model/layers.32/Add_1                               add                            [-2125.27f, 13672.9f]                      16
/model/layers.33/input_layernorm/Mul_1               contrib.epu.rms_norm           [-187.593f, 150.813f]                      23
/model/layers.33/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-8.17666f, 7.40637f]                      25
/model/layers.33/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-7.68112f, 7.72248f]                      25
/model/layers.33/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-6.35676f, 6.26976f]                      25
CustomOp/linalg::channelwiseQuantMatMul<37>135       contrib.epu.quadric_custom_op  [-22.3657f, 115.159f]                      24
/model/layers.33/Add                                 add                            [-2125.11f, 13717f]                        16
/model/layers.33/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-29.27f, 27.8766f]                        26
/model/layers.33/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-4.70675f, 3.16855f]                      26
CustomOp/nn::gateProj<37, 26>133                     contrib.epu.quadric_custom_op  [-0.278465f, 20.9414f]                     26
/model/layers.33/mlp/up_proj/MatMul_smooth_mul       multiply                       [-7.67804f, 4.66365f]                      27
CustomOp/linalg::channelwiseQuantMatMul<36>132       contrib.epu.quadric_custom_op  [-21.9644f, 28.9324f]                      26
/model/layers.33/mlp/Mul                             multiply                       [-191.84f, 180.474f]                       21
/model/layers.33/mlp/down_proj/MatMul_smooth_mul     multiply                       [-16.8717f, 15.8574f]                      24
CustomOp/linalg::channelwiseQuantMatMul<34>134       contrib.epu.quadric_custom_op  [-224.43f, 154.382f]                       23
/model/layers.33/Add_1                               add                            [-2126.49f, 13736f]                        16
/model/layers.34/input_layernorm/Mul_1               contrib.epu.rms_norm           [-190.924f, 150.511f]                      23
/model/layers.34/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-9.52248f, 9.91184f]                      25
/model/layers.34/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-7.38449f, 7.55077f]                      26
/model/layers.34/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-5.32747f, 5.83561f]                      26
CustomOp/linalg::channelwiseQuantMatMul<36>139       contrib.epu.quadric_custom_op  [-134.891f, 252.592f]                      23
/model/layers.34/Add                                 add                            [-2127.57f, 13665.9f]                      16
/model/layers.34/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-102.457f, 31.8715f]                      24
/model/layers.34/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-11.9706f, 3.27663f]                      24
CustomOp/nn::gateProj<35, 25>137                     contrib.epu.quadric_custom_op  [-0.278465f, 43.9184f]                     25
/model/layers.34/mlp/up_proj/MatMul_smooth_mul       multiply                       [-15.5429f, 4.71439f]                      24
CustomOp/linalg::channelwiseQuantMatMul<34>136       contrib.epu.quadric_custom_op  [-33.824f, 68.3458f]                       24
/model/layers.34/mlp/Mul                             multiply                       [-1174.96f, 792.102f]                      19
/model/layers.34/mlp/down_proj/MatMul_smooth_mul     multiply                       [-110.529f, 78.4391f]                      22
CustomOp/linalg::channelwiseQuantMatMul<31>138       contrib.epu.quadric_custom_op  [-18980.2f, 553.5f]                        16
/model/layers.34/Add_1                               add                            [-7343.94f, 5351.69f]                      16
/model/layers.35/input_layernorm/Mul_1               contrib.epu.rms_norm           [-298.728f, 121.485f]                      22
/model/layers.35/self_attn/q_proj/MatMul_smooth_mul  multiply                       [-40.8404f, 9.94293f]                      24
/model/layers.35/self_attn/k_proj/MatMul_smooth_mul  multiply                       [-34.7046f, 8.17391f]                      25
/model/layers.35/self_attn/v_proj/MatMul_smooth_mul  multiply                       [-6.21749f, 4.66202f]                      25
CustomOp/linalg::channelwiseQuantMatMul<36>143       contrib.epu.quadric_custom_op  [-282.141f, 824.661f]                      21
/model/layers.35/Add                                 add                            [-7101.39f, 5072.41f]                      16
/model/layers.35/post_attention_layernorm/Mul_1      contrib.epu.rms_norm           [-601.076f, 47.2325f]                      21
/model/layers.35/mlp/gate_proj/MatMul_smooth_mul     multiply                       [-42.4844f, 4.64554f]                      22
CustomOp/nn::gateProj<34, 24>141                     contrib.epu.quadric_custom_op  [-0.278465f, 109.277f]                     24
/model/layers.35/mlp/up_proj/MatMul_smooth_mul       multiply                       [-58.8737f, 4.981f]                        22
CustomOp/linalg::channelwiseQuantMatMul<33>140       contrib.epu.quadric_custom_op  [-159.195f, 162.484f]                      23
/model/layers.35/mlp/Mul                             multiply                       [-1002.33f, 1755.35f]                      16
/model/layers.35/mlp/down_proj/MatMul_smooth_mul     multiply                       [-73.9084f, 97.6708f]                      21
CustomOp/linalg::channelwiseQuantMatMul<29>142       contrib.epu.quadric_custom_op  [-4812.36f, 1990.75f]                      18
/model/layers.35/Add_1                               add                            [-6990.34f, 2581.91f]                      16
/model/norm/Mul_1                                    contrib.epu.rms_norm           [-139.062f, 130.451f]                      23
/Gather                                              take                           [-93.9597f, 66.4554f]                      23
/lm_head/MatMul_smooth_mul                           multiply                       [-5.11149f, 4.01418f]                      24
CustomOp/linalg::channelwiseQuantMatMul<36>144       contrib.epu.quadric_custom_op  [-15.0257f, 24.2741f]                      26

2026-10-23 01:04 - INFO - epu - util - START====================build_cpu_runnable_fx_relay
2026-10-23 01:04 - INFO - epu - util - START=======================quantize_to_chimera_fx
2026-10-23 01:05 - INFO - epu - util - START=================================relay_to_tir
2026-10-23 01:05 - INFO - epu - util - START===========================relay_to_epu_relay
2026-10-23 01:05 - INFO - epu - util - START==============================adapt_and_order
2026-10-23 01:05 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 01:05 - INFO - epu - util - START=============================plan_lrm_virtual
2026-10-23 01:05 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 01:05 - INFO - epu - util - START===============================lrm_alloc_loop
2026-10-23 01:06 - INFO - epu - util - START==============================amend_ctrl_flow
2026-10-23 01:06 - INFO - epu - util - START================================lrm_splitting
2026-10-23 01:07 - INFO - epu - util - START=====================LrmUsageChecker.__call__
2026-10-23 01:07 - INFO - epu - util - START==============================ext_split_relay
2026-10-23 01:07 - INFO - epu - util - START=========================OcmAllocator.analyze
2026-10-23 01:07 - INFO - epu - util - START===============================Compiler.build
2026-10-23 01:07 - INFO - epu - util - START========================RelayToPrimFunc.build
2026-10-23 01:07 - INFO - epu - util - START=========================OcmAllocator.analyze
2026-10-23 01:07 - INFO - epu - util - START====================================build_tir
2026-10-23 01:07 - INFO - epu - chimera_job - Compilation of qwen3_custom_ops_seq1024_QC_P_1d7_2MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful



╒═════════════════════╤════════════════════════════════════════════════════════════════════════╕
│ Module Name         │ qwen3_custom_ops_seq1024_QC_P_1d7_2MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ ONNX File           │ qwen3_custom_ops_seq1024.onnx                                          │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Product Target      │ QC-P                                                                   │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Number of Cores     │ 1                                                                      │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700                                                                  │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ L2M Size            │ 2MB                                                                    │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ LRM Size            │ 4kB                                                                    │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ External Read BW    │ 128GBps                                                                │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ External Write BW   │ 128GBps                                                                │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ MACS per PE         │ 16                                                                     │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Max L2M             │ 0.172MB                                                                │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Max LRM             │ 0.609kB                                                                │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes  │ 0.583MB                                                                │
├─────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Network GMACs       │ 7.870                                                                  │
╘═════════════════════╧════════════════════════════════════════════════════════════════════════╛

╒═════╤════════╤══════════════════════════╤═══════════════════╤══════════════════════════╤═══════╕
│     │ Type   │ Name                     │ shape             │ type                     │ mse   │
╞═════╪════════╪══════════════════════════╪═══════════════════╪══════════════════════════╪═══════╡
│   0 │ Input  │ input_ids                │ [1, 1]            │ tensor[int32]            │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   1 │ Input  │ attention_mask           │ [1, 1, 1024]      │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   2 │ Input  │ sin                      │ [1, 1, 128]       │ tensor[FixedPoint32<30>] │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   3 │ Input  │ cos                      │ [1, 1, 128]       │ tensor[FixedPoint32<30>] │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   4 │ Input  │ past_key_values.0.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   5 │ Input  │ past_key_values.0.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   6 │ Input  │ past_key_values.1.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   7 │ Input  │ past_key_values.1.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   8 │ Input  │ past_key_values.2.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│   9 │ Input  │ past_key_values.2.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  10 │ Input  │ past_key_values.3.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  11 │ Input  │ past_key_values.3.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  12 │ Input  │ past_key_values.4.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  13 │ Input  │ past_key_values.4.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  14 │ Input  │ past_key_values.5.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  15 │ Input  │ past_key_values.5.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  16 │ Input  │ past_key_values.6.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  17 │ Input  │ past_key_values.6.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  18 │ Input  │ past_key_values.7.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  19 │ Input  │ past_key_values.7.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  20 │ Input  │ past_key_values.8.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  21 │ Input  │ past_key_values.8.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  22 │ Input  │ past_key_values.9.key    │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  23 │ Input  │ past_key_values.9.value  │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  24 │ Input  │ past_key_values.10.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  25 │ Input  │ past_key_values.10.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  26 │ Input  │ past_key_values.11.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  27 │ Input  │ past_key_values.11.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  28 │ Input  │ past_key_values.12.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  29 │ Input  │ past_key_values.12.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  30 │ Input  │ past_key_values.13.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  31 │ Input  │ past_key_values.13.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  32 │ Input  │ past_key_values.14.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  33 │ Input  │ past_key_values.14.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  34 │ Input  │ past_key_values.15.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  35 │ Input  │ past_key_values.15.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  36 │ Input  │ past_key_values.16.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  37 │ Input  │ past_key_values.16.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  38 │ Input  │ past_key_values.17.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  39 │ Input  │ past_key_values.17.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  40 │ Input  │ past_key_values.18.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  41 │ Input  │ past_key_values.18.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  42 │ Input  │ past_key_values.19.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  43 │ Input  │ past_key_values.19.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  44 │ Input  │ past_key_values.20.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  45 │ Input  │ past_key_values.20.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  46 │ Input  │ past_key_values.21.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  47 │ Input  │ past_key_values.21.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  48 │ Input  │ past_key_values.22.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  49 │ Input  │ past_key_values.22.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  50 │ Input  │ past_key_values.23.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  51 │ Input  │ past_key_values.23.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  52 │ Input  │ past_key_values.24.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  53 │ Input  │ past_key_values.24.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  54 │ Input  │ past_key_values.25.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  55 │ Input  │ past_key_values.25.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  56 │ Input  │ past_key_values.26.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  57 │ Input  │ past_key_values.26.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  58 │ Input  │ past_key_values.27.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  59 │ Input  │ past_key_values.27.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  60 │ Input  │ past_key_values.28.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  61 │ Input  │ past_key_values.28.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  62 │ Input  │ past_key_values.29.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  63 │ Input  │ past_key_values.29.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  64 │ Input  │ past_key_values.30.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  65 │ Input  │ past_key_values.30.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  66 │ Input  │ past_key_values.31.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  67 │ Input  │ past_key_values.31.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  68 │ Input  │ past_key_values.32.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  69 │ Input  │ past_key_values.32.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  70 │ Input  │ past_key_values.33.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  71 │ Input  │ past_key_values.33.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  72 │ Input  │ past_key_values.34.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  73 │ Input  │ past_key_values.34.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  74 │ Input  │ past_key_values.35.key   │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  75 │ Input  │ past_key_values.35.value │ [1, 8, 1023, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  76 │ Output │ logits                   │ [1, 151936]       │ tensor[FixedPoint32<26>] │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  77 │ Output │ present.0.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  78 │ Output │ present.0.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  79 │ Output │ present.1.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  80 │ Output │ present.1.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  81 │ Output │ present.2.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  82 │ Output │ present.2.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  83 │ Output │ present.3.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  84 │ Output │ present.3.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  85 │ Output │ present.4.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  86 │ Output │ present.4.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  87 │ Output │ present.5.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  88 │ Output │ present.5.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  89 │ Output │ present.6.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  90 │ Output │ present.6.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  91 │ Output │ present.7.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  92 │ Output │ present.7.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  93 │ Output │ present.8.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  94 │ Output │ present.8.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  95 │ Output │ present.9.key            │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  96 │ Output │ present.9.value          │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  97 │ Output │ present.10.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  98 │ Output │ present.10.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│  99 │ Output │ present.11.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 100 │ Output │ present.11.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 101 │ Output │ present.12.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 102 │ Output │ present.12.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 103 │ Output │ present.13.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 104 │ Output │ present.13.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 105 │ Output │ present.14.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 106 │ Output │ present.14.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 107 │ Output │ present.15.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 108 │ Output │ present.15.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 109 │ Output │ present.16.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 110 │ Output │ present.16.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 111 │ Output │ present.17.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 112 │ Output │ present.17.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 113 │ Output │ present.18.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 114 │ Output │ present.18.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 115 │ Output │ present.19.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 116 │ Output │ present.19.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 117 │ Output │ present.20.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 118 │ Output │ present.20.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 119 │ Output │ present.21.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 120 │ Output │ present.21.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 121 │ Output │ present.22.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 122 │ Output │ present.22.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 123 │ Output │ present.23.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 124 │ Output │ present.23.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 125 │ Output │ present.24.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 126 │ Output │ present.24.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 127 │ Output │ present.25.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 128 │ Output │ present.25.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 129 │ Output │ present.26.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 130 │ Output │ present.26.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 131 │ Output │ present.27.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 132 │ Output │ present.27.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 133 │ Output │ present.28.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 134 │ Output │ present.28.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 135 │ Output │ present.29.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 136 │ Output │ present.29.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 137 │ Output │ present.30.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 138 │ Output │ present.30.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 139 │ Output │ present.31.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 140 │ Output │ present.31.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 141 │ Output │ present.32.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 142 │ Output │ present.32.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 143 │ Output │ present.33.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 144 │ Output │ present.33.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 145 │ Output │ present.34.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 146 │ Output │ present.34.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 147 │ Output │ present.35.key           │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 148 │ Output │ present.35.value         │ [1, 8, 1024, 128] │ n/a                      │ n/a   │
╘═════╧════════╧══════════════════════════╧═══════════════════╧══════════════════════════╧═══════╛

6. Prepare the Prompt

The tokenizer runs on the host, in the examples environment (transformers 4.51 or newer, which is where Qwen3 landed). prepare_inputs.py writes the four files the runner reads: input_logits.bin (token ids), sin.bin / cos.bin (RoPE tables for every position in the context) and prompt_len.bin.

The prompt is deliberately short — "The tallest" is two tokens — because every token costs a full pass through 8 billion parameters on the ISS.

!python3 prepare_inputs.py --prompt "The tallest" --new_tokens 1
2026-10-23 01:08:02.026503: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2026-10-23 01:08:02.028725: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2026-10-23 01:08:02.067618: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.
2026-10-23 01:08:02.196602: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-10-23 01:08:02.851555: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
[  785 81617]
Prompt length: 2

7. Run Autoregressive Decode on the ISS

qwen3.cpp is the host-side runner around the CGC-generated kernel. It loads the packed weights, then loops over the prompt and the generated tokens: one kernel launch per token, the KV cache appended on-chip, the next token chosen with an on-chip top-1 over the 151,936 logits — prompt_len + new_tokens − 1 passes in total, all inside a single ISS session. Each generated token is streamed back to the host through a kernel parameter register as soon as it is picked.

sdk source compiles the runner and the kernel with Quadric LLVM for the Chimera GPNPU backend and executes them on the ISS with the 4-core QC-P configuration from section 1. The weights CGC emitted first move next to the runner, where it looks for them.

Compile: ~10 minutes · Run: ~30 minutes

stage_const_tensor("ccl_build/qwen3_custom_ops_seq1024*")
Staged const_tensor_data.bin (6.29 GB of INT4/INT8 weights) for the runner





PosixPath('const_tensor_data.bin')
%%bash
set -euo pipefail
sdk source -v qwen3.cpp \
    --include-cgc-headers \
    --target QC-P \
    --num-cores 4 \
    --ocm-size 2MB \
    --macs-per-pe 16 \
    --clock-freq-ghz 1 \
    --ext-read-bw 24GBps \
    --ext-write-bw 24GBps \
    --ddr-axi-width 256 \
    --quiet
2026-10-23 01:08 - DEBUG - sdk - cli - Executing command: cmake CMakeLists.txt -B /quadric/sdk-cli/examples/models/qwen/qwen3_8b/qwen3_QC-P_1d0_2MB_4kB_24GBps_24GBps_16_OFF_x1_x4/build -DNUM_GPNPUS=4 -DNUM_CORES=16 -DNUM_BORDERS=2 -DEPU_VERSION=2.0.0 -DQLLVM_ROOT_PATH=/quadric/llvm -DOCM_SIZE_KIBIBYTES=2048 -DNUM_PE_MACS=16 -DASSERT_MLS_WIDTH_LINE_ALIGN=ON -DSTACKOVERFLOW_ERROR=OFF -DHARDWARE_TARGET=OFF -DELS_AXI_WIDTH_BITS=256 -DELS_RD_BW_MBPS=196608 -DELS_WR_BW_MBPS=196608 -DCYCLE_TIME_NS=1.0 
2026-10-23 01:08 - DEBUG - sdk - cli - Executing command: make -j8


[SDK-CLI] : Executing on QC-P simulator


2026-10-23 01:11 - DEBUG - sdk - cli - Executing command: ./qwen3_host -c --ddrRdBwTotal 196608.0 --ddrWrBwTotal 196608.0 --ddrAxiWidth 256 --instMemDepth 1310720 --ocmSize 2097152 --cycleTimeNS 1.0 --no-check --ddrRdAvgPct 100 --ddrRdMaxPct 100 --ddrWrAvgPct 100 --ddrWrMaxPct 100 --postKernelFlowTimeoutCycles 4000000 --clusterSize 4 --numClusters 1 
2026-10-23 01:19 - WARNING - epu - core - No profile.json found. Not able to show performance results.
2026-10-23 01:19 - INFO - epu - core - If you would like to see performance results, add profiling statements to your source code.


[SDK-CLI] : Execution completed.

8. Decode the Output and Read the Profile

output_seq.bin holds the generated token ids; the tokenizer turns them back into text. The ISS profile then gives the cycle breakdown of core 0 — compute, MAC, data movement, stalls — and, from the total across all four cores, the per-token throughput. The run above made two passes through the network (a two-token prompt plus one generated token), so the cycles per token are the total divided by two.

!python3 decode_outputs.py
Token ids: [4752]
Generated text:  building
plot_decode_profile("qwen3_QC-P*/**/profile_core0.json", clock_freq_ghz=1.0)
[SDK-CLI] : TotalCycles: 100,604,124
[SDK-CLI] : Executions/second: 10

compute      : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 18.25M
data_array   : ▇▇▇▇▇▇▇ 7.108M
mac          : ▇ 1.077M
data_ocm     : ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 44.978M
data_external: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 29.189M

'qwen3_QC-P_1d0_2MB_4kB_24GBps_24GBps_16_OFF_x1_x4/output/profile_core0.json'
prompt_tokens = int(np.fromfile("prompt_len.bin", dtype=np.int32)[0])
report_decode_throughput(
    "qwen3_QC-P*/**/profile_core*.json",
    passes=prompt_tokens + NEW_TOKENS - 1,
    clock_freq_ghz=1.0,
    silicon_clock_ghz=1.7,
)
Total cycles        :     100,604,136  (2 passes, 4 core(s))
Cycles per token    :      50,302,068
@ 1.0 GHz ISS clock    :       50.3 ms/token  ->  19.9 tokens/s
@ 1.7 GHz silicon clock:       29.6 ms/token  ->  33.8 tokens/s





50302068.0

9. Compare Decode Against ONNX Runtime

A generated token that reads well is not a correctness check. The reference is the same shape-fixed graph before custom-op replacement, run in ONNX Runtime on the host: the prompt is replayed token by token with the same left-padded KV cache and the same per-position RoPE tables the runner consumed, and the logits of the last prompt pass are compared with the ones the runner wrote to output_logits.bin. The gate: the top-1 token must match, at least three of the top-5 tokens must be shared, and the two logit vectors must correlate with Pearson r ≥ 0.7 — so a divergence anywhere in the decoder stack surfaces here, not in a sentence someone has to judge by eye. Fixed-point custom kernels do not reproduce ONNX Runtime's fp32 execution of the quantized graph bit for bit, which is why the correlation threshold, not equality, is the gate.

compare_decode_with_ort(decode_seq_onnx)
Prompt tokens         : [785, 81617]
ONNX Runtime top-5    : [4752, 315, 11, 883, 825]
ISS top-5             : [4752, 16301, 883, 11, 323]
Top-1 match           : True   top-5 overlap: 3/5
Logit Pearson r       : 0.7655   max |diff|: 12.054
PASS: ISS decode matches the ONNX Runtime reference





{'ort_top5': [4752, 315, 11, 883, 825],
 'iss_top5': [4752, 16301, 883, 11, 323],
 'pearson': 0.7654599343142197,
 'top5_overlap': 3}

Summary

ModelQwen3-8B — 36 layers, hidden size 4096, 32 query / 8 KV heads, 151,936-token vocabulary
QuantizationW4A8 SmoothQuant (α = 0.36): INT4 weights packed v8i4, INT8 activations
TargetQC-P × 4 cores, 2 MB OCM per core, 16 MACs/PE, 1 GHz, 24 GB/s DDR per core
DecodeFull autoregressive loop on the ISS — ~18–19 tokens/s average, 20–21 peak; logits match ONNX Runtime (top-1, top-5, Pearson r)
Custom Opsnn::qwen3Attention, nn::gateProj, linalg::channelwiseQuantMatMul

Key takeaways

  1. One export, two graphs. Decode and prefill differ only in their fixed shapes; weights, tensor ranges and custom ops are shared, so the prefill program in qwen3_8b_prefill.ipynb is one shape-fix away from this decode kernel.
  2. Fused attention keeps the KV cache on-chip. nn::qwen3Attention appends to the cache and attends over it inside OCM, so the runner never round-trips cache state through the host between tokens.
  3. INT4 weights are a bandwidth play. Packing eight weights per 32-bit word halves the DDR traffic of an INT8 model per token — the quantity that sets decode speed on a memory-bound 8B model.
  4. Every GPNPU result is checked against a host reference. Decode logits are compared with an ONNX Runtime replay of the same prompt, on top-1, top-5 and Pearson correlation.
  5. sdk source closes the loop. One command compiles the CGC output and a hand-written host runner with Quadric LLVM and executes them on the cycle-accurate multicore ISS.

Citation

@article{qwen3technicalreport,
  title   = {Qwen3 Technical Report},
  author  = {Qwen Team},
  journal = {arXiv preprint arXiv:2505.09388},
  year    = {2025}
}

Sign in to your account

Don't have an account? 
By signing in, you are agreeing to our Terms of Use and Privacy Policy.
Quadric // One architecture. Every algorithm.

Develop.

Simulate.

Profile.

Collaborate.