NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/qwen/qwen3_prefill/qwen3_prefill_all_dec.ipynb.
Qwen3 Prefill - All Decoders CGC Compilation
Why compile all 36 decoders?
This notebook demonstrates the CGC (Chimera Graph Compiler) compilation of the full decoder stack of the Qwen3-8B prefill model. Where the single-decoder notebook validated correctness on a single isolated layer, this notebook scales up to compile all 36 decoder layers in one pass, producing the complete prefill inference graph.
In this notebook we demonstrate:
- Converting ONNX operators to Quadric Custom Ops via the
custom_op_matchscript for efficient execution on the Chimera architecture. - Lowering the resulting custom-op ONNX graph through the Chimera Graph Compiler (CGC) to generate optimized C++ targeting the QC-P hardware.
Target Hardware: QC-P (Quadric Chimera Perform) with 4 cores, 2MB OCM, 16 MACs/PE
Pipeline Overview
+--------------------------------+
| qwen3_8b_prefill_1024_context | Quantized, shape-fixed model from
| .onnx (Prerequisite) | qwen3_single_decoder.ipynb
+----------+---------------------+
|
v
+---------------------+
| Custom Op | Replace ONNX ops with Quadric-optimized
| Matching | custom kernels (attention, gate/up proj,
| | RMSNorm, matmul, etc.)
+----------+----------+
|
v
+---------------------+
| CGC Compile | Lower the custom-op graph to C++ for
| (Graph -> C++) | QC-P hardware; schedule ops and optimize
| | memory layout within 2MB OCM constraint
+---------------------+
1. Setup and Model Download
The starting point of this notebook is qwen3_8b_prefill_1024_context.onnx — the shape-fixed, quantized Qwen3-8B prefill model (sequence length 1024). The qwen3_single_decoder.ipynb notebook explains in detail the pre-processing steps needed to prouce this graph. Here we download it directly from S3, so the shape-fixing step does not need to be repeated.
Download the pre-built prefill model from S3:
qwen3_8b_prefill_1024_context.onnx- the pre-built prefill model (seq_len=1024)27fe33ec-1773-11f1-b0a3-ea22fe300241- the external weights data fileqwen3_0_36_tranges- the tranges for the quantized model
import os
from examples.models.zoo.zoo_utils import download_file
## Base URL for model files
base_url = "https://sdk-cli-models.s3.us-east-2.amazonaws.com/"
## File names
prefill_onnx_filename = "qwen3_8b_prefill_1024_context.onnx"
prefill_data_filename = "27fe33ec-1773-11f1-b0a3-ea22fe300241"
prefill_tranges_filename = "qwen_0_36.onnx.tranges"
current_dir = os.getcwd()
prefill_onnx_path = os.path.join(current_dir, prefill_onnx_filename)
prefill_data_path = os.path.join(current_dir, prefill_data_filename)
prefill_tranges_path = os.path.join(current_dir, prefill_tranges_filename)
## Download files
print(f"Downloading {prefill_onnx_filename}...")
download_file(base_url + prefill_onnx_filename, prefill_onnx_path)
print(f" ✓ Downloaded {prefill_onnx_filename}")
print(f"Downloading {prefill_data_filename}...")
download_file(base_url + prefill_data_filename, prefill_data_path)
print(f" ✓ Downloaded {prefill_data_filename}")
print(f"Downloading {prefill_tranges_filename}...")
download_file(base_url + prefill_tranges_filename, prefill_tranges_path)
print(f" ✓ Downloaded {prefill_tranges_filename}")
print("\nAll prefill model files downloaded successfully!")
Downloading qwen3_8b_prefill_1024_context.onnx...
✓ Downloaded qwen3_8b_prefill_1024_context.onnx
Downloading 27fe33ec-1773-11f1-b0a3-ea22fe300241...
✓ Downloaded 27fe33ec-1773-11f1-b0a3-ea22fe300241
Downloading qwen_0_36.onnx.tranges...
✓ Downloaded qwen_0_36.onnx.tranges
All prefill model files downloaded successfully!
2. Custom Op Replacement
Replace standard ONNX operators across all 36 decoder layers with Quadric-optimized custom kernels. This is the same replacement applied in qwen3_single_decoder.ipynb, now applied uniformly to the full decoder stack.
The qwen3_custom_op_replacer script pattern-matches subgraphs in all_dec.onnx and substitutes them with custom ops. For the Qwen3-8B prefill at sequence length 1024, the key replacements per decoder are:
| ONNX Subgraph | Custom Op |
|---|---|
| QKV projections + RoPE + scaled dot-product attention | qwen3PrefillAttention |
| Gate proj x Up proj -> SiLU -> Down proj (FFN) | qwen3PrefillGateUpProjection |
| RMSNorm (pre-attention and pre-FFN) | Fused into adjacent custom ops |
Inputs:
qwen3_8b_prefill_1024_context.onnx- the 36-decoder graph from Step 1qwen_0_36.onnx.tranges- quantization tensor ranges used to parameterize the fixed-point custom op arguments (e.g.FixedPoint32scale factors seen in the generated C++)
Output: qwen3_custom_op_all_dec.onnx - the custom-op graph ready for CGC compilation
from custom_op_match import qwen3_custom_op_replacer
## QWEN3 8B architecture parameters
num_heads = 8
embed_dim = 4096
seq_length = 1024
num_decoders = 36
input_onnx_path = "qwen3_8b_prefill_1024_context.onnx"
custom_onnx_path = "qwen3_custom_op_all_dec.onnx"
tranges_path = "qwen_0_36.onnx.tranges"
qwen3_custom_op_replacer(
input_onnx_path,
custom_onnx_path,
tranges_path,
num_heads=num_heads,
embed_dim=embed_dim,
seq_length=seq_length,
num_decoders=num_decoders,
include_lm_head=True,
)
Warning: /model/layers.0/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.1/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.2/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.3/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.4/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.5/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.6/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.7/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.8/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.9/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.10/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.11/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.12/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.13/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.14/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.15/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.16/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.17/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.18/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.19/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.20/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.21/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.22/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.23/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.24/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.25/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.26/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.27/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.28/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.29/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.30/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.31/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.32/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.33/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.34/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Warning: /model/layers.35/self_attn/o_proj/MatMul_smooth_output_QuantizeLinear_Output not found in tranges, using default frac_bits=15
Processing matmul ops for decoder 0...
Decoder 0 gate_up frac bits:
gate_combined_scale=5.489234e-04 frac_bits=41
up_combined_scale=3.307317e-04 frac_bits=42
gate_im_frac_bits=31
up_im_frac_bits=31
gate_output_frac_bits=28
up_output_frac_bits=28
Processing matmul ops for decoder 1...
Decoder 1 gate_up frac bits:
gate_combined_scale=7.408716e-03 frac_bits=38
up_combined_scale=3.896641e-03 frac_bits=39
gate_im_frac_bits=28
up_im_frac_bits=29
gate_output_frac_bits=27
up_output_frac_bits=26
Processing matmul ops for decoder 2...
Decoder 2 gate_up frac bits:
gate_combined_scale=5.992835e-02 frac_bits=35
up_combined_scale=5.992835e-02 frac_bits=35
gate_im_frac_bits=27
up_im_frac_bits=27
gate_output_frac_bits=24
up_output_frac_bits=24
Processing matmul ops for decoder 3...
Decoder 3 gate_up frac bits:
gate_combined_scale=1.153846e-02 frac_bits=37
up_combined_scale=9.374998e-03 frac_bits=37
gate_im_frac_bits=28
up_im_frac_bits=28
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 4...
Decoder 4 gate_up frac bits:
gate_combined_scale=1.012771e-02 frac_bits=37
up_combined_scale=1.079110e-02 frac_bits=37
gate_im_frac_bits=28
up_im_frac_bits=28
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 5...
Decoder 5 gate_up frac bits:
gate_combined_scale=1.041709e-02 frac_bits=37
up_combined_scale=5.855937e-03 frac_bits=38
gate_im_frac_bits=28
up_im_frac_bits=29
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 6...
Decoder 6 gate_up frac bits:
gate_combined_scale=6.929483e-02 frac_bits=34
up_combined_scale=7.354605e-02 frac_bits=34
gate_im_frac_bits=27
up_im_frac_bits=27
gate_output_frac_bits=24
up_output_frac_bits=24
Processing matmul ops for decoder 7...
Decoder 7 gate_up frac bits:
gate_combined_scale=7.519060e-03 frac_bits=38
up_combined_scale=1.755872e-03 frac_bits=40
gate_im_frac_bits=28
up_im_frac_bits=29
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 8...
Decoder 8 gate_up frac bits:
gate_combined_scale=2.171312e-03 frac_bits=39
up_combined_scale=1.607593e-03 frac_bits=40
gate_im_frac_bits=29
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=28
Processing matmul ops for decoder 9...
Decoder 9 gate_up frac bits:
gate_combined_scale=2.659734e-03 frac_bits=39
up_combined_scale=2.695962e-03 frac_bits=39
gate_im_frac_bits=29
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=28
Processing matmul ops for decoder 10...
Decoder 10 gate_up frac bits:
gate_combined_scale=2.187623e-03 frac_bits=39
up_combined_scale=1.581833e-03 frac_bits=40
gate_im_frac_bits=29
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 11...
Decoder 11 gate_up frac bits:
gate_combined_scale=1.958575e-03 frac_bits=39
up_combined_scale=1.510549e-03 frac_bits=40
gate_im_frac_bits=29
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 12...
Decoder 12 gate_up frac bits:
gate_combined_scale=1.847970e-03 frac_bits=40
up_combined_scale=2.534580e-03 frac_bits=39
gate_im_frac_bits=29
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 13...
Decoder 13 gate_up frac bits:
gate_combined_scale=1.677463e-03 frac_bits=40
up_combined_scale=3.220730e-03 frac_bits=39
gate_im_frac_bits=30
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=25
Processing matmul ops for decoder 14...
Decoder 14 gate_up frac bits:
gate_combined_scale=2.226458e-03 frac_bits=39
up_combined_scale=2.970458e-03 frac_bits=39
gate_im_frac_bits=30
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=25
Processing matmul ops for decoder 15...
Decoder 15 gate_up frac bits:
gate_combined_scale=2.298211e-03 frac_bits=39
up_combined_scale=6.424773e-03 frac_bits=38
gate_im_frac_bits=29
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=25
Processing matmul ops for decoder 16...
Decoder 16 gate_up frac bits:
gate_combined_scale=8.712117e-03 frac_bits=37
up_combined_scale=9.366387e-03 frac_bits=37
gate_im_frac_bits=28
up_im_frac_bits=28
gate_output_frac_bits=23
up_output_frac_bits=24
Processing matmul ops for decoder 17...
Decoder 17 gate_up frac bits:
gate_combined_scale=2.006680e-03 frac_bits=39
up_combined_scale=1.808745e-03 frac_bits=40
gate_im_frac_bits=30
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=25
Processing matmul ops for decoder 18...
Decoder 18 gate_up frac bits:
gate_combined_scale=2.155281e-03 frac_bits=39
up_combined_scale=1.964622e-03 frac_bits=39
gate_im_frac_bits=29
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=26
Processing matmul ops for decoder 19...
Decoder 19 gate_up frac bits:
gate_combined_scale=1.855934e-03 frac_bits=40
up_combined_scale=2.282373e-03 frac_bits=39
gate_im_frac_bits=30
up_im_frac_bits=30
gate_output_frac_bits=27
up_output_frac_bits=28
Processing matmul ops for decoder 20...
Decoder 20 gate_up frac bits:
gate_combined_scale=1.904722e-03 frac_bits=40
up_combined_scale=1.601911e-03 frac_bits=40
gate_im_frac_bits=29
up_im_frac_bits=29
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 21...
Decoder 21 gate_up frac bits:
gate_combined_scale=2.035758e-03 frac_bits=39
up_combined_scale=2.340634e-03 frac_bits=39
gate_im_frac_bits=30
up_im_frac_bits=29
gate_output_frac_bits=27
up_output_frac_bits=27
Processing matmul ops for decoder 22...
Decoder 22 gate_up frac bits:
gate_combined_scale=2.405553e-03 frac_bits=39
up_combined_scale=2.320086e-03 frac_bits=39
gate_im_frac_bits=30
up_im_frac_bits=30
gate_output_frac_bits=26
up_output_frac_bits=27
Processing matmul ops for decoder 23...
Decoder 23 gate_up frac bits:
gate_combined_scale=2.210377e-03 frac_bits=39
up_combined_scale=2.788087e-03 frac_bits=39
gate_im_frac_bits=30
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=27
Processing matmul ops for decoder 24...
Decoder 24 gate_up frac bits:
gate_combined_scale=3.017297e-03 frac_bits=39
up_combined_scale=3.114498e-03 frac_bits=39
gate_im_frac_bits=29
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=27
Processing matmul ops for decoder 25...
Decoder 25 gate_up frac bits:
gate_combined_scale=3.542291e-03 frac_bits=39
up_combined_scale=3.030597e-03 frac_bits=39
gate_im_frac_bits=29
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 26...
Decoder 26 gate_up frac bits:
gate_combined_scale=2.672773e-03 frac_bits=39
up_combined_scale=4.103271e-03 frac_bits=38
gate_im_frac_bits=29
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=27
Processing matmul ops for decoder 27...
Decoder 27 gate_up frac bits:
gate_combined_scale=3.190623e-03 frac_bits=39
up_combined_scale=4.202626e-03 frac_bits=38
gate_im_frac_bits=29
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=27
Processing matmul ops for decoder 28...
Decoder 28 gate_up frac bits:
gate_combined_scale=4.199372e-03 frac_bits=38
up_combined_scale=4.026828e-03 frac_bits=38
gate_im_frac_bits=29
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 29...
Decoder 29 gate_up frac bits:
gate_combined_scale=3.686063e-03 frac_bits=39
up_combined_scale=4.187949e-03 frac_bits=38
gate_im_frac_bits=30
up_im_frac_bits=29
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 30...
Decoder 30 gate_up frac bits:
gate_combined_scale=4.159406e-03 frac_bits=38
up_combined_scale=7.111935e-03 frac_bits=38
gate_im_frac_bits=29
up_im_frac_bits=28
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 31...
Decoder 31 gate_up frac bits:
gate_combined_scale=4.884809e-03 frac_bits=38
up_combined_scale=8.509487e-03 frac_bits=37
gate_im_frac_bits=29
up_im_frac_bits=28
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 32...
Decoder 32 gate_up frac bits:
gate_combined_scale=5.781837e-03 frac_bits=38
up_combined_scale=7.929652e-03 frac_bits=37
gate_im_frac_bits=28
up_im_frac_bits=28
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 33...
Decoder 33 gate_up frac bits:
gate_combined_scale=9.199777e-03 frac_bits=37
up_combined_scale=2.238982e-02 frac_bits=36
gate_im_frac_bits=28
up_im_frac_bits=28
gate_output_frac_bits=26
up_output_frac_bits=26
Processing matmul ops for decoder 34...
Decoder 34 gate_up frac bits:
gate_combined_scale=4.481895e-02 frac_bits=35
up_combined_scale=6.740215e-02 frac_bits=34
gate_im_frac_bits=27
up_im_frac_bits=27
gate_output_frac_bits=25
up_output_frac_bits=24
Processing matmul ops for decoder 35...
Decoder 35 gate_up frac bits:
gate_combined_scale=1.067153e-01 frac_bits=34
up_combined_scale=1.776727e-01 frac_bits=33
gate_im_frac_bits=25
up_im_frac_bits=25
gate_output_frac_bits=24
up_output_frac_bits=23
Successfully saved modified model to /quadric/sdk-cli/examples/models/qwen/qwen3_prefill/qwen3_custom_op_all_dec.onnx
3. CGC Compilation (ONNX -> C++)
The Chimera Graph Compiler (CGC) lowers qwen3_custom_op_all_dec.onnx into optimized C++ targeting the QC-P hardware.
Hardware Configuration:
- Product: QC-P (Quadric Chimera Perform)
- OCM Size: 2MB on-chip memory
- MACs per PE: 16 multiply-accumulate units per processing element
- Target Language: QIL (Quadric Intermediate Language)
3.1 Increase Stack Size
The 36-decoder custom-op graph is large enough to exhaust the default system stack during compilation. Increase the limit before running CGC.
import resource
## Increase stack size for large model compilation
resource.setrlimit(resource.RLIMIT_STACK, (32768 * 1024, 32768 * 1024))
3.2 Run CGC Compilation
from tvm.contrib.epu.chimera_job.chimera_job import ChimeraJob
from tvm.contrib.epu.chimera_job.hw_config import HWConfig
## Configure hardware target
num_decoders = 36
custom_onnx_path = "qwen3_custom_op_all_dec.onnx"
tranges_path = "qwen_0_36.onnx.tranges"
hw_config = HWConfig(product="QC-P", ocm_size="2MB", macs_per_pe=16)
## Specify I/O tensors to ignore during compilation (KV cache management)
io_to_ignore = ["attention_mask"]
for i in range(num_decoders):
io_to_ignore.extend(
[
f"present.{i}.key",
f"present.{i}.value",
f"past_key_values.{i}.key",
f"past_key_values.{i}.value",
]
)
## Create and run CGC compilation job
cgc_job = ChimeraJob(
custom_onnx_path,
hw_config=hw_config,
trange_file=tranges_path,
target_lang="QIL", # Quadric Intermediate Language
io_to_ignore=io_to_ignore,
)
print("Starting CGC compilation...")
cgc_job.compile()
print("\nCompilation complete!")
print(cgc_job)
Starting CGC compilation...
2026-06-19 05:12 - INFO - epu - chimera_job - START==================================onnx_ingest
2026-06-19 05:12 - INFO - epu - chimera_job - Numerical ranges provided
/usr/local/lib/python3.10/dist-packages/tvm/relay/frontend/onnx.py:6270: UserWarning: This protobuf of onnx model is too large (>2GB). Call check_model with model path instead.
warnings.warn(str(e))
2026-06-19 05:13 - INFO - epu - codegen - START===============================optimize_relay
2026-06-19 05:13 - INFO - epu - codegen - START====================quantize_to_cpu_runnable_fx
2026-06-19 05:13 - INFO - epu - fx -
Source name Op Output 0 Range Output 0 Frac Bits
--------------------------------------------------------------- ----------------------------- ---------------------- --------------------
/model/embed_tokens/Gather contrib.epu.embedding [-0.628906f, 0.8125f] 31
/model/layers.0/input_layernorm/Mul_1 contrib.epu.rms_norm [-0.359118f, 0.37561f] 31
CustomOp/linalg::channelwiseQuantMatMul<43>2 contrib.epu.quadric_custom_op [-2.25614f, 3.85482f] 28
CustomOp/add<MultiCoreMode::All>2 contrib.epu.quadric_custom_op [-2.30253f, 4.25466f] 28
/model/layers.0/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-1.23024f, 1.59211f] 29
CustomOp/nn::qwen3PrefillGateUpProjection<31,31,28,28,41,42>0 contrib.epu.quadric_custom_op [-11.3828f, 9.95375f] 27
CustomOp/linalg::channelwiseQuantMatMul<38>1 contrib.epu.quadric_custom_op [-5.18762f, 16.7897f] 26
CustomOp/add<MultiCoreMode::All>1 contrib.epu.quadric_custom_op [-6.67269f, 19.2419f] 26
/model/layers.1/input_layernorm/Mul_1 contrib.epu.rms_norm [-1.30106f, 1.14032f] 30
CustomOp/linalg::channelwiseQuantMatMul<43>5 contrib.epu.quadric_custom_op [-1.41003f, 1.53298f] 29
CustomOp/add<MultiCoreMode::All>5 contrib.epu.quadric_custom_op [-6.78994f, 19.2634f] 26
/model/layers.1/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-51.933f, 47.6116f] 25
CustomOp/nn::qwen3PrefillGateUpProjection<28,29,27,26,38,39>3 contrib.epu.quadric_custom_op [-22.4947f, 19.6069f] 26
CustomOp/linalg::channelwiseQuantMatMul<37>4 contrib.epu.quadric_custom_op [-12.4679f, 38.1379f] 25
CustomOp/add<MultiCoreMode::All>4 contrib.epu.quadric_custom_op [-18.1108f, 57.4013f] 25
/model/layers.2/input_layernorm/Mul_1 contrib.epu.rms_norm [-0.862898f, 1.28555f] 29
CustomOp/linalg::channelwiseQuantMatMul<42>8 contrib.epu.quadric_custom_op [-1.03252f, 1.31911f] 29
CustomOp/add<MultiCoreMode::All>8 contrib.epu.quadric_custom_op [-18.1085f, 57.4384f] 25
/model/layers.2/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-82.5232f, 81.3135f] 24
CustomOp/nn::qwen3PrefillGateUpProjection<27,27,24,24,35,35>6 contrib.epu.quadric_custom_op [-26.306f, 24.5606f] 26
CustomOp/linalg::channelwiseQuantMatMul<37>7 contrib.epu.quadric_custom_op [-10.8904f, 26.5761f] 26
CustomOp/add<MultiCoreMode::All>7 contrib.epu.quadric_custom_op [-23.2706f, 81.9635f] 24
/model/layers.3/input_layernorm/Mul_1 contrib.epu.rms_norm [-1.24747f, 3.00854f] 28
CustomOp/linalg::channelwiseQuantMatMul<42>11 contrib.epu.quadric_custom_op [-1.84038f, 1.97427f] 29
CustomOp/add<MultiCoreMode::All>11 contrib.epu.quadric_custom_op [-23.1594f, 81.7177f] 24
/model/layers.3/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-53.3995f, 62.6125f] 25
CustomOp/nn::qwen3PrefillGateUpProjection<28,28,26,26,37,37>9 contrib.epu.quadric_custom_op [-23.0629f, 23.7062f] 26
CustomOp/linalg::channelwiseQuantMatMul<39>10 contrib.epu.quadric_custom_op [-7.28584f, 7.4731f] 27
CustomOp/add<MultiCoreMode::All>10 contrib.epu.quadric_custom_op [-23.3718f, 85.3067f] 24
/model/layers.4/input_layernorm/Mul_1 contrib.epu.rms_norm [-1.80589f, 2.00268f] 29
CustomOp/linalg::channelwiseQuantMatMul<42>14 contrib.epu.quadric_custom_op [-3.53964f, 2.62418f] 29
CustomOp/add<MultiCoreMode::All>14 contrib.epu.quadric_custom_op [-23.2038f, 85.5677f] 24
/model/layers.4/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-32.953f, 41.0647f] 25
CustomOp/nn::qwen3PrefillGateUpProjection<28,28,27,27,37,37>12 contrib.epu.quadric_custom_op [-20.2387f, 20.039f] 26
CustomOp/linalg::channelwiseQuantMatMul<39>13 contrib.epu.quadric_custom_op [-6.69227f, 12.1684f] 27
CustomOp/add<MultiCoreMode::All>13 contrib.epu.quadric_custom_op [-22.4392f, 87.0669f] 24
/model/layers.5/input_layernorm/Mul_1 contrib.epu.rms_norm [-2.14681f, 3.19879f] 28
CustomOp/linalg::channelwiseQuantMatMul<42>17 contrib.epu.quadric_custom_op [-6.97809f, 4.76181f] 28
CustomOp/add<MultiCoreMode::All>17 contrib.epu.quadric_custom_op [-21.9713f, 84.54f] 24
/model/layers.5/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-24.6416f, 19.9332f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<28,29,27,27,37,38>15 contrib.epu.quadric_custom_op [-18.2442f, 18.1234f] 26
CustomOp/linalg::channelwiseQuantMatMul<38>16 contrib.epu.quadric_custom_op [-37.2974f, 18.5171f] 25
CustomOp/add<MultiCoreMode::All>16 contrib.epu.quadric_custom_op [-18.6574f, 63.1892f] 24
/model/layers.6/input_layernorm/Mul_1 contrib.epu.rms_norm [-1.99346f, 7.23117f] 27
CustomOp/linalg::channelwiseQuantMatMul<40>20 contrib.epu.quadric_custom_op [-31.7815f, 8.99221f] 26
CustomOp/add<MultiCoreMode::All>20 contrib.epu.quadric_custom_op [-12.6824f, 38.3116f] 25
/model/layers.6/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-9.84406f, 199.177f] 23
CustomOp/nn::qwen3PrefillGateUpProjection<27,27,24,24,34,34>18 contrib.epu.quadric_custom_op [-1282.21f, 4841.01f] 18
CustomOp/linalg::channelwiseQuantMatMul<27>19 contrib.epu.quadric_custom_op [-2118.69f, 9638.07f] 17
CustomOp/add<MultiCoreMode::All>19 contrib.epu.quadric_custom_op [-2121.44f, 9654.1f] 17
/model/layers.7/input_layernorm/Mul_1 contrib.epu.rms_norm [-5.1855f, 10.441f] 27
CustomOp/linalg::channelwiseQuantMatMul<41>23 contrib.epu.quadric_custom_op [-4.27446f, 3.6406f] 28
CustomOp/add<MultiCoreMode::All>23 contrib.epu.quadric_custom_op [-2121.23f, 9654.21f] 17
/model/layers.7/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-6.89828f, 54.4274f] 25
CustomOp/nn::qwen3PrefillGateUpProjection<28,29,27,27,38,40>21 contrib.epu.quadric_custom_op [-16.9021f, 17.2346f] 26
CustomOp/linalg::channelwiseQuantMatMul<37>22 contrib.epu.quadric_custom_op [-6.76926f, 12.1386f] 27
CustomOp/add<MultiCoreMode::All>22 contrib.epu.quadric_custom_op [-2121.18f, 9654.34f] 17
/model/layers.8/input_layernorm/Mul_1 contrib.epu.rms_norm [-4.59068f, 9.9222f] 27
CustomOp/linalg::channelwiseQuantMatMul<41>26 contrib.epu.quadric_custom_op [-3.77026f, 7.3769f] 27
CustomOp/add<MultiCoreMode::All>26 contrib.epu.quadric_custom_op [-2120.87f, 9653.89f] 17
/model/layers.8/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-7.8127f, 6.98587f] 28
CustomOp/nn::qwen3PrefillGateUpProjection<29,30,27,28,39,40>24 contrib.epu.quadric_custom_op [-15.8356f, 18.0968f] 26
CustomOp/linalg::channelwiseQuantMatMul<38>25 contrib.epu.quadric_custom_op [-10.3458f, 12.5126f] 27
CustomOp/add<MultiCoreMode::All>25 contrib.epu.quadric_custom_op [-2120.62f, 9654.73f] 17
/model/layers.9/input_layernorm/Mul_1 contrib.epu.rms_norm [-4.52299f, 10.3721f] 27
CustomOp/linalg::channelwiseQuantMatMul<39>29 contrib.epu.quadric_custom_op [-7.31632f, 7.46158f] 27
CustomOp/add<MultiCoreMode::All>29 contrib.epu.quadric_custom_op [-2120.34f, 9654.1f] 17
/model/layers.9/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-8.45564f, 8.75462f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,30,27,28,39,39>27 contrib.epu.quadric_custom_op [-15.1261f, 14.7165f] 27
CustomOp/linalg::channelwiseQuantMatMul<38>28 contrib.epu.quadric_custom_op [-9.86182f, 8.2895f] 27
CustomOp/add<MultiCoreMode::All>28 contrib.epu.quadric_custom_op [-2120.34f, 9656.53f] 17
/model/layers.10/input_layernorm/Mul_1 contrib.epu.rms_norm [-7.60531f, 17.5566f] 26
CustomOp/linalg::channelwiseQuantMatMul<40>32 contrib.epu.quadric_custom_op [-7.399f, 5.84284f] 28
CustomOp/add<MultiCoreMode::All>32 contrib.epu.quadric_custom_op [-2119.36f, 9655.58f] 17
/model/layers.10/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-8.33987f, 9.41487f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,30,27,27,39,40>30 contrib.epu.quadric_custom_op [-15.7846f, 20.8058f] 26
CustomOp/linalg::channelwiseQuantMatMul<37>31 contrib.epu.quadric_custom_op [-5.03255f, 18.6078f] 26
CustomOp/add<MultiCoreMode::All>31 contrib.epu.quadric_custom_op [-2119.79f, 9656.97f] 17
/model/layers.11/input_layernorm/Mul_1 contrib.epu.rms_norm [-5.77417f, 10.4802f] 27
CustomOp/linalg::channelwiseQuantMatMul<40>35 contrib.epu.quadric_custom_op [-7.13255f, 7.60828f] 27
CustomOp/add<MultiCoreMode::All>35 contrib.epu.quadric_custom_op [-2118.77f, 9655.72f] 17
/model/layers.11/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-9.30534f, 7.62255f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,30,27,27,39,40>33 contrib.epu.quadric_custom_op [-12.4377f, 12.5137f] 27
CustomOp/linalg::channelwiseQuantMatMul<38>34 contrib.epu.quadric_custom_op [-7.59268f, 11.5153f] 27
CustomOp/add<MultiCoreMode::All>34 contrib.epu.quadric_custom_op [-2118.11f, 9657.75f] 17
/model/layers.12/input_layernorm/Mul_1 contrib.epu.rms_norm [-6.76928f, 11.1257f] 27
CustomOp/linalg::channelwiseQuantMatMul<40>38 contrib.epu.quadric_custom_op [-7.8656f, 13.7375f] 27
CustomOp/add<MultiCoreMode::All>38 contrib.epu.quadric_custom_op [-2117.66f, 9657.01f] 17
/model/layers.12/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-10.1594f, 7.15519f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,30,27,27,40,39>36 contrib.epu.quadric_custom_op [-60.1034f, 12.2375f] 25
CustomOp/linalg::channelwiseQuantMatMul<37>37 contrib.epu.quadric_custom_op [-12.1023f, 17.851f] 26
CustomOp/add<MultiCoreMode::All>37 contrib.epu.quadric_custom_op [-2118.72f, 9659.33f] 17
/model/layers.13/input_layernorm/Mul_1 contrib.epu.rms_norm [-5.04587f, 9.31552f] 27
CustomOp/linalg::channelwiseQuantMatMul<40>41 contrib.epu.quadric_custom_op [-5.54142f, 8.70166f] 27
CustomOp/add<MultiCoreMode::All>41 contrib.epu.quadric_custom_op [-2117.86f, 9659.14f] 17
/model/layers.13/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-11.9317f, 6.23108f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<30,29,26,25,40,39>39 contrib.epu.quadric_custom_op [-16.9783f, 33.7621f] 25
CustomOp/linalg::channelwiseQuantMatMul<38>40 contrib.epu.quadric_custom_op [-11.2142f, 11.9456f] 27
CustomOp/add<MultiCoreMode::All>40 contrib.epu.quadric_custom_op [-2120.76f, 9664.39f] 17
/model/layers.14/input_layernorm/Mul_1 contrib.epu.rms_norm [-6.68111f, 13.0119f] 27
CustomOp/linalg::channelwiseQuantMatMul<40>44 contrib.epu.quadric_custom_op [-8.29877f, 19.2297f] 26
CustomOp/add<MultiCoreMode::All>44 contrib.epu.quadric_custom_op [-2120.07f, 9663.34f] 17
/model/layers.14/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-14.412f, 6.25216f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<30,29,26,25,39,39>42 contrib.epu.quadric_custom_op [-84.1848f, 13.7615f] 24
CustomOp/linalg::channelwiseQuantMatMul<36>43 contrib.epu.quadric_custom_op [-18.0127f, 15.0041f] 26
CustomOp/add<MultiCoreMode::All>43 contrib.epu.quadric_custom_op [-2123.29f, 9666.16f] 17
/model/layers.15/input_layernorm/Mul_1 contrib.epu.rms_norm [-7.40155f, 13.516f] 27
CustomOp/linalg::channelwiseQuantMatMul<39>47 contrib.epu.quadric_custom_op [-6.83366f, 13.2874f] 27
CustomOp/add<MultiCoreMode::All>47 contrib.epu.quadric_custom_op [-2122.7f, 9664.69f] 17
/model/layers.15/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-16.0488f, 6.0668f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<29,29,26,25,39,38>45 contrib.epu.quadric_custom_op [-27.8356f, 48.4799f] 25
CustomOp/linalg::channelwiseQuantMatMul<36>46 contrib.epu.quadric_custom_op [-16.8693f, 20.4889f] 26
CustomOp/add<MultiCoreMode::All>46 contrib.epu.quadric_custom_op [-2123.21f, 9668.25f] 17
/model/layers.16/input_layernorm/Mul_1 contrib.epu.rms_norm [-8.15026f, 15.6546f] 26
CustomOp/linalg::channelwiseQuantMatMul<38>50 contrib.epu.quadric_custom_op [-39.5976f, 22.6377f] 25
CustomOp/add<MultiCoreMode::All>50 contrib.epu.quadric_custom_op [-2122.92f, 9666.15f] 17
/model/layers.16/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-36.8359f, 6.53895f] 25
CustomOp/nn::qwen3PrefillGateUpProjection<28,28,23,24,37,37>48 contrib.epu.quadric_custom_op [-191.33f, 4661.68f] 18
CustomOp/linalg::channelwiseQuantMatMul<27>49 contrib.epu.quadric_custom_op [-1202.39f, 13581.2f] 17
CustomOp/add<MultiCoreMode::All>49 contrib.epu.quadric_custom_op [-2130f, 13602.6f] 17
/model/layers.17/input_layernorm/Mul_1 contrib.epu.rms_norm [-7.88683f, 13.0965f] 27
CustomOp/linalg::channelwiseQuantMatMul<37>53 contrib.epu.quadric_custom_op [-14.5229f, 61.5807f] 25
CustomOp/add<MultiCoreMode::All>53 contrib.epu.quadric_custom_op [-2129.24f, 13637f] 17
/model/layers.17/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-12.2827f, 6.15562f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<30,30,27,25,39,40>51 contrib.epu.quadric_custom_op [-13.8208f, 9.58101f] 27
CustomOp/linalg::channelwiseQuantMatMul<38>52 contrib.epu.quadric_custom_op [-21.4484f, 22.413f] 26
CustomOp/add<MultiCoreMode::All>52 contrib.epu.quadric_custom_op [-2128.21f, 13636.9f] 17
/model/layers.18/input_layernorm/Mul_1 contrib.epu.rms_norm [-10.6242f, 15.4063f] 26
CustomOp/linalg::channelwiseQuantMatMul<40>56 contrib.epu.quadric_custom_op [-11.2483f, 15.3732f] 26
CustomOp/add<MultiCoreMode::All>56 contrib.epu.quadric_custom_op [-2127.25f, 13638.1f] 17
/model/layers.18/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-11.7031f, 6.12549f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,30,27,26,39,39>54 contrib.epu.quadric_custom_op [-9.16472f, 64.1296f] 24
CustomOp/linalg::channelwiseQuantMatMul<35>55 contrib.epu.quadric_custom_op [-27.1561f, 62.6349f] 25
CustomOp/add<MultiCoreMode::All>55 contrib.epu.quadric_custom_op [-2127.74f, 13638.1f] 17
/model/layers.19/input_layernorm/Mul_1 contrib.epu.rms_norm [-16.3269f, 23.5058f] 26
CustomOp/linalg::channelwiseQuantMatMul<39>59 contrib.epu.quadric_custom_op [-9.69087f, 19.3656f] 26
CustomOp/add<MultiCoreMode::All>59 contrib.epu.quadric_custom_op [-2127.19f, 13637.7f] 17
/model/layers.19/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-11.4451f, 6.57414f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<30,30,27,28,40,39>57 contrib.epu.quadric_custom_op [-10.5208f, 11.7491f] 27
CustomOp/linalg::channelwiseQuantMatMul<36>58 contrib.epu.quadric_custom_op [-28.7991f, 32.1844f] 25
CustomOp/add<MultiCoreMode::All>58 contrib.epu.quadric_custom_op [-2127.2f, 13637.6f] 17
/model/layers.20/input_layernorm/Mul_1 contrib.epu.rms_norm [-17.2382f, 20.6114f] 26
CustomOp/linalg::channelwiseQuantMatMul<40>62 contrib.epu.quadric_custom_op [-11.7166f, 31.5278f] 25
CustomOp/add<MultiCoreMode::All>62 contrib.epu.quadric_custom_op [-2126.62f, 13634.6f] 17
/model/layers.20/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-12.2707f, 7.12023f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,29,27,27,40,40>60 contrib.epu.quadric_custom_op [-18.5987f, 19.8616f] 26
CustomOp/linalg::channelwiseQuantMatMul<38>61 contrib.epu.quadric_custom_op [-16.8789f, 16.8205f] 26
CustomOp/add<MultiCoreMode::All>61 contrib.epu.quadric_custom_op [-2125.87f, 13634.7f] 17
/model/layers.21/input_layernorm/Mul_1 contrib.epu.rms_norm [-23.1988f, 24.3195f] 26
CustomOp/linalg::channelwiseQuantMatMul<39>65 contrib.epu.quadric_custom_op [-8.88642f, 21.276f] 26
CustomOp/add<MultiCoreMode::All>65 contrib.epu.quadric_custom_op [-2124.82f, 13636.5f] 17
/model/layers.21/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-10.9728f, 7.28116f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<30,29,27,27,39,39>63 contrib.epu.quadric_custom_op [-41.594f, 53.735f] 25
CustomOp/linalg::channelwiseQuantMatMul<37>64 contrib.epu.quadric_custom_op [-32.024f, 16.017f] 25
CustomOp/add<MultiCoreMode::All>64 contrib.epu.quadric_custom_op [-2124.91f, 13636.6f] 17
/model/layers.22/input_layernorm/Mul_1 contrib.epu.rms_norm [-30.8369f, 32.7385f] 25
CustomOp/linalg::channelwiseQuantMatMul<39>68 contrib.epu.quadric_custom_op [-12.5842f, 31.0796f] 25
CustomOp/add<MultiCoreMode::All>68 contrib.epu.quadric_custom_op [-2124.1f, 13635.5f] 17
/model/layers.22/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-10.0459f, 8.97793f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<30,30,26,27,39,39>66 contrib.epu.quadric_custom_op [-42.2895f, 37.618f] 25
CustomOp/linalg::channelwiseQuantMatMul<37>67 contrib.epu.quadric_custom_op [-24.8563f, 17.5565f] 26
CustomOp/add<MultiCoreMode::All>67 contrib.epu.quadric_custom_op [-2123.61f, 13635.4f] 17
/model/layers.23/input_layernorm/Mul_1 contrib.epu.rms_norm [-32.5652f, 31.0209f] 25
CustomOp/linalg::channelwiseQuantMatMul<39>71 contrib.epu.quadric_custom_op [-8.13639f, 35.0445f] 25
CustomOp/add<MultiCoreMode::All>71 contrib.epu.quadric_custom_op [-2123.06f, 13632.6f] 17
/model/layers.23/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-10.1758f, 10.4433f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<30,29,26,27,39,39>69 contrib.epu.quadric_custom_op [-64.6526f, 60.3269f] 24
CustomOp/linalg::channelwiseQuantMatMul<37>70 contrib.epu.quadric_custom_op [-24.6908f, 31.6436f] 25
CustomOp/add<MultiCoreMode::All>70 contrib.epu.quadric_custom_op [-2122.79f, 13632.6f] 17
/model/layers.24/input_layernorm/Mul_1 contrib.epu.rms_norm [-50.8646f, 39.9522f] 25
CustomOp/linalg::channelwiseQuantMatMul<38>74 contrib.epu.quadric_custom_op [-16.7965f, 31.3655f] 25
CustomOp/add<MultiCoreMode::All>74 contrib.epu.quadric_custom_op [-2123.99f, 13635.1f] 17
/model/layers.24/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-10.2601f, 12.5924f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,29,26,27,39,39>72 contrib.epu.quadric_custom_op [-107.554f, 90.9104f] 24
CustomOp/linalg::channelwiseQuantMatMul<36>73 contrib.epu.quadric_custom_op [-22.1531f, 38.053f] 25
CustomOp/add<MultiCoreMode::All>73 contrib.epu.quadric_custom_op [-2123.99f, 13635.3f] 17
/model/layers.25/input_layernorm/Mul_1 contrib.epu.rms_norm [-40.2525f, 33.3074f] 25
CustomOp/linalg::channelwiseQuantMatMul<39>77 contrib.epu.quadric_custom_op [-10.4602f, 17.5029f] 26
CustomOp/add<MultiCoreMode::All>77 contrib.epu.quadric_custom_op [-2123.75f, 13635.5f] 17
/model/layers.25/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-11.309f, 12.7584f] 27
CustomOp/nn::qwen3PrefillGateUpProjection<29,29,26,26,39,39>75 contrib.epu.quadric_custom_op [-117.555f, 109.813f] 24
CustomOp/linalg::channelwiseQuantMatMul<36>76 contrib.epu.quadric_custom_op [-19.0471f, 36.3233f] 25
CustomOp/add<MultiCoreMode::All>76 contrib.epu.quadric_custom_op [-2123.73f, 13635.5f] 17
/model/layers.26/input_layernorm/Mul_1 contrib.epu.rms_norm [-50.6021f, 38.8768f] 25
CustomOp/linalg::channelwiseQuantMatMul<40>80 contrib.epu.quadric_custom_op [-8.51294f, 19.8805f] 26
CustomOp/add<MultiCoreMode::All>80 contrib.epu.quadric_custom_op [-2124.04f, 13636.4f] 17
/model/layers.26/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-12.5561f, 15.4058f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<29,29,26,27,39,38>78 contrib.epu.quadric_custom_op [-105.01f, 132.743f] 23
CustomOp/linalg::channelwiseQuantMatMul<36>79 contrib.epu.quadric_custom_op [-21.3917f, 49.6387f] 25
CustomOp/add<MultiCoreMode::All>79 contrib.epu.quadric_custom_op [-2124.03f, 13636.6f] 17
/model/layers.27/input_layernorm/Mul_1 contrib.epu.rms_norm [-61.2838f, 48.1027f] 25
CustomOp/linalg::channelwiseQuantMatMul<39>83 contrib.epu.quadric_custom_op [-8.8824f, 19.0919f] 26
CustomOp/add<MultiCoreMode::All>83 contrib.epu.quadric_custom_op [-2124.55f, 13638.8f] 17
/model/layers.27/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-14.5542f, 17.2044f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<29,29,26,27,39,38>81 contrib.epu.quadric_custom_op [-137.571f, 122.284f] 23
CustomOp/linalg::channelwiseQuantMatMul<36>82 contrib.epu.quadric_custom_op [-29.7459f, 42.6272f] 25
CustomOp/add<MultiCoreMode::All>82 contrib.epu.quadric_custom_op [-2124.49f, 13639.2f] 17
/model/layers.28/input_layernorm/Mul_1 contrib.epu.rms_norm [-65.7556f, 56.4749f] 24
CustomOp/linalg::channelwiseQuantMatMul<39>86 contrib.epu.quadric_custom_op [-16.1447f, 22.5356f] 26
CustomOp/add<MultiCoreMode::All>86 contrib.epu.quadric_custom_op [-2125.41f, 13642.1f] 17
/model/layers.28/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-13.3559f, 17.9234f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<29,29,26,26,38,38>84 contrib.epu.quadric_custom_op [-132.766f, 140.229f] 23
CustomOp/linalg::channelwiseQuantMatMul<35>85 contrib.epu.quadric_custom_op [-38.7672f, 58.2323f] 25
CustomOp/add<MultiCoreMode::All>85 contrib.epu.quadric_custom_op [-2125.36f, 13642.5f] 17
/model/layers.29/input_layernorm/Mul_1 contrib.epu.rms_norm [-87.7913f, 63.9043f] 24
CustomOp/linalg::channelwiseQuantMatMul<39>89 contrib.epu.quadric_custom_op [-15.1006f, 23.0286f] 26
CustomOp/add<MultiCoreMode::All>89 contrib.epu.quadric_custom_op [-2126.23f, 13643.9f] 17
/model/layers.29/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-14.5663f, 19.9309f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<30,29,26,26,39,38>87 contrib.epu.quadric_custom_op [-150.54f, 123.012f] 23
CustomOp/linalg::channelwiseQuantMatMul<35>88 contrib.epu.quadric_custom_op [-49.9533f, 76.5029f] 24
CustomOp/add<MultiCoreMode::All>88 contrib.epu.quadric_custom_op [-2126.19f, 13644f] 17
/model/layers.30/input_layernorm/Mul_1 contrib.epu.rms_norm [-97.8058f, 70.9147f] 24
CustomOp/linalg::channelwiseQuantMatMul<38>92 contrib.epu.quadric_custom_op [-26.3882f, 42.82f] 25
CustomOp/add<MultiCoreMode::All>92 contrib.epu.quadric_custom_op [-2126.46f, 13648.6f] 17
/model/layers.30/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-16.1268f, 19.8625f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<29,28,26,26,38,38>90 contrib.epu.quadric_custom_op [-147.928f, 141.815f] 23
CustomOp/linalg::channelwiseQuantMatMul<35>91 contrib.epu.quadric_custom_op [-53.484f, 106.991f] 24
CustomOp/add<MultiCoreMode::All>91 contrib.epu.quadric_custom_op [-2126.44f, 13648.7f] 17
/model/layers.31/input_layernorm/Mul_1 contrib.epu.rms_norm [-119.533f, 93.6401f] 24
CustomOp/linalg::channelwiseQuantMatMul<38>95 contrib.epu.quadric_custom_op [-17.2974f, 50.1237f] 25
CustomOp/add<MultiCoreMode::All>95 contrib.epu.quadric_custom_op [-2126.83f, 13653.5f] 17
/model/layers.31/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-17.2516f, 18.0006f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<29,28,26,26,38,37>93 contrib.epu.quadric_custom_op [-147.492f, 171.108f] 23
CustomOp/linalg::channelwiseQuantMatMul<34>94 contrib.epu.quadric_custom_op [-42.7217f, 101.298f] 24
CustomOp/add<MultiCoreMode::All>94 contrib.epu.quadric_custom_op [-2126.65f, 13653.4f] 17
/model/layers.32/input_layernorm/Mul_1 contrib.epu.rms_norm [-129.345f, 108.642f] 23
CustomOp/linalg::channelwiseQuantMatMul<37>98 contrib.epu.quadric_custom_op [-26.9792f, 52.8894f] 25
CustomOp/add<MultiCoreMode::All>98 contrib.epu.quadric_custom_op [-2125.55f, 13676.5f] 17
/model/layers.32/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-17.899f, 23.7326f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<28,28,26,26,38,37>96 contrib.epu.quadric_custom_op [-190.096f, 285.781f] 22
CustomOp/linalg::channelwiseQuantMatMul<34>97 contrib.epu.quadric_custom_op [-67.0631f, 111.421f] 24
CustomOp/add<MultiCoreMode::All>97 contrib.epu.quadric_custom_op [-2125.27f, 13672.9f] 17
/model/layers.33/input_layernorm/Mul_1 contrib.epu.rms_norm [-187.593f, 150.813f] 23
CustomOp/linalg::channelwiseQuantMatMul<37>101 contrib.epu.quadric_custom_op [-22.3657f, 115.159f] 24
CustomOp/add<MultiCoreMode::All>101 contrib.epu.quadric_custom_op [-2125.11f, 13717f] 17
/model/layers.33/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-29.27f, 27.8766f] 26
CustomOp/nn::qwen3PrefillGateUpProjection<28,28,26,26,37,36>99 contrib.epu.quadric_custom_op [-191.84f, 180.474f] 23
CustomOp/linalg::channelwiseQuantMatMul<34>100 contrib.epu.quadric_custom_op [-224.43f, 154.382f] 23
CustomOp/add<MultiCoreMode::All>100 contrib.epu.quadric_custom_op [-2126.49f, 13736f] 17
/model/layers.34/input_layernorm/Mul_1 contrib.epu.rms_norm [-190.924f, 150.511f] 23
CustomOp/linalg::channelwiseQuantMatMul<36>104 contrib.epu.quadric_custom_op [-134.891f, 252.592f] 23
CustomOp/add<MultiCoreMode::All>104 contrib.epu.quadric_custom_op [-2127.57f, 13665.9f] 17
/model/layers.34/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-102.457f, 31.8715f] 24
CustomOp/nn::qwen3PrefillGateUpProjection<27,27,25,24,35,34>102 contrib.epu.quadric_custom_op [-1174.96f, 792.102f] 20
CustomOp/linalg::channelwiseQuantMatMul<31>103 contrib.epu.quadric_custom_op [-18980.2f, 553.5f] 16
CustomOp/add<MultiCoreMode::All>103 contrib.epu.quadric_custom_op [-7343.94f, 5351.69f] 18
/model/layers.35/input_layernorm/Mul_1 contrib.epu.rms_norm [-298.728f, 121.485f] 22
CustomOp/linalg::channelwiseQuantMatMul<36>107 contrib.epu.quadric_custom_op [-282.141f, 824.661f] 21
CustomOp/add<MultiCoreMode::All>107 contrib.epu.quadric_custom_op [-7101.39f, 5072.41f] 18
/model/layers.35/post_attention_layernorm/Mul_1 contrib.epu.rms_norm [-601.076f, 47.2325f] 21
CustomOp/nn::qwen3PrefillGateUpProjection<25,25,24,23,34,33>105 contrib.epu.quadric_custom_op [-1002.33f, 1755.35f] 20
CustomOp/linalg::channelwiseQuantMatMul<29>106 contrib.epu.quadric_custom_op [-4812.36f, 1990.75f] 18
CustomOp/add<MultiCoreMode::All>106 contrib.epu.quadric_custom_op [-6990.34f, 2581.91f] 18
/model/norm/Mul_1 contrib.epu.rms_norm [-139.062f, 130.451f] 23
CustomOp/nn::gather<Direction::Height>108 contrib.epu.quadric_custom_op [-93.9597f, 66.4554f] 24
CustomOp/linalg::channelwiseQuantMatMul<36>108 contrib.epu.quadric_custom_op [-15.0257f, 24.2741f] 26
2026-06-19 05:13 - INFO - epu - codegen - START====================build_cpu_runnable_fx_relay
2026-06-19 05:13 - INFO - epu - codegen - START=======================quantize_to_chimera_fx
2026-06-19 05:13 - INFO - epu - codegen - START=================================relay_to_tir
2026-06-19 05:13 - INFO - epu - codegen - START===========================relay_to_epu_relay
2026-06-19 05:13 - INFO - epu - codegen - START==============================adapt_and_order
2026-06-19 05:14 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 05:14 - INFO - epu - codegen - START=============================plan_lrm_virtual
2026-06-19 05:15 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 05:15 - INFO - epu - codegen - START===============================lrm_alloc_loop
2026-06-19 05:17 - INFO - epu - codegen - START==============================amend_ctrl_flow
2026-06-19 05:17 - INFO - epu - codegen - START================================lrm_splitting
2026-06-19 05:20 - INFO - epu - codegen - START==============================ext_split_relay
2026-06-19 05:22 - INFO - epu - codegen - START====================================build_tir
2026-06-19 05:22 - INFO - epu - chimera_job - Compilation of qwen3_custom_op_all_dec_QC_P_1d7_2MB_4kB_128GBps_128GBps_16_OFF_x1_x1 successful
Compilation complete!
╒═════════════════════╤═══════════════════════════════════════════════════════════════════════╕
│ Module Name │ qwen3_custom_op_all_dec_QC_P_1d7_2MB_4kB_128GBps_128GBps_16_OFF_x1_x1 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ ONNX File │ qwen3_custom_op_all_dec.onnx │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Product Target │ QC-P │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Number of Cores │ 1 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ ISS Clock Frequency │ 1.700 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ L2M Size │ 2MB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ LRM Size │ 4kB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ External Read BW │ 128GBps │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ External Write BW │ 128GBps │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ MACS per PE │ 16 │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Max L2M │ 0.000MB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Max LRM │ 0.000kB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Max Temp Ext Bytes │ 128.000MB │
├─────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ Network GMACs │ 7,422.326 │
╘═════════════════════╧═══════════════════════════════════════════════════════════════════════╛
╒═════╤════════╤══════════════════════════╤═══════════════════╤══════════════════════════╤═══════╕
│ │ Type │ Name │ shape │ type │ mse │
╞═════╪════════╪══════════════════════════╪═══════════════════╪══════════════════════════╪═══════╡
│ 0 │ Input │ input_ids │ [1, 1024] │ tensor[int32] │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 1 │ Input │ attention_mask │ [1, 1024, 1024] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 2 │ Input │ sin │ [1, 1024, 128] │ tensor[FixedPoint32<30>] │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 3 │ Input │ cos │ [1, 1024, 128] │ tensor[FixedPoint32<30>] │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 4 │ Input │ past_key_values.0.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 5 │ Input │ past_key_values.0.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 6 │ Input │ past_key_values.1.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 7 │ Input │ past_key_values.1.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 8 │ Input │ past_key_values.2.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 9 │ Input │ past_key_values.2.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 10 │ Input │ past_key_values.3.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 11 │ Input │ past_key_values.3.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 12 │ Input │ past_key_values.4.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 13 │ Input │ past_key_values.4.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 14 │ Input │ past_key_values.5.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 15 │ Input │ past_key_values.5.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 16 │ Input │ past_key_values.6.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 17 │ Input │ past_key_values.6.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 18 │ Input │ past_key_values.7.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 19 │ Input │ past_key_values.7.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 20 │ Input │ past_key_values.8.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 21 │ Input │ past_key_values.8.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 22 │ Input │ past_key_values.9.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 23 │ Input │ past_key_values.9.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 24 │ Input │ past_key_values.10.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 25 │ Input │ past_key_values.10.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 26 │ Input │ past_key_values.11.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 27 │ Input │ past_key_values.11.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 28 │ Input │ past_key_values.12.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 29 │ Input │ past_key_values.12.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 30 │ Input │ past_key_values.13.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 31 │ Input │ past_key_values.13.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 32 │ Input │ past_key_values.14.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 33 │ Input │ past_key_values.14.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 34 │ Input │ past_key_values.15.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 35 │ Input │ past_key_values.15.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 36 │ Input │ past_key_values.16.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 37 │ Input │ past_key_values.16.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 38 │ Input │ past_key_values.17.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 39 │ Input │ past_key_values.17.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 40 │ Input │ past_key_values.18.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 41 │ Input │ past_key_values.18.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 42 │ Input │ past_key_values.19.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 43 │ Input │ past_key_values.19.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 44 │ Input │ past_key_values.20.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 45 │ Input │ past_key_values.20.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 46 │ Input │ past_key_values.21.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 47 │ Input │ past_key_values.21.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 48 │ Input │ past_key_values.22.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 49 │ Input │ past_key_values.22.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 50 │ Input │ past_key_values.23.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 51 │ Input │ past_key_values.23.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 52 │ Input │ past_key_values.24.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 53 │ Input │ past_key_values.24.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 54 │ Input │ past_key_values.25.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 55 │ Input │ past_key_values.25.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 56 │ Input │ past_key_values.26.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 57 │ Input │ past_key_values.26.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 58 │ Input │ past_key_values.27.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 59 │ Input │ past_key_values.27.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 60 │ Input │ past_key_values.28.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 61 │ Input │ past_key_values.28.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 62 │ Input │ past_key_values.29.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 63 │ Input │ past_key_values.29.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 64 │ Input │ past_key_values.30.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 65 │ Input │ past_key_values.30.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 66 │ Input │ past_key_values.31.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 67 │ Input │ past_key_values.31.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 68 │ Input │ past_key_values.32.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 69 │ Input │ past_key_values.32.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 70 │ Input │ past_key_values.33.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 71 │ Input │ past_key_values.33.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 72 │ Input │ past_key_values.34.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 73 │ Input │ past_key_values.34.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 74 │ Input │ past_key_values.35.key │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 75 │ Input │ past_key_values.35.value │ [1, 8, 0, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 76 │ Output │ logits │ [1, 151936] │ tensor[FixedPoint32<26>] │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 77 │ Output │ present.0.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 78 │ Output │ present.0.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 79 │ Output │ present.1.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 80 │ Output │ present.1.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 81 │ Output │ present.2.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 82 │ Output │ present.2.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 83 │ Output │ present.3.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 84 │ Output │ present.3.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 85 │ Output │ present.4.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 86 │ Output │ present.4.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 87 │ Output │ present.5.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 88 │ Output │ present.5.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 89 │ Output │ present.6.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 90 │ Output │ present.6.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 91 │ Output │ present.7.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 92 │ Output │ present.7.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 93 │ Output │ present.8.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 94 │ Output │ present.8.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 95 │ Output │ present.9.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 96 │ Output │ present.9.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 97 │ Output │ present.10.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 98 │ Output │ present.10.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 99 │ Output │ present.11.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 100 │ Output │ present.11.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 101 │ Output │ present.12.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 102 │ Output │ present.12.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 103 │ Output │ present.13.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 104 │ Output │ present.13.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 105 │ Output │ present.14.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 106 │ Output │ present.14.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 107 │ Output │ present.15.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 108 │ Output │ present.15.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 109 │ Output │ present.16.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 110 │ Output │ present.16.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 111 │ Output │ present.17.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 112 │ Output │ present.17.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 113 │ Output │ present.18.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 114 │ Output │ present.18.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 115 │ Output │ present.19.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 116 │ Output │ present.19.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 117 │ Output │ present.20.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 118 │ Output │ present.20.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 119 │ Output │ present.21.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 120 │ Output │ present.21.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 121 │ Output │ present.22.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 122 │ Output │ present.22.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 123 │ Output │ present.23.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 124 │ Output │ present.23.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 125 │ Output │ present.24.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 126 │ Output │ present.24.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 127 │ Output │ present.25.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 128 │ Output │ present.25.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 129 │ Output │ present.26.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 130 │ Output │ present.26.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 131 │ Output │ present.27.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 132 │ Output │ present.27.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 133 │ Output │ present.28.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 134 │ Output │ present.28.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 135 │ Output │ present.29.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 136 │ Output │ present.29.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 137 │ Output │ present.30.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 138 │ Output │ present.30.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 139 │ Output │ present.31.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 140 │ Output │ present.31.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 141 │ Output │ present.32.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 142 │ Output │ present.32.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 143 │ Output │ present.33.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 144 │ Output │ present.33.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 145 │ Output │ present.34.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 146 │ Output │ present.34.value │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 147 │ Output │ present.35.key │ [1, 8, 1024, 128] │ n/a │ n/a │
├─────┼────────┼──────────────────────────┼───────────────────┼──────────────────────────┼───────┤
│ 148 │ Output │ present.35.value │ [1, 8, 1024, 128] │ n/a │ n/a │
╘═════╧════════╧══════════════════════════╧═══════════════════╧══════════════════════════╧═══════╛
