NOTE: The Jupyter Notebook below is included in the Chimera SDK and can be run interactively by running the following CLI command:
$ quadric sdk notebook
From the Jupyter Notebook window in your browser, select the notebook named /quadric/sdk-cli/examples/models/qwen3_6/qwen3_6.ipynb.
Qwen3.6-35B-A3B Decode on Chimera GPNPU
This notebook runs Qwen3.6-35B-A3B on the Chimera ISS through ChiPy: a real prompt token goes in, a predicted token comes out, and the whole 40-layer decoder runs as one kernel per token.
It is a hybrid stack — three Gated DeltaNet (linear-attention) layers for every one gated-attention layer, an A-A-A-B pattern — with a 256-expert top-8 Mixture-of-Experts FFN in every layer. The delivery is calibrated W4A16: routed experts int4, shared expert and attention projections int8, activations fp16.
Pipeline
Model: Qwen3.6-35B-A3B (35B total / 3B active, 40 decoder layers, hidden size 2048, 256 experts with top-8 routing, shared-expert intermediate 512, 248,320-token vocabulary, rope_theta 1e7), W4A16 weight-only quantization.
| Section | What runs | Scored against |
|---|---|---|
| 3-4 | the whole model, one kernel per token, real W4A16 weights | the delivery's own reference logits |
| 5-11 | one op or one layer at a time, synthetic weights | a float64 reference on the ISS |
| 12-13 | the embedding gather and the vocabulary projection, real weights | the delivery's captured activations |
Every weight in the 40-layer kernel is a baked constant rather than an input edge, which is what makes the depth affordable: constants are materialised as offsets off one base pointer, so the kernel takes the same entry pointers at forty layers as at four.
Note — split pipeline.
- On the Chimera GPNPU: the embedding gather, all 40 decoder layers, the final RMSNorm and the vocabulary projection — the entire network.
- On the host CPU: picking a token from the logits and scoring it against the reference, carrying the state between tokens, and decoding ids back to text.
The full pipeline can run end-to-end on the GPNPU — see
yolox_e2e_chipy.ipynborretina_net_e2e_chipy.ipynbfor examples that compose CCL custom ops with ChiPy to keep every stage on-chip.
1. Setup
Make the qwen3_6 package importable from this directory.
import pathlib, sys
_pkg_parent = str(pathlib.Path.cwd().parent.resolve())
if _pkg_parent not in sys.path:
sys.path.insert(0, _pkg_parent)
2. Download the W4A16 delivery and the tokenizer
The 40-layer run needs full_model.pth as well as the test vectors: only the checkpoint carries weights past layers 0 and 3. Together that is ~52.5 GB, fetched once and skipped when already present, so a machine that points $QWEN36_W4A16_BUNDLE at an existing copy downloads nothing.
The delivery ships weights and captured activations but no tokenizer, so the published model's tokenizer.json (12.2 MB) is fetched alongside it — that is what turns the predicted ids back into text in section 3. It lands beside this notebook rather than beside the delivery, because the delivery is often a read-only shared mount. Reading it needs only the tokenizers library, not transformers: a tokenizer.json is self-contained.
import os
from pathlib import Path
from qwen3_6 import delivery
from qwen3_6.generate import tokenizer as tok_loader
delivery.download_bundle(full_model=True) # ~52.5 GB; skips whatever is already cached
os.environ.setdefault("QWEN36_TOKENIZER", str(Path.cwd() / "tokenizer.json"))
if not tok_loader.available():
tok_loader.download_tokenizer(dest=os.environ["QWEN36_TOKENIZER"])
## The name, not the path: the notebook's outputs are published verbatim, and an absolute
## path names whoever's machine happened to run it.
print(f"tokenizer: {Path(tok_loader.tokenizer_path()).name}")
fetching into checkpoint/qwen3_5_35b_a3b_W4A16_delivery_v1 (383.8 GB free)
Downloading test_vectors.safetensors ...
test_vectors.safetensors (17.6 GB) — done
Downloading full_model.pth ...
full_model.pth (34.9 GB) — done
tokenizer: tokenizer.json
3. Run decode on the ISS
Decodes four tokens, one kernel call per token, with the DeltaNet recurrent state and the KV cache threaded between them — each token is only right if every call before it was. Each token is its own kernel invocation: the state leaves on its own edges and returns as the next token's seed.
One compile serves every position: the decode position reaches the kernel through the pos edge.
The loop is teacher-forced: each call is given the prompt token for its position, and its prediction is scored against the reference rather than fed back as the next input.
It prints the prompt and the predicted tokens as text, using the tokenizer fetched above, next to the ids they came from.
from qwen3_6.pipeline.decode_pipeline import validate_decode_loop
assert validate_decode_loop(num_tokens=4) # 40 real-weight layers, four tokens, caches threaded
packed layer 0
packed layer 1
packed layer 2
packed layer 3
packed layer 4
packed layer 5
packed layer 6
packed layer 7
packed layer 8
packed layer 9
packed layer 10
packed layer 11
packed layer 12
packed layer 13
packed layer 14
packed layer 15
packed layer 16
packed layer 17
packed layer 18
packed layer 19
packed layer 20
packed layer 21
packed layer 22
packed layer 23
packed layer 24
packed layer 25
packed layer 26
packed layer 27
packed layer 28
packed layer 29
packed layer 30
packed layer 31
packed layer 32
packed layer 33
packed layer 34
packed layer 35
packed layer 36
packed layer 37
packed layer 38
packed layer 39
compiling one kernel for all 4 tokens
2026-10-23 23:15 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-23 23:15 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-23 23:15 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-23 23:15 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
token 0 device vs host float64: cosine 0.99898391 norm ratio 1.007201
token 0 chain: cosine 0.95429557
token 0 (id 19): PASS -> 24 '9'
token 1 chain: cosine 0.99905460
token 1 (id 19): PASS -> 15 '0'
token 2 chain: cosine 0.99921648
token 2 (id 1116): PASS -> 220 ' '
token 3 chain: cosine 0.99942577
token 3 (id 551): PASS -> 557 ' was'
prompt '44 , he'
predicted ['9', '0', ' ', ' was']
perplexity: device 1794.55 vs reference 1845.53 (-2.76%) over 4 positions
decode_loop (40 layers, 4 tokens, caches threaded): PASS
4. Compare decode against the delivery's reference
The run above prints two gated numbers per token, plus the perplexity for the run.
The token. The device's token must maximise the delivery's reference logits for that position.
The chain cosine. ≥ 0.95 against lm_head:float_input, the delivery's post-norm hidden. The device exports the pre-norm residual and the host applies the final RMSNorm, so this scores the stack, not the kernel's norm. A floor against a bf16 capture. Reads 0.9578 here.
The perplexity. Against the delivery's own logits over the same positions, scored the same way. Printed, not gated.
At token 0 the run also prints the device against our own float64 host chain — cosine and norm ratio. Printed, not gated: it is the only number that isolates our fixed-point error rather than agreement with a lossy capture.
A single 40-layer kernel has no observable layer boundaries, so per-op and per-layer scoring lives in sections 5 to 13. A shorter run scores its final residual against the delivery's capture only where the stack ends on a captured layer — num_layers 4; anything else prints ungated.
5. Validate the MoE FFN block
Runs the router, the routed half on its own, and the complete Qwen3.6 MoE block on the Chimera ISS against a NumPy reference. The block is qwen36::moeFfnBlockDecode, which computes out = routed + sigmoid(x · W_sg) * shared: the top-k routed experts plus a dense shared-expert SwiGLU applied to every token, scaled by a sigmoid output gate. One fused matmul over [W_gate | W_up | W_sg | W_router] produces every first-layer projection at once (shared gate/up, the output-gate logit and all E router logits), and routed/shared are summed on-chip so only the final hidden reaches DDR.
The routed half also runs on its own: routed without the shared expert, as the router → expert-path pair. Inputs and weights are synthetic (random, in a representative range), so this checks kernel correctness on hardware rather than model quality, which needs real weights and the full stack.
from qwen3_6.pipeline.decode_pipeline import (
validate_moe_block,
validate_moe_routed_path,
validate_router,
)
assert validate_router() # qwen36::moeRouter vs the TopK oracle
assert validate_moe_routed_path() # routed half alone (router → routed-expert path)
assert validate_moe_block() # the complete block: routed + sigmoid-gated shared expert
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-23 23:55 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
router: PASS (max score err 2.97e-05)
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
moe_routed_path: PASS (cosine 1.000000, norm ratio 1.000213)
moe_block: PASS (cosine 1.000000, norm ratio 1.000112, shared-branch delta 125.794% vs expected 125.775%)
5.1 Score the block against the real W4A16 weights
Loads the real Qwen3.5-35B-A3B W4A16 delivery and scores the ISS output against the float activations shipped alongside it, component by component -- shared expert, routed experts and output gate scored separately.
The test vectors (17.6 GB) are fetched on first use and cached. A machine that points $QWEN36_W4A16_BUNDLE at an existing copy already has them and downloads nothing.
from qwen3_6 import delivery
from qwen3_6.moe import loader
from qwen3_6.pipeline.decode_pipeline import validate_moe_block_real_weights
if not loader.available():
delivery.download_bundle() # test vectors only; full_model=True adds all 40 layers
assert validate_moe_block_real_weights()
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-23 23:56 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
moe_block_real (layer 0, token 0): PASS (ISS vs dequant ref: cosine 1.000000 ratio 1.000043; vs delivered float: quant 3.948e-03, ISS 3.937e-03; per component: shared 2.116e-03, routed 4.462e-03, gate 5.941e-04; shares routed 87.1% / shared 24.8%)
5.2 Check the reference against the published model's own implementation
Scores the reference against the transformers implementation of this architecture — the router's softmax → top-k → renormalize ordering, the SwiGLU, and the sigmoid output gate.
The Qwen3_5Moe classes ship only in transformers >= 5, installed into a notebook-local directory and run in a fresh interpreter.
import glob
import os
import shutil
import subprocess
import sys
TARGET = os.path.abspath("qwen36_t5141")
subprocess.run(
[sys.executable, "-m", "pip", "install", "--quiet", "--target", TARGET, "transformers==5.14.1"],
check=True,
)
## `pip --target` resolves dependencies without crediting the environment, so it drops its own
## numpy 2.x alongside transformers. torch here is built against the environment's numpy 1.x, so
## the environment's has to win; transformers only asks for numpy>=1.17, which it satisfies.
for stale in glob.glob(os.path.join(TARGET, "numpy*")):
shutil.rmtree(stale, ignore_errors=True)
env = dict(os.environ, PYTHONPATH=os.pathsep.join([TARGET, os.path.abspath("..")]))
done = subprocess.run(
[sys.executable, "moe/hf_reference_check.py"], env=env, capture_output=True, text=True
)
print(done.stdout.strip() or done.stderr.strip()[-2000:])
assert done.returncode == 0, "reference disagrees with transformers Qwen3_5MoeSparseMoeBlock"
moe_reference_vs_hf: PASS (rel 2.377e-08 vs gate 1e-06)
6. Validate the Gated DeltaNet decode layer
Runs the full fused Gated DeltaNet decode layer (qwen36::gatedDeltaNetDecode: in_proj → causal conv1d + SiLU → gates → recurrent gated delta rule with index-based GQA → gated RMSNorm → out_proj) on the Chimera ISS and compares against the op's CPU reference (itself bit-exact vs the deltanet_decode oracle). Synthetic inputs in a representative range — this checks kernel correctness on hardware, not model quality.
from qwen3_6.pipeline.decode_pipeline import validate_deltanet
assert validate_deltanet() # full Gated DeltaNet decode layer (qwen36::gatedDeltaNetDecode)
assert validate_deltanet(num_steps=4) # multi-step: intra-call conv ring + state recurrence
2026-10-24 00:00 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:00 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:00 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:00 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
/quadric/sdk-cli/examples/models/qwen3_6/deltanet/deltanet_chipy.py:195: RuntimeWarning: overflow encountered in cast
.astype(np.float16)
/quadric/sdk-cli/examples/models/qwen3_6/deltanet/deltanet_chipy.py:253: RuntimeWarning: invalid value encountered in matmul
mixed_qkv = (xs @ w_qkv)[0] # [conv_dim]
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
deltanet (hidden=256, steps=1): PASS (cosine 0.999999, norm ratio 0.999916)
/quadric/sdk-cli/examples/models/qwen3_6/deltanet/deltanet_chipy.py:181: RuntimeWarning: overflow encountered in cast
.astype(np.float16)
/quadric/sdk-cli/examples/models/qwen3_6/deltanet/deltanet_chipy.py:254: RuntimeWarning: invalid value encountered in matmul
z_proj = (xs @ w_z)[0] # [value_dim]
/quadric/sdk-cli/examples/models/qwen3_6/deltanet/deltanet_chipy.py:255: RuntimeWarning: invalid value encountered in matmul
b_proj = (xs @ w_b)[0] # [Hv]
/quadric/sdk-cli/examples/models/qwen3_6/deltanet/deltanet_chipy.py:256: RuntimeWarning: invalid value encountered in matmul
a_proj = (xs @ w_a)[0] # [Hv]
deltanet (hidden=256, steps=4): PASS (cosine 0.999999, norm ratio 1.000050)
7. Validate the full linear decoder layer
Assembles the whole linear decoder layer — input RMSNorm → Gated DeltaNet → residual → post-attn RMSNorm → MoE FFN → residual — from the validated ops (qwen36::rmsNorm, qwen36::rmsNormFp16, qwen36::add around the DeltaNet and MoE ops) and runs it on the Chimera ISS, then a 2-layer stack. Each layer is checked against a float64 reference (cosine ≥ 0.98). The layer runs as two register-fitting sub-blocks.
from qwen3_6.pipeline.decode_pipeline import validate_decoder_layer
assert validate_decoder_layer(n_layers=1) # full linear decoder layer
assert validate_decoder_layer(n_layers=2) # N-layer stack
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
layer 0: PASS (cosine 0.999999, ratio 1.000020, mixer 0.999999)
decoder_layer (n=1): PASS
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:01 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
layer 0: PASS (cosine 0.999999, ratio 1.000020, mixer 0.999999)
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
layer 1: PASS (cosine 1.000000, ratio 0.999920, mixer 0.999999)
decoder_layer (n=2): PASS
8. Validate cross-call DeltaNet state resume
One split-state DeltaNet call per step, reading the evolved conv-ring / recurrent state back to the host and feeding it in as the next call's seed. Checks that two separate 1-step calls match a single 2-step call (cosine ≥ 0.98 and norm ratio within 5%). Synthetic weights at hidden 256, and the DeltaNet state only — the KV cache is covered by section 3.
from qwen3_6.pipeline.decode_pipeline import validate_resume
assert validate_resume() # per-call state resume == one 2-step call
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:02 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
resume (per-call): PASS (step0 cos 0.999999 ratio 0.999902, step1 cos 0.999999 ratio 1.000030, state cos 1.000000 ratio 0.999978)
9. Validate the gated-attention decode op
Runs the full fused gated-attention decode layer (qwen36::gatedAttentionDecode: q/k/v proj → (1+weight) QK-RMSNorm → partial RoPE → KV-cache append → GQA scaled-dot-product attention → sigmoid output gate → o_proj) on the Chimera ISS against the op's float64 reference. Synthetic inputs in a representative range, at num_tokens=1 and num_tokens=2.
from qwen3_6.pipeline.decode_pipeline import validate_attention
assert validate_attention(num_tokens=1) # gated-attention decode op (qwen36::gatedAttentionDecode)
assert validate_attention(num_tokens=2) # real multi-key softmax over the growing KV cache
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
/quadric/sdk-cli/examples/models/qwen3_6/attention/attention_chipy.py:224: RuntimeWarning: overflow encountered in cast
return arr.astype(np.float16).astype(np.float64)
/quadric/sdk-cli/examples/models/qwen3_6/attention/attention_chipy.py:252: RuntimeWarning: invalid value encountered in matmul
query = (x_row @ q_w_a)[0].reshape(nq, hd)
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
attention (embed=256, hidden=256, tokens=1): PASS (cosine 0.999948, norm ratio 0.988827)
/quadric/sdk-cli/examples/models/qwen3_6/attention/attention_chipy.py:253: RuntimeWarning: invalid value encountered in matmul
gate = (x_row @ gate_w_a)[0].reshape(nq, hd)
/quadric/sdk-cli/examples/models/qwen3_6/attention/attention_chipy.py:254: RuntimeWarning: invalid value encountered in matmul
key = (x_row @ k_w_a)[0].reshape(nkv, hd)
/quadric/sdk-cli/examples/models/qwen3_6/attention/attention_chipy.py:255: RuntimeWarning: invalid value encountered in matmul
value = (x_row @ v_w_a)[0].reshape(nkv, hd)
attention (embed=256, hidden=256, tokens=2): PASS (cosine 0.999923, norm ratio 0.989294)
10. Validate the gated-attention decoder layer
Assembles the whole gated-attention decoder layer — input RMSNorm → gated attention → residual → post-attn RMSNorm → MoE FFN → residual — from the validated ops and runs it on the Chimera ISS. Each layer is checked against a float64 reference (cosine ≥ 0.98 and norm ratio within 5%). The layer runs as two register-fitting sub-blocks.
from qwen3_6.pipeline.decode_pipeline import validate_attention_layer
assert validate_attention_layer() # full gated-attention decoder layer
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:03 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
attn layer 0: PASS (cosine 0.999999, ratio 1.000515, mixer 0.999998)
attention_layer (n=1, hidden=256): PASS
11. Validate the hybrid A-A-A-B decoder
Stacks the real Qwen3.6 layer pattern: three linear-attention (A) DeltaNet decoder layers followed by one gated-attention (B) layer, threading the hidden state layer-to-layer via a host round-trip. Runs on the Chimera ISS; each layer is checked against a float64 reference (cosine ≥ 0.98 and norm ratio within 5%).
from qwen3_6.pipeline.decode_pipeline import validate_hybrid_decoder
assert validate_hybrid_decoder() # hybrid A-A-A-B decoder stack
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
hybrid layer 0 (A): PASS (cosine 0.999999, ratio 1.000020, mixer 0.999999)
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:04 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
hybrid layer 1 (A): PASS (cosine 1.000000, ratio 0.999920, mixer 0.999999)
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
hybrid layer 2 (A): PASS (cosine 1.000000, ratio 0.999995, mixer 0.999999)
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
hybrid layer 3 (B): PASS (cosine 1.000000, ratio 1.000035, mixer 0.999999)
shared-expert share over 4 layers: min 24.824%, median 46.280%
hybrid_decoder (A-A-A-B): PASS
12. Embed real tokens
Everything above starts from a hidden state — synthetic, or one lifted from the delivery's captured activations. This is the op that produces it: qwen36::embedTokens, a cgc::embedding gather over the vocabulary table, the same core op llama's and qwen's ONNX graphs lower their leading Gather onto.
Checked for bit-exactness against the delivery's oracle, with the dequantized row rounded through bf16 as the reference model is.
The dictionary is FX32 at the residual stream's frac, so the gathered rows are layer 0's input as they stand — nothing converts between them. It holds the ids under test here; the whole 248320-row table is bit-exact too.
Needs full_model.pth: the dictionary's w_scale lives only in the checkpoint.
from qwen3_6.pipeline.decode_pipeline import validate_embed_tokens
assert validate_embed_tokens() # real token ids -> hidden states, gated on bit-exactness
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
embed_tokens (8 tokens, vocab 7): PASS (bit-exact vs delivered float: True, max|err| 0.000e+00)
13. Project to the vocabulary
The model's other end. qwen36::lmHead is the largest dense matmul in it — [1, 2048] × [2048, 248320], 508M MACs per token, with none of the top-k sparsity that keeps the MoE affordable. FP16 MACs over a weight the host dequantizes, since the delivery quantizes lm_head weight-only; at 1017 MB it is also the largest single edge in the pipeline.
Gated on which token wins: the chosen token must attain the reference's maximum. Cosine rides alongside, to catch a projection that is wrong everywhere but keeps its maximum.
Runs 8192 of the 248320 vocabulary rows. The whole vocabulary passes too — top-1 8/8, cosine 0.9999982.
The final RMSNorm is not part of this op — the delivery's oracle takes the post-norm hidden as its input. That norm is qwen36::rmsNorm, validated above.
from qwen3_6.pipeline.decode_pipeline import validate_lm_head
assert validate_lm_head() # hidden -> vocabulary logits, gated on the chosen token
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max macs per_pe 8, forcing 8 macs per pe
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max clock speed of 1GHz forcing 1GHz
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
2026-10-24 00:05 - WARNING - epu - hw_config - QB-series products have a max ext bw of 32GBps, forcing 32GBps
lm_head (8 tokens, vocab 8192): PASS (top-1 8/8, cosine 0.99999837, widest reference tie 2)
Summary
The whole 40-layer Qwen3.6 decoder runs as one kernel per token on the Chimera ISS, on the calibrated W4A16 delivery, and the token it picks attains the reference's top logit.
Pass gates for the component sections: router — exact top-k match + score error < 2e-3; MoE routed half — cosine ≥ 0.98 and output-norm ratio within 2%; the full MoE block — cosine ≥ 0.999, norm ratio within 2%, and the shared-expert branch must move the output at least half as far from routed as the reference does; Gated DeltaNet layer, full decoder layer, cross-call resume, gated-attention op / layer, and the hybrid decoder — cosine ≥ 0.98 and norm ratio within 5%. The embedding is gated on bit-exactness and the vocabulary projection on which token wins.
Key takeaways
- Forty layers fit in one kernel because every weight is a baked constant rather than an input edge. Constants cost no entry pointers, so depth is bounded by memory rather than by the 15-pointer edge limit that caps a weight-edge kernel at four layers.
- The caches cross between tokens on their own edges. Each token is a separate kernel invocation, so the DeltaNet recurrent state and the KV cache leave as outputs and return as the next token's seed — a stricter test of them than holding them resident would be.
- Quantization is per-component, not uniform: routed experts int4, shared expert and attention projections int8, activations fp16. The routed experts are 92.9% of the model, which is what makes int4 there worth its calibration cost.
- Every component is also scored on its own. A single fused kernel has no observable layer boundaries, so sections 5 to 13 check each op and each layer against its own reference — the router, both mixers, a decoder layer of each kind, and the hybrid A-A-A-B stack.
Citation
@misc{qwen3_6_35b_a3b,
title = {Qwen3.5-35B-A3B},
author = {Qwen Team},
howpublished = {\url{https://huggingface.co/Qwen/Qwen3.5-35B-A3B}},
year = {2026}
}
