Models / qwen3.6-35b-a3b

Qwen3.6 35B-A3B

A 35B-parameter hybrid mixture of experts (3B active): three Gated DeltaNet (linear attention) layers for every gated full-attention layer, 256 experts with 8 chosen per token plus a gated shared expert, and an untied output head; the text model of Qwen3.6-35B-A3B, without its vision tower.

34.7B parametersqwen3_5_moeApache-2.0text-generationdecoder-onlymixture-of-expertslinear-attentiongated-deltanethybrid

Hugging Face GitHub Registry entry

No sample has been recorded for Qwen3.6 35B-A3B yet. What it produces, next to the reference stack's answer, will appear here once the benchmark harness has run it.

Load

from linnet import nest

model = nest.load("qwen3.6-35b-a3b")

Checkpoint

Qwen/Qwen3.6-35B-A3B
model-00001-of-00026.safetensors, model-00002-of-00026.safetensors, model-00003-of-00026.safetensors, model-00004-of-00026.safetensors, model-00005-of-00026.safetensors, model-00006-of-00026.safetensors, model-00007-of-00026.safetensors, model-00008-of-00026.safetensors, model-00009-of-00026.safetensors, model-00010-of-00026.safetensors, model-00011-of-00026.safetensors, model-00012-of-00026.safetensors, model-00013-of-00026.safetensors, model-00014-of-00026.safetensors, model-00015-of-00026.safetensors, model-00016-of-00026.safetensors, model-00017-of-00026.safetensors, model-00018-of-00026.safetensors, model-00019-of-00026.safetensors, model-00020-of-00026.safetensors, model-00021-of-00026.safetensors, model-00022-of-00026.safetensors, model-00023-of-00026.safetensors, model-00024-of-00026.safetensors, model-00025-of-00026.safetensors, model-00026-of-00026.safetensors

Entries

forward
next_token
decode
prefill
decode_rows
prefill_slots

Generics

Vocab = 248320
H = 2048
Heads = 16
KvHeads = 2
HeadDim = 256
KeyHeads = 16
ValueHeads = 32
KeyDim = 128
ValueDim = 128
Conv = 8192
Width = 4
Experts = 256
TopK = 8
ExpertInner = 512
SharedInner = 512
Groups = 10
Batch = 1
MaxSeq = 262144
T = bf16