Nestmodels for Linnet
Models / qwen3.6-35b-a3b

Qwen3.6 35B-A3B

A 35B-parameter hybrid mixture of experts (3B active): three Gated DeltaNet (linear attention) layers for every gated full-attention layer, 256 experts with 8 chosen per token plus a gated shared expert, and an untied output head; the text model of Qwen3.6-35B-A3B, without its vision tower.

34.7B parametersqwen3_5_moeApache-2.0text-generationdecoder-onlymixture-of-expertslinear-attentiongated-deltanethybrid
Hugging FaceGitHub Registry entry

The text model of Qwen3.6-35B-A3B, a 35B-parameter hybrid mixture of experts of the Qwen3.5-MoE architecture (qwen3_5_moe), about 3B active per token, without its vision tower or its multi-token-prediction layer. Its 40 layers come in fours: three of linear attention (Gated DeltaNet) and one of gated full attention, the same blocks as qwen3.8-27b at width 2048 (16 query/key heads and 32 value heads in the linear attention; 16 query heads and 2 key/value heads in the full attention).

Each layer's feed-forward block (src/moe.linnet) is a mixture of 256 experts: the router picks 8 per token by its softmax, renormalized over the chosen, and each expert is a SwiGLU of width 512 whose gate and up projections the checkpoint stores side by side (experts.gate_up_proj, [256, 1024, 2048]). A shared expert of width 512 runs on every token, scaled by its own sigmoid gate. The chosen experts go through std.nn.moe: a grouped product on a GPU, the dense form elsewhere.

Caches, states and entries are as in qwen3.8-27b; bindings.json maps the layers the same way.

Backends ​

The same source and checkpoint in each framework; numerics and compile are the loaders' options.

PyTorch ​

python
from linnet import nest

model = nest.load("qwen3.6-35b-a3b", backend="torch", numerics="fast", compile="inductor")
logits = model(tokens)

JAX ​

python
from linnet import nest

f = nest.load("qwen3.6-35b-a3b", backend="jax")          # StableHLO compiled by XLA
logits = f(tokens)

Flax NNX ​

python
from linnet import nest

model = nest.load("qwen3.6-35b-a3b", backend="nnx")
logits = model(tokens)

Command line ​

bash
python -m linnet.nest pull qwen3.6-35b-a3b
linnet stablehlo --root Model --entry forward --bind Vocab=248320 --bind H=2048 --bind Heads=16 --bind KvHeads=2 --bind HeadDim=256 --bind KeyHeads=16 --bind ValueHeads=32 --bind KeyDim=128 --bind ValueDim=128 --bind Conv=8192 --bind Width=4 --bind Experts=256 --bind TopK=8 --bind ExpertInner=512 --bind SharedInner=512 --bind Groups=10 --bind Batch=1 --bind MaxSeq=262144 --bind T=bf16 --bind B=1 --bind S=8 <model dir>/src/lib.linnet