Models / qwen3.8-27b

Qwen3.8 27B

A 27B-parameter hybrid decoder: three Gated DeltaNet (linear attention) layers for every gated full-attention layer, zero-centered RMS norms, SwiGLU, and an untied output head; the text model of Qwen3.8-27B, without its vision tower.

26.9B parametersqwen3_5Apache-2.0text-generationdecoder-onlylinear-attentiongated-deltanethybrid

Hugging Face GitHub Registry entry

No sample has been recorded for Qwen3.8 27B yet. What it produces, next to the reference stack's answer, will appear here once the benchmark harness has run it.

Load

from linnet import nest

model = nest.load("qwen3.8-27b")

Checkpoint

Qwen/Qwen3.8-27B
model-00001-of-00018.safetensors, model-00002-of-00018.safetensors, model-00003-of-00018.safetensors, model-00004-of-00018.safetensors, model-00005-of-00018.safetensors, model-00006-of-00018.safetensors, model-00007-of-00018.safetensors, model-00008-of-00018.safetensors, model-00009-of-00018.safetensors, model-00010-of-00018.safetensors, model-00011-of-00018.safetensors, model-00012-of-00018.safetensors, model-00013-of-00018.safetensors, model-00014-of-00018.safetensors, model-00015-of-00018.safetensors, model-00016-of-00018.safetensors, model-00017-of-00018.safetensors, model-00018-of-00018.safetensors

Entries

forward
next_token
decode
prefill
decode_rows
prefill_slots

Generics

Vocab = 248320
H = 5120
Heads = 24
KvHeads = 4
HeadDim = 256
KeyHeads = 16
ValueHeads = 48
KeyDim = 128
ValueDim = 128
Conv = 10240
Width = 4
Inner = 17408
Groups = 16
Batch = 1
MaxSeq = 262144
T = bf16