Models / qwen3.8-27b

Qwen3.8 27B

A 27B-parameter hybrid decoder: three Gated DeltaNet (linear attention) layers for every gated full-attention layer, zero-centered RMS norms, SwiGLU, and an untied output head; the text model of Qwen3.8-27B, without its vision tower.

26.9B parametersqwen3_5Apache-2.0text-generationdecoder-onlylinear-attentiongated-deltanethybrid

Hugging Face GitHub Registry entry

Qwen3.8 27B has not been measured yet. Its numbers will appear here once the benchmark harness has run it on a GPU against the stacks people already use.

Load

from linnet import nest

model = nest.load("qwen3.8-27b")

Checkpoint

Qwen/Qwen3.8-27B
model-00001-of-00018.safetensors, model-00002-of-00018.safetensors, model-00003-of-00018.safetensors, model-00004-of-00018.safetensors, model-00005-of-00018.safetensors, model-00006-of-00018.safetensors, model-00007-of-00018.safetensors, model-00008-of-00018.safetensors, model-00009-of-00018.safetensors, model-00010-of-00018.safetensors, model-00011-of-00018.safetensors, model-00012-of-00018.safetensors, model-00013-of-00018.safetensors, model-00014-of-00018.safetensors, model-00015-of-00018.safetensors, model-00016-of-00018.safetensors, model-00017-of-00018.safetensors, model-00018-of-00018.safetensors

Entries

forward
next_token
decode
prefill
decode_rows
prefill_slots

Generics

Vocab = 248320
H = 5120
Heads = 24
KvHeads = 4
HeadDim = 256
KeyHeads = 16
ValueHeads = 48
KeyDim = 128
ValueDim = 128
Conv = 10240
Width = 4
Inner = 17408
Groups = 16
Batch = 1
MaxSeq = 262144
T = bf16