Nestmodels for Linnet
Models / llama-3.1-8b-instruct-fp8

Llama 3.1 8B Instruct FP8

Llama 3.1 8B Instruct with every decoder projection in FP8 E4M3, one scale per output row: RedHatAI's FP8-dynamic checkpoint. Half the weight memory of bf16, faster single-sequence decoding.

8B parametersllamallama3.1text-generationdecoder-onlygrouped-query-attentionchatfp8
Hugging FaceGitHubarXiv Registry entry

Greedy decoding of one chat prompt, through Linnet and through transformers (KV cache, argmax, bf16).

Prompt

Explain in two sentences why the sky is blue.

Linnet

The sky appears blue because of a phenomenon called Rayleigh scattering, where shorter (blue) wavelengths of light are scattered more than longer (red) wavelengths by the tiny molecules of gases in the Earth's atmosphere. As a result, when sunlight enters the atmosphere, the blue light is dispersed in all directions, giving the sky its blue color.

transformers (KV cache, argmax, bf16)

The sky appears blue because of a phenomenon called Rayleigh scattering, where shorter (blue) wavelengths of light are scattered more than longer (red) wavelengths by the tiny molecules of gases in the Earth's atmosphere. As a result, when sunlight enters the atmosphere, the blue light is dispersed in all directions, giving the sky its blue color.

All 69 generated tokens agree with the reference.