LLM Anatomy Atlas Die strukturelle Anatomie offener Sprachmodelle

Modellindex

Das vollständige Verzeichnis der in diesem Atlas erfassten Architekturen. Sortieren Sie jede Spalte, filtern Sie nach Labor oder Attention-Familie und öffnen Sie ein Modell für sein vollständiges Strukturprofil. Die Parameterzahl bezeichnet die Gesamtsumme; „Aktiv“ ist die Anzahl der pro Token beteiligten Parameter bei sparsamen Modellen.

80
GPT-2 OpenAI 1.24 1.24 1K MHA12:12 Dicht
GPT-3 175B (paper) OpenAI 175 175 2K MHA96:96 Dicht
LLaMA 1 7B Meta 6.74 6.74 2K MHA32:32 Dicht
Llama 2 70B Meta 68.98 68.98 4K GQA64:8 Dicht
Mistral 7B Mistral 7.24 7.24 32K GQA32:8 Dicht
ChatGLM3 6B GLM 6.2 6.2 8K GQA32:2 Dicht
DeepSeek-Coder 6.7B DeepSeek 6.7 6.7 16K MHA32:32 Dicht
DeepSeek LLM 7B DeepSeek 5.9 5.9 4K MHA32:32 Dicht
DeepSeekMoE 16B DeepSeek 16.2 2.6 4K MHA16:16 64e top6 +2shared
Qwen1.5 7B Qwen 6.6 6.6 32K MHA32:32 Dicht
Mixtral 8x22B Mistral 140.6 39.1 64K GQA48:8 8e top2
DeepSeek-V2 DeepSeek 236 21 160K MLA 160e top6 +2shared
DeepSeek-V2 Lite DeepSeek 15.6 2.5 160K MLA 64e top6 +2shared
GLM-4 9B GLM 9.4 9.4 8K GQA32:2 Dicht
Qwen2 72B Qwen 71.4 71.4 32K GQA64:8 Dicht
Llama 3.1 70B Meta 70.55 70.55 128K GQA64:8 Dicht
Llama 3.1 8B Meta 8.03 8.03 128K GQA32:8 Dicht
Qwen2.5 72B Qwen 71.4 71.4 32K GQA64:8 Dicht
Qwen2.5 7B Qwen 7.6 7.6 32K GQA28:4 Dicht
DeepSeek-V3 DeepSeek 671 37 160K MLA 256e top8 +1shared
DeepSeek-V3 Base DeepSeek 671 37 160K MLA 256e top8 +1shared
DeepSeek-R1 DeepSeek 671 37 160K MLA 256e top8 +1shared
MiniMax-Text-01 MiniMax 456 45.9 10000K GQA64:8 32e top2
Moonlight 16B Kimi 15.8 2.8 8K MLA 64e top6 +2shared
Qwen3 0.6B Qwen 0.6 0.6 40K GQA16:8 Dicht
Qwen3 235B A22B Qwen 234.6 21.7 256K GQA64:4 128e top8
Qwen3 30B A3B Qwen 30.4 3.2 256K GQA32:4 128e top8
Qwen3 32B Qwen 32.1 32.1 40K GQA64:8 Dicht
Qwen3 8B Qwen 7.9 7.9 40K GQA32:8 Dicht
MiniMax-M1 MiniMax 456 45.9 10000K GQA64:8 32e top2
GLM-4.5 GLM 360.4 32.7 128K GQA96:8 160e top8 +1shared
GLM-4.5-Air GLM 107.7 13 128K GQA96:8 128e top8 +1shared
Kimi K2 Base Kimi 1,040.2 32.8 128K MLA 384e top8 +1shared
Step-3 StepFun 321 38 64K MHA 48e top3 +1shared
DeepSeek-V3.1 DeepSeek 671 37 160K MLA 256e top8 +1shared
Hunyuan 7B Hunyuan 7.8 7.8 32K GQA32:8 Dicht
GPT-OSS 120B OpenAI 116.8 5.1 128K GQA64:8 128e top4
GPT-OSS 20B OpenAI 20.9 3.6 128K GQA64:8 32e top4
DeepSeek-V3.2 DeepSeek 671 37 160K MLA 256e top8 +1shared
Kimi K2 0905 Kimi 1,040.2 32.8 256K MLA 384e top8 +1shared
Qwen3-Next 80B A3B Qwen 78.7 2.9 256K GQA16:2 512e top10 +1shared
GLM-4.6 GLM 360.4 32.7 198K GQA96:8 160e top8 +1shared
Kimi Linear 48B A3B Kimi 50 2.8 256K MLA 256e top8 +1shared
MiniMax-M2 MiniMax 228.3 10.6 192K GQA48:8 256e top8
GLM-4.7-Flash GLM 29.4 3.2 198K MLA 64e top4 +1shared
Qwen3-Coder-Next Qwen 78.7 2.9 256K GQA16:2 512e top10 +1shared
GLM-5 GLM 744 40 198K MLA 256e top8 +1shared
Kimi K2.5 Kimi 1,040.2 32.8 256K MLA 384e top8 +1shared
MiniMax-M2.5 MiniMax 228.3 10.6 192K GQA48:8 256e top8
Qwen3.5 122B A10B Qwen 119.9 7.6 256K GQA32:2 256e top8 +1shared
Qwen3.5 27B Qwen 23.7 23.7 256K GQA24:4 Dicht
Qwen3.5 35B A3B Qwen 33.9 2.7 256K GQA16:2 256e top8 +1shared
Qwen3.5 397B A17B Qwen 392.6 13.6 256K GQA32:2 512e top10 +1shared
Qwen3.5 9B Qwen 7.9 7.9 256K GQA16:4 Dicht
Qwen3.5 9B Base Qwen 7.9 7.9 256K GQA16:4 Dicht
Step-3.5-Flash StepFun 196 11 256K MHA64:64 288e top8 +1shared
DeepSeek-V4-Flash DeepSeek 284 13 1024K MQA64:1 256e top6 +1shared
DeepSeek-V4-Flash Base DeepSeek 284 13 1024K MQA64:1 256e top6 +1shared
DeepSeek-V4-Pro DeepSeek 1,606.6 49 1024K MQA128:1 384e top6 +1shared
GLM-5.1 GLM 744 40 198K MLA 256e top8 +1shared
Hunyuan 3 Preview Hunyuan 296.3 19.9 256K GQA64:8 192e top8 +1shared
Hunyuan 3 Preview Base Hunyuan 296.3 19.9 256K GQA64:8 192e top8 +1shared
Kimi K2.6 Kimi 1,040.2 32.8 256K MLA 384e top8 +1shared
MiniMax-M2.7 MiniMax 228.3 10.6 200K GQA48:8 256e top8
Qwen3.6 27B Qwen 23.7 23.7 256K GQA24:4 Dicht
Qwen3.6 35B A3B Qwen 33.9 2.7 256K GQA16:2 256e top8 +1shared
Hunyuan-TurboS 30B A3B Hunyuan 30.3 3.3 256K GQA32:4 128e top8 +1shared
Hunyuan-TurboS 7B Hunyuan 7.8 7.8 256K GQA32:8 Dicht
Step-3.7-Flash StepFun 198 11 256K MHA 288e top8 +1shared
GLM-5.2 GLM 767.4 40.2 1024K MLA 256e top8 +1shared
MiniMax-M3 MiniMax 443.4 25.5 1024K GQA64:4 128e top4 +1shared
MiniMax-M3 MXFP8 MiniMax 443.4 25.5 1024K GQA64:4 128e top4 +1shared
Hunyuan 3 Hunyuan 296.3 19.9 256K GQA64:8 192e top8 +1shared
Kimi K3 Kimi 2,780 103 1024K MLA 896e top16 +2shared
GLM-5.3 GLM 767.4 40.2 1024K MLA 256e top8 +1shared
GLM-5.3-Flash GLM 328.9 13 1024K MLA 288e top8 +1shared
Hunyuan 4 Preview Hunyuan 770 49 1024K MLA 256e top8 +1shared
Qwen3.8 2.4T A95B Qwen 2,399.6 95 256K GQA64:4 512e top10 +1shared
Qwen3.8 27B Qwen 23.7 23.7 256K GQA24:4 Dicht
Qwen3.8-Flash-Next Qwen 123.6 5.1 256K GQA24:2 512e top10 +1shared