AI NEWS.cctiny-random-DeepseekV2ForCausalLM
Randomly initialised `DeepseekV2ForCausalLM` for testing MLA (multi-head latent attention) and MoE code paths without loading a 29 GiB checkpoint. Used by vLLM's `tests/models/transformers/test_backend.py::test_mla`. The attention head dimensions are copied verbatim from deepseek-ai/DeepSeek-V2-Lite-Chat so that `kv_lora_rank + qk_rope_head_dim == 576`, the latent head size every MLA kernel is built around. Only the width, depth and expert counts are shrunk. Layer 0 is dense and the rest are MoE (`first_k_dense_replace=1`), matching the real model. The tokenizer is DeepSeek-V2-Lite-Chat's, unmodified.
Randomly initialised `DeepseekV2ForCausalLM` for testing MLA (multi-head latent attention) and MoE code paths without loading a 29 GiB checkpoint. Used by vLLM's `tests/models/transformers/test_backend.py::test_mla`. The attention head dimensions are copied verbatim from deepseek-ai/DeepSeek-V2-Lite-Chat so that `kv_lora_rank + qk_rope_head_dim == 576`, the latent head size every MLA kernel is built around. Only the width, depth and expert counts are shrunk. Layer 0 is dense and the rest are MoE (`first_k_dense_replace=1`), matching the real model. The tokenizer is DeepSeek-V2-Lite-Chat's, unmodified.