Moonlight is the open-source 3B/16B MoE LLMs from Moonshot AI, trained with the Muon optimizer for ~2x compute efficiency compared to AdamW. Pretrained, instruct-tuned, and intermediate checkpoints available.
Moonlight is the open-source 3B/16B MoE LLMs from Moonshot AI, trained with the Muon optimizer for ~2x compute efficiency compared to AdamW. Pretrained, instruct-tuned, and intermediate checkpoints available.
