Zac Zuo

3mo ago

Moonlight - Efficient, Open-Source LLMs from Moonshot AI

Moonlight is the open-source 3B/16B MoE LLMs from Moonshot AI, trained with the Muon optimizer for ~2x compute efficiency compared to AdamW. Pretrained, instruct-tuned, and intermediate checkpoints available.