Qwen 2.5 0.5B is one of the best compact open-source models available. However, its 151K vocabulary size creates significant embedding memory overhead on constrained edge devices.
Nova-Mini uses a curated 16,384 BPE vocabulary, keeping embedding matrices lightweight and maximizing attention capacity across its 22 layers.