yulan-team/YuLan-Mini
Text Generation
•
Updated
•
566
•
26
YuLan-Mini
A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.
Note A highly capable 2.4B lightweight LLM using only 1T pre-training data.
-
yulan-team/YuLan-Mini-Datasets
Updated • 261 • 8 -
YuLan-Mini: An Open Data-efficient Language Model
Paper • 2412.17743 • Published • 59
yulan-team/YuLan-Mini-Before-Annealing
Updated • 34 • 5Note The model & optimizer states of the last curriculum phase before learning rate annealing.
yulan-team/YuLan-Mini-Phase20
Updated • 8 • 2Note The model & optimizer states of the 20th curriculum phase.