gm8xx8



Apple has released a 7B base model trained on 2.5T tokens from open datasets, primarily in English, with a 2048 context window. the model is trained using PyTorch with the OpenLM framework, combines data from DCLM-BASELINE, StarCoder, and ProofPile2, achieving an MMLU score of 0.6372. apple/DCLM-7B, is fully open-sourced, including pretraining dataset. it is available on 🤗 and released under an open license, including the Apple Sample Code License. 

https://huggingface.co/apple/DCLM-7B