Stability-AI/StableLM: StableLM: Stability AI Language Models

It looks like this is using The Pile and a few other datasets.