Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Data-Selection
/
data_scorer
like
0
Follow
Data Selection
3
togethercomputer/RedPajama-Data-1T
English
arxiv:
2410.07064
License:
apache-2.0
Model card
Files
Files and versions
Community
main
data_scorer
1 contributor
History:
4 commits
t1101675
Update README.md
d9b2b78
verified
26 days ago
.gitattributes
1.52 kB
initial commit
26 days ago
README.md
968 Bytes
Update README.md
26 days ago
config.json
97 Bytes
Upload folder using huggingface_hub
26 days ago
data_scorer_model.pt
pickle
Detected Pickle imports (3)
"torch._utils._rebuild_tensor_v2"
,
"torch.HalfStorage"
,
"collections.OrderedDict"
What is a pickle import?
247 MB
LFS
Upload folder using huggingface_hub
26 days ago
merges.txt
456 kB
Upload folder using huggingface_hub
26 days ago
special_tokens_map.json
694 Bytes
Upload folder using huggingface_hub
26 days ago
tokenizer.json
2.11 MB
Upload folder using huggingface_hub
26 days ago
tokenizer_config.json
1.23 kB
Upload folder using huggingface_hub
26 days ago
vocab.json
798 kB
Upload folder using huggingface_hub
26 days ago