FineWeb2-C: Help Build Better Language Models in Your Language
β’
10
Daniel van Strien PRO
davanstrien
AI & ML interests
Machine Learning Librarian
Recent Activity
updated
a dataset
about 2 hours ago
data-is-better-together/fineweb-c-progress
updated
a dataset
about 6 hours ago
librarian-bots/dataset_cards_with_metadata
liked
a model
about 18 hours ago
Qwen/QVQ-72B-Preview
Articles
Organizations
davanstrien's activity
replied to
their
post
5 days ago
There are some already in the Argilla instance!
You can also join the discussions here: https://huggingface.co/spaces/HuggingFaceFW/discussion :)
replied to
their
post
5 days ago
Thanks to the hard work of @ivykopal , the first 1,000 annotations for Slovak have been completed! Make sure to give Ivan a follow :)
reacted to
nicolay-r's
post with β€οΈ
5 days ago
Post
2031
π’ Deligted to share the most recent milestone on quick deployment of Named Entity Recognition (NER) in Gen-AI powered systems.
Releasing the bulk-ner 0.25.0 which represent a tiny framework that would save you time for deploing NER with any model.
π Why is this important? In the era of GenAI the handling out textual output might be challenging. Instead, recognizing named-entities via domain-oriented systems for your donwstream LLM would be preferable option.
π¦: https://pypi.org/project/bulk-ner/0.25.0/
π: https://github.com/nicolay-r/bulk-ner
I noticed that the direct adaptaion of the LM for NER would result in spending signifcant amount of time on formatting your texts according to the NER-model needs.
In particular:
1. Processing CONLL format with B-I-O tags from model outputs
2. Input trimming: long input content might not be completely fitted
To cope with these problems, in version 0.25.0 I made a huge steps forward by providing:
β π Python API support: see screenshot below for a quick deployment (see screenshot below πΈ)
β πͺΆ No-string: dependencies are now clear, so it is purely Python implementation for API calls.
β π Simplified output formatting: we use lists to represent texts with inner lists that refer to annotated objects (see screenshot below πΈ)
π We have a colab for a quick start here (or screenshot for bash / Python API πΈ)
https://colab.research.google.com/github/nicolay-r/ner-service/blob/main/NER_annotation_service.ipynb
π The code for pipeline deployment is taken from the AREkit project:
https://github.com/nicolay-r/AREkit
Releasing the bulk-ner 0.25.0 which represent a tiny framework that would save you time for deploing NER with any model.
π Why is this important? In the era of GenAI the handling out textual output might be challenging. Instead, recognizing named-entities via domain-oriented systems for your donwstream LLM would be preferable option.
π¦: https://pypi.org/project/bulk-ner/0.25.0/
π: https://github.com/nicolay-r/bulk-ner
I noticed that the direct adaptaion of the LM for NER would result in spending signifcant amount of time on formatting your texts according to the NER-model needs.
In particular:
1. Processing CONLL format with B-I-O tags from model outputs
2. Input trimming: long input content might not be completely fitted
To cope with these problems, in version 0.25.0 I made a huge steps forward by providing:
β π Python API support: see screenshot below for a quick deployment (see screenshot below πΈ)
β πͺΆ No-string: dependencies are now clear, so it is purely Python implementation for API calls.
β π Simplified output formatting: we use lists to represent texts with inner lists that refer to annotated objects (see screenshot below πΈ)
π We have a colab for a quick start here (or screenshot for bash / Python API πΈ)
https://colab.research.google.com/github/nicolay-r/ner-service/blob/main/NER_annotation_service.ipynb
π The code for pipeline deployment is taken from the AREkit project:
https://github.com/nicolay-r/AREkit
reacted to
their
post with β€οΈ
5 days ago
Post
2872
πΈπ° Hovorte po slovensky? Help build better AI for Slovak!
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
-
- 3 replies
posted
an
update
5 days ago
Post
2872
πΈπ° Hovorte po slovensky? Help build better AI for Slovak!
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
-
-
- 3 replies
posted
an
update
12 days ago
Post
1638
Introducing FineWeb-C ππ, a community-built dataset for improving language models in ALL languages.
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! π
data-is-better-together/fineweb-c
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! π
data-is-better-together/fineweb-c
reacted to
anton-l's
post with π₯
13 days ago
Post
2072
Introducing ππ
π’π§πππππ‘: the best public math pre-training dataset with 50B+ tokens!
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
HuggingFaceTB/finemath
Math remains challenging for LLMs and by training on FineMath we see considerable gains over other math datasets, especially on GSM8K and MATH.
We build the dataset by:
π οΈ carefully extracting math data from Common Crawl;
π iteratively filtering and recalling high quality math pages using a classifier trained on synthetic annotations to identify math reasoning and deduction.
We conducted a series of ablations comparing the performance of Llama-3.2-3B-Base after continued pre-training on FineMath and observe notable gains compared to the baseline model and other public math datasets.
We hope this helps advance the performance of LLMs on math and reasoning! π
Weβre also releasing all the ablation models as well as the evaluation code.
HuggingFaceTB/finemath-6763fb8f71b6439b653482c2
reacted to
stefan-it's
post with β€οΈ
23 days ago
Post
1163
My latest project is the outcome of the last 2+ years working with TPUs from the amazing TPU Research Cloud (TRC) program and training Encoder-only LMs with the TensorFlow Model Garden library.
π Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
π Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with β€οΈ and π₯¨.
π Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
π Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with β€οΈ and π₯¨.
reacted to
davidberenstein1957's
post with π₯
23 days ago
Post
2058
Open Preference Dataset for Text-to-Image Generation by the π€ Community
Open Image Preferences is an Apache 2.0 licensed dataset for text-to-image generation. This dataset contains 10K text-to-image preference pairs across common image generation categories, while using different model families and varying prompt complexities.
https://huggingface.co/blog/image-preferences
Open Image Preferences is an Apache 2.0 licensed dataset for text-to-image generation. This dataset contains 10K text-to-image preference pairs across common image generation categories, while using different model families and varying prompt complexities.
https://huggingface.co/blog/image-preferences
reacted to
thomwolf's
post with π
23 days ago
Post
4447
We are proud to announce
HuggingFaceFW/fineweb-2: A sparkling update to
HuggingFaceFW/fineweb with 1000s of π£οΈlanguages.
We applied the same data-driven approach that led to SOTA English performance inπ· FineWeb to thousands of languages.
π₯ FineWeb2 has 8TB of compressed text data and outperforms other multilingual datasets in our experiments.
The dataset is released under the permissive π ODC-By 1.0 license, and the π» code to reproduce it and our evaluations is public.
We will very soon announce a big community project, and are working on a π blogpost walking you through the entire dataset creation process. Stay tuned!
In the mean time come ask us question on our chat place: HuggingFaceFW/discussion
H/t @guipenedo @hynky @lvwerra as well as @vsabolcec Bettina Messmer @negar-foroutan and @mjaggi
We applied the same data-driven approach that led to SOTA English performance inπ· FineWeb to thousands of languages.
π₯ FineWeb2 has 8TB of compressed text data and outperforms other multilingual datasets in our experiments.
The dataset is released under the permissive π ODC-By 1.0 license, and the π» code to reproduce it and our evaluations is public.
We will very soon announce a big community project, and are working on a π blogpost walking you through the entire dataset creation process. Stay tuned!
In the mean time come ask us question on our chat place: HuggingFaceFW/discussion
H/t @guipenedo @hynky @lvwerra as well as @vsabolcec Bettina Messmer @negar-foroutan and @mjaggi
- 2 replies
posted
an
update
about 1 month ago
Post
503
Increasingly, LLMs are becoming very useful for helping scale annotation tasks, i.e. labelling and filtering. When combined with the structured generation, this can be a very scalable way of doing some pre-annotation without requiring a large team of human annotators.
However, there are quite a few cases where it still doesn't work well. This is a nice paper looking at the limitations of LLM as an annotator for Low Resource Languages: On Limitations of LLM as Annotator for Low Resource Languages (2411.17637).
Humans will still have an important role in the loop to help improve models for all languages (and domains).
However, there are quite a few cases where it still doesn't work well. This is a nice paper looking at the limitations of LLM as an annotator for Low Resource Languages: On Limitations of LLM as Annotator for Low Resource Languages (2411.17637).
Humans will still have an important role in the loop to help improve models for all languages (and domains).
reacted to
andito's
post with π₯
about 1 month ago
Post
1801
SmolVLM speeding locally on a laptop thanks to mlx-vlm and
@Gradio ! Try it with two lines:
pip install git+https://github.com/andimarafioti/mlx-vlm.git@stream-generate-fix
python -m mlx_vlm.chat_ui --model mlx-community/SmolVLM-Instruct-8bit
Gotta love the MLX community! Big thanks to @pcuenq and @prince_canuma !
@Gradio ! Try it with two lines:
pip install git+https://github.com/andimarafioti/mlx-vlm.git@stream-generate-fix
python -m mlx_vlm.chat_ui --model mlx-community/SmolVLM-Instruct-8bit
Gotta love the MLX community! Big thanks to @pcuenq and @prince_canuma !
reacted to
MohamedRashad's
post with π
about 1 month ago
Post
1625
A while back i shared this model
MohamedRashad/arabic-small-nougat that was a finetune from
facebook/nougat-small for the Arabic Language.
Today this humble project has been scaled with new models, new datasets, new space, and a new paper
Check everything throught this collection here:
MohamedRashad/arabic-nougat-673a3f540bd92904c9b92a8e
Today this humble project has been scaled with new models, new datasets, new space, and a new paper
Check everything throught this collection here:
MohamedRashad/arabic-nougat-673a3f540bd92904c9b92a8e
- 1 reply
reacted to
AdinaY's
post with π€
about 1 month ago
Post
1108
Zhipu AI, the Chinese generative AI startup behind CogVideo, just launched their first productized AI Agent - AutoGLM π₯
π https://agent.aminer.cn
With simple text or voice commands, it:
β¨ Simulates phone operations effortlessly
β¨ Autonomously handles 50+ step tasks
β¨ Seamlessly operates across apps
Powered by Zhipu's "Decoupled Interface" and "Self-Evolving Learning Framework" to achieve major performance gains in Phone Use and Web Browser Use!
Meanwhile, GLM4-Edge is now on Hugging Face hubπ
π THUDM/glm-edge-6743283c5809de4a7b9e0b8b
Packed with advanced dialogue + multimodal models:
π± 1.5B / 2B models: Built for mobile & in-car systems
π» 4B / 5B models: Optimized for PCs
π https://agent.aminer.cn
With simple text or voice commands, it:
β¨ Simulates phone operations effortlessly
β¨ Autonomously handles 50+ step tasks
β¨ Seamlessly operates across apps
Powered by Zhipu's "Decoupled Interface" and "Self-Evolving Learning Framework" to achieve major performance gains in Phone Use and Web Browser Use!
Meanwhile, GLM4-Edge is now on Hugging Face hubπ
π THUDM/glm-edge-6743283c5809de4a7b9e0b8b
Packed with advanced dialogue + multimodal models:
π± 1.5B / 2B models: Built for mobile & in-car systems
π» 4B / 5B models: Optimized for PCs
reacted to
merve's
post with π
about 1 month ago
Post
2172
The authors of ColPali trained a retrieval model based on SmolVLM π€
vidore/colsmolvlm-alpha
TLDR;
- ColSmolVLM performs better than ColPali and DSE-Qwen2 on all English tasks
- ColSmolVLM is more memory efficient than ColQwen2 π
TLDR;
- ColSmolVLM performs better than ColPali and DSE-Qwen2 on all English tasks
- ColSmolVLM is more memory efficient than ColQwen2 π
reacted to
davidberenstein1957's
post with π₯
about 1 month ago
Post
1571
π₯ Dataset Drop - Open Image Preferences
BlackForest Labs Flux Dev VS. Stability AI Stable Diffusion Large 3.5
Together with the β data-is-better-together community, we've worked on an Apache 2.0 licensed open image preference dataset based on the fal ai imgsys prompts dataset. Thanks to the awesome community, we have managed to get 5K preference pairs in less than 2 days. The annotation alignment among annotators is great too.
Aashish Kumar won a month of Hugging Face Pro by making the most contributions! Congrats from the entire team π₯
The best thing?! We are not done yet! Let's keep the annotations coming for 5K more in the second part of the sprint! (with more prices to go around).
Dataset: https://huggingface.co/datasets/data-is-better-together/image-preferences-results
BlackForest Labs Flux Dev VS. Stability AI Stable Diffusion Large 3.5
Together with the β data-is-better-together community, we've worked on an Apache 2.0 licensed open image preference dataset based on the fal ai imgsys prompts dataset. Thanks to the awesome community, we have managed to get 5K preference pairs in less than 2 days. The annotation alignment among annotators is great too.
Aashish Kumar won a month of Hugging Face Pro by making the most contributions! Congrats from the entire team π₯
The best thing?! We are not done yet! Let's keep the annotations coming for 5K more in the second part of the sprint! (with more prices to go around).
Dataset: https://huggingface.co/datasets/data-is-better-together/image-preferences-results
reacted to
andito's
post with π₯
about 1 month ago
Post
3278
Let's go! We are releasing SmolVLM, a smol 2B VLM built for on-device inference that outperforms all models at similar GPU RAM usage and tokens throughputs.
- SmolVLM generates tokens 7.5 to 16 times faster than Qwen2-VL! π€―
- Other models at this size crash a laptop, but SmolVLM comfortably generates 17 tokens/sec on a macbook! π
- SmolVLM can be fine-tuned on a Google collab! Or process millions of documents with a consumer GPU!
- SmolVLM even outperforms larger models in video benchmarks, despite not even being trained on videos!
Check out more!
Demo: HuggingFaceTB/SmolVLM
Blog: https://huggingface.co/blog/smolvlm
Model: HuggingFaceTB/SmolVLM-Instruct
Fine-tuning script: https://github.com/huggingface/smollm/blob/main/finetuning/Smol_VLM_FT.ipynb
- SmolVLM generates tokens 7.5 to 16 times faster than Qwen2-VL! π€―
- Other models at this size crash a laptop, but SmolVLM comfortably generates 17 tokens/sec on a macbook! π
- SmolVLM can be fine-tuned on a Google collab! Or process millions of documents with a consumer GPU!
- SmolVLM even outperforms larger models in video benchmarks, despite not even being trained on videos!
Check out more!
Demo: HuggingFaceTB/SmolVLM
Blog: https://huggingface.co/blog/smolvlm
Model: HuggingFaceTB/SmolVLM-Instruct
Fine-tuning script: https://github.com/huggingface/smollm/blob/main/finetuning/Smol_VLM_FT.ipynb
reacted to
nataliaElv's
post with π
about 1 month ago
Post
1631
Would you like to get a high-quality dataset to pre-train LLMs in your language? π
At Hugging Face we're preparing a collaborative annotation effort to build an open-source multilingual dataset as part of the Data is Better Together initiative.
Follow the link below, check if your language is listed and sign up to be a Language Lead!
https://forms.gle/s9nGajBh6Pb9G72J6
At Hugging Face we're preparing a collaborative annotation effort to build an open-source multilingual dataset as part of the Data is Better Together initiative.
Follow the link below, check if your language is listed and sign up to be a Language Lead!
https://forms.gle/s9nGajBh6Pb9G72J6
reacted to
their
post with β€οΈ
about 1 month ago
Post
2477
First dataset for the new Hugging Face Bluesky community organisation:
bluesky-community/one-million-bluesky-posts π¦
π 1M public posts from Bluesky's firehose API
π Includes text, metadata, and language predictions
π¬ Perfect to experiment with using ML for Bluesky π€
Excited to see people build more open tools for a more open social media platform!
π 1M public posts from Bluesky's firehose API
π Includes text, metadata, and language predictions
π¬ Perfect to experiment with using ML for Bluesky π€
Excited to see people build more open tools for a more open social media platform!
posted
an
update
about 1 month ago
Post
2477
First dataset for the new Hugging Face Bluesky community organisation:
bluesky-community/one-million-bluesky-posts π¦
π 1M public posts from Bluesky's firehose API
π Includes text, metadata, and language predictions
π¬ Perfect to experiment with using ML for Bluesky π€
Excited to see people build more open tools for a more open social media platform!
π 1M public posts from Bluesky's firehose API
π Includes text, metadata, and language predictions
π¬ Perfect to experiment with using ML for Bluesky π€
Excited to see people build more open tools for a more open social media platform!