PleIAs/pleias_3b_literature
Updated
•
22
PleIAs
company
AI & ML interests
Open Science LLMs
Recent Activity
PleIAs's activity
Pclanglais
updated
a
model
about 17 hours ago
Pclanglais
updated
a
model
4 days ago
Pclanglais
updated
a
dataset
6 days ago
davanstrien
posted
an
update
8 days ago
Post
2943
🇸🇰 Hovorte po slovensky? Help build better AI for Slovak!
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
We only need 90 more annotations to include Slovak in the next Hugging Face FineWeb2-C dataset ( data-is-better-together/fineweb-c) release!
Your contribution will help create better language models for 5+ million Slovak speakers.
Annotate here: data-is-better-together/fineweb-c.
Read more about why we're doing it: https://huggingface.co/blog/davanstrien/fineweb2-community
-
- 3 replies
davanstrien
posted
an
update
14 days ago
Post
1671
Introducing FineWeb-C 🌐🎓, a community-built dataset for improving language models in ALL languages.
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! 🌍
data-is-better-together/fineweb-c
Inspired by FineWeb-Edu the community is labelling the educational quality of texts for many languages.
318 annotators, 32K+ annotations, 12 languages - and growing! 🌍
data-is-better-together/fineweb-c
Post
1183
My latest project is the outcome of the last 2+ years working with TPUs from the amazing TPU Research Cloud (TRC) program and training Encoder-only LMs with the TensorFlow Model Garden library.
👉 Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
👉 Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with ❤️ and 🥨.
👉 Link: https://github.com/stefan-it/model-garden-lms
An overview of some features:
- Cheatsheet for setting-up a TPU VM Pod (with all necessary dependencies) to pretrain LMs with TF Model Garden
- Conversion scripts that convert TF Model Garden weights to Hugging Face Transformers-compatible models
- Supported architectures include BERT, BERT with Token Dropping and TEAMS
I also released BERT-based models pretrained on the great Hugging Face FineWeb and FineWeb-Edu datasets (10BT subset). With more to come!
👉 Model Hub Link: https://huggingface.co/model-garden-lms
If you find these resources useful, please give them a like!
Made from Bavarian Oberland with ❤️ and 🥨.
davanstrien
posted
an
update
about 1 month ago
Post
505
Increasingly, LLMs are becoming very useful for helping scale annotation tasks, i.e. labelling and filtering. When combined with the structured generation, this can be a very scalable way of doing some pre-annotation without requiring a large team of human annotators.
However, there are quite a few cases where it still doesn't work well. This is a nice paper looking at the limitations of LLM as an annotator for Low Resource Languages: On Limitations of LLM as Annotator for Low Resource Languages (2411.17637).
Humans will still have an important role in the loop to help improve models for all languages (and domains).
However, there are quite a few cases where it still doesn't work well. This is a nice paper looking at the limitations of LLM as an annotator for Low Resource Languages: On Limitations of LLM as Annotator for Low Resource Languages (2411.17637).
Humans will still have an important role in the loop to help improve models for all languages (and domains).
davanstrien
posted
an
update
about 1 month ago
Post
2481
First dataset for the new Hugging Face Bluesky community organisation:
bluesky-community/one-million-bluesky-posts 🦋
📊 1M public posts from Bluesky's firehose API
🔍 Includes text, metadata, and language predictions
🔬 Perfect to experiment with using ML for Bluesky 🤗
Excited to see people build more open tools for a more open social media platform!
📊 1M public posts from Bluesky's firehose API
🔍 Includes text, metadata, and language predictions
🔬 Perfect to experiment with using ML for Bluesky 🤗
Excited to see people build more open tools for a more open social media platform!
davanstrien
posted
an
update
about 1 month ago
Post
1351
The Bluesky AT Protocol unlocks exciting possibilities:
- Building custom feeds using ML
- Creating dashboards for data exploration
- Developing custom models for Bluesky
To gather Bluesky resources on the Hub, I've created a community org: https://huggingface.co/bluesky-community
My first rather modest contribution is a dashboard that shows the number of posts every second. Drinking straight from the firehose API 🚰
bluesky-community/bluesky-posts-over-time
- Building custom feeds using ML
- Creating dashboards for data exploration
- Developing custom models for Bluesky
To gather Bluesky resources on the Hub, I've created a community org: https://huggingface.co/bluesky-community
My first rather modest contribution is a dashboard that shows the number of posts every second. Drinking straight from the firehose API 🚰
bluesky-community/bluesky-posts-over-time
- 1 reply
davanstrien
posted
an
update
about 2 months ago
Post
1310
huggingface.co/DIBT is dead! Long live https://huggingface.co/data-is-better-together!
We're working on some very cool projects so we're doing a bit of tidying of the Data is Better Together Hub org 🤓
catherinearnett
authored
3
papers
2 months ago
BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training
Paper
•
2409.04599
•
Published
•
1
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models
Paper
•
2311.09194
•
Published
Toxicity of the Commons: Curating Open-Source Pre-Training Data
Paper
•
2410.22587
•
Published
•
10
eliotj
authored
a
paper
2 months ago
davanstrien
posted
an
update
2 months ago
Post
2529
Excited to see my weird
davanstrien/ufo-ColPali dataset featured in a video by
@sabrinaesaquino
!
The video covers using ColPali with Binary Quantization in Qdant to accelerate retrieval. 2x speed up with no performance drop in results 🛸
Video: https://youtu.be/_A90A-grwIc?si=oB3JAhJG8VQUZGLz
Blog post: https://danielvanstrien.xyz/posts/post-with-code/colpali-qdrant/2024-10-02_using_colpali_with_qdrant.html
The video covers using ColPali with Binary Quantization in Qdant to accelerate retrieval. 2x speed up with no performance drop in results 🛸
Video: https://youtu.be/_A90A-grwIc?si=oB3JAhJG8VQUZGLz
Blog post: https://danielvanstrien.xyz/posts/post-with-code/colpali-qdrant/2024-10-02_using_colpali_with_qdrant.html
-
- 2 replies
davanstrien
posted
an
update
3 months ago
Post
1248
ColPali is an exciting new approach to multimodal document retrieval, but some doubt its practical use with existing vector DBs.
It turns out it's super easy to use Qdrant to index and search ColPali embeddings efficiently.
Blog post here: https://danielvanstrien.xyz/posts/post-with-code/colpali-qdrant/2024-10-02_using_colpali_with_qdrant.html
Very silly demo: davanstrien/ufo-ColPali-Search
It turns out it's super easy to use Qdrant to index and search ColPali embeddings efficiently.
Blog post here: https://danielvanstrien.xyz/posts/post-with-code/colpali-qdrant/2024-10-02_using_colpali_with_qdrant.html
Very silly demo: davanstrien/ufo-ColPali-Search
davanstrien
posted
an
update
3 months ago
Post
2189
Yesterday, I shared a blog post on generating data for fine-tuning ColPali using the
Qwen/Qwen2-VL-7B-Instruct model.
To simplify testing this approach, I created a Space that lets you generate queries from an input document page image: davanstrien/ColPali-Query-Generator
I think there is much room for improvement, but I'm excited about the potential for relatively small VLMs to create synthetic data.
You can read the original blog post that goes into more detail here: https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html
To simplify testing this approach, I created a Space that lets you generate queries from an input document page image: davanstrien/ColPali-Query-Generator
I think there is much room for improvement, but I'm excited about the potential for relatively small VLMs to create synthetic data.
You can read the original blog post that goes into more detail here: https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html
davanstrien
posted
an
update
3 months ago
Post
3170
ColPali is revolutionizing multimodal retrieval, but could it be even more effective with domain-specific fine-tuning?
Check out my latest blog post, where I guide you through creating a ColPali fine-tuning dataset using Qwen/Qwen2-VL-7B-Instruct to generate queries for a collection of UFO documents sourced from the Internet Archive.
The post covers:
- Introduction to data for ColPali models
- Using Qwen2-VL for retrieval query generation
- Tips for better query generation
Check out the post here:
https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html
The resulting Hugging Face dataset: davanstrien/ufo-ColPali
Check out my latest blog post, where I guide you through creating a ColPali fine-tuning dataset using Qwen/Qwen2-VL-7B-Instruct to generate queries for a collection of UFO documents sourced from the Internet Archive.
The post covers:
- Introduction to data for ColPali models
- Using Qwen2-VL for retrieval query generation
- Tips for better query generation
Check out the post here:
https://danielvanstrien.xyz/posts/post-with-code/colpali/2024-09-23-generate_colpali_dataset.html
The resulting Hugging Face dataset: davanstrien/ufo-ColPali
- 1 reply
davanstrien
posted
an
update
4 months ago
Post
450
🛸 I'm working on a pipeline for creating domain-specific ColPali fine-tuning datasets using a collection of UFO newsletters from the Internet Archive as a case study.
I will have a full notebook to share on Monday, but you can already take a look at the dataset here: davanstrien/ufo-ColPali
I will have a full notebook to share on Monday, but you can already take a look at the dataset here: davanstrien/ufo-ColPali
davanstrien
posted
an
update
4 months ago
Post
1691
Almost ready: search for a Hugging Face dataset on the Hub from information in the datasets viewer preview!
Soon, you can find deep-cut datasets even if they don't have a full dataset card (you should still document your datasets!)
You can help improve this project by rating synthetic user search queries for hub datasets.
If you have a Hub login, you can start annotating in Argilla
in < 5 seconds here: https://davanstrien-my-argilla.hf.space/dataset/1100a091-7f3f-4a6e-ad51-4e859abab58f/annotation-mode
I need to do some tidying, but I'll share all the code and in-progress datasets for this soon!
Soon, you can find deep-cut datasets even if they don't have a full dataset card (you should still document your datasets!)
You can help improve this project by rating synthetic user search queries for hub datasets.
If you have a Hub login, you can start annotating in Argilla
in < 5 seconds here: https://davanstrien-my-argilla.hf.space/dataset/1100a091-7f3f-4a6e-ad51-4e859abab58f/annotation-mode
I need to do some tidying, but I'll share all the code and in-progress datasets for this soon!