illuin/grouse
Viewer
•
Updated
•
160
•
79
•
1
Illuin Technology
company
Verified
AI & ML interests
None defined yet.
Recent Activity
View all activity
illuin's activity
antonioloison
updated
a
dataset
25 days ago
manu
authored
a
paper
3 months ago
gautierviaud
authored
a
paper
4 months ago
antonioloison
authored
a
paper
4 months ago
Post
735
ColPali: A new approach to efficient and intelligent document retrieval 🚀
Our latest research paper, "ColPali: Efficient Document Retrieval with Vision Language Models," introduces a groundbreaking approach to large-scale visual document analysis. By leveraging Vision Language Models (VLMs), we have created a new framework for document retrieval that's both powerful and efficient.
Key Insights:
💡 ColPali combines ColBERT's multi-vector strategy with VLMs' document understanding capabilities
⚙️ ColPali is based on PaliGemma-3B (SigLIP, Gemma-2B) + a linear projection layer and is trained to maximize the similarity between the document and the query embeddings
📊 The Vision Document Retrieval benchmark (ViDoRe) is a challenging dataset that spans various industry topics and aims at matching real-life retrieval scenarios
🏆 ColPali outperforms existing models on all datasets in ViDoRe (average NDCG@5 of 81.3% vs 67.0% for the best baseline model)
⚡ ColPali is faster at document embedding compared to traditional PDF parser pipelines, making ColPali viable for industrial use
🔍 ColPali is highly interpretable thanks to patch-based similarity maps
Dive deeper into ColPali and explore our resources:
📑 Full paper: arxiv.org/abs/2407.01449
🛠️ Datasets, model weights, evaluation code, leaderboard, demos: huggingface.co/vidore
Shoutout to my amazing co-authors Manuel Faysse ( @manu ) and Hugues Sibille ( @HugSib ). We are grateful for the invaluable feedback from Bilel Omrani, Gautier Viaud, Celine Hudelot, and Pierre Colombo. This work is sponsored by ILLUIN Technology. ✨
Our latest research paper, "ColPali: Efficient Document Retrieval with Vision Language Models," introduces a groundbreaking approach to large-scale visual document analysis. By leveraging Vision Language Models (VLMs), we have created a new framework for document retrieval that's both powerful and efficient.
Key Insights:
💡 ColPali combines ColBERT's multi-vector strategy with VLMs' document understanding capabilities
⚙️ ColPali is based on PaliGemma-3B (SigLIP, Gemma-2B) + a linear projection layer and is trained to maximize the similarity between the document and the query embeddings
📊 The Vision Document Retrieval benchmark (ViDoRe) is a challenging dataset that spans various industry topics and aims at matching real-life retrieval scenarios
🏆 ColPali outperforms existing models on all datasets in ViDoRe (average NDCG@5 of 81.3% vs 67.0% for the best baseline model)
⚡ ColPali is faster at document embedding compared to traditional PDF parser pipelines, making ColPali viable for industrial use
🔍 ColPali is highly interpretable thanks to patch-based similarity maps
Dive deeper into ColPali and explore our resources:
📑 Full paper: arxiv.org/abs/2407.01449
🛠️ Datasets, model weights, evaluation code, leaderboard, demos: huggingface.co/vidore
Shoutout to my amazing co-authors Manuel Faysse ( @manu ) and Hugues Sibille ( @HugSib ). We are grateful for the invaluable feedback from Bilel Omrani, Gautier Viaud, Celine Hudelot, and Pierre Colombo. This work is sponsored by ILLUIN Technology. ✨
tonywu71
authored
a
paper
6 months ago
manu
authored
a
paper
6 months ago
BrunoHays
updated
a
dataset
10 months ago
manu
authored
3
papers
11 months ago
Post
These past months, I've been busy baking a special sort of Croissant 🥐 with an awesome team !
🥐 CroissantLLM is a truly bilingual language model trained on 3 trillion tokens of French and English data. In its size category (<2B), it is the best model in French, but it also rivals the best monolingual English models !
💾 To train it, we collected, filtered and cleaned huge quantities of permissively licensed French data, across various domains (legal, administrative, cultural, scientific), and different text modalities (speech transcriptions, movie subtitles, encyclopedias, forums, webpages)...
⚖️ Assessing LLM performance is not easy, especially outside of English, and to this end we crafted a novel evaluation benchmark, FrenchBench, aiming to assess reasoning, factual knowledge, and linguistic capabilities of models in French !
🔎 The best current LLMs are hidden behind a shroud of mystery, trained with undisclosed training data mixes or strategies. We go the opposite way, releasing all of the project's artefacts (model checkpoints, data, training details, evaluation benchmarks...) We obtain 81 % of the Stanford FMTI transparency criterias, far ahead of even most open initiatives !
🧪Beyond a powerful industrial resource, our transparent initiative is a stepping stone for many scientific questions ! How does teaching a model two languages instead of one splits its monolingual ability ? Does training on so much French help the model integrate French-centric knowledge and cultural biases ? How does the model memorize the training data ?
Many more things to say, for those interested, I recommend checking out:
🗞️ The blogpost: https://huggingface.co/blog/manu/croissant-llm-blog
📖 The 45 page report with lots of gems: https://arxiv.org/abs/2402.00786
🤖 Models, Data, Demo: https://huggingface.co/croissantllm
🥐 CroissantLLM is a truly bilingual language model trained on 3 trillion tokens of French and English data. In its size category (<2B), it is the best model in French, but it also rivals the best monolingual English models !
💾 To train it, we collected, filtered and cleaned huge quantities of permissively licensed French data, across various domains (legal, administrative, cultural, scientific), and different text modalities (speech transcriptions, movie subtitles, encyclopedias, forums, webpages)...
⚖️ Assessing LLM performance is not easy, especially outside of English, and to this end we crafted a novel evaluation benchmark, FrenchBench, aiming to assess reasoning, factual knowledge, and linguistic capabilities of models in French !
🔎 The best current LLMs are hidden behind a shroud of mystery, trained with undisclosed training data mixes or strategies. We go the opposite way, releasing all of the project's artefacts (model checkpoints, data, training details, evaluation benchmarks...) We obtain 81 % of the Stanford FMTI transparency criterias, far ahead of even most open initiatives !
🧪Beyond a powerful industrial resource, our transparent initiative is a stepping stone for many scientific questions ! How does teaching a model two languages instead of one splits its monolingual ability ? Does training on so much French help the model integrate French-centric knowledge and cultural biases ? How does the model memorize the training data ?
Many more things to say, for those interested, I recommend checking out:
🗞️ The blogpost: https://huggingface.co/blog/manu/croissant-llm-blog
📖 The 45 page report with lots of gems: https://arxiv.org/abs/2402.00786
🤖 Models, Data, Demo: https://huggingface.co/croissantllm
- 3 replies
gautierviaud
authored
a
paper
11 months ago
manu
authored
a
paper
about 1 year ago
BrunoHays
updated
3
datasets
over 1 year ago
manu
updated
a
model
over 1 year ago
BrunoHays
updated
a
dataset
almost 2 years ago