Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search
Paper
•
2412.18319
•
Published
•
34
Huanjin Yao
HuanjinYao
AI & ML interests
None yet
Recent Activity
authored
a paper
8 days ago
Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via
Collective Monte Carlo Tree Search
Organizations
HuanjinYao's activity
authored
a
paper
8 days ago
upvoted
a
paper
12 days ago
liked
a
Space
7 months ago
updated
a
model
7 months ago
About video instruction data
2
#15 opened 7 months ago
by
HuanjinYao
reacted to
merve's
post with 👀
7 months ago
Post
2060
Do we fully leverage ViT encoders in vision language models?
A new paper (by @HuanjinYao et al) built a dense connector that does it better! HuanjinYao/DenseConnector-v1.5-8B
HuanjinYao/denseconnector-66500e173fc8c9f05dc98dea
VLMs consist of an image encoder block, a projection layer that projects image embeddings to text embedding space and then a text decoder sequentially connected 📖
This paper explores using intermediate states of image encoder and not a single output 🤩
The authors explore three different ways of instantiating dense connector: sparse token integration, sparse channel integration and dense channel integration. (see paper on how they do it Dense Connector for MLLMs (2405.13800))
They explore all three of them integrated to LLaVA 1.5 and found out each of the new models are superior to the original LLaVA 1.5 🥹 I tried the model and it seems to work very well. As part of the release, the authors have released various ckpts based on different decoders (Vicuna 7/13B and Llama 3-8B) that you can find in the collection 🤗
A new paper (by @HuanjinYao et al) built a dense connector that does it better! HuanjinYao/DenseConnector-v1.5-8B
HuanjinYao/denseconnector-66500e173fc8c9f05dc98dea
VLMs consist of an image encoder block, a projection layer that projects image embeddings to text embedding space and then a text decoder sequentially connected 📖
This paper explores using intermediate states of image encoder and not a single output 🤩
The authors explore three different ways of instantiating dense connector: sparse token integration, sparse channel integration and dense channel integration. (see paper on how they do it Dense Connector for MLLMs (2405.13800))
They explore all three of them integrated to LLaVA 1.5 and found out each of the new models are superior to the original LLaVA 1.5 🥹 I tried the model and it seems to work very well. As part of the release, the authors have released various ckpts based on different decoders (Vicuna 7/13B and Llama 3-8B) that you can find in the collection 🤗
updated
a
Space
7 months ago
updated
a
collection
7 months ago
updated
a
model
7 months ago
Space GPU grant
3
#1 opened 7 months ago
by
merve
updated
a
collection
8 months ago
updated
2
models
8 months ago
upvoted
a
paper
8 months ago
updated
a
collection
8 months ago
updated
a
model
8 months ago
Link the model to the paper on Daily Paper page to increase visibility
#1 opened 8 months ago
by
AdinaY
authored
a
paper
8 months ago
updated
a
collection
8 months ago