OFIR SHIN's picture

1 1

OFIR SHIN

ofirst

·

AI & ML interests

None yet

Recent Activity

new activity about 2 months ago

microsoft/OmniParser:Missing configuration files for transformer support

reacted to merve's post with 🔥 about 2 months ago

Microsoft released a groundbreaking model that can be used for web automation, with MIT license 🔥 https://huggingface.co/microsoft/OmniParser Interesting highlight for me was Mind2Web (a benchmark for web navigation) capabilities of the model, which unlocks agentic behavior for RPA agents. no need for hefty web automation pipelines that get broken when the website/app design changes! Amazing work. Lastly, the authors also fine-tune this model on open-set detection for interactable regions and see if they can use it as a plug-in for VLMs and it actually outperforms off-the-shelf open-set detectors like GroundingDINO. 👏 OmniParser is a state-of-the-art UI parsing/understanding model that outperforms GPT4V in parsing.

View all activity

Organizations

None yet

models

None public yet

datasets

None public yet