Microsoft OmniParser: Local Install with Miniconda3 (Easy Setup!)
Learn how to install and run Microsoft's OmniParser locally on your computer using Miniconda3. This powerful screen parsing tool lets you extract information from screenshots and control your UI with AI. In this tutorial, we'll cover:
Setting up a conda environment
Installing the necessary dependencies
Downloading and converting the pre-trained model weights
Running a simple demo with Gradio
Get started with OmniParser today and unlock the potential of AI-powered UI interaction!
Install environment:
conda create -n "omni" python==3.12
conda activate omni
pip install -r requirements.txt
Then download the model ckpts files in: https://huggingface.co/microsoft/Omni..., and put them under weights/, default folder structure is: weights/icon_detect, weights/icon_caption_florence, weights/icon_caption_blip2.
Finally, convert the safetensor to .pt file.
python weights/convert_safetensor_to_pt.py
To run gradio demo, simply run:
python gradio_demo.py
https://github.com/microsoft/OmniParser
#omniparser #microsoft #ai #screenparsing #uiautomation #computervision #deeplearning #machinelearning #tutorial #conda #python #gradio #opensource