Instructions to use microsoft/kosmos-2.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/kosmos-2.5 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="microsoft/kosmos-2.5")# Load model directly from transformers import AutoProcessor, AutoModelForImageTextToText processor = AutoProcessor.from_pretrained("microsoft/kosmos-2.5") model = AutoModelForImageTextToText.from_pretrained("microsoft/kosmos-2.5") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use microsoft/kosmos-2.5 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "microsoft/kosmos-2.5" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/kosmos-2.5", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/microsoft/kosmos-2.5
- SGLang
How to use microsoft/kosmos-2.5 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "microsoft/kosmos-2.5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/kosmos-2.5", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "microsoft/kosmos-2.5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/kosmos-2.5", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use microsoft/kosmos-2.5 with Docker Model Runner:
docker model run hf.co/microsoft/kosmos-2.5
vllm?
#25 opened 8 months ago
by
ansuglutor
New bigger Model?
#24 opened 8 months ago
by
digitalesingulary
can you provide onnx model weights?
#23 opened 9 months ago
by
ningpp
Poor results, what do i do wrong?
1
#22 opened 9 months ago
by
hanshupe
Limit for max_new_tokens
#21 opened 9 months ago
by
hanshupe
How can i install transformers>=4.56, is that stable version?
2
#20 opened 9 months ago
by
duynvh2k
Replace **dtype** with **torch_dtype** in Model Loading
1
#19 opened 9 months ago
by
DaVinciCode
When will Kosmos-2.5 be integrated into Transformers?
π 1
7
#16 opened 10 months ago
by
digitalesingulary
what is the possible max new token?
#15 opened over 1 year ago
by
PF94
Update pipeline tag
#14 opened over 1 year ago
by
nielsr
Guide to train or fine-tune Kosmos-2.5 for other language
5
#13 opened over 1 year ago
by
GardensOfBabylon29
ImportError: cannot import name 'Kosmos2_5ForConditionalGeneration' from 'transformers'
1
#12 opened over 1 year ago
by
ksarao
Table extraction
1
#11 opened over 1 year ago
by
Jkppp
Upload receipt_00008.png
#10 opened over 1 year ago
by
callsys
Batch inferencing bug: All outputs in the same batch has the same prediction, even for different images
#8 opened almost 2 years ago
by
weihf
Error: 'Kosmos2_5ForConditionalGeneration' from 'transformers' when using the latest transformer version
β 11
12
#7 opened almost 2 years ago
by
Nidhin117
Kosmos-2.5 - Containerized & made available over an API
π 6
2
#6 opened almost 2 years ago
by
AbheekG
Batch pred slower than single image inference on 1x4090?
#5 opened almost 2 years ago
by
04RR
Add example code / notebook
π 6
1
#4 opened almost 2 years ago
by
EwoutH
will you be releasing a transformers version?
π 3
2
#3 opened about 2 years ago
by
jshenoy
No config.json
1
#2 opened about 2 years ago
by
abalogh
Apply for community grant: Academic project
π 3
1
#1 opened about 2 years ago
by
estyle