Instructions to use HuggingFaceM4/idefics-80b-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceM4/idefics-80b-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-80b-instruct")

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-80b-instruct")
model = AutoModelForImageTextToText.from_pretrained("HuggingFaceM4/idefics-80b-instruct")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use HuggingFaceM4/idefics-80b-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceM4/idefics-80b-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b-instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/HuggingFaceM4/idefics-80b-instruct

SGLang

How to use HuggingFaceM4/idefics-80b-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceM4/idefics-80b-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b-instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceM4/idefics-80b-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceM4/idefics-80b-instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use HuggingFaceM4/idefics-80b-instruct with Docker Model Runner:
```
docker model run hf.co/HuggingFaceM4/idefics-80b-instruct
```

idefics-80b-instruct

Commit History

set the correct vision_config.hidden_act

eca1af7

stas commited on Aug 28, 2023

Update readme and doc from the 80b repo

3bcd940

VictorSanh commited on Aug 22, 2023

Update readme and doc from the 80b repo

1648537

VictorSanh commited on Aug 22, 2023

Update readme and doc from the 80b repo

83f9046

VictorSanh commited on Aug 21, 2023

Update readme and doc from the 80b repo

a2a5c48

VictorSanh commited on Aug 21, 2023

Update readme and doc from the 80b repo

b56116a

VictorSanh commited on Aug 21, 2023

Update readme and doc from the 80b repo

64c8644

VictorSanh commited on Aug 21, 2023

Update readme and doc from the 80b repo

52afb70

VictorSanh commited on Aug 20, 2023

Update readme and doc from the 80b repo

1138740

VictorSanh commited on Aug 19, 2023

Update README.md

a7e32ef

VictorSanh commited on Aug 18, 2023

revert

4dce551

VictorSanh commited on Aug 18, 2023

add tokenizer.json for fast tokenizer

e128d75

VictorSanh commited on Aug 18, 2023

Update README from idefics-80b

6d380f1

Leyo commited on Aug 18, 2023

Update README from idefics-80b

4cf4362

Leyo commited on Aug 18, 2023

Create README.md

676b50a

VictorSanh commited on Aug 15, 2023

add safetensors

db61980

hugo@huggingface.co commited on Aug 8, 2023

update 80b instruct to latest model trained

efd2373

VictorSanh commited on Aug 2, 2023

Update tokenizer_config.json

489226f

VictorSanh commited on Jul 31, 2023

Update config.json (#5)

6b82e8d

VictorSanh commited on Jul 27, 2023

s|IdeficsForCausalLM|IdeficsForVisionText2Text|g

a9325f0

VictorSanh commited on Jul 27, 2023

Create preprocessor_config.json

5674075

VictorSanh commited on Jul 26, 2023

Update config.json

0979cbf

VictorSanh commited on Jul 26, 2023

First commit

aca8e0b

VictorSanh commited on Jul 25, 2023

initial commit

d53eea0

Victor Sanh commited on Jul 25, 2023

Commit History

set the correct vision_config.hidden_act eca1af7

Update readme and doc from the 80b repo 3bcd940

Update readme and doc from the 80b repo 1648537

Update readme and doc from the 80b repo 83f9046

Update readme and doc from the 80b repo a2a5c48

Update readme and doc from the 80b repo b56116a

Update readme and doc from the 80b repo 64c8644

Update readme and doc from the 80b repo 52afb70

Update readme and doc from the 80b repo 1138740

Update README.md a7e32ef

revert 4dce551

add tokenizer.json for fast tokenizer e128d75

Update README from idefics-80b 6d380f1

Update README from idefics-80b 4cf4362

Create README.md 676b50a

add safetensors db61980

update 80b instruct to latest model trained efd2373

Update tokenizer_config.json 489226f

Update config.json (#5) 6b82e8d

s|IdeficsForCausalLM|IdeficsForVisionText2Text|g a9325f0

Create preprocessor_config.json 5674075

Update config.json 0979cbf

First commit aca8e0b

initial commit d53eea0

set the correct vision_config.hidden_act

eca1af7

Update readme and doc from the 80b repo

3bcd940

Update readme and doc from the 80b repo

1648537

Update readme and doc from the 80b repo

83f9046

Update readme and doc from the 80b repo

a2a5c48

Update readme and doc from the 80b repo

b56116a

Update readme and doc from the 80b repo

64c8644

Update readme and doc from the 80b repo

52afb70

Update readme and doc from the 80b repo

1138740

Update README.md

a7e32ef

revert

4dce551

add tokenizer.json for fast tokenizer

e128d75

Update README from idefics-80b

6d380f1

Update README from idefics-80b

4cf4362

Create README.md

676b50a

add safetensors

db61980

update 80b instruct to latest model trained

efd2373

Update tokenizer_config.json

489226f

Update config.json (#5)

6b82e8d

s|IdeficsForCausalLM|IdeficsForVisionText2Text|g

a9325f0

Create preprocessor_config.json

5674075

Update config.json

0979cbf

First commit

aca8e0b

initial commit

d53eea0