Instructions to use hrithik-dev8/tiny-random-MiniCPM-o-2_6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hrithik-dev8/tiny-random-MiniCPM-o-2_6 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="hrithik-dev8/tiny-random-MiniCPM-o-2_6", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("hrithik-dev8/tiny-random-MiniCPM-o-2_6", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use hrithik-dev8/tiny-random-MiniCPM-o-2_6 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hrithik-dev8/tiny-random-MiniCPM-o-2_6"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hrithik-dev8/tiny-random-MiniCPM-o-2_6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/hrithik-dev8/tiny-random-MiniCPM-o-2_6

SGLang

How to use hrithik-dev8/tiny-random-MiniCPM-o-2_6 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hrithik-dev8/tiny-random-MiniCPM-o-2_6" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hrithik-dev8/tiny-random-MiniCPM-o-2_6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hrithik-dev8/tiny-random-MiniCPM-o-2_6" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hrithik-dev8/tiny-random-MiniCPM-o-2_6",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use hrithik-dev8/tiny-random-MiniCPM-o-2_6 with Docker Model Runner:
```
docker model run hf.co/hrithik-dev8/tiny-random-MiniCPM-o-2_6
```

tiny-random-MiniCPM-o-2_6 / tokenizer_config.json

hrithik-dev8

Upload folder using huggingface_hub

1c546e7 verified 4 months ago

raw

history blame contribute delete

13.1 kB

	{
	"add_bos_token": false,
	"add_prefix_space": false,
	"added_tokens_decoder": {
	"4900": {
	"content": "<unk>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4901": {
	"content": "<\|endoftext\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4902": {
	"content": "<\|im_start\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4903": {
	"content": "<\|im_end\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4904": {
	"content": "<\|object_ref_start\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4905": {
	"content": "<\|object_ref_end\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4906": {
	"content": "<\|box_start\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4907": {
	"content": "<\|box_end\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4908": {
	"content": "<\|quad_start\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4909": {
	"content": "<\|quad_end\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4910": {
	"content": "<\|vision_start\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4911": {
	"content": "<\|vision_end\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4912": {
	"content": "<\|vision_pad\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4913": {
	"content": "<\|image_pad\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4914": {
	"content": "<\|video_pad\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4915": {
	"content": "<image>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4916": {
	"content": "</image>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4917": {
	"content": "<ref>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4918": {
	"content": "</ref>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4919": {
	"content": "<box>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4920": {
	"content": "</box>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4921": {
	"content": "<quad>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4922": {
	"content": "</quad>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4923": {
	"content": "<point>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4924": {
	"content": "</point>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4925": {
	"content": "<slice>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4926": {
	"content": "</slice>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4927": {
	"content": "<image_id>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4928": {
	"content": "</image_id>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4929": {
	"content": "<unit>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4930": {
	"content": "</unit>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4931": {
	"content": "<asr>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4932": {
	"content": "</asr>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4933": {
	"content": "<query>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4934": {
	"content": "</query>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4935": {
	"content": "<\|audio_start\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4936": {
	"content": "<\|audio\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4937": {
	"content": "<\|audio_end\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4938": {
	"content": "<\|spk_bos\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4939": {
	"content": "<\|spk\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4940": {
	"content": "<\|spk_eos\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4941": {
	"content": "<\|tts_bos\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4942": {
	"content": "<\|tts_eos\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4943": {
	"content": "<\|listen\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4944": {
	"content": "<\|speak\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4945": {
	"content": "<\|interrupt\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4946": {
	"content": "<\|vad_start\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4947": {
	"content": "<\|vad_end\|>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4948": {
	"content": "<reserved_43>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	},
	"4949": {
	"content": "<reserved_53>",
	"lstrip": false,
	"normalized": false,
	"rstrip": false,
	"single_word": false,
	"special": true
	}
	},
	"additional_special_tokens": [
	"<image>",
	"</image>",
	"<ref>",
	"</ref>",
	"<box>",
	"</box>",
	"<quad>",
	"</quad>",
	"<point>",
	"</point>",
	"<slice>",
	"</slice>",
	"<image_id>",
	"</image_id>",
	"<unit>",
	"</unit>",
	"<asr>",
	"</asr>",
	"<query>",
	"</query>",
	"<\|audio_start\|>",
	"<\|audio\|>",
	"<\|audio_end\|>",
	"<\|spk_bos\|>",
	"<\|spk\|>",
	"<\|spk_eos\|>",
	"<\|tts_bos\|>",
	"<\|tts_eos\|>",
	"<\|listen\|>",
	"<\|speak\|>",
	"<\|interrupt\|>",
	"<\|vad_start\|>",
	"<\|vad_end\|>",
	"<reserved_43>",
	"<reserved_53>"
	],
	"auto_map": {
	"AutoTokenizer": [
	"openbmb/MiniCPM-o-2_6--tokenization_minicpmo_fast.MiniCPMOTokenizerFast",
	null
	]
	},
	"bos_token": "<\|im_start\|>",
	"chat_template": "{%- if tools %}\n {{- '<\|im_start\|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool \| tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><\|im_end\|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<\|im_start\|>system\\n' + messages[0]['content'] + '<\|im_end\|>\\n' }}\n {%- else %}\n {{- '<\|im_start\|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<\|im_end\|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<\|im_start\|>' + message.role + '\\n' + message.content + '<\|im_end\|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<\|im_start\|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments \| tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<\|im_end\|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<\|im_start\|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<\|im_end\|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<\|im_start\|>assistant\\n' }}\n{%- endif %}\n",
	"clean_up_tokenization_spaces": false,
	"eos_token": "<\|im_end\|>",
	"errors": "replace",
	"extra_special_tokens": {},
	"model_max_length": 131072,
	"pad_token": "<\|endoftext\|>",
	"split_special_tokens": false,
	"tokenizer_class": "MiniCPMOTokenizer",
	"unk_token": "<unk>",
	"bos_token_id": 4901,
	"eos_token_id": 4903
	}