Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloom")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom") model = AutoModelForCausalLM.from_pretrained("bigscience/bloom") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use bigscience/bloom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloom
- SGLang
How to use bigscience/bloom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloom with Docker Model Runner:
docker model run hf.co/bigscience/bloom
Commit History
Fixing italics that aren't rendering for "BigScience Large Open-science Open-access Multilingual Language Model" 833b211
Update README.md (#18) 44651d2
ybelkada commited on
Update README.md (#15) 0aa20cd
ybelkada commited on
Update README.md f25baa0
Younes Belkada commited on
Update README.md 290df47
Younes Belkada commited on
Update README.md (#16) 45cf2c9
ybelkada commited on
Adding blurb about mistaking sentience. (#12) dc534f6
Update README.md (#14) 9d151d5
Update README.md a339d03
Younes Belkada commited on
Update README.md 826ae56
Younes Belkada commited on
Update README.md 215bee9
Edit intermediary-checkpoint warning 55d00a5
Teven Le Scao commited on
Update README.md e8fa22e
Younes Belkada commited on
Update README.md cf39b50
Younes Belkada commited on
Update README.md 39b727a
Younes Belkada commited on
Update README.md 945bfb3
Younes Belkada commited on
Update README.md 4320d65
Younes Belkada commited on
Update README.md 3351c7c
Younes Belkada commited on
Update README.md 6dbe969
Younes Belkada commited on
Add correct pipeline tag (#8) 0bc3d2e
ybelkada commited on
Open-source -> Open-science in citation 8ffc30e
Update date. 908fd7c
Moves Chris up in the author list 9b8168a
Changing name from "Open-source" to "Open-science" 2701b5e
Updating the loss and ppl values for today. 114e931
Updates the BLOOM model card to sync with the updated https://github.com/bigscience-workshop/model_card (#2) 472cdd0
Update README.md e9247ea
Update README.md a655f9e
Younes Belkada commited on
Update Model Card 536a915
Younes Belkada commited on
initial commit 0b49f22
Younes Belkada commited on