Instructions to use MadMarx37/deepseek-coder-1.3b-python-peft with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MadMarx37/deepseek-coder-1.3b-python-peft with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MadMarx37/deepseek-coder-1.3b-python-peft")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("MadMarx37/deepseek-coder-1.3b-python-peft")
model = AutoModelForCausalLM.from_pretrained("MadMarx37/deepseek-coder-1.3b-python-peft")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use MadMarx37/deepseek-coder-1.3b-python-peft with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MadMarx37/deepseek-coder-1.3b-python-peft"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MadMarx37/deepseek-coder-1.3b-python-peft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MadMarx37/deepseek-coder-1.3b-python-peft

SGLang

How to use MadMarx37/deepseek-coder-1.3b-python-peft with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MadMarx37/deepseek-coder-1.3b-python-peft" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MadMarx37/deepseek-coder-1.3b-python-peft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MadMarx37/deepseek-coder-1.3b-python-peft" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MadMarx37/deepseek-coder-1.3b-python-peft",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MadMarx37/deepseek-coder-1.3b-python-peft with Docker Model Runner:
```
docker model run hf.co/MadMarx37/deepseek-coder-1.3b-python-peft
```

deepseek-coder-1.3b-python-peft / README.md

MadMarx37

Update README.md

570f146 verified about 2 years ago

preview code

raw

history blame contribute delete

3.41 kB

	---
	library_name: transformers
	tags:
	- code
	license: mit
	datasets:
	- ArtifactAI/arxiv_python_research_code
	language:
	- en
	pipeline_tag: text-generation
	---

	# Model Card for Model ID

	<!-- Provide a quick summary of what the model is/does. -->
	A parameter-efficient finetune (using LoRA) of DeepSeek Coder 1.3B finetuned on Python code.


	## Model Details

	### Model Description

	<!-- Provide a longer summary of what this model is. -->

	A finetune of DeepSeek Coder 1.3B finetuned on 1000 examples of Python code from the ArtifactAI/arxiv_python_research_code dataset.

	- Model type: Text Generation
	- Language(s) (NLP): English, Python
	- Finetuned from model: deepseek-ai/deepseek-coder-1.3b-base

	### Model Sources [optional]

	<!-- Provide the basic links for the model. -->

	- Repository: https://github.com/kevin-v96/python-codecomplete-lm

	## Uses

	<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
	To generate Python code

	## How to Get Started with the Model

	Use the code below to get started with the model.

	```
	from transformers import pipeline

	model_name = "MadMarx37/deepseek-coder-1.3b-python-peft"

	def generate_output(input):
	# Run text generation pipeline with our next model
	pipe = pipeline(task="text-generation", model=model_name, tokenizer=model_name, max_length=max_length)
	result = pipe(input)
	print(result[0]['generated_text'])
	```

	## Training Details

	#### Training Hyperparameters

	- Training regime: fp16 mixed-precision with original model loaded in 4bits with bitsandbytes <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
	- learning_rate = 2e-3
	- lr_scheduler_type = 'cosine_with_restarts'
	- max_grad_norm = 0.001
	- weight_decay = 0.001
	- num_train_epochs = 15
	- eval_strategy = "steps"
	- eval_steps = 25

	#### Speeds, Sizes, Times [optional]

	<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

	1.3B parameters. Training time of ~2 hours on an RTX3080.

	## Evaluation

	<!-- This section describes the evaluation protocols and provides the results. -->

	### Testing Data, Factors & Metrics

	#### Testing Data

	<!-- This should link to a Dataset Card if possible. -->

	https://huggingface.co/datasets/ArtifactAI/arxiv_python_research_code

	#### Metrics

	<!-- These are the evaluation metrics being used, ideally with a description of why. -->

	Standard training and eval loss from the HF SFTTrainer.

	### Results

	Training Loss: 0.074100
	Validation Loss: 0.022271

	#### Summary

	The training had some unstability in the gradient norms, but the overall trend in both training and validation loss
	were downward, and validation loss has almost plateaud, which is ideally where we want our model. The code generation on the same
	prompts that we tested the original model on also seem better with the finetuned model. A good way to make the model better, if
	we wanted to increase the finetuning data, would be to also increase the epochs.

	The training run metrics can be seen here:
	https://wandb.ai/kevinv3796/python-autocomplete-deepseek/reports/Supervised-Finetuning-run-for-DeepSeek-Coder-1-3B-on-Python-Code--Vmlldzo3NzQ4NjY0?accessToken=bo0rlzp0yj9vxf1xe3fybfv6rbgl97w5kkab478t8f5unbwltdczy63ba9o9kwjp