| --- |
| library_name: peft |
| base_model: meta-llama/Llama-2-7b-hf |
| datasets: |
| - jtatman/python-code-dataset-500k |
| license: mit |
| --- |
| |
| ## Citation |
|
|
| ```bibtex |
| @misc{Molino2019, |
| author = {Piero Molino and Yaroslav Dudin and Sai Sumanth Miryala}, |
| title = {Ludwig: a type-based declarative deep learning toolbox}, |
| year = {2019}, |
| eprint = {arXiv:1909.07930}, |
| } |
| ``` |
| <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
| # Model Card for Model ID |
|
|
| <!-- Provide a quick summary of what the model is/does. --> |
|
|
|
|
|
|
|
|
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| <!-- Provide a longer summary of what this model is. --> |
|
|
| This model focuses on fine-tuning the Llama-2 7B large language model for Python code generation. The project leverages Ludwig, an open-source toolkit, and a dataset of 500k Python code samples from Hugging Face. The model applies techniques such as prompt templating, zero-shot inference, and few-shot learning, enhancing the model's performance in generating Python code snippets efficiently. |
|
|
| - **Developed by:** Kevin Geejo, Aniket Yadav, Rishab Pandey |
|
|
| - **Model type:** Fine-tuned Llama-2 7B for Python code generation |
| - **Language(s) (NLP):** Python (for code generation tasks) |
| - **License:** Not explicitly mentioned, but Llama-2 models are typically governed by Meta AI’s open-source licensing |
| - **Finetuned from model [optional]:** Llama-2 7B (Meta AI, 2023) |
|
|
| ### Model Sources [optional] |
|
|
| <!-- Provide the basic links for the model. --> |
|
|
| - **Repository:** Hugging Face |
|
|
|
|
| ## Uses |
|
|
| <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
| ### Direct Use |
|
|
| <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. --> |
|
|
| - Python code generation for software development |
| - Automation of coding tasks |
| - Developer productivity enhancement |
|
|
| ### Downstream Use [optional] |
|
|
| <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app --> |
|
|
| - Code completion, bug fixing, and Python code translation |
|
|
| ### Out-of-Scope Use |
|
|
| <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. --> |
|
|
| - Non-Python programming tasks |
| - Generation of sensitive, legal, or medical content |
|
|
| ## Bias, Risks, and Limitations |
|
|
| <!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
|
| - Limited to Python programming tasks |
| - Dataset biases from Hugging Face's Python Code Dataset |
| - Environmental impact from computational costs during fine-tuning |
|
|
| ### Recommendations |
|
|
| <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
| Users should be aware of computational efficiency trade-offs and potential limitations in generalizing to new Python tasks. |
|
|
| ## How to Get Started with the Model |
|
|
| Use the code below to get started with the model: |
|
|
| ```python |
| # Example setup (simplified) |
| import ludwig |
| from transformers import AutoModel |
| |
| model = AutoModel.from_pretrained("llama-2-7b-python") |
| ``` |
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
|
| - 500k Python code samples sourced from Hugging Face |
|
|
| ### Training Procedure |
|
|
| <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
|
|
| - **Preprocessing [optional]:** Hugging Face Python Code Dataset |
| - **Training regime:** Parameter Efficient Fine-Tuning (PEFT) and Low-Rank Adaptation (LoRA) |
|
|
| #### Speeds, Sizes, Times [optional] |
|
|
| - Not explicitly mentioned in the document |
|
|
| ## Evaluation |
|
|
| <!-- This section describes the evaluation protocols and provides the results. --> |
|
|
| ### Testing Data, Factors & Metrics |
|
|
| #### Testing Data |
|
|
| - Derived from Python code datasets on Hugging Face |
|
|
| #### Factors |
|
|
| - Python code generation tasks |
|
|
| #### Metrics |
|
|
| - Code correctness and efficiency |
|
|
| ### Results |
|
|
| - Fine-tuning improved Python code generation performance |
|
|
| #### Summary |
|
|
| The fine-tuned model showed enhanced proficiency in generating Python code snippets, reflecting its adaptability to specific coding tasks. |
|
|
| ## Model Examination [optional] |
|
|
| <!-- Relevant interpretability work for the model goes here --> |
|
|
| [More Information Needed] |
|
|
| ## Environmental Impact |
|
|
| <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> |
|
|
| Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute). |
|
|
|
|
|
|
|
|
|
|
| ### Model Architecture and Objective |
|
|
| - Llama-2 7B model architecture fine-tuned for Python code generation |
|
|
| ### Compute Infrastructure |
|
|
| - Not explicitly mentioned |
|
|
| #### Hardware |
|
|
| - Not specified |
|
|
| #### Software |
|
|
| - Ludwig toolkit and Hugging Face integration |
|
|
| **BibTeX:** |
|
|
| [More Information Needed] |
|
|
| **APA:** |
|
|
| [More Information Needed] |
|
|
| ## Glossary [optional] |
|
|
| <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. --> |
|
|
| - **Llama-2:** Open-source large language model by Meta AI |
| - **LoRA (Low-Rank Adaptation):** Efficient fine-tuning method modifying fewer model parameters |
| - **PEFT:** Parameter-efficient fine-tuning technique |
|
|
| ## More Information [optional] |
|
|
| [More Information Needed] |
|
|
| ## Model Card Authors [optional] |
|
|
| Kevin Geejo, Aniket Yadav, Rishab Pandey |
|
|
| ## Model Card Contact |
|
|
| kevingeejo02@gmail.com, aniketwater07@gmail.com, rishabpandey@gmail.com |
|
|
| ### Framework versions |
|
|
| - **Llama-2 version:** 7B |
| - **Ludwig version:** 0.8 |
| - **Hugging Face integration:** Latest |
|
|
