This repository demonstrates fine-tuning StarCoder2-3b for SQL tasks using LORA, which reduces model size for efficient training on smaller datasets. StarCoder2 models are trained on over 600 programming languages and natural language text from sources like Wikipedia and GitHub.
From the README
Finetune-StarCoder2 Fine-tune StarCoder2-3b for SQL tasks on limited resources with LORA. LORA reduces model size for faster training on smaller datasets. StarCoder2 is a family of code generation models (3B, 7B, and 15B), trained on 600+ programming languages from The Stack v2 and some natural language text such as Wikipedia, Arxiv, and GitHub issues.
Skills
- Jupyter Notebook
- fine-tuning
- lora
- peft
- peft-fine-tuning-llm
- stack
- starcoder2
