Stanford Alpaca: Instruction-following LLaMA model training
Research project that fine-tunes LLaMA models to follow instructions using self-generated training data.
Learn more about Stanford Alpaca
Stanford Alpaca is a fine-tuned version of Meta's LLaMA model trained specifically for instruction-following tasks. The model uses a 52K instruction dataset generated through a modified Self-Instruct approach, where GPT-3's text-davinci-003 creates diverse instruction-response pairs. The fine-tuning process employs standard Hugging Face training code with specific hyperparameters optimized for 7B and 13B parameter models. The project is designed for research purposes and includes data generation pipelines, training scripts, and weight recovery tools.
Self-Generated Dataset
Uses a modified Self-Instruct pipeline with text-davinci-003 to generate 52K diverse instruction-following examples at reduced cost (under $500).
Complete Training Pipeline
Provides end-to-end code for data generation, model fine-tuning, and weight recovery with documented hyperparameters for reproducible results.
Research-Focused Implementation
Built specifically for academic research with CC BY NC 4.0 licensing and detailed documentation of methodology and limitations.
See how people are using Stanford Alpaca
Top in AI & ML
Related Repositories
Discover similar tools and frameworks used by developers
Chat SDK
Open-source Next.js template for AI chatbots with auth, persistence, and multi-provider support.
GFPGAN
PyTorch framework for blind face restoration using StyleGAN2 priors.
Optuna
Define-by-run Python framework for automated hyperparameter tuning.
Transformers
Unified API for pre-trained transformer models across frameworks.
X Recommendation Algorithm
Open source implementation of X's recommendation algorithm for timeline and notification ranking.