Overview

Soup is a groundbreaking tool designed to radically simplify and democratize Large Language Model (LLM) fine-tuning. It empowers users to fine-tune and post-train LLMs with a single command and a straightforward YAML configuration, eliminating the need for complex SSH setups or convoluted configuration files.

Key Features

• Unprecedented Resource Efficiency: Fine-tune an 8B model on a 4 GB laptop GPU. Soup's innovative "layer streaming" technology keeps the base model out of VRAM, feeding it to the GPU layer by layer. This enables remarkable performance, with an Llama-3.1-8B-Instruct + NF4 model achieving 119.6 tok/s on a 4 GB RTX 3050 Laptop GPU, matching a full resident run on an H100.
• Simplified Workflow: Go from configuration to a trained model in one command. Soup handles batch size, GPU detection, quantization, and more automatically.
• Zero SSH, Zero Config Hell: Eliminate the frustration of managing remote infrastructure. A single YAML file is all you need to define your training process.
• Local Training Power: Train on your own GPU using QLoRA, no cloud subscription required.
• Versatile Data Handling: Supports a wide range of data formats (Alpaca, ShareGPT, ChatML, preference pairs, and more) automatically detected from JSONL, JSON, CSV, Parquet, or TXT files.
• Extensive Model Support: Works with any text-generation model from the HuggingFace Hub that loads with AutoModelForCausalLM. Ready-made recipes are available for over 100 models.
• Powerful Evaluation and Governance: Includes tools like soup ship for rigorous model evaluation and provenance-binding, ensuring you know if your model has improved or regressed.
• Flexible Export Options: Export trained models to various formats like GGUF, ONNX, TensorRT, AWQ, and GPTQ for seamless deployment.

Typical Use Cases

• Individual Developers and Researchers: Fine-tune LLMs on personal hardware, even with limited VRAM, to experiment with new models and techniques.
• Small to Medium-Sized Teams: Streamline LLM development workflows, reducing infrastructure overhead and accelerating model iteration cycles.
• AI Enthusiasts: Easily adapt powerful open-source LLMs to specific tasks or datasets without requiring deep DevOps or ML engineering expertise.
• Rapid Prototyping: Quickly iterate on model performance for applications like chatbots, code generation, content creation, and more, leveraging the efficiency of layer streaming.
• Educational Purposes: Provide an accessible platform for learning and practicing LLM fine-tuning on commodity hardware.