Unsloth Setup Guide for RTX 3090

INSTALLED AND VERIFIED ON LOCAL-AI SERVER

Research completed: 2025-12-28 Installation verified: 2025-12-28

Server Details

Hostname: local-ai
Tailscale IP: 100.89.34.86
LAN IP: 10.0.0.250
User: lars
Password: LARS25 (stored in Locker: f9ca71df)
GPUs: 2x RTX 3090 (48GB total VRAM)

Installed Versions

Unsloth: 2025.12.9
PyTorch: 2.9.1+cu128
CUDA: Enabled, 2 GPUs detected

Installation Command Used

pip3 install unsloth

This installed all dependencies including: - torch 2.9.1 - transformers 4.57.3 - accelerate 1.12.0 - bitsandbytes 0.49.0 - peft 0.18.0 - trl 0.24.0 - xformers 0.0.33.post2

Verification Command

python3 -c "from unsloth import FastLanguageModel; print('Unsloth loaded')"

Key Features for RTX 3090

3x faster training with Triton kernels
30% less VRAM with padding-free packing
FP8 GRPO reinforcement learning
Docker image available

OOM Prevention

If out of memory: 1. Set batch_size to 1, 2, or 3 2. Use 4-bit quantization (QLoRA) 3. Reduce context length

Training Script Template

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen2.5-Coder-14B-Instruct-bnb-4bit",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
)

Locker: Local AI Server (f9ca71df)
Track: LARS Implementation Sprint (fc0dd483)
KB: Nexus AI Engine (e213b1c0)