tune.fyi - LLM Training Telemetry and VRAM Calculator

Product Tool

PRODUCT MODE: You are using the actual tune.fyi tool. Running simulations here consumes GPU hours.

Tracker
Data Drop
VRAM Estimator
Interactive Console
Calibration Audit DB
Throughput
Cost & Data
LR Schedule
Test Suite

Live Tracker & Duration

Input Mode

Dataset Size

Target Epochs

Number of GPUs

Micro Batch

Grad Accumulation

Current Speed (s/it)

Steps Completed

Total Steps

Global Batch: -

Remaining

Estimated ETA

Total Run Time: -

2. Data & Telemetry Drop

📂

Drop trainer_state.json here

or adapter_config.json to auto-fill model settings

VRAM Estimator (Per GPU) Standard Estimation Formulas

Model Params (B)

Seq Length

Hidden Dim

Num Layers

Attn Heads

Method

Distributed

Optimizer

LoRA Rank

Target Modules

Hardware

Grad Checkpointing

Flash Attention 2

Mixed-Prec Master Copy

Estimated VRAM / GPU-

Limit: - GB

Weights:-

Gradients:-

Optimizer States:-

Activations:-

Master Weights:-

CUDA Overhead:~1.5 GB

Interactive Console (Simulation)

Ready...

Calibration Audit DB

Throughput & Efficiency MFU = (tokens/s x 6N) / Peak TFLOPS

Tokens / Second

Samples / Second

Model FLOPs Utilization

Cost & Data Estimator Chinchilla: optimal tokens ~ 20 x params

Cost per GPU-hour ($)

Total GPU-Hours

Estimated Cost

Total Tokens

Learning Rate Schedule Preview

Peak LR

Min LR

Warmup Steps

Schedule

9. Validation & Test Suite Client-Side

Click "Run All Tests" to validate calculations.