VaultGemma: The world's most capable differentially private LLM

VaultGemma, a 1B-parameter model, emerges as the world's largest open differentially private language model. By establishing new scaling laws, researchers determined optimal configurations between privacy, compute, and data budgets, finding that DP training requires smaller models with larger batch sizes than traditional approaches. The model delivers strong privacy guarantees while performing comparably to non-private models from five years ago.