How DoorDash uses AI Models to Understand Restaurant Menus

DoorDash's AI system converts menu photos to digital data, reducing manual work. It combines OCR with language models but faced challenges with inconsistent menus and image quality. They created a "guardrail model" to predict transcription accuracy, with a simple LightGBM model outperforming complex networks. The system now uses both traditional OCR+LLM and multimodal LLMs, with the guardrail ensuring output quality and balancing efficiency with accuracy.