ChatGPT/Gemini can now draw on your screen to help you navigate complex software
SMRTR summary
SketchVLM enables AI models to annotate images with arrows, labels, and shapes instead of text-only responses, making explanations clearer and improving reasoning accuracy by up to 28.5 points on tasks like navigation and object counting.
SMRTR provides this summary for quick context. The original article belongs to Hacker News.
Read the original article