Apple is currently levelling up Siri’s ‘onscreen awareness,’ enabling it to interact with your screen

Apple is developing an "onscreen awareness" feature for Siri, enabling it to understand and interact with on-screen content. This will allow Siri to answer questions and perform actions related to displayed items without detailed user instructions. Apple is providing APIs for third-party app integration and testing ChatGPT integration for screenshot-based queries. While not yet available in current betas, onscreen awareness may arrive in iOS 18.4 in early 2025, potentially enhancing Siri's capabilities as a digital assistant.