Apple has announced major advancements in Visual Intelligence, introducing a new Siri mode in the Camera app that can analyze real-world objects and perform actions based on what the iPhone observes. During the WWDC keynote, Apple’s vice president of software engineering, Sebastien Marineau-Mes, highlighted the new Siri mode’s ability to interpret the camera’s view and offer contextually relevant actions using image understanding powered by Apple’s foundational models. A key feature is bill splitting, where users can point their iPhone camera at a restaurant check to instantly divide the total among friends, with Apple Cash integration enabling immediate payments. Additionally, users can receive nutritional information by pointing the camera at a plate of food. This Visual Intelligence capability is also being integrated into visionOS, allowing Siri to recognize real-world objects in a user’s environment and provide pertinent information as needed.
How is Apple enhancing Visual Intelligence with new features?
