Apple has introduced a new feature called Visual Intelligence, which is exclusive to the iPhone 16 and iPhone 16 Pro models. This innovative feature relies on the Camera Control button and is available as of iOS 18.2. In this guide, we will explore what Visual Intelligence can do, how to activate it, and its various uses.
Activating and Using Visual Intelligence
To use Visual Intelligence, you need to hold down on the Camera Control button for a few seconds to activate the Visual Intelligence mode. Simply pressing the Camera Control button opens up the camera with Camera Control, so you need a distinct press-and-hold gesture to get to it. Make sure you’re not already in the Camera app, as it won’t work if the camera is already active.
The Visual Intelligence Interface
The Visual Intelligence interface features a view from the camera, a button to capture a photo, and dedicated ‘Ask’ and ‘Search’ buttons. The Ask queries ChatGPT, while Search sends an image to Google Search.
Capturing Photos
Using Visual Intelligence requires taking a photo of whatever you’re looking at. You need to snap a photo using the Camera Control button, and select an option. It does not work with a live camera view, and you cannot use photos that you took previously.
Getting Details About Places
If you’re out somewhere and want to get more information about a restaurant or a retail store, click and hold on the Camera Control button, and then click Camera Control again to take a photo or tap the name of the location at the top of the display. From there, you can see the hours when the business is open, place an order for delivery at relevant locations, view the menu, view offered services, make a reservation, call the business, or visit the location’s website.
Summarizing Text
Take a photo of text from the Visual Intelligence interface and choose the ‘Summarize’ option to get a summary of what’s written. The Summarize option is useful for long blocks of text but is similar to other Apple Intelligence summaries, making it brief and not particularly in-depth.
Reading Text Out Loud
Whenever you take a Camera Control image of text, there is an option to hear it read aloud. To use this, simply tap the ‘Read Aloud’ button at the bottom of the display, and Siri will read it out loud in your selected Siri voice.
Translating Text
If text that you capture with Visual Intelligence is not in your language (limited to English at this time), you’ll see a ‘Translate’ option. You can tap it to get an instant translation.
Navigating Links and Contacts
If there’s a link in an image that you capture with Visual Intelligence, you’ll see a link that you can tap to visit the website. If there is an email address in an image, you can tap it to compose an email in the Mail app. Similarly, if there is a phone number, you’ll see an option to call it.
Creating Calendar Events
Using Visual Intelligence on something that has a date will give you an option to add that event to your calendar.
Detecting and Saving Contact Info
For phone numbers, email addresses, and addresses, Apple says you can add the information to a contact in the Contacts app. You can also open an address in the Maps app.
Scanning QR Codes
Visual Intelligence can be used to scan a QR code. With QR codes, you don’t actually need to snap an image; simply point the camera at the QR code and then tap the link that pops up. QR code scanning also works in the Camera app without Visual Intelligence active.
Asking ChatGPT
You can take a photo of anything and tap on the ‘Ask’ option to send it to ChatGPT while asking a question about it. If you take a picture of an item with Visual Intelligence and want to know what it is, for example, you tap on Ask and then type in ‘What is this?’ to get to a ChatGPT interface.
ChatGPT will respond, and you can type back if you have follow-up questions. Visual Intelligence uses the ChatGPT Siri integration, which is opt-in. By default, no data is collected, but if you sign in with an OpenAI account, ChatGPT can remember conversations.
Searching Google for Items
You can take a picture of any item that you see and tap on the ‘Search’ option to use Google Image Search to find it on the web. This feature is useful for locating items that you might want to buy.
Conclusion
Visual Intelligence is an exciting new feature that unlocks the power of AI on iPhone. With its various uses, from summarizing text to scanning QR codes, this feature has the potential to revolutionize the way we interact with our devices. In this guide, we have explored what Visual Intelligence can do and how to use it effectively.
Related Resources
- iOS 18
- iPadOS 18
- iPhone 16
- iPhone 16 Pro
- Buyer’s Guide: iPhone 16 (Buy Now), iPhone 16 Pro (Buy Now)
- Related Forums: iOS 18, iPadOS 18
