Apple Intelligence | What you get and what you need to know

Apple Intelligence. What a ride!

I enabled it on my MacBook Air and iPhone on 21 Dec 2024. It was released 2 weeks earlier for the Australian and New Zealand region as part of IOS 18.2. It is still the beta version so do expect changes. Shortly after 18.3 was released to developers.

If you have not cottoned on, note the name “Apple Intelligence” as in AI. Not “Copilot” for Microsoft or “Gemini” for Google. And why no? I suspect no one guessed what it would be called.

Straight off the bat, I was impressed. Some of the features were outstanding, some not. Here is my take on the ride. First some background.

Much of Apple Intelligence is part of AI (Artificial Intelligence) referred to as generative AI. It helps create, compose, proofread and correct articles on the fly. Plus the ability to contextualise the tone and sentiment that is close to what the user wants.

Prior to Apple Intelligence and other recent ones such as Microsoft’s Copilot and Google’s Gemini, the first such application is Chat GPT, released on November 2022.

Chat GPT took the World by storm, albeit there were fears about its abuse and inaccurate information. This is my article on Chat GPT that I posted on March 2023. It did catch many by surprise what Large Language Models (LLM) in AI could do. Chat GPT set a new paradigm.

Besides text related enhancements, this Apple release also covers photos and other things using AI. Clean Up for iPhotos is particularly impressive.

Availability

Many including me had the impression that Apple Intelligence will only be available in iPhone 16 onwards. It is available in the following devices as well.

iPhone 15 Pro or iPhone 15 Pro Max;
any iPad with A17 Pro or M1 and later;
Mac with M1 or later. (Apple silicon chip)

Apple Intelligence covers all the later versions of Mac, iPad and iPhone devices

Besides having these devices there are 2 other things to note.

Your OS must be the latest, iOS 18.1 or later, iPadOS 18.1 or later, or MacOS Sequoia 15.1 or later. Second is that it is not available in all regions. Note other regions might need higher OS version such as Australia which requires Sequoia 15.2.

The US had it first followed by other English speaking countries. Non-English languages are expected to be available in April 2025 . If you are not in the region, you can access it by changing your region in the device setting.

If you have a MacBook the meets the specs, try Apple Intelligence on it rather than on the iPhone. Much easier to grasp with the larger screen except for Camera Control.

Apprehension

One fear I suspect some have is that AI is going to takeover your device and you have little control. Or you unsure what it can reveal about you. Actually it is more an enabler of things you already do on your device. My guess, in time it will become second nature to many of us. A good analogy is grammar and spell check when it was first introduced years ago.

The overall theme is productivity, efficiency with more polished and presentable outcome. To work smarter. The second theme is giving context in order to shape and personalise your work to reflect desired tone and temperament. It is not a library of stock responses and answers. Unless you make it so. The third theme is utility such as the Clean Up tool, Image Search and Memory Creation in iPhotos and Visual Intelligence with Camera Control.

Key features

Writing tools, the main feature covering all forms of text such as emails, documents, notes and messages. With Chat GPT option
Enhancing Mail and Messaging with AI driven summaries and smart replies. And prioritising emails
Enhancing Siri with AI to respond more naturally and getting the context right. With Chat GPT option
Summaries for notifications, webpages, notes, recorded phone calls and audio recordings
Natural language image search, cleanup tool and memory movie creation for iPhotos
Visual Intelligence with Camera Control allowing for information on objects and places that are in the frame. With Chat GPT option
Image playground, a fun creative tool using your image or image of someone
Custom emojis called Genmoji and drawing on Notes – limited to iPhone devices from 15 pro onwards

Things that make a difference

Writing Tools is a big win as it covers all forms of text. Multiple options to present your views in the best possible light. You can have your article or email completely rewritten by AI and decide which is better. You could also adopt a friendly, professional or concise approach. It allows you to try out various options, amend parts of the suggested text before sending out what works for you best.

Siri, re-born with AI is clearly another impressive change. The enhancement is more than perceptible. I had not used Siri since its inception except to try it when it was first introduced. It’s now my first port of call for many things. The newly introduced LLM (large language model) and the resulting natural language processing is so much cleaner, engaging and it understands context much better. You don’t have to focus on how to structure your request or instructions as before.

Enabling access to Chat GPT for both Writing Tools and Siri pushed both features a level up. It’s an option. It is superior to what Apple Intelligence LLM can offer as it’s learned data repository is much larger.

Siri will actually offer Chat GPT if it thinks that the latter can provide more or better information. In settings, I changed the option of prompting the user to consider Chat GPT to go straight to it if Siri feels it is a better option. I did this after doing a few trials and the lift was significant.

You have 2 options to access Chat GPT. One is Apple direct access, the second is your own Chat GPT login access whether it is the free version or paid version. My recommendation is to enable the Apple direct to Chat GPT for both Writing tools and Siri.

There is one stand-out new utility which uses AI and that is the Clean Up tool with iPhoto. For the first time, without the use of additional high end software, time and effort, you can remove distracting items from an iPhoto image with good results in seconds. The AI not only identifies possible distracting items but it also backfills the void left with identical or similar settings of the surrounding area.

There are occasions it cannot get the desired results but this a new feature and still impressive and necessary. iPhotos also comes with a image search function that takes text description to identify photos. You can further narrow down the search with more specific attributes such as “girl with red hat in Bali”. How good is that?

Visual intelligence with Camera Control is yet another feature where you can press the Camera Control button, aim at a cafe you are passing by and get information about the place, its address, contact details, opening hours and view the menu.

Things that make little difference

I found the stand-alone app Image Playground gimmicky and toyish. You probably play with it occasionally for the fun of it. Custom emojis however is simple and good. You can set the tone and sentiments better with customised emoji which Apple calls Genmoji. A personal touch with little effort. A customised emoji can even be your personal signature.

Summaries across the various mediums including notifications is not necessary. I felt the Apple having developed this feature decided to paint the town red with it. Probably more useful to get the key points from media articles especially opinion pieces which sometimes can be overwhelming. The other useful purpose is to summarise work related lengthy documents or research related documents. Thankfully there are toggle switches to get the ones you want.

Prioritising in email is another one that would work for some and not for others. Prioritising which is already a long existing feature has been enhanced with AI. I prefer to switch it off as I do not want to miss out anything important.. And not all important messages have descriptive text and dates suggesting urgency that AI can pick up. I do like to work with the latest item on the top and I know I will need to eventually cover all unread items

Moving forward

I would recommend trying out all the features of Apple Intelligence and find out what helps. Here is an excellent guide from Apple that covers all the new features. It will take you about 2 hours to understand and try out the features. I suggest bookmarking it as a ready reference for the next few weeks.

My plan is to use Writing tools across the board to deliver a better text related outcome including the right tone and temperament. With Chat GPT, I expect a lift in more data, relevancy and quality of information.

Using Siri would be next. No more texting queries for everything. It’s best value is when you are not at work or home and no access to a PC. Rather than fiddle with the small keyboard of the phone, you are better off using Siri. I would however need to first break my habit of heading to Google and remember there is Siri.

I will also use the iPhoto Clean Up tool, image search function and memory movie creation and in that order. My guess is you will realise the value of Clean Up when you take photos on a trip and a few stand out. But they have distracting elements. Either someone walked past behind unnoticed and photo bombed the shot or you realise that the image would look better by removing some elements to allow focus on the main subject or another subject.

A little bit more on the the Clean Up tool. It identifies possible items to remove or you can pick what you want removed. Once again it does not work sometimes due to the complexity of the image. See the comparison below. Despite the clutter it did the work on the first attempt within seconds.

Apple privacy

There is no other for-profit commercial entity that I am aware of that is religious about privacy as Apple. Seen it before.

Apple Pay is a good example. It has been an outstanding success since its launch in Sept 2014. It has 3 primary functions – fraud prevention, data privacy and convenience at point of sales terminals. No physical card needed. Apple does not say anything much about the first 2 despite leading the industry in it. See my article on Apple Pay that covers it.

The same philosophy is behind the deployment of Apple Intelligence architecture. Much of the AI processing is done by downloaded models in the individual’s device. These models are dropped to devices after the user activates the feature and each feature needs to be individually activated.

There are elements that require processing to be done outside the devices. These are done by Apple’s innovative PCC *Private Cloud Computing”. Controls are in place that only allows the device owner to access the data in PCC. Not even Apple or a 3rd party can access the data. More to this here.

It should be noted that Apple Intelligence came in later than Microsoft’s Copilot and Google’s Gemini and others. Gemini is said by peers to be better than Apple’s version but the approach is different. The latter provides AI as a suite to enhance its existing features and not use the Chatbot approach. Nevertheless we all better for it for having choices across competing platforms.

Enjoy tinkering with Apple Intelligence and finding out what works for you.