Apple’s Worldwide Developers Conference (WWDC) 2024 kicked off with a keynote packed full of exciting software and hardware announcements that will shape the Apple ecosystem in the coming year. The biggest surprise was the unveiling of Apple Intelligence, Apple’s new AI system that combines the power of generative models with users’ personal context to deliver incredibly useful and relevant intelligence across iPhone, iPad, and Mac.
Apple CEO Tim Cook emphasized that Apple Intelligence goes beyond conventional artificial intelligence. “It must comprehend you and be deeply rooted in your daily life, your relationships, your communications, and more. It transcends artificial intelligence. It’s about personal intelligence,” he declared.
Let’s dive into all the major updates unveiled for iOS 18, iPadOS 18, macOS 15, watchOS 11, the new Apple Vision Pro AR/VR headset, and most importantly, the groundbreaking Apple Intelligence initiative.
Apple Intelligence
The headlining announcement at WWDC was the introduction of Apple Intelligence, a comprehensive AI system deeply integrated into iOS 18, iPadOS 18, and macOS Sequoia. Apple Intelligence harnesses the power of generative models to understand and create language and images, take action across apps, and draw from users’ personal context to simplify and accelerate everyday tasks.
A cornerstone of Apple Intelligence is on-device processing, which delivers personal intelligence without collecting users’ data. For more complex requests, Apple introduced Private Cloud Compute, which sets a new standard for privacy in AI. It allows Apple Intelligence to flex and scale computational capacity between on-device processing and larger server-based models running on dedicated Apple silicon servers, all while ensuring user data remains secure and unexposed.
Apple revealed that Apple Intelligence is comprised of multiple highly-capable generative models that are specialized for users’ everyday tasks and can adapt on the fly for their current activity. This includes a ~3 billion parameter on-device language model and a larger server-based language model available with Private Cloud Compute. These foundation models have been fine-tuned for user experiences such as writing and refining text, prioritizing and summarizing notifications, creating playful images, and taking in-app actions.
One of the key features powered by Apple Intelligence is the new systemwide writing tools. Users can now rewrite, proofread, and summarize text across various apps like Mail, Notes, Safari, Pages, and Keynote. The rewrite tool offers different versions of text and can adjust tone, while the proofread feature checks grammar and suggests edits. Summarize creates concise recaps of content.
Apple Intelligence also introduces new language capabilities to help users communicate more effectively. Mail gains features like prioritized notifications for urgent messages and quick email summaries. The Notes and Phone apps can now record, transcribe, and summarize audio, making it easy to recall key points from lectures or calls.
For visual expression, Apple Intelligence enables delightful new ways to create images and personalized emoji. Users can generate fun, original images in the Messages app based on the conversation context. A new feature called Genmoji allows the creation of personalized emoji characters that can be customized with prompts for themes, accessories, and more. The Image Wand tool in Notes turns rough sketches into related images that complement notes.
Siri + ChatGPT Integration
Siri is getting a major upgrade thanks to Apple Intelligence. With an all-new design, richer language understanding, and the ability to communicate via text, interacting with Siri is more natural than ever. Siri gains awareness of the user’s personal context, can take action within and across apps, and has deep knowledge of device features and settings.
Impressively, Apple has partnered with OpenAI to integrate ChatGPT into Siri and the new writing tools. Users can tap into ChatGPT’s vast knowledge base for complex queries that require domain expertise, like generating recipe ideas or interior design advice. Siri will ask permission before sharing any user information with ChatGPT. The ChatGPT features will be free for all users, with subscribers able to connect accounts to access premium capabilities.
OpenAI CEO Sam Altman expressed excitement about the partnership, stating, “We’re thrilled to collaborate with Apple to bring ChatGPT to their users in a new way. Apple shares our commitment to safety and innovation, and this partnership aligns with OpenAI’s mission to make advanced AI accessible to everyone.”
Apple plans to support other AI models in the future as well, but OpenAI is the only one mentioned at launch. Siri will be able to leverage external AI models to enhance its functionality while maintaining user privacy.
Privacy is Paramount
Apple emphasized that privacy is at the core of Apple Intelligence at every step. On-device processing ensures personal data isn’t collected, and Private Cloud Compute enables complex AI tasks while protecting user information with dedicated Apple silicon. Users have control over when ChatGPT is used and must provide permission before data is shared.
Craig Federighi, Apple’s senior vice president of software engineering, introduced Private Cloud Compute, a technology designed to safeguard data even for more intensive AI tasks requiring the cloud. “Your data is never stored or made accessible to Apple,” he assured.
The Apple silicon servers that form the foundation of Private Cloud Compute provide unprecedented cloud security. This starts with the Secure Enclave, which protects critical encryption keys on the server just as it does on a user’s iPhone, while Secure Boot ensures the OS running on the server is signed and verified, just like in iOS. Trusted Execution Monitor makes sure only signed and verified code runs, and attestation enables a user’s device to securely verify the identity and configuration of a Private Cloud Compute cluster before sending a request. And to verify Apple’s privacy promise, independent experts can inspect the code that runs on Private Cloud Compute servers.
Apple also raised the bar for privacy with new features aimed at helping users protect sensitive areas of their phones. With locked and hidden apps, users can lock an app to protect its contents from view, or hide an app to help prevent others from seeing it. When a user locks an app, if someone tries to tap it, they will be required to authenticate using Face ID, Touch ID, or a passcode. For additional privacy, a user can also hide an app, moving it to a locked, hidden apps folder that requires Face ID, Touch ID, or a passcode to open.
“We relentlessly deliver on our commitment to give users the strongest and most innovative privacy protections,” said Erik Neuenschwander, Apple’s director of User Privacy. “This year is no exception, and the ability to lock and hide apps is just one example of Apple helping users remain in control of their information, even when they are sharing their devices with others.”
For years, Apple has worked to ensure that a user can share exactly what they want to, and with whom. In 2020, Apple introduced the Photos picker, which lets a user select photos and videos to use in an app, without requiring full Photo Library access. This year, Apple has two new features that expand on this protection. With Contacts permission improvements in iOS 18, Apple is putting users in control by allowing them to select specific contact fields to share with an app, rather than their entire contact card.
iOS 18
The next generation of Apple’s mobile operating system, iOS 18, is a massive release that makes iPhone more personal, powerful, and intelligent than ever before.
The iOS 18 update starts with the Lock Screen, which gains interactive widgets to view and control apps without unlocking the device. Users have extensive options to customize the Lock Screen with styles, colors, and fonts. The new widget system enables glanceable information and quick actions.
The Home Screen also receives enhancements for easier organization and personalization. Users can now place apps and widgets anywhere on the Home Screen, customize the dock, and access an updated library of widgets. The new App Library makes it a snap to find and launch apps.
iOS 18 introduces a major redesign of the Photos app, making it smarter and more organized. The new “Days” tab intelligently curates users’ photo libraries to highlight meaningful moments and memories. Photos are arranged in a grid that mixes intelligently resized images for visual interest. Users can customize their Days view by pinning favorite photos, which can be shared with family via iCloud Shared Photo Library.
iMessage gains a slew of new features to enhance communication. Users can now edit recently sent messages, undo delivery of a message, and mark messages as unread to revisit later. The biggest change is the ability to collaborate and communicate entirely within iMessage threads using SharePlay. This allows users to chat, watch videos, listen to music, and more while messaging. RCS as an afterthought is sneekily announced as FINALLY being supported.
iOS 18 brings support for web push notifications, allowing web apps added to the Home Screen to send notifications just like native apps. The Health app adds a new Medications feature for managing and tracking prescriptions, vitamins, and supplements. Users can create custom schedules, set reminders, and view interaction warnings.
The Maps app introduces multi-stop routing, enabling users to plan trips with up to 15 stops and automatically sync routes from Mac to iPhone. The new “Look Around” window lets users virtually explore cities with high-resolution 3D imagery.
Other notable iOS 18 features include an Apple Pay Later installment option, Shared Tab Groups in Safari, Freeform for collaborative brainstorming with friends, Dictation enhancements, multi-user support for iPad, and much more.
iPadOS 18
iPadOS, the dedicated operating system for Apple’s tablets, receives some major productivity and multitasking enhancements in version 18.
The headline feature is Stage Manager, a new way to multitask with overlapping, resizable windows on iPad. It automatically organizes apps and windows, making it faster and easier to switch between tasks. Users can create groups of apps to be used in specific contexts like work or gaming. External display support is also enhanced, with the ability to run up to eight apps simultaneously across the iPad and external monitor.
Apple is introducing a new app called Freeform, a collaborative whiteboard tool with tight integration into the Apple ecosystem. It allows users to brainstorm together in real-time on a flexible canvas with support for Apple Pencil, images, documents, links, and more.
The Weather app finally makes its way to iPad with a design that takes advantage of the larger screen real estate. Pro users will appreciate the Reference Mode on iPad Pro with the Liquid Retina XDR display for color grading and compositing workflows.
iPadOS 18 also brings a suite of powerful new features geared toward productivity. Users can now connect a physical keyboard and trackpad to their iPad for a more laptop-like experience. The Files app gains support for external drives and file servers. And the new App Library automatically organizes apps into categories for easier access.
Other iPadOS 18 updates include enhancements to Messages, Safari web apps, pro video workflows with multicam support and Reference Mode, and desktop-class apps enabling more powerful app experiences.
macOS 15
The next major release of macOS is version 15, codenamed “Sequoia”. The update refines the user experience while introducing some handy new productivity features.
Continuity, the feature that enables seamless transition between Apple devices, gets a major upgrade. With Continuity Camera, users can now use their iPhone as a webcam for their Mac. It works wirelessly and takes advantage of the iPhone’s superior camera system, with support for Center Stage, Portrait mode, and the new Studio Light feature.
Sequoia brings some updates to core system apps. Mail introduces undo send, scheduled send, follow-up suggestions, and improved search. Safari adds shared tab groups for collaborating on collections of websites and Passkeys for more secure passwordless sign-in. iCloud Shared Photo Library makes it easier to share photos and videos with up to five family members in the Photos app.
Spotlight is becoming more intelligent, with richer results for artists, movies, actors, and businesses, along with the ability to search within Live Text and images system-wide. Users can even search for images based on the text they contain on the web.
Sequoia also focuses on gaming, with improvements to Metal 3, Apple’s hardware-accelerated 3D graphics framework. MetalFX Upscaling enables developers to render rich, immersive graphics while maintaining high frame rates. No Man’s Sky and Resident Evil Village are coming to Mac later this year, taking advantage of Apple silicon performance.
Other macOS Sequoia features include an updated System Preferences app with a more streamlined design, the Clock and Weather apps from iOS, support for physical security keys, and new accessibility tools.
watchOS 11
Apple Watch gets some exciting updates in watchOS 11 with new watch faces, bands, and health and fitness features.
There are four new watch faces: Astronomy, Lunar, Metropolitan, and Playtime, offering creative ways to personalize the Apple Watch. The popular Modular face gets an upgrade with more color options and complications. Apple is also introducing more expressive Memoji watch faces.
watchOS 11 brings more detailed metrics for runners, including stride length, ground contact time, and vertical oscillation. Fitness+ is expanding to all iPhone users, even without an Apple Watch, and adding onscreen guidance for HIIT, cycling, rowing, and treadmill workouts.
The Sleep app adds sleep stage tracking, measuring time spent in REM, Core, and Deep sleep, powered by the heart rate sensor and accelerometer. Users can see nightly sleep stage data ou can see nightly sleep stage data on Apple Watch in the Sleep app and more detailed information including sleep stage history on iPhone.
With the new Medications feature in the Health app, you can discreetly track your medication intake, create schedules, and receive alerts directly on Apple Watch. Custom schedules can be created for each medication, whether it needs to be taken multiple times a day, once a week, or as needed.
watchOS 11 is also gaining support for more smart home devices with an updated Home app, enabling you to view and control accessories directly from your wrist.
visionOS 2
Apple previewed visionOS 2, a major update to the operating system powering the Apple Vision Pro mixed reality headset. Just months after the initial U.S. release of Vision Pro, visionOS 2 introduces a range of exciting new capabilities and experiences.
One of the standout features is the ability to easily transform regular 2D photos into immersive spatial photos using AI. The redesigned Photos app makes it simple for users to revisit favorite memories with added depth and realism. Spatial Personas with SharePlay also allows users to share photos together in a virtual space using 3D avatars.
VisionOS 2 brings enhancements to the Mac virtual display functionality. A new ultra-wide display option provides the equivalent of two side-by-side 4K monitors that extend in front of the user like a curved display. This significantly expands the virtual workspace for improved productivity.
Navigation becomes more intuitive with the addition of new hand gestures to quickly access the home screen, settings, and control center. Travel Mode gains optimizations for train rides, expanding the scenarios where Vision Pro can be used on the go.
For developers, visionOS 2 introduces new tools and APIs to create richer spatial apps that are more volumetric and shareable. This includes a Volumetric API for running complex 3D apps simultaneously. Canon is also releasing a custom lens designed specifically for capturing spatial video content for Vision Pro, which can be edited in Final Cut Pro.
The Apple TV app on Vision Pro will soon support multiview for watching up to five sports streams at once. The Mindfulness app is adding a “Follow Your Breathing” feature with dynamic visuals and sounds that respond to the user’s breathing patterns.
Other updates include the ability to view content from other Apple devices using AirPlay, systemwide Live Captions for spoken dialogue, and support for popular business and productivity apps like Slack and Microsoft 365.
Apple announced that the Vision Pro will become available in more countries, starting with China, Japan, and Singapore on June 28, followed by Australia, Canada, France, Germany, and the UK on July 12. The App Store now has over 2,000 apps built specifically for Vision Pro, along with 1.5 million compatible iPad apps.
Other Announcements
In addition to the major platform updates, Apple made some other notable announcements at WWDC 2024:
- Apple introduced the M3 chip, the next generation of Apple silicon. The M3 delivers up to 40% faster CPU performance, up to 30% faster graphics performance, and up to 40% lower power consumption compared to the M2. The new 15-inch MacBook Air and updated 13-inch MacBook Pro are the first Macs powered by M3.
- Apple Arcade is getting more than 30 new titles in the coming months, including Teenage Mutant Ninja Turtles: Splintered Fate, Disney SpellStruck, Brave Hand, and Frogger.
- Apple Pay Later is a new financing option that lets you split the cost of purchases into four equal payments over six weeks with no interest or fees. It will be built into Apple Pay and available everywhere it’s accepted online and in-app.
- Apple is expanding its car key feature to more vehicle brands, including Ford and Honda. Car keys are getting more functionality like vehicle status info and the ability to adjust seats, mirrors, climate control, and more.
- Apple Music is adding a new feature called Apple Music Sessions, exclusive live performances recorded in Spatial Audio. Artists include Carrie Underwood, Tenille Townes, and Ronnie Dunn.
- Apple News introduces Sports section in the Today tab and score and schedule cards for MLB and NBA games. My Sports enables you to follow your favorite teams across Apple News, Stocks, and Apple TV.
- Accessibility updates include Personal Voice for people at risk of losing their ability to speak, Live Speech for conversations, Assistive Access to simplify the iPhone interface, and more.
Apple’s “Cautious” AI Strategy to the Forefront
While Apple’s AI(branding Apple Intelligence, I see what you did there Apple) announcements at WWDC 2024 were significant, they reflected the company’s characteristically cautious approach to rolling out new technologies. Rather than making Apple Intelligence available to all users immediately, the company is starting with a limited beta release this fall, initially only in the US.
Apple executives emphasized that privacy and security are paramount in the design of Apple Intelligence. On-device processing will handle many AI tasks without sending personal data to the cloud. For more complex queries that do require the cloud, Apple introduced Private Cloud Compute, a secure platform running on dedicated Apple silicon servers. User data is never stored or accessible to Apple.
The integration of OpenAI’s ChatGPT into Siri is also a measured step. Users will have control over when their queries are sent to ChatGPT, and no user data will be stored by OpenAI for model training. Apple also made clear that while ChatGPT is the first AI model they are integrating, it won’t be the only one – they plan to support other models in the future as well.
Some analysts believe Apple’s go-slow approach to AI is a smart strategy that plays to the company’s strengths in user experience, privacy, and ecosystem control. By gradually rolling out AI features that are deeply integrated into its platforms, Apple can deliver an elegant experience that still feels uniquely “Apple”. The company may not always be first to market with new technologies, but it often succeeds by offering a more polished and thoughtful implementation.
Other observers, however, worry that Apple is at risk of falling behind as rivals like Google and Microsoft race ahead with generative AI. The concern is that Apple’s careful approach could look more like caution in a few years if the company doesn’t keep pace with the rapid advancements in the field. Balancing innovation and restraint is a challenge for any tech giant in the age of AI.
The Road Ahead for Apple Intelligence
Regardless of the differing opinions on Apple’s AI strategy, it’s clear that Apple Intelligence is just the beginning of the company’s ambitions in this “AI” space. As Apple’s machine learning chief John Giannandrea said in a rare interview last year, “AI will transform every product and service we have.”
In the near term, Apple Intelligence will continue to expand and evolve. The beta launching this fall is focused on the core experiences of Siri, writing assistance, and image generation. But Apple has a vast ecosystem of apps and services where AI could be impactful, from health and fitness to music and TV recommendations to accessibility features. Expect to see Apple Intelligence steadily make its way into more corners of the Apple universe.
Longer term, it’s not hard to imagine Apple Intelligence becoming one of the key tentpoles of the company’s product strategy alongside pillars like Apple silicon and services. Just as Apple has spent the last decade building world-class chips to power its devices, it may well spend the next decade building world-class AI to make those devices more capable and intelligent than ever.
Of course, Apple will have to navigate significant challenges and unknowns along the way. The path forward for the AI industry is exciting but uncertain, with major questions around ethics, bias, copyright, and more still unresolved. As a company that champions privacy and prides itself on its values, Apple will be closely scrutinized in how it approaches these issues.
There are also open questions about the business model for AI, especially for a company like Apple that is used to charging premium prices for differentiated hardware and software. Will users pay extra for AI features? Will Apple need to monetize user data and attention in new ways to fund its AI development? How will Apple’s AI strategy intersect with its lucrative App Store and the third-party developer ecosystem?
These are all questions that Apple will have to grapple with in the years ahead as it goes deeper on AI. What’s not in question, though, is that Apple has officially entered the AI race with Apple Intelligence. It may be starting at the back of the pack, but it has the resources, the talent, and the ambition to be a serious contender. The future of the iPhone, and of Apple, will be shaped by artificial intelligence – and that future is now underway.
To ReCap
WWDC 2024 showcased how Apple continues to advance its platforms with innovative features that empower developers and enrich the lives of users. From the game-changing Vision Pro headset to the productivity enhancements in iOS, iPadOS and macOS, there’s a lot to be excited about in the Apple ecosystem.
Developers can access beta versions of iOS 18, iPadOS 18, macOS 15, watchOS 11, starting that day, with public betas coming in July. The free software updates will be available to all users in the fall.
It’s clear that Apple’s vision of the future revolves around ever more seamless integration between devices, powerful tools for work and play, and deeply personalized experiences. As Apple said in its keynote, its goal is to create technology that empowers people to do amazing things and make meaningful connections. WWDC 2024 showed that Apple is well on its way to achieving that mission.
WWDC 2024 will undoubtably be remembered as a milestone moment for Apple and for the tech industry at large. With the introduction of Apple Intelligence and the company’s new AI strategy, Apple is embarking on a journey that will transform its products and its business in the coming years.
The integration of generative AI into Siri, the system-wide writing enhancements, the image generation features, and the ChatGPT partnership are just the first steps. But they represent a major shift in how Apple is thinking about the role of machine learning and AI in its ecosystem.
As always, Apple will take a uniquely Apple approach, prioritizing privacy, security, elegance, and the end-to-end user experience. It won’t be a straightforward or easy path, but it’s one the company is now committed to traveling.
For developers, Apple Intelligence opens up exciting new possibilities to incorporate AI into their apps and create more powerful, more personalized experiences. For users, it promises to make the iPhone, iPad, and Mac more helpful and more delightful than ever before.
The age of AI has arrived, and with Apple Intelligence, Apple is ready to embrace it – on its own terms and in its own way. The future is intelligent, and it’s starting now. Welcome to the era of Apple Intelligence.
