Apple’s fancy new CarPlay will only work wirelessly

Apple’s been talking about its next generation of CarPlay for two years now with very little to show for it — the system is designed to unify the interfaces on every screen in your car, including the instrument cluster, but so far only Aston Martin and Porsche have said they’ll ship cars with the system, without any specific dates in the mix.

And the public response from the rest of the industry towards next-gen CarPlay has been pretty cool overall. I talk to car CEOs on Decoder quite often, and most of them seem fairly skeptical about allowing Apple to get between them and their customers. “We have Apple CarPlay,” Mercedes-Benz CEO Ola Källenius told me in April. “If, for some of the functions, you feel more comfortable with that and will switch back and forth, be my guest. But to give up the whole cockpit head unit — in our case, a passenger screen and everything — to somebody else? The answer is no.”

That industry skepticism seems to have hit home for Apple, which posted two WWDC 2024 videos detailing the architecture and design of next-gen CarPlay. Both made it clear that automakers will have a lot of control over how things look and work, and even have the ability to just use their own interfaces for various features using something called “punch-through UI.” The result is an approach to CarPlay that’s much less “Apple runs your car” and much more “Apple built a design toolkit for automakers to use however they want.”

See, right now CarPlay is basically just a second monitor for your phone – you connect to your car, and your phone sends a video stream to the car. This is why those cheap wireless CarPlay dongles work – they’re just wireless display adapters, basically.

But if you want to integrate things like speedometers and climate controls, CarPlay needs to actually collect data from your car, display it in realtime, and be able to control various features like HVAC directly. So for next-gen CarPlay, Apple’s split things into what it calls “layers,” some of which run on your iPhone, but others which run locally on the car so they don’t break if your phone disconnects. And phone disconnects are going to be an issue, because next-generation CarPlay only supports wireless connections. “The stability and performance of the wireless connection are essential,” Apple’s Tanya Kancheva says while talking about the next-gen architecture. Given that CarPlay connectivity issues are still the most common issue in new cars and wireless made it worse, that’s something Apple needs to keep an eye on.

There are two layers that run locally on the car, in slightly different ways. There’s the “overlay UI,” which has things like your turn signals and odometer in it. These can be styled, but everything about it is entirely run on your car, and otherwise untouchable. Then there is the “local UI,” which has things like your speedometer and tachometer — things related to driving that need to update all the time, basically. Automakers can customize these in several ways – there are different gauge styles and layouts, from analog to digital, and they can include logos and so on. Interestingly, there’s only one font choice: Apple’s San Francisco, which can be modified in various ways, but can’t be swapped out.

Apple’s goal for next-gen CarPlay is to have it start instantaneously — ideally when the driver opens the door — so the assets for these local UI elements are loaded onto the car from your phone during the pairing process. Carmakers can update how things look and send refreshed assets through the phone over time as well — exactly how and how often is still a bit unclear.

Then there’s what Apple calls “remote UI,” which is all stuff that runs on your phone: maps, music, trip info. This is the most like CarPlay today, except now it can run on any other screen in your car. 

The final layer is called “punch-through UI,” and it’s where Apple is ceding the most ground to automakers. Instead of coming up with its own interface ideas for things like backup cameras and advanced driver-assistance features, Apple’s allowing carmakers to simply feed their existing systems through to CarPlay. When you shift to reverse, the interface will simply show you your car’s backup camera screen, for example:

But carmakers can use punch-through UI for basically anything they want, and even deeplink CarPlay buttons to their own interfaces. Apple’s example here is a vision of multiple colliding interface ideas all at once: a button in CarPlay to control massage seats that can either show native CarPlay controls, or simply drop you into the car’s own interface.

Or a hardware button to pick drive modes could send you to either CarPlay settings, deeplink you into the automaker’s iPhone app, or just open the native car settings:

Apple’s approach to HVAC is also what amounts to a compromise: the company isn’t really rethinking anything about how HVAC controls work. Instead, it’s allowing carmakers to customize controls from a toolkit to match the car system and even display previews of a car interior that match trim and color options. If you’ve ever looked at a car with a weird SYNC button that keeps various climate zones paired up, well, the next generation of CarPlay has a weird SYNC button too.

All of this is kept running at 60fps (or higher, if the car system supports it) by a new dedicated UI timing channel, and a lot of the underlying compositing relies on OpenGL running on the car itself.

All in all, it’s a lot of info, and what feels like a lot of Apple realizing that carmakers aren’t going to just give up their interfaces — especially since they’ve already invested in designing these sorts of custom interfaces for their native systems, many of which now run on Unreal Engine with lots of fun animations, and have Google services like Maps integrated right in. Allowing automakers to punch those interfaces through CarPlay might finally speed up adoption – and it also might create a mix-and-match interface nightmare. 

All that said, it’s telling that no one has seen anything but renders of next-gen CarPlay anywhere yet. We’ll have to see what it’s like if this Porsche and Aston ever arrive, and if that tips anyone else into adopting it.