Support NeoGAF

[Eddie-Griffin]

https://www.idropnews.com/rumors/ap...-to-interact-with-your-iphone-and-mac/192272/

Features like Continuity and Handoff have become legendary, delivering an almost magical experience when working with multiple Apple devices together. Open an app on your iPhone or Apple Watch, and you’ll see an icon for it in your Mac’s dock, allowing you to transfer over whatever you’re doing without missing a beat. For instance, if you start an email on your iPhone and then decide that it’s too long to peck out on the touchscreen keyboard, you can transfer that draft to your MacBook with a single click and pick up right where you left off.

There’s also a plethora of other little touches that Apple has added over the years, from unlocking your Mac with your Apple Watch and handing off music from your iPhone to a HomePod to using your iPhone to enter text on your Apple TV and sharing files with AirDrop.

Hence, it’s a given that Apple’s upcoming mixed-reality headset will offer continuity-like features. However, a new European patent application is giving us some more insight into what Apple’s working on, and it looks like the company plans to take this seamlessness to a whole new level for its headset.

‘Extended Reality Systems’​

The patent, which is titled Multi-Device Continuity for Use with Extended Reality Systems, describes a level of user experience that goes way beyond simply selecting an icon to transfer an app. Instead, it defines an “extended reality system” in which users will be able to interact with devices and move apps and data around simply by looking at things.

Apple provides some cool examples in the patent as to how it envisions this working, from visualizing data on pages that appear blank to everyone else to starting music playback by merely glancing at and gesturing toward a HomePod in the same room.

In one example, Apple describes how a user editing an email on their iPhone could quickly transfer their work in progress, put away the iPhone, and continue on the headset using an extended reality (XR) view that replicates a virtual screen suspended in front of them that they would interact with via hand and finger gestures to finish composing the email. This virtual display could be as large as the wearer’s field of view allows, with a full-sized virtual keyboard to replicate the typing experience. The user would simply move their fingers as if typing on a real keyboard, with the headset’s sophisticated camera system would pick up those movements and translating them into keystrokes.

In another example, Apple describes transferring music playback from an iPhone to a HomePod simply by looking at and gesturing toward the speaker in question — even from the other end of the room.

Three-dimensional information regarding the devices in the physical environment can be gathered by the XR device, and used to facilitate smooth and continuous transfer of control and/or content between the devices and/or the XR device.

The patent goes beyond HomePod speakers, describing how the headset could map out every device in the room. One example illustrates how it could overlay information in “accessory windows” that appear alongside an actual display, such as an iMac or a TV. This could be used to display contextual information that would be visible only to the wearer, such as identifying the actors in a movie or television show or displaying more personal and confidential information related to a file or email that’s open on a desktop computer.

Of course, as with most patent applications, it’s important to note that there are never any guarantees that these technologies will see the light of day, and we’re not holding our breath that all of this will arrive in Apple’s first-generation headset, which could be announced as soon as this year’s Worldwide Developer’s Conference (WWDC) in June. However, patents like these provide great insight into Apple’s thinking, and there’s little doubt the company is aiming to provide a user experience like this for its headset, providing it can work out all the kinks.

So a person can play gaming on a MAC, then continue on the headset, or play games on the headset and continue on the Iphone and back and forth, and then can type a message to a friend about the big game in <insert sport> Sunday night, and then finish typing it with their VR headset's virtually keyboard, while using XR to walk to their kitchen and drink some OJ, because he is lose once again and not in jail, and can listen to their CrippledBiscuit album as they take their ice cream out the microwave, and then glance over to a Homepod to play it on speaker, so you can go back to playing or surfing the superhighway on VR.

Apple is using continuity and hand-off features to integrate your IOS devices, and possibly some other devices not from Apple if they are smart.

This 'Virtual information superhighway'(C)(R)(TM) is integration we have yet to see from any VR headset yet.

"if you start an email on your iPhone and then decide that it’s too long to peck out on the touchscreen keyboard, you can transfer that draft to your MacBook with a single click and pick up right where you left off."

I mean that's pretty incredible if it's executed correctly and actually fast and seamless, like near instant.

They really are going all in on this headset.

"describing how the headset could map out every device in the room. One example illustrates how it could overlay information in “accessory windows” that appear alongside an actual display, such as an iMac or a TV. This could be used to display contextual information that would be visible only to the wearer, such as identifying the actors in a movie or television show or displaying more personal and confidential information related to a file or email that’s open on a desktop computer."

This is crazy ambitious stuff, if they can get half this stuff to work as well as implied that's a pretty big deal.

 

[Buggy Loop]

So a person can play gaming on a MAC, then continue on the headset

If its a closed ecosystem not supporting PCVR/SteamVR, no way i buy it.

 

[Magic Carpet]

Took Apple years to get the seamless airpods connect from ipad to iphone. Many updates and it finally works as promised.

 

[DemonGamerTTT]

Interact with your phone and Mac...
"High-end" gaming accessorie here folks.

 

[Eddie-Griffin]

If its a closed ecosystem not supporting PCVR/SteamVR, no way i buy it.

Macs support steam tho.

 

[Calverz]

Macs support steam tho.

They do but it’s absolutely dire.

 

[pauljeremiah]

They do but it’s absolutely dire.

Exactly, I have 700+ games on steam but when I open it on my iMac it’s “here’s the 30 games you can play from your library”

 

[ReBurn]

I wouldn't have any doubt that whatever integration they have would be seamless. I just wonder if it will be useful. What could I be doing on my phone that I would want to hand off to a VR headset? Watching YouTube maybe?

 

[midnightAI]

https://www.idropnews.com/rumors/ap...-to-interact-with-your-iphone-and-mac/192272/




So a person can play gaming on a MAC, then continue on the headset, or play games on the headset and continue on the Iphone and back and forth, and then can type a message to a friend about the big game in <insert sport> Sunday night, and then finish typing it with their VR headset's virtually keyboard, while using XR to walk to their kitchen and drink some OJ, because he is lose once again and not in jail, and can listen to their CrippledBiscuit album as they take their ice cream out the microwave, and then glance over to a Homepod to play it on speaker, so you can go back to playing or surfing the superhighway on VR.

Apple is using continuity and hand-off features to integrate your IOS devices, and possibly some other devices not from Apple if they are smart.

This 'Virtual information superhighway'(C)(R)(TM) is integration we have yet to see from any VR headset yet.

"if you start an email on your iPhone and then decide that it’s too long to peck out on the touchscreen keyboard, you can transfer that draft to your MacBook with a single click and pick up right where you left off."

I mean that's pretty incredible if it's executed correctly and actually fast and seamless, like near instant.

They really are going all in on this headset.

"describing how the headset could map out every device in the room. One example illustrates how it could overlay information in “accessory windows” that appear alongside an actual display, such as an iMac or a TV. This could be used to display contextual information that would be visible only to the wearer, such as identifying the actors in a movie or television show or displaying more personal and confidential information related to a file or email that’s open on a desktop computer."

This is crazy ambitious stuff, if they can get half this stuff to work as well as implied that's a pretty big deal.

I still don't see any evidence this is aimed at gaming at all. (Playing 2D mobile games in a window in VR does not count)

 

[Eddie-Griffin]

Exactly, I have 700+ games on steam but when I open it on my iMac it’s “here’s the 30 games you can play from your library”

I wouldn't be surprised if they work on that with SteamVR. Would improve headset engagement, especially for those steam users already in Apples ecosystem in some form.

 

[Tams]

A load of that is shit I don't want or need. I compartmentalise most of my time and activities. All I really want synchronised are messages, photos, and documents. And I don't need any of them on my gaming devices. Nor do I really need my games on my other devices.

But hey, if there are people out there who want this, good for them.

 

[theHFIC]

I wouldn't be surprised if they work on that with SteamVR. Would improve headset engagement, especially for those steam users already in Apples ecosystem in some form.

Apple and Valve partnered up for Steam VR on Mac along with HTC in 2017 but by 2020 Valve announced it was ending support of Steam VR on Mac to focus o. Windows / Linux - it didn’t sound that amicable.