at the end of this video, the question was asked: what magic will apple have to pull off to make you care about their headset?
it may sound mundane, but as someone who uses a computer daily, i’m most excited about the ergonomic benefits of a headset vs a computer or phone. and “holographic” facetime is a huge draw because i’m so tired of video calls. they’re so bad.
so what i’d most like to see from apple’s headset is the ability to use my mac, freed from the concept of screens. if i’m gonna pick the headset over my computer, the headset will need the following:
- FLOATING APPS — put apps anywhere in view, not bound by any screen, and have them either follow me or stay where i put them, depending on the mode i choose
- FAMILIAR INPUT — the ability to use my mouse and keyboard
- UHD PASSTHROUGH — nearly indistinguishable from reality, especially wrt resolution, color & latency — i don’t want to work in VR
- RESOLUTION — if i’m to prefer this over my mac for creativity & productivity, it’s gotta be able to render text and graphics at least close to par with current macbook displays
- NO JITTER — the system shouldn’t struggle to track the world and keep apps in place or to smoothly move them around as my head moves
- WEIGHT & FIT — again, if i’m to grab this over my mac, i don’t want to be distracted by discomfort. this is about ergonomics for me, it’s gotta be light and stay in place easily
honestly if it could do all that to extend my computer, i would buy it. the ergonomic and productivity wins (no more switching between screens!) would make the sale. but the headset will probably also be a standalone device, and here’s what i’d like to see when not connected to a computer:
EYE-TRACKING & RING CONTROL
ergonomics again: if i’m to choose the headset over other devices, its control mechanism has gotta be at least as good as the trackpads and touchscreens we’re used to. for me, that means that we use our eyes as a cursor and a ring to tap, swipe, scroll. we don’t move the cursor with the ring. and we don’t have to use our hands to grab things mid-air (which sounds tiring AF), though we can if we want. the ring will provide reliable touch input and its glass surface will offer comfortable long-term use, just like a touchscreen or trackpad.
i don’t think it makes sense to rely on existing surfaces like a desk as a touch surface because of two major issues: unreliable tracking and eroding skin with extended use. imagine swiping for an hour on a wooden table, ow.
FLOATING (IMMERSIVE) APPS
again, the ergonomics of being able to browse the web, watch videos, read books, and play games with my head UP & BACK (finally!). i have frequent neck/back pain and i’m over it. this would even let me lay back to read a book/article hands free or watch videos lying in bed. sign me up!
apple could take this to another level if they went all-in on immersion, rebuilding stock apps without the design constraints of rectangular screens — e.g. a music app that displays shelves of records that i could swipe through with my ring; 3D visualizations created by the artist could appear in my room while the music plays; and eventually, 3D photorealistic renderings of the artist could sing right to me (with optional eye-contact!) — a personal concert with every song!
PRIVACY
since this kind of eye-tracking will produce precise data of where i look and for how long, i want complete control over how that data is used. the movement patterns could even be fed into an AI algorithm to guess at how i’m feeling about what i see. i don’t want apps — or even apple — to build targeted experiences based on this deep knowledge of such an intimate part of my consciousness. it would be too easy to manipulate me with this detailed insight.
HOLOGRAPHIC CALLS
photorealistic facetime calls is one of the most exciting features. if i could regularly call my mom while i’m traveling and see her life-size, make natural eye contact, and have crystal clear audio, this would feel fabulously futuristic and i would use it all the time.
SHARE-PLAY
pair holo-calls with share-play video streaming or music listening, and i’m sold. level it up with a VR mode where i can watch a movie with a friend in an IMAX theater on the ocean at night under the stars with a full moon near the horizon… or listening to music and dancing together in a neon-lit alien landscape with exotic creatures, bioluminescent plants, and massive ringed moons in the sky… and i’m super sold… and working together in shared digital space is an obvious, oft-hyped scenario and would be stunningly useful.
CREATIVITY & PRODUCTIVITY
obviously this is where apple shines, and i think there is a lot of space in the market to develop something truly fantastic, even with the tech that’s available today.
imagine a video editing app that doesn’t have to squish all its functions into a rectangle: professional video editing studios, with full-table setups could be recreated in 3D, placing advanced tools all over and floating above your desk…
imagine the headset works with apple pencil and can turn any surface into a drawing pad…
imagine music production with the full suite of tools available in pro recording studios recreated on your desk at home.
this is the untapped power of 3D apps that could democratize access to professional equipment at a tiny fraction of the cost to consumers and the environment!
GAMES
while i think apple will focus on their strengths — photo & video, communication, creativity, and productivity — i think they have done a good job with mobile gaming and will probably offer some significant gaming experiences, but i don’t think this will or should be the focus of this device.
VOLUMETRIC VIDEO
i don’t think this will be ready for v1, but eventually the ability to record 360-degree depth-video from my perspective and save or stream that video for others to “watch” is a mind-bendingly awesome feature. any sort of forward movement will cause motion sickness, so i’m not sure how this will work with non-stationary video. but for stationary video, viewers will be able to experience something exactly as the creator experienced it, precisely from their perspective and this could change everything.
