Unlocking a New Dimension

Back in 2008, I left the corporate world to jump into the just-opening world of the App Store, and I have spent the years since working first on mobile games and then mobile applications/graphics here with About Objects for some of the best-known companies in the world. As the world leaps into Apple’s Vision Pro in 2024, I’m thrilled to be part of the first wave of development for what promises to be the next frontier of computing.

We’re currently developing mixed reality (MR) experiences integrating 3D content with traditional interfaces for Vision Pro, going far beyond what can be done on flat screens and monitors. Taking 3D geography models and enhancing them with real-time data enables spatially-aware planning and monitoring of real-world scenarios. Essentially, we take incoming live positioning data, process it through custom Bezier curves and other geometry algorithms, and use the resulting MeshBuffers to generate new MeshResources. As our scenarios get more complicated, we are beginning to push the limits of RealityKit. Still, with a bit of throttling, current experiences are supported well enough, but there is room for improvement in the latency of generating dynamic MeshResources. With the upcoming release of VisionOS 2.0 and the power of the hardware, we expect to see a marked decrease in that latency and overhead. After processing, the data feeds spatial annotations on the 3D USDZ models, and the custom-generated 3D geometry provides additional context for users – letting them evaluate changes and make on-the-fly updates. We’re also leveraging full immersion, dropping users into breathtaking environments that increase their understanding of real-world locations, which is useful for training and situational awareness.

Spatial Computing on Vision Pro is a paradigm shift for interfaces reminiscent of the leap we saw when the iPhone revolutionized mobile computing. I look forward to seeing what we’ll be able to deliver, what new capabilities Apple will introduce as this technology matures, and what WWDC 2024 will bring!