Reimagining America’s Pastime for the Spatial Era
From flat screens to immersive worlds. The MLB App on XR transforms live baseball into a living, spatial experience—where fans don’t just watch the game, they step inside it.
CREDITS:
Vice President, Design and Innovation
Alexander Reyna
Role
EP, Product Design, Feature Prototype, UX Experiments
Collaborators
David Santana, Product Design , Art Direction, UX/UI
Christina Goefron, UX/UI
Chris Makris, Technical Art
Kamil Durzynski, Lead Art
Geoff Erickson, Lead Engineering
Galal Hassan, Lead Engineering Hawkeye
Chuck Noble, Lead Engineering ML + Data
Mike Bell, Lead Character Technical Direction
Nathan Tompkins, Senior Producer
A Multi-Year Evolution in Immersive Sports
What began with the launch of MLB on Apple Vision Pro in February 2024 grew into a cross-platform ecosystem that redefined how fans experience baseball.
February 2024 — Apple Vision Pro: Introduced spatial viewing and Statcast-powered immersion.
September 2024 — Meta Quest: Extended the experience to accessible VR, connecting a wider audience.
October 2025 — Galaxy XR: Delivered the highest-fidelity version yet—featuring full-limb player tracking and real-time, field-wide digital recreation.
Each launch built upon the last—expanding fidelity, deepening interaction, and uniting design, data, and storytelling across devices.
Challenge
Meeting the Fans of Tomorrow
Baseball’s greatest opportunity isn’t just to innovate—it’s to meet the fans of tomorrow where they already are. In digital spaces. On immersive devices. Inside interactive worlds that merge data, drama, and connection.
Baseball’s greatest opportunity isn’t just to innovate—it’s to meet the fans of tomorrow where they already are. In digital spaces. On immersive devices. Inside interactive worlds that merge data, drama, and connection.
From Passive Viewing to Personal Connection
Younger audiences crave participation, personalization, and presence. Yet traditional broadcasts kept them at a distance—confined to static screens and one-size-fits-all perspectives.
Building for a New Reality
Developing for emerging spatial platforms required reimagining interaction, performance, and production pipelines from the ground up. Apple’s Vision Pro demanded intuitive gaze controls and cinematic fidelity; Meta Quest required optimization for comfort and accessibility.
Challenges of a Groundbreaking Platform
Integrating real-time baseball data into XR pushed technical limits. Machine learning powered Hawkeye motion cleanup, refining optical tracking into smooth, human-authentic animation. A unified player-likeness pipeline merged photogrammetry, rigging, and shading for lifelike avatars, while the multiview system enabled fans to follow multiple games and angles simultaneously—an architectural feat demanding precision sync and GPU efficiency at 8K.
Cross-Platform Parity—and Divergence
The early Vision Pro and Quest versions used lightweight player “chips”—center-of-mass abstractions optimized for streaming performance. But the Galaxy XR release (October 2025) represented a creative and technical milestone: full-limb player tracking for every athlete on the field, rendered in real time. Each player’s anatomy, motion, and lighting were reconstructed with unmatched fidelity, achieving the sport’s most realistic digital recreation to date.
Passthrough minifield example
Home Plate Camera view
I typically lead technical art and development of all our products and set the tone for what's possible in XR with early experiments leading to functional prototypes. The MLB App on XR devices achieved significant technical art milestones by integrating advanced rendering and animation pipelines into a demanding, real-time XR environment.
Our targets required aggressive optimization to keep both high frame rate and high quality rendering.
- Capture and 3d scans of home plate for high fidelity geometry and texture
- The development of a time of day lighting system using latitude and longitude to deliver a look that matches real world broadcast lighting
- Ultra performant spotlight shadows and associated pass through transparency shaders that allow for unique “areas of interest”.
- The development of a player database with customized values for all MLB athletes.
- Support for a uniform lettering system that includes all team jersey variations including team lettering font
- Skin shader development, driven by the player database to power unique likeness.
Home Plate dirt details
Time of Day Sky System tests
Shader lighting and transparency test
All Star player editor
Player Lettering
Skin shader example
A New Dimension in Capture
MLB’s 2025 postseason became the testbed for our 120 FPS 8K spatial video project
We pioneered the first-ever live HLS delivery of stereoscopic 8K spatial replays. This collaboration between our capture, encoding, and XR design teams enabled dual stereo spatial highlight clips from the World Series to play directly within the MLB App environments. For the first time, fans could relive the game from “impossible angles” built with an eye towards a production broadcast solution.
New Content for New Platforms
The 2024 World Series was the perfect time to tell an immersive story in 8K 180
We’ve been shooting VR 360 and 180 since 2016 and constantly evaluate new capture tech. Because 2024 was a major year with the AVP announcement and the new options available for high fidelity 8K capture, we decided to capture all of World Series 2024 in a three camera shoot at 90FPS . This led to a seven minute edited featurette covering the rise and fall of New York as they tried to hold off the Dodgers inevitable march to victory.