Timeline:
July 2023 (3 days)
The Team:
Simran Jagnik (Unity developer)
Leo(Backend Developer)
Linhao(UX Designer)
Dallas(UX Designer)
Simon(Unity developer)
Responsibilities
Unity AR prototype including 3D worldbuilding, animations, trigger interactions, physics, and UI.
Tools used:
Unity
C#
MRTK
Background
Image generators like Dall-E 2 and Stable Diffusion make imagining incredible fantastic new realities easier than ever, so how can we bring these alternate realities into our own?
Imagine being in your everyday space and suddenly interacting with Salvador Dali’s creations, exploring the cosmos, or experiencing what it would be like to have an alien cyborg riding an elephant right next to you.
Imagine being stuck designing an interior for a client, when the thought occurs, what if there was a medieval archway instead of a door?
By bridging the world generation of Stable Diffusion with the immersive experience of AR, we aim to open new possibilities into how our spaces can inspire us and help us make it a collaborative ARchive.
Imagining other worlds, surrealist environments, and the fantastic supernatural versions of our mundane reality has long been a central goal of novelists, playwrights, and filmmakers, but our experience brings these concepts into a physical space.
“What if our current environment belonged to another world?”
We built a custom node.js backend that runs the stable diffusion models on replicate with our custom parameters, to allow real-time image generation in under 4 seconds. Our experience interacts with our node project to click an actual angled image capturing its location and line of sight.
We tried different AR headsets before choosing Microsoft HoloLens 2 for its wider field of view, improved resolution, and more comfortable design. We used its hand-tracking technology to pinch and click interactions that create the portals.
Video Walkthrough
Showcase at MIT Reality Hack 2023
We got to showcase the AR at the MIT Immersive public exhibition. It was exhilarating to get to know what people liked about the concept and execution.