NASA partners with Microsoft to launch “mixed reality” Mars exhibit

NASA has collaborated with Microsoft to create a new mixed-reality exhibit known as “Destination: Mars“, allowing the public to explore the red planet with holographic tour guides. The exhibit will open this summer in Florida at NASA’s Kennedy Space Center Visitor Complex.

Featuring the second person on the moon, Dr. Buzz Aldrin, and guided by Curiosity Mars Rover driver Erisa Hines, viewers will check out real sites on Mars where scientists have made important discoveries. The experience was created using imagery taken from the Curiosity Rover which landed on Mars in August 2012.

Erisa Hines talks to participants in Destination Mars / Image courtesy of NASA/JPL

Erisa Hines talks to participants in “Destination: Mars” / Image courtesy of NASA/JPL

“Destination: Mars” makes use of OnSight software, a Mars rover mission operations tool co-developed by NASA’s Jet Propulsion Laboratory (JPL) and Microsoft. With OnSight, NASA scientists have a more intuitive way to explore the area around Mars rovers. Users can interact with a 3-D simulation of the Martian environment and use a first-person perspective to plan and conduct science experiments.

In a press release, NASA noted that up until this point, “rover operations required scientists to examine Mars imagery on a computer screen, and make inferences about what they are seeing.” Virtual reality can open up an entirely new way of conducting research on a planet over 100 million miles away from Earth.

“OnSight gives our rover scientists the ability to walk around and explore Mars right from their offices. It fundamentally changes our perception of Mars, and how we understand the Mars environment surrounding the rover.” Dave Lavery, program executive for Solar System Exploration at NASA Headquarters

OnSight is already becoming integrated in robotic Mars missions. Recently, Abigail Fraeman, a Curiosity science team member at JPL, used OnSight to identify a potential transition path for the rover between two Martian rock formations.

A screen view of OnSight / Image courtesy of NASA/JPL

A screen view of OnSight / Image courtesy of NASA/JPL

Microsoft has previously worked with NASA to send one of their virtual reality headsets, the HoloLens, to the International Space Station. Astronaut Scott Kelly tested the headset during his 340-day stay on station.

HoloLens projects light directly on the user’s eyeball in order to create a mixed reality experience. The headset overlays images on top of the actual environment around you.

Astronaut Scott Kelly (right) testing out Microsoft HoloLens on ISS / Courtesy of NASA

Astronaut Scott Kelly (right) testing out Microsoft HoloLens on ISS / Courtesy of NASA

In an interview after he returned to Earth, Kelly said that virtual reality has a lot of potential when it comes to space exploration. He explained that the headset had cameras on it which could share what you’re seeing to someone back on Earth.

“A person on the ground could be drawing things in your field of view and pointing to things. I could be doing the same. I could say ‘Hey, is this the bolt or connector you’re talking about?’ and the other person could just write an arrow in your field of view. And it worked great, I was really surprised.” Astronaut Scott Kelly on the Microsoft HoloLens

As for other strategic partnerships, it comes as no surprise that Dr. Buzz Aldrin would want to be involved in the “Destination: Mars” project. Through his books, social media accounts, and speaking engagements, Aldrin has proven to be a steadfast advocate for making Mars, rather than the moon, the next destination for human exploration.

While NASA had originally focused on first sending humans back to the moon under the Bush Administration, the focus was changed to Mars under the Obama Administration. NASA is currently centering time and resources on the Journey to Mars initiative.

“As we prepare to send humans to Mars in the 2030s, the public will now be able to preview the experience the astronauts will have as they walk and study the Martian surface.” Dave Lavery, program executive for Solar System Exploration at NASA Headquarters

Wherever the ultimate destination, NASA engineers are working to transform scientific and exploration efforts in space with the addition of virtual reality and mixed reality tools. With exhibits like “Destination: Mars” and tools like OnSight and HoloLens, exploring and working in space from Earth is becoming a much more realistic and immersive experience.