EIGHT YEARS AGO, the 2014 edition of IQT Quarterly, the publication of In-Q-Tel, notes that we are still “far away from a true Star Trek Holodeck experience,” yet:
A perfect simulated reality that is indistinguishable from real life will ultimately take one of two forms: it will either manipulate real light and real matter, like the Star Trek Holodeck, or it will remove the “middleman” of wearable VR inputs and instead directly manipulate our perceptions through a machine-brain interface, like that envisioned in The Matrix. Between those perfect simulations and the current state of the art, we envision the emergence of hybrids, such as the manipulation of real light (holograms) combined with haptic gloves, or the direct manipulation of the brain’s sense of touch combined with VR/AR contact lenses, or many other such combinations involving other senses. Given where VR is now compared to just 10 years ago, and the historical pace of technological change since the Industrial Revolution, it’s astounding to consider how VR might continue to evolve. We think these “perfect systems” may emerge within the next two centuries, and that the current state of the art provides a strong foundation to build upon.
The rapid investments from government sources in augmented and virtual reality reflect the vision laid out eight years ago. Mojo Vision, a startup based in Saratoga, California, is developing an augmented reality contact lens using “tiny microLED” displays the size of a grain of sand to project images directly onto the retina. The company is backed financially by the Defense Advanced Research Projects Agency, or DARPA, a research and development arm of the Pentagon.
In 2018, In-Q-Tel backed a startup called Immersive Wisdom that provides communication and data sharing interfaces. “Allowing multiple users to be anywhere in the world, while still being connected via the same virtual space containing shared maps, video feeds, and real-time data, offers a significant new edge,” In-Q-Tel said in a statement at the time.
In-Q-Tel also invested in DigiLens, the producers of a low-cost holographic lens used for augmented and virtual reality glasses. Last year, the Air Force awarded $1.2 million in contracts to the company.
According to the new disclosure, In-Q-Tel also invested over $1.9 million in Dreamscape Immersive, a Los Angeles-based virtual reality company. Co-founded by former Disney executive Bruce Vaughn, the company provides story-based virtual reality experiences, including the VR-based “Men in Black” feature released in 2019. As previously reported by Forbes, former Raytheon executive Dave Wajsgras — who became a member of Dreamscape Immersive’s board — has said, “The US department of defense is aggressively increasing spending on synthetic digital training which prepares personnel for real-life situations.”
Military interest in holographic imaging, in particular, has grown rapidly in recent years. The IDST article reported that military planners in China and the U.S. have touted holographic technology to project images “to incite fear in soldiers on a battlefield.” Other uses involve the creation of three-dimensional maps of villages of specific buildings and to analyze blast forensics.
PERHAPS NO INVESTMENT is as illustrative of the industry’s commitment to production despite potential red flags as the Defense Department’s flagship augmented reality project: the Army’s Integrated Visual Augmentation System, or IVAS, goggles. IVAS provides headsets to guide soldiers through unfamiliar territory, along with machine learning to instantly distinguish between friend or foe, and targeting systems for tracking vehicles and objects on the battlefield. (Investor presentations show the IVAS contract is also supported by Ultralife Corp., a battery provider, and Intevac, which produces night vision sensors and cameras for the defense industry.)
The Army has touted the system as a way to “fight, rehearse, and train” using “advanced eyewear that places simulated images in a Soldier’s view of real-world environments,” allowing soldiers to see through smoke or complete darkness.
The initial contract for the IVAS system, which is based on Microsoft’s HoloLens technology, was awarded to Microsoft and a number of other firms.
But the IVAS contract, which could one day cost upward of $22 billion, has faced chronic delays and failures. Last October, the Army pushed the official launch date to September 2022 for the product for operational fielding and testing.
The Defense Department’s 2021 annual report from its Director of Operational Test & Evaluation, or DOT&E, painted a sobering picture. According to the report, “Soldiers continue to lack confidence in their ability to complete the most essential warfighting functions effectively and safely while wearing the IVAS in all mission scenarios.” At worst, the goggles led to soldiers being unable to “distinguish enemy from friendly forces.”
But all such damning details were — despite congressional appeal — redacted from the publicly available version of the DOT&E’s report through being labeled as “controlled unclassified information,” or CUI. And while the Department of Defense Office of Inspector General made headlines last month through a report based on DOT&E’s critique, the underlying details remained redacted.
The CUI version of the Defense Department’s testing report was only made public through a leak to the nonprofit Project on Government Oversight, where Grazier works. POGO’s primary reason for releasing the document was to help expose the failure of the F-35 program, but Grazier stated that he ultimately released the entire document in hopes of crowdsourcing the analysis of broader instances of fraud, waste, and abuse.
“If you can’t distinguish between a friendly troop and an enemy troop, then you’re going to have a very difficult time distinguishing a civilian,” noted Grazier. “If a soldier can’t identify friends or enemies with their vision system,” he added, “that tells me that system probably needs a lot more work or possibly needs to be scrapped altogether.”
Soon after Congress put $349 million of its IVAS funding on hold in March, Insider reported on a leaked Microsoft memo in which a manager wrote that the company “expect[s] soldier sentiment to continue to be negative as reliability improvements have been minimal from previous events.” HoloLens boss Alex Kipman reportedly described his team as “[s]o depressed, so demoralized, so broken.” The Wall Street Journal reported that roughly 100 Microsoft HoloLens employees left to work at Facebook parent company Meta Platforms Inc. in 2021.
In February, Secretary of the U.S. Air Force Christine Wormuth also poured cold water on the project, at an event for the Center for a New American Security, tempering expectations.
“Remember early satellite phones from the 1980s that wealthy people had in their cars?” said Wormuth during the event. “They were big and clunky and now we have iPhones. It took us some time to get there. The first iteration of IVAS may not be quite as streamlined as we want it to be ultimately, but it’s the alpha version, and we need to start there.”
The military, including soldiers at Fort Benning, have found some productive applications of the IVAS system. Rather than forming the basis of futuristic cyborg warriors, the HoloLens goggles have been an expensive thermal sensor for rapidly detecting soldier temperatures — as a way to screen Covid-19 cases.