If we’ve learned anything about technology over the last few decades, it’s that we don’t prepare for the downsides until the problems are so egregious we can’t ignore them. The poster child is social media, which was hailed as utopian when it first arrived, but is now widely considered a destructive and destabilizing force in society. It took over a decade for this to sink in, but these days a large majority of Americans believe social media has a mostly negative effect on our world.  

The reasons cited include the spread of misinformation, hate, harassment, polarization and partisanship. Of course, there’s nothing about the technology itself that creates these problems. It’s the business models behind social media that have driven platforms to mediate information flow across society, filtering and amplifying content in ways that distort our thinking. This is a form of mind control, and it’s about to get much worse. I’m talking about the metaverse. 

Unless regulated, the metaverse could become the most dangerous tool of persuasion ever created. I don’t make this warning lightly. I’ve been a technologist in this field for over 30 years, starting as a researcher at Stanford, NASA and the U.S. Air Force and then founding a number of early companies in the space. I genuinely believe the metaverse can be a positive force for humanity, but if we wait for the problems to become egregious as with social media, it will be too late to undo the damage.  

To raise awareness about the issues, I’ve written many articles about the dangers of the metaverse and the need to protect human rights, but I have not explained from a technical perspective why immersive technologies are so much more dangerous than traditional social media. To do so, I’d like to introduce a basic engineering concept called feedback control. 

Event

GamesBeat Summit Next 2022

Join gaming leaders live this October 25-26 in San Francisco to examine the next big opportunities within the gaming industry.

Register Here

It comes from a technical discipline called control theory, which is the method used by engineers to control the behaviors of a system. Think of the thermostat in your house. You set a temperature goal and if your house falls below that goal, the heat turns on. If your house gets too hot, it turns off. When working properly, the thermostat keeps your house close to the goal you set. That’s feedback control. 

Of course, engineers like to make things more complex than they need to, so the simple concept above is generally represented in a standard format csystem a control system.

Generic “control system” diagram. Image via Wikipedia

In the heating example, your house would be the system, a thermometer would be the sensor, and the thermostat would be the controller. An input signal called the reference is the temperature you set as the goal. The goal is compared to the actual temperature in your house (i.E., measured output). The difference between the goal and measured temperature is fed into the thermostat which determines what the heater should do. If the house is too cold, its heater turns on. If it’s too hot its heater turns off. That’s a classic control system. 

Of course, control systems can get very sophisticated, enabling airplanes to fly on autopilot and cars to drive themselves—even allowing robotic rovers to land on Mars. These systems need sophisticated sensors to detect driving conditions or flying conditions or whatever else is appropriate for the task. These systems also need powerful controllers to process the sensor data and influence system behaviors in subtle ways. These days, the controllers increasingly use AI algorithms at their core. 

With that background, let’s jump back into the metaverse.

Referring back to the standard diagram above, we see that only a few elements are needed to effectively control a system, whether it’s a simple thermostat or a sophisticated robot. The two most important elements are a sensor to detect the system’s real-time behaviors, and a controller that can influence those behaviors. The only other elements needed are the feedback loops that continually detect behaviors and impart influences, guiding the system towards desired goals. 

The human in the loop

As you may have guessed, when considering the danger of the metaverse, the system being controlled is you — the human in the loop. After all, when you put on a headset and sink into the metaverse, you’re immersing yourself in an environment that has the potential to act upon you more than you act upon it. Said another way, you become an inhabitant of an artificial world run by a third party that can monitor and influence your behaviors in real time. That’s a very dangerous si

Control system diagram with “human” in the loop

In the figure above, system input to the human user are the immersive sights, sounds and touch sensations that are fed into your eyes, ears, hands and body. This is overwhelming input — possibly the most extensive and intimate input we could imagine other than using surgical brain implants. This means the ability to influence the system (i.E. You) is equally extensive and intimate. On the other side of the user in the diagram above is the system output — that’s your actions and reactions. 

This brings us to the sensor box in the diagram above. In the metaverse, sensors will track everything you do in real time — the physical motions of your head, hands and body. That includes the direction you’re looking in, how long your gaze lingers, the faint motion of your eyes, the dilation of your pupils, the changes in your posture and gait — even your vital signs are likely to be tracked in the metaverse, including your heart rate, respiration rate and blood pressure. 

In addition, the metaverse will monitor your facial expressions and vocal inflections to track your emotions in real time. This goes beyond sensing expressions that other people notice; it also includes subconscious expressions that are too subtle for humans to recognize. Known as “micro-expressions,” these events can reveal emotions that users do not intend to convey. Users may not even be aware of feeling those emotions, enabling metaverse platforms to know your inner feelings better than you do.

This means when you immerse yourself into the metaverse, sensors will track almost everything you do and know exactly how you feel while doing it. tuation.