"Cyberspace. A consensual hallucination experienced daily by billions of legitimate operators, in every nation, by children being taught mathematical concepts... A graphic representation of data abstracted from the banks of every computer in the human system. Unthinkable complexity. Lines of light ranged in the nonspace of the mind, clusters and constellations of data. Like city lights, receding..."

-- William Gibson, Neuromancer

As a science fiction buff and generally imaginative person, my friend Bruce enjoys a good virtual reality tale. Whether it’s Neromancer's cyberspace, Stephenson’s Metaverse or just your run-of-the-mill Matrix, he enjoys the philosophical conundrums of the mind-body duality as much as the next guy.

But as a computer programmer, just don’t get him started on the feasibility of actually building massively populated virtual worlds. He happens to know a thing or two about latency, and knows too well that it’s the long-term lethal flaw awaiting futuristic virtual world building.

I call his issue “Bruce’s Virtual Reality Lag Snag”.

The Lag Snag is this: Every step you take toward creating a more plausible online, counter-reality will suffer from increasing, un-realistic, performance lag.

Or put into a slightly longer format: As you add more and more to the virtual environment to make it real--more people, more types of clothes, waving blades of grass, birds in the air and bunnies on the ground--the more data you create and the more slowly the world responds to your presence.

In practical terms, this means that the experience of the data that comprises a virtual world suffers from an experiential lag compared to what you’d expect. The time it takes for light to shine off a tree in front of you may be so quick that it appears instantaneous in real life. But online, that virtual light takes a noticeable amount of time to reach you when it comes from a server in Korea.

The cumulative effect of this is that a virtual world which otherwise looked real would be experienced online as if time were stuck in jelly, was dreamlike or just plain weird. You’d tell your avatar to reach for a switch and a noticeable fraction of time later, they’d reach out. Or perhaps, your character would reach for the switch in real time, but the light would turn on after a moment’s hesitation. Or maybe time would just stutter along. Some things would seem to happen in real time, other things would happen curiously just a little in the future, or perhaps a little later in the past.

And the reasons for this are all technical. It takes longer for virtual reality to travel through the network and work its way through your computer, video card and out your monitor than we are used to in “real” reality.

At the heart of this lag in experience is “network latency.”

A good definition of network latency reads:

“The minimum time a network needs to send the smallest possible amount of data. Latency depends on things like line speed and the receive and retransmit delay in routers and modems. A low latency indicates a high network efficiency.” (http://www.numion.com/faq/ping.html).

The key terms here are “minimum time” and “smallest possible amount of data”. So, when you read a ping number in a game, that ping is a measure of the network latency. That’s the fastest that data is moving up and down, give or take, between your machine and the game server.

Network (or Internet) latency is then simply the notion that it takes a measurable amount of time to ship data from place to place. Although the data may move down the wire at the speed of light, it encounters all kinds of slowdowns—routers, switches and firewalls—that reduce the pace of the data from place to place. Even the transmission of the signal itself takes some time. To send a signal at the speed of light all around the Earth would take about 134 milliseconds, if the routing were 100% efficient. That’s a ping rate of 134 just to send a signal at maximum speed around the fatest part of the globe. (And if it seems unreasonable to quote an unnecessary roundtrip around the planet, imagine the same lag issues when bouncing signals off satellites that sit about 22,000 miles above the planet. It would take about twice as long as trip around the world to bounce a signal up and down off a satellite that far away. Take that Wintermute!)

So, how fast is fast enough?

People in the telephone business have been studying that question for quite some time. The conclusion is that anything under 200 milliseconds (.2 seconds) is usually acceptable and anything over 400 is not. Or, to put it in specific terms, if a telephone call lags by more than about .2 second between the time you say something and the time the other person hears it, it gets annoying. Once the lag reaches over a half-second, it’s hard to communicate. (http://www.tmcnet.com/articles/itmag/0899/0899brook.htm)

And that’s for voice communication. It’s likely that our tolerance for lag is even lower in other areas. Imagine how disconcerting it is to see an umpire signal a strike, only to hear the word “Strike” catch up a moment later. We are used to it, but it’s weird.

Take these lag scenarios online and they get even more unusual. Imagine you’re a pitcher in a virtual game of baseball. It all looks real enough. You can count the stitches on the ball and feel a cool breeze blowing in from left field. But as you throw a high fastball, a subtle virtual Doppler effect occurs--the pitch seems to slow as it reaches the batter, then suddenly flings back toward the mound at an unexpected velocity as the batter connects.

These kinds of data lag effects could even mess up simple bow and curtsey efforts in an RPG. The gentleman bows, the lady walks past, then after hesitating ever so slightly, curtseys. Was she being demure? Was the response polite, rather than affectionate? These simple dips in time could pool up with murky meaning.

So, to put those ping stats into perspective I checked up on Battlefield Vietnam today and the ping rates were largely in the range of 100 – 200 milliseconds. That’s in the acceptable range, right?

Not exactly. Remember ping measures the minimum time for a minimum amount of data. So, in practice, in terms of the experience of what is happening onscreen, things are probably running a lot slower.

Game designers work around many of these issues through a variety of clever tricks. For instance, they can prioritize certain types of data and make sure that this data is very small and more likely to travel quickly. In this case, the location of your opponent might be transmitted with a small packet of compressed info that just might make it in the minimum amount of time. Or, sometimes an online system can just guess. If you shoot a missile, the program can send the rocket’s destination info and let the computer on the other end calculate the hit.

There are plenty of tricks. But they all come up against the same basic problem—it takes a while for info to travel across the network, from one place to the next, from the game server out to player machines.

Of course network latency isn’t the biggest problem today with massively multiplayer virtual world building. A sticky issue that holds back the evolution of these worlds has to do with coming up with computing architectures that are big enough and fast enough to keep track of all the data used to make the world. Even if you assumed an infinitely fast network, we’d still have all kinds of play lag as centralized severs struggle to keep all the player machines up to date with the latest computed state of affairs.

This problem may be on the road to solution.

At E3 this year Masa Chatani, the chief technology officer for Sony Computer Entertainment talked about the Cell processor at the heart of the next generation PlayStation. One thing he made clear was that the Cell architecture was designed to work as a part of a parallel computing system. Now, that could mean that multiple Cell chips in a single box could give unprecedented parallel processing power in the home. On the other hand, imagine a world with 10 million Cell-based home game machines, all linked to the Internet. Massively multiplayer game play might finally live up to it's promise. With every PlayStation 3 working in collaboration with every other PS3, the possibility to generate virtual worlds at level of detail unseen so far becomes possible.

So today, one of the current barriers to virtual world building is that everyone in the world needs a copy of the world on their computer—at least in terms of the graphics adventures. If you buy Everquest, all those discs you load up are a copy of the world in as many reusable parts as the developer can provide. If someone invents something new, sooner or later it has to be shipped down to you computer. And so, massively multiplayer gamers have grown used to periodic chunky downloads of new assets and the mix-and-match nature of scenery, costume and character.

Something like a Cell processor network might make that unnecessary. Machines could work together to continually process an ongoing stream of data (possibly doing nothing more sophisticated than running highly efficient compression on the data). When I set up my home in a virtual world, I could populate it with all kinds of handcrafted items. When you come to visit me, my machine could talk to your machine to ensure that you see what I see. No central processor tracking the whole world would be necessary. Each player would bring their own processing power to the mix to collaboratively compute the results. While I’m checking my inventory prior to a quest, my machine could be contributing to the ray tracing of a cool dragon somewhere else or pitching in on some AI calculation for a computer controlled merchant halfway around the globe.

But even as the parallel computing power becomes available and these worlds become startlingly realistic, Bruce’s Lag Snag will kick in. All that data has to move around. When some stranger visits your virtual apartment and knocks over a chair, that data has to move. When someone rents a virtual plane to tow a “Will you marry me?” banner across the virtual skies, data has to move. When you jostle through a crowd, data moves.

Bigger pipes will help. By some estimates the human nervous system can take in about 10 million bits of information a second. So, it’s reasonable to imagine a world where every home has a T-3 pipe delivering 5 megabytes a second (or potentially about five times the amount of sense data that we can perceive). This, of course, skips over the ample issues and questions related to what kinds of sensory data we could push and need to push through the pipe to make something real, nor the actual interface between the raw digital data and the raw nerve endings (or brain surface) that we’d need. No, even after we assume all those massive hurdles overcome, we’d still face the Lag Snag.

Today, we live at the edge of impressive advances in virtual world creation and participation. Technologies like the Cell processor make it clear that the kind of processing power we need are reaching consumer levels and that the bandwidth we need is not in the realm of science fiction. We can imagine building some of the complex, immersive virtual worlds long dreamed of. But the closer we get, the more potent the Lag Snag

It might look real from moment to moment, but the flow of data across the network will not be able to keep up.

There is a fundamental phenomenological disjunction between the spatialization of experience in real and virtual environments

The future may hold some interesting and surprising solutions, of course. To give him credit, Gibson didn’t envision the Matrix—virtual life as simulacrum. No, he thought about an abstract dataspace that looked more like Rez and less like the Holodeck. Simplier representations should require less data and could feasibly keep us below the noticeable lag threshold.

But based on our obsession with artistic realism, we’ll need to look for other answers. Perhaps a form of quantum computing will allow two processors on either side of the globe to communicate instantaneously. Maybe advances in switching and materials will decrease the transmission times for large loads of data such that we remain below our 200-millisecond bar. Or maybe the herky-jerky dreamtime world of online life will grow familiar and comfortable. It will seem as real as a dream, but we’ll always know that we are dreaming.

Whatever the case, the Lag Snag lies in wait us, patiently promising to one day to delay the satisfaction of our virtual desires.