The Steam Frame has a heap of new features to dig into, but I want to focus on one here: foveated streaming. It sounds like a medical issue but I assure you, it’s all about ensuring the Steam Frame’s wireless connection is strong, stable, and appears high quality.
The Steam Frame offers a few ways to play games, both locally on its own Arm chip and streamed from a nearby gaming PC. As the name suggests, foveated streaming is used during the latter.
Rather than send two views (one for each eye), which is usually what’s required for VR, Valve is sending four. Two views are sent in low resolution and represent the entire field of view. The other two are only a fragment of the complete picture, but sent in high resolution.
“We send a bunch more bits to that area than we do to the low-resolution imagery,” Selan says.
It’s like a magnifying glass, offering a better glimpse of whatever it hovers over, and following a user’s vision.
“The trick is that the eye tracking is fast enough that it essentially beats where you’re going. So, by the time you look to the right, we’ve already sent the bits at the very high resolution to where you’re looking to the right. When we send if you look to the left, we beat your eyes looking there where we already have landed the bits at the full resolution.”
I’ve used the Steam Frame and I never noticed any low-resolution areas in my vision. I had almost forgotten about the foveated streaming until it was mentioned to me. I tried darting my eyes from left to right to try and trick the eye tracking, but it totally kept up.
Image 1 of 2
“We find that even for the most observant viewers, it’s very, very hard to see what’s happening,” Selan says.
I take this as a bit of a challenge, but I still can’t see any difference in quality anywhere on the screen. Even when Selan enabled a special mode in Half-Life Alyx that highlighted the high-quality area, where I’m looking, I still couldn’t make out the low-quality areas. I don’t know if you’ve ever tried to view something in your peripheral vision without looking at it, but it’s actually quite tough. Moreover, that part of your vision isn’t very detailed—your brain fills in a lot of the gaps. That’s what foveated streaming is taking advantage of.
Foveated rendering, a feature you might be more familiar with, works in a similar way with eye tracking. What’s known as quad view rendering is essentially the exact same system. The key difference here is that foveated streaming doesn’t lower the performance requirements for the PC that’s rendering the VR game to be sent to the Steam Frame—it’s working overtime to deliver the full, crisp image.
That’s a good and a bad thing. On the one hand, if your gaming PC is struggling to play VR games, foveated streaming won’t help. On the other, foveated streaming doesn’t require a developer to do anything. Though I wouldn’t worry about picking between them: Valve has confirmed you will be able to use both at the same time in games that support foveated rendering.
“Most critically, the software developer, the partners who ship on Steam, don’t have to do any work to take advantage of that,” Selan says.
Having spent time without a cable to anchor me down, I can see why Valve is pushing to remove wires from the VR experience. It’s far more convincing for a lack of it. So too does the streaming experience feel the most convincing part of the Steam Frame, even more so than its new onboard processing capabilities. Not the least bit because the battery life while streaming should be many more hours than playing games on the headset itself. That feels like an easy win, as does foveated streaming.