Support NeoGAF

[thehillissilent]

For those interested, Digital Foundry did a comparison article for 2 cloud gaming services: OnLive and Gaikai. They both have different business models. OnLive is more like a retailer and Gaikai is utilized by publishers but will be allowing full game streaming soon.

http://www.eurogamer.net/articles/digitalfoundry-face-off-gaikai-vs-onlive

Graphics settings/Image quality: Gaikai is the clear winner due to maxed out graphics settings

One of the most compelling arguments in favour of cloud gaming is that hardware is upgraded server-side, potentially allowing for better-than-consoles visuals. Right now, only Gaikai is really delivering in terms of high-end visuals.

OnLive is running games on low settings. Only plackpack titles can be maxed out.

We came away disappointed with OnLive in this regard when we first looked at the service, and in the here and now, nothing much has really changed. Far from delivering high-end, console-beating experiences, we are treated to a mixture of downgraded visuals with higher than console frame-rates.
In order to maintain the target 60FPS as closely as possible, graphics settings are dialled back to a standard that is in some cases visibly worse than what we are seeing on the current consoles. In previous analyses, we deduced that OnLive servers most likely use dual core CPUs combined with something along the lines of an NVIDIA 9800GT or 9800GTX. These conclusions were based on matching graphics between the cloud service and the original PC game, then measuring performance. As OnLive blocks off graphics settings menus, it was the only way we could compare.

Gaikai is running games at max setting, which can be adjustable.

Interestingly, Gaikai takes a different approach in allowing the user to tweak the video settings in some titles, while also providing an upper-end baseline for games where the user is unable to make any changes in this area. A look at the display settings menu in From Dust hosted on Gaikai reveals that an NVIDIA GeForce GTX 560Ti is used in the terminals to run the game, while the refresh rate is set up to allow for a 60FPS update - a Core i5 and GTX 560ti combo would be enough to run all of the titles featured on Gaikai at 60Hz, most of them at 1080p.....
In all of the games we've tried the graphics settings are generally set to maximum (or thereabouts), with high levels of texture filtering present and with 8x multi-sampling anti-aliasing (MSAA) enabled. The batch of screenshots below give you a good idea of just how much of a visual upgrade you are getting over what is routinely available in OnLive, and on current console hardware

Framerate and Performance: OnLive is the winner due to encoding video at 60FPS.

OnLive's most obvious advantage is temporal resolution - it aims for 60FPS on our comparison games and does a decent enough job of maintaining it, while Gaikai offers an inconsistent update closer to 30FPS.

OnLive is smoother since the video is encoded at 60FPS.

Running at 60FPS, the action is smoother with OnLive and the visible judder that is present with the game on Gaikai is completely absent. Even when OnLive does drop frames, it's far less noticeable compared to its rival. What we describe as a 'perceptual' 60FPS is in play here, where the dips in performance are so small that they can go unnoticed by the player when engrossed in gameplay.

Gaikai is less smooth and framerate dips are much more noticable due to lower video encoding rate. However, Gaikai mentions running their games at 60FPS in the datacenter in many articles. http://www.edge-online.com/news/gaikais-perry-cloud-latency-not-concern

Compression Quality: Gaikai is the clear winner. Gaikai is using x264 software compression and OnLive is using a custom hardware encoder although both are rendering h.256 video.

In terms of video quality, Gaikai's advantage is clear. While OnLive often looks muddy, blurry and is filled with heavy compression artifacts during the general run of play, video quality is much more solid with Gaikai - in fast-paced scenes with lots of complex scenery, detail is maintained, and despite there being some visible artifacts on show at times, we really get the impression that we are looking at something closer to a native high definition presentation.

Gaikai seems to encode the game at a lower framerate in order to allow more video bandwidth for a boost in video quality. However, Gaikai did state that they were running games at 60FPS in the datacenter (http://www.edge-online.com/news/gaikais-perry-cloud-latency-not-concern). Also, Gaikai can use more bandwidth than OnLive however, it does hover around 5mbps on average from my personal test.

On the other hand, Gaikai appears to aim for a more manageable 30FPS, seemingly relying on the use of more local datacentres to keep the level of input lag in check. Our theory is that the games themselves may actually be running at 60FPS server-side with Gaikai, but with the video encoder not encoding every frame generated, giving a lower frame-rate client-side.

Basically, fewer unique frames means less work for the compressor to deal with when encoding the video stream. From another perspective, dropping down to 30FPS also provides double the amount of bandwidth for image quality and thus delivers overall clarity closer to the experience of gaming on local hardware.

Latency: Gaikai is the overall winner due to the varied location of their datacenters. However, there are some cases where OnLive can match the latency of Gaikai. Although, Gaikai was actually able to match console latency in some test concerning Bulletstorm (133ms).

Higher frame-rates should give OnLive a clear latency advantage over Gaikai, but testing demonstrates that this is not the case. Gaikai's more local datacentres offer the service an important advantage over its rival

AC Brotherhood
OnLive: 183ms, 183ms, 183ms Gaikai: 167ms, 183ms, 167ms

AC Brotherhood (under load)
OnLive:200ms, 183ms, 200ms Gaikai: 216ms, 167ms, 216ms

Orcs Must Die
OnLive: 283ms, 250ms, 283ms Gaikai: 183ms, 183ms, 183ms

Orcs Must Die (under load)
OnLive: 283ms, 300ms, 283ms Gaikai: 183ms, 183ms, 183ms

The results are rather interesting. Not only does the amount of latency present in some of the titles running on Gaikai equal that of OnLive, but there are also times where controller response is significantly faster despite the frame-rate being lower due to the 30FPS video encode. Taking a closer look at Assassin's Creed on Gaikai reveals that when the frame-rate is consistently hitting the target, the level of latency hovers around the 167ms to 183ms mark - it's playable, but inconsistent. With the game running flat-out at 60FPS, OnLive is more consistent, if slightly slower than Gaikai. When performance drops, both platforms are impacted, but it's Gaikai that seems to suffer the most.

On the other hand, when looking at Orcs Must Die we get far better latency results and as such a completely different feeling of controller response. With Gaikai, baseline latency isn't exactly wonderful at 183ms, but we found it to be consistent and it was possible to adjust to the gameplay experience. Gaikai annihilated OnLive in terms of response here with a consistent 83ms-100ms advantage - remarkable.

Other tests were equally polarising when it came to seeing how well Gaikai fared when put up against games running on the Xbox 360 and natively on PC. Sometimes we got some incredibly good results: playing Bulletstorm, we found controller response to be slightly slower than the 360 game, but sometimes it hit parity - a truly remarkable achievement. The quickest response time we measured was 133ms (identical to Bulletstorm 360) but it could drift to 150ms, in-line with The Darkness 2 on the 360 or Killzone 2 on PS3 - and we also recorded the odd 166ms measurement too. A matter of 17 or 33ms in additional lag might seem miniscule but you can definitely feel it in the inconsistency of the response.

Overall, I would have to go with Gaikai for this Face-Off. However, both services can't beat the local PC experience.

Steve Perlman's group takes honours for frame-rate, while David Perry's team command clear advantages in game spec, video encoding and overall latency. The signs are there that this can work: Bulletstorm on Gaikai running with the same latency as the Xbox 360 game - however inconsistent it may be - is as clear an indicator as any that cloud could evolve into a viable contender. The technology here is achieving miracles as is, and with advances in infrastructure and improved engineering, it's only going to get better.

TLR: Gaikai is running games at max settings (which are adjustable)and has better latency than OnLive. However, OnLive has a much more stable framerate since they are encoding the video at 60 fps. OnLive has been running certain Playpass games on low settings like Assasins Creed. Also, OnLive has recently been in the habit of launching games at lower graphics settings than increasing them months after launch. I beleive this has to do with server capacity issues. This has happened with Batman Arhkam City and Saints Row The third. However, users can adjust graphics settings in Playpack subscription games.

 

[thehillissilent]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

Obviously, OnLive needs to improve on the graphics/image quality and latency as far as DF article is concerned. Gaikai needs to improve framerate and also latency (not as much as OnLive though).

 

[Gravijah]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

i will be. i'm just waiting for the services to mature and have a wider range of games.

 

[jett]

Man that's a massive difference in video quality between gaikai and onlive. I'll take 30fps in this case.

Well i won't be taking anything because this service isn't available where I live, but still.

 

[U n i o n 0015]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

The concept is great, though I'd be concerned about latency issues. It would have to 100% (all the time) indistinguishable from playing locally for me to jump aboard. My internet is fast at my current place, but the sad fact is that it's likely at the next place it won't. Internet speeds across the U.S. are, by and large, garbage.

And it massively annoys me that OnLive isn't using the highest-spec hardware for games right off the bat. That's one of the huge reasons for using this kind of service and it sounds like they're not even holding up that part of the bargain.

 

[Sophia]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

I'll start using Cloud gaming once they have the latency issues worked out. While Gaikai's been way better than OnLive, both of them are still unacceptable to borderline unplayable where I live.

 

[crazy monkey]

The biggest problem with cloud gaming is not with cloud gaming, It is with internet service providers. GB limits and speeds are issue. Also Owning of the game. If steam goes on cloud it will be a hit.

 

[I H8 Memes]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

Obviously, OnLive needs to improve on the graphics/image quality and latency as far as DF article is concerned. Gaikai needs to improve framerate and also latency (not as much as OnLive though).

Much like Digital Distribution, I would only be interested if it was offered by a privately owned, stable, and well trusted company. Like Valve. I would never commit to having a significant library tied to a publicly owned company.

Also the service would have to provide near zero latency. I wont play a game with mouse lag. The controls need to be so crisp that I can forget there is a mouse in my hand.

 

[thehillissilent]

The biggest problem with cloud gaming is not with cloud gaming, It is with internet service providers. GB limits and speeds are issue. Also Owning of the game. If steam goes on cloud it will be a hit.

I would have to agree that ISPs stand in the way of progress. However, I could see ISPs launching their own cloud gaming portal as well.

Much like Digital Distribution, I would only be interested if it was offered by a privately owned, stable, and well trusted company. Like Valve. I would never commit to having a significant library tied to a publicly owned company.

Also the service would have to provide near zero latency. I wont play a game with mouse lag. The controls need to be so crisp that I can forget there is a mouse in my hand.

Latency issues will always be a sticking point. On my connection, Gaikai definately has less lag than OnLive. I tend to play games on Gaikai using mouse\keyboard and there is a huge difference. I tried playing certain games on OnLive like DNF and it was basically unplayable with mouse\keyboard lag. However, games like F3AR on OnLive seemed native. The main issue is that cloud gaming is so inconsistent due to people's internet connections.

 

[Ardenyal]

The biggest problem with cloud gaming is not with cloud gaming, It is with internet service providers. GB limits and speeds are issue. Also Owning of the game. If steam goes on cloud it will be a hit.

I dunno, even on a gigabit connection the cloud gaming services are a bit iffy. Gaikai was a lot better with regards to latency but I still wouldn't switch to gaming in the cloud at this moment.

 

[thehillissilent]

Well, I hope both OnLive and Gaikai are paying attention to this article. They both need to work on latency. OnLive has a lot more work cut out for them though. Gaikai definately does seem to have the advantage due to multiple datacenters, but both services still need to work in order for mainstream gamers take it seriously. I did find it curious that Gaikai was encoding games at half of the framerate (30 fps) than they run the games in the datacenter (60fps). Digital Foundry was correct about their speculation

Our findings strongly suggest that the service is also rendering its games natively at 60FPS on its servers, but with a much lower frame-rate on the actual video encode. In theory, this introduces slightly more latency, although Gaikai claims that by having its datacentres positioned closer to the player, this additional lag is cancelled out, and the result is overall close to the native 30FPS console experience.

http://www.edge-online.com/news/gaikais-perry-cloud-latency-not-concern

Another solution is rather more surprising. "Imagine your game's running at 30 fps on your console, but we run it at 60 fps [in the data centre], the amount of time that the game took while it was running," he explains. "The engine itself took less time because we're running it faster.

"So we take that time and use it for compression and sending, and you can suddenly see how the math starts to work in our favour. The faster we run the game, the more we overclock it, the lower the feel of the latency. This is one of many, many ways we've discovered to tighten up the feel of it."

As far as image quality is concerned, OnLive really needs to step it up. It was downright embarrassing seeing the difference between Gaikai and OnLive's graphic settings. The fact that Assassins Creed is using low settings on OnLive should not be acceptable. I can speculate why OnLive uses lower graphics settings (60 fps framerate, older gpus, and server capacity issues).

As far as performance is concerned, I have to congratulate OnLive for not only running the game at 60 fps in the datacenter, but encoding the video at 60 fps. Games "feels" smoother on OnLive, although I personally notice faster response time on Gaikai using mouse/keyboard. By Gaikai encoding the game at 30 fps, it can send the frames faster and dramatically increase image quality but you can more easily notice framerate drops. Gaikai seems to compensate the added latency of using a 30fps encode by dramatically increasing the number of datacenters. I do wonder what would happen if Gaikai would encode the video at 60fps. I would hope that OnLive would introduce more datacenters to minimize lag further. These are interesting tradeoffs. I wonder what techniques other competitors will use. I'm especially looking forward to Crytek's GFACE service because the game engine itself will generate h.264 video which could be superior.

The huge gap in frame-rate between both services is clearly visible in Assasin's Creed Brotherhood, where the drops in smoothness are far more noticeable on Gaikai. There's a real sense that the lower frame-rate and visible judder can seriously impact on how fun the game is to play, while on the other hand, OnLive not only looks smoother, but there is also more consistency between what the player is seeing and what they are 'feeling'.

The results are rather interesting. Not only does the amount of latency present in some of the titles running on Gaikai equal that of OnLive, but there are also times where controller response is significantly faster despite the frame-rate being lower due to the 30FPS video encode.

 

[PistolPete]

How much bandwidth does an hour of gaming on these services consume? Is it equivalent to a typical Netflix movie?

This could be a problem for people in markets where bandwidth is capped or metered. Just a thought...

 

[thehillissilent]

How much bandwidth does an hour of gaming on these services consume? Is it equivalent to a typical Netflix movie?

This could be a problem for people in markets where bandwidth is capped or metered. Just a thought...

Gaikai and OnLive use similar amounts of bandwidth although Gaikai can use more bandwidth at times.

Here are the measurements for OnLive:

Based off of OnLive's recommendations (2mbps-5mbps) (Here's the following):
2mbps= 900MB per hour
3mbps= 1.35GB per hour
5mbps= 2.25GB per hour
6mbps= 2.7GB per hour (I'm not sure if they use this much but they could)

Using 250GB (Comcast) cap as an example (not counting any other internet usage for the month):
2mbps= 277 hours max
3mbps= 185 hours max
5mbps= 111 hours max
6mbps= 92 hours max

Here's my personal analysis of Gaikai when I was playing The Darkness 2 demo:

http://www.neogaf.com/forum/showpost.php?p=35190334&postcount=206

I was using Gaikai last night and finally got to track some bandwidth usage in Ubuntu. It was much lower than I initally thought and seems comparable to OnLive. Gaikai uses anywhere from 55KB/s to 1.2MB/s. However, it usually hovers between 250KB/s and 650KB/s with the occasional spike to 800KB/s and rarely over 1.0MB/s.

I was playing The Darkness II demo in 720p in Google Chromium.

If we choose 650kb/s as average, here are the stats:

39MB per minute/2.3GB per hour

It was basically running slightly over 5mbps on average, put it can peak occcasionally at around 10mbps.

I assume for 1080p, 10mbps would probably be average. 1080P and 3D (which they showed on LG 3DTVs) is proabably even higher.

 

[Durante]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

No, I won't use it. The reason isn't one of those you listed (I have an uncapped 50mbps connection), it's simply image quality and especially latency. To give you an idea of my perspective: I play games at 120 Hz on PC, and I am the author of a program intended to reduce input lag on the order of milliseconds. I specifically shop for displays with <16ms input lag. Playing with 150 ms+ of lag sounds terrible.

In terms of image quality, I already get hugely annoyed at console ports that use video cutscenes that actually look worse than the game rendered in realtime on PC. I don't want to imagine getting the same quality (or worse!) throughout all gameplay.

 

[infinityBCRT]

Wow. I'm shocked that DF didn't pick up on this, but OnLive looks like it uses motion smoothing. At least thats how it looked when I tried out ACB. Its 30 FPS gameplay thats being interpolated to 60 FPS.

 

[thehillissilent]

Wow. I'm shocked that DF didn't pick up on this, but OnLive looks like it uses motion smoothing. At least thats how it looked when I tried out ACB. Its 30 FPS gameplay thats being interpolated to 60 FPS.

I wonder if that is true. If it is, OnLive is being really lazy.

 

[Metalmurphy]

I'm surprised the latency was that similar in their tests, for me the delay, or lack of one, was much more noticeable when using Gaikai. Definitely much better then OnLive.

 

[2San]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

Obviously, OnLive needs to improve on the graphics/image quality and latency as far as DF article is concerned. Gaikai needs to improve framerate and also latency (not as much as OnLive though).

I'm not even that serious of a gamer, but the input lag that comes from turning on Vsync already drives me insane. No self respecting gamer is going to play a fighter or a competitive shooter via cloud gaming. We have way too many factors that induce lag even without the Internet involved.

 

[thehillissilent]

I'm surprised the latency was that similar in their tests, for me the delay, or lack of one, was much more noticeable when using Gaikai. Definitely much better then OnLive.

I would have to agree. I was suprised that they were similar when Gaikai seems to have a lot less lag for me personally.

 

[Phife Dawg]

Do you guys think that you will use Cloud gaming in the future? Why or why not? If not what are the reasons: (bandwidth caps and internet speed, latency Lack of physical ownership, lack of exclusives, game catalog).

Obviously, OnLive needs to improve on the graphics/image quality and latency as far as DF article is concerned. Gaikai needs to improve framerate and also latency (not as much as OnLive though).

Tried Gaikai and the experience was pretty much horrible due to latency. If we get a solution for this I'd contemplate a cheap subscription model in the future.

 

[theBishop]

i'm honestly surprised by these results.

 

[thehillissilent]

i'm honestly surprised by these results.

What suprised you?

 

[fizzelopeguss]

So much for "custom silicone".

 

[thehillissilent]

So much for "custom silicone".

Yeah, it's pretty sad:

In order to best demonstrate the difference we've put together a head-to-head video, with the settings in Assassin's Creed Brotherhood on Gaikai being dialled down to match those found in the OnLive version of the game as closely as possible. For the record, we needed to pull down pretty much every setting to the lowest level to match the make-of OnLive's visuals.

OnLive sacrifices graphical detail and use of higher precision effects in order to maintain a 60FPS update as closely as possible while keeping server-side costs down (top). Meanwhile, Gaikai games appear to be running at maximum settings (or close to) on considerably more powerful hardware, thus delivering a level of graphical polish above the level usually found on the Xbox 360 and PlayStation 3.

 

[Perkel]

Dat latency. Unberable for me.

300ms Is unplayable and anything above 130-150 in online just unplayable. Damn even 100ms is high...

 

[thehillissilent]

Dat latency. Unberable for me.

300ms Is unplayable and anything above 130-150 in online just unplayable. Damn even 100ms is high...

Well, you should look at this article for console lag measurements:
http://www.eurogamer.net/articles/digitalfoundry-lag-factor-article

Game Latency Measurement (Consoles)

Burnout Paradise 67ms
BioShock (frame-locked) 133ms
BioShock (unlocked) as low as 67ms
Call of Duty 4: Modern Warfare 67ms-84ms
Call of Duty: World at War 67ms-100ms
Call of Juarez: Bound in Blood 100ms
Forza Motorsport 2 67ms
Geometry Wars 2 67ms
Guitar Hero: Aerosmith 67ms
Grand Theft Auto IV 133ms-200ms
Halo 3 100ms-150ms
Left 4 Dead 100ms-133ms
LEGO Batman 133ms
Mirror's Edge 133ms
Street Fighter IV 67ms
Soul Calibur IV 67ms-84ms
Unreal Tournament 3 100ms-133ms
X-Men Origins: Wolverine 133ms

Gaikai is closest to obtaining console-like lag, with measurements like 133ms from Bulletstorm and 167 from Crysis 2. However, fighting games and competitive fps like COD will suffer from additional lag on cloud gaming services. Most 60FPS games have a 66.67ms latency and games that are 30FPS start at a minumum lag of 100ms.

It seems that, at best, cloud gaming can provide latency on par with 30fps games.

 

[Zaptruder]

No, I won't use it. The reason isn't one of those you listed (I have an uncapped 50mbps connection), it's simply image quality and especially latency. To give you an idea of my perspective: I play games at 120 Hz on PC, and I am the author of a program intended to reduce input lag on the order of milliseconds. I specifically shop for displays with <16ms input lag. Playing with 150 ms+ of lag sounds terrible.

In terms of image quality, I already get hugely annoyed at console ports that use video cutscenes that actually look worse than the game rendered in realtime on PC. I don't want to imagine getting the same quality (or worse!) throughout all gameplay.

Must be a burden to be so hyper-critical of input lags

 

[thehillissilent]

Must be a burden to be so hyper-critical of input lags

LOL! Yeah, cloud gaming isn't for him at all! Right now, the best cloud gaming can do is provide latency which you would normally get with 30fps games. It seems that both OnLive and Gaikai run their games at 60fps (67ms) in the datacenter and add from 67-83ms or more during the compression and tranmission phase.