Fake frames: An uncomfortable truth about DLSS 5

[erek]

“In summary:​

• PCWorld examines a ComputerBase study revealing that gamers often prefer DLSS 4.5’s AI-generated frames over native rendering in blind tests, despite online criticism of “fake” frames.
• This finding suggests a disconnect between gamers’ expressed opinions and actual visual preferences, with DLSS 4.5 receiving over 50% of votes in three out of six tested games.
• The results support Nvidia CEO Jensen Huang’s vision that neural rendering represents the future of game graphics technology.

I looked at the “blind” game comparisons, as well as earlier side-by-side comparisons with labeling. For me, the DLSS 4.5 scenes sometimes look over-rendered. Take Assassin’s Creed Shadows as an example—all of the grass looks more defined with DLSS 4.5, yes. But the human eye doesn’t focus on everything in view. It concentrates on a certain portion, and whatever is not in that area starts to lose detail. In games, I prefer a hierarchy of visuals, so I’m willing to sacrifice a bit of crispness to approximate that. And that’s how the natively rendered version gets my vote.

But what if I didn’t have to choose native rendering for that desired outcome? What if I could instead expect upscaling to simulate both what I want (accuracy of focus) andwhat others want (accuracy of detail)? I wonder if we’d care as much about traditional raster vs. AI upscaling performance. (At least, when it comes to visuals.)“

Source: https://www.pcworld.com/article/3072853/fake-frames-may-be-better-than-you-think.html

 

[LukeTbk]

One way to have an idea of gamers expressed opinions would be what have been winning the last 20 years, upscaling has been the way to go for most games-gamers all this time.

80%+ of RTX owners do use DLSS, there is not necessarily an disconnect between expressed opinion (putting it to on being the ultimate expression of it), people that express negative opinion of it could very well not like it in a blind tests as well, it is not like the tech was not well liked by the vast majority of gamers and doing well in blind test for something to be explained.

It does well in blind test and it is used by almost everyone when available (and been used by console gamers since HD tv became populars).

 

[Prince Valiant]

“In summary:​

• PCWorld examines a ComputerBase study revealing that gamers often prefer DLSS 4.5’s AI-generated frames over native rendering in blind tests, despite online criticism of “fake” frames.
• This finding suggests a disconnect between gamers’ expressed opinions and actual visual preferences, with DLSS 4.5 receiving over 50% of votes in three out of six tested games.
• The results support Nvidia CEO Jensen Huang’s vision that neural rendering represents the future of game graphics technology.

I looked at the “blind” game comparisons, as well as earlier side-by-side comparisons with labeling. For me, the DLSS 4.5 scenes sometimes look over-rendered. Take Assassin’s Creed Shadows as an example—all of the grass looks more defined with DLSS 4.5, yes. But the human eye doesn’t focus on everything in view. It concentrates on a certain portion, and whatever is not in that area starts to lose detail. In games, I prefer a hierarchy of visuals, so I’m willing to sacrifice a bit of crispness to approximate that. And that’s how the natively rendered version gets my vote.

But what if I didn’t have to choose native rendering for that desired outcome? What if I could instead expect upscaling to simulate both what I want (accuracy of focus) andwhat others want (accuracy of detail)? I wonder if we’d care as much about traditional raster vs. AI upscaling performance. (At least, when it comes to visuals.)“

Source: https://www.pcworld.com/article/3072853/fake-frames-may-be-better-than-you-think.html

I don't follow their reasoning here. There's no need to approximate in a game engine what your eyes do in the first place. As far as the upscaling and frame generation goes, as long as it's not mandatory I don't care what Jensen peddles.

 

[Twisted Kidney]

Frame rate is every bit as important as detail in gaming.

 

[DukenukemX]

Frame rate is every bit as important as detail in gaming.

Only if the frames are real. The whole point of a higher frame rate is to reduce input lag. Fake frames make real input lag.

 

[Twisted Kidney]

Only if the frames are real. The whole point of a higher frame rate is to reduce input lag. Fake frames make real input lag.

I'm not really talking about input lag, I just mean a stable frame rate. You're right though, input lag from phony frames is a whole separate, very smelly, kettle of fish.

 

[Meeho]

1. Did they just watch the monitors or actually played the games?

2. Public generally has shit taste in picture quality. Overprocessed, oversharpened, oversaturated, dynamic contrast visual garbage will always win in votes as "looks better".

 

[GotNoRice]

IMO the biggest downside of "fake frames" has always been latency. I'm wondering how this would be revealed in a "blind test". Are these people actually playing the game or just looking at it?

 

[sfsuphysics]

So out of curiosity, if you're running a game that doesn't stress your card in the least, and you run it at a moderate 60fps or something. Will it still introduce fake frames into that?

 

[GotNoRice]

So out of curiosity, if you're running a game that doesn't stress your card in the least, and you run it at a moderate 60fps or something. Will it still introduce fake frames into that?

What is your monitor refresh rate? Are you using GSync?

 

[kac77]

I think this really depends on how sensitive you are to the technology. I'm sensitive to the up scaling (DLSS and FSR) and frame generation. The ghosting makes my eyes twitch kind of like the scene in the Matrix with the cat. Add frame generation onto that and its worse cause now there's lag among other things. If I can play without either technology I prefer it and I only turn it on if I have to.

 

[Meeho]

IMO the biggest downside of "fake frames" has always been latency. I'm wondering how this would be revealed in a "blind test". Are these people actually playing the game or just looking at it?

I don't think this was about Frame Generation FPS multipliers out of thin air, just DLSS upscaling, but I could be mistaken.

 

[GoldenTiger]

Only if the frames are real. The whole point of a higher frame rate is to reduce input lag. Fake frames make real input lag.

It's also to improve animation and motion detail.

 

[GotNoRice]

I don't think this was about Frame Generation FPS multipliers out of thin air, just DLSS upscaling, but I could be mistaken.

I can see how the arguments about the two could become intermingled, but a frame isn't generally considered to be "fake" just because it was upscaled. A frame is "fake" because it was generated from other frames.

PCWorld examines a ComputerBase study revealing that gamers often prefer DLSS 4.5’s AI-generated frames over native rendering in blind tests, despite online criticism of “fake” frames.

 

[GoldenTiger]

Isn't this just a poorly done analysis of the computer base blind dlss upscaling video test that's been making the rounds for several days now?

 

[Zarathustra[H]]

I think passively viewed side by side blind tests are irrelevant for frame generation.

There is no doubt that AI can generate decent looking frames. It may have some interacting on occasion, but they can look good.

The issue is the input lag, and that you are never going to feel unless you are the one holding the mouse and playing the game.

At least to me, frame rate has always had much less to do with visual smoothness than it does with reducing input lag.

Frame generation only helps with visual smoothness. It does nothing to improve input lag (and can actually even make it slightly worse)

In order to get acceptable input lag you need minimum frame rates (however you define them. 0.1% minimums?) of at least 60fps. While frame generation can feel just fine If you enable it on top of already having 60fps minimums, if you are not achieving 60fps minimums without it, and enable frame generation to try to boost your frame rates to an acceptable level, the game will still feel like utter shit.

In other words, Frame Generation is a technology that only helps when you don't need it.

 

[sharknice]

If you've ever worked on game engines you understand that even the frames rendered by the game engine itself are "fake". Game engines filled with approximations and tricks to render things at playable frame rates. It is more than possible for AI generated frames and AI upscaling to not only make better looking images but more accurately rendered images.

The problem is you can do all the fake frames you want, but you still need a good base framerate to have good input lag.

IMO the future of frame gen is the game engine making a Quake 3 looking image at an extremely high frame rate and AI image generation making it look photo realistic or stylized in whatever way the developer chooses. The game engine could even feed the AI generation with double the frames it is outputting.

 

[Cactusj]

I think the part getting missed is that its not just DLSS vs Native, its DLSS vs Native with massive TAA blur. TAA is a scourge on the gaming industry and is a result of lazy game development. DLSS looks as good as it does, because it removes TAA before its own processing.

 

[WilyKit]

I think the part getting missed is that its not just DLSS vs Native, its DLSS vs Native with massive TAA blur. TAA is a scourge on the gaming industry and is a result of lazy game development. DLSS looks as good as it does, because it removes TAA before its own processing.

Native rendering + DLAA ftw

 

[AVATARAT]

I prefer good native (or DLAA) + only fake frames IF they are implemented correctly, and Nvidia does not perform well in this aspect.
DLSS is something I dislike because its implementations are rarely good.

The main problem here is that Nvidia and Sony (UE5) are working against us as a team.

 

[LukeTbk]

IMO the future of frame gen is the game engine making a Quake 3 looking image at an extremely high frame rate and AI image generation making it look photo realistic or stylized in whatever way the developer chooses. The game engine could even feed the AI generation with double the frames it is outputting.

for game that want photo-realistic results with huge training data easily available we can see this happening soon, they could also probably make an pixar type rendering engine that not use in realtime-not shipped with the game, only used to train a model for stylised/non real data to train on look and do something similar, stick model for the characther with others data and so on.

Say an photo realistic NBA game, could maybe even be stick skeleton, motion vector, torque on joints and list of non we usually see in the usual rendering input that the rendering generator would receive a bit like this:

View: https://youtu.be/0WkixvqnPXw?t=84

Would be trained on how individual player gate and others particularly moving mechanics they have

 

[LukeTbk]

I think passively viewed side by side blind tests are irrelevant for frame generation.

There is no doubt that AI can generate decent looking frames. It may have some interacting on occasion, but they can look good.

The issue is the input lag, and that you are never going to feel unless you are the one holding the mouse and playing the game.

by fake frame here they mean regular dlss upscaling which do not change latency like the current added interpolating frames tech.

In other words, Frame Generation is a technology that only helps when you don't need it.

that not what blindtest (with people playing on a computer, the only way to test that tech) shows, that were motion smoothness of the tech help the most has well, lot of people saying 35fps->60 fps using frame generation feel terrible, also think that playing at 35fps feel terrible, if they would test between the 2, many still prefer the added motion clarity.

the question is not just about 35->60 being a great experience, it is also is it better than playing at 35 fps... for many it is

 

[sharknice]

for game that want photo-realistic results with huge training data easily available we can see this happening soon, they could also probably make an pixar type rendering engine that not use in realtime-not shipped with the game, only used to train a model for stylised/non real data to train on look and do something similar, stick model for the characther with others data and so on.

Say an photo realistic NBA game, could maybe even be stick skeleton, motion vector, torque on joints and list of non we usually see in the usual rendering input that the rendering generator would receive a bit like this:

View: https://youtu.be/0WkixvqnPXw?t=84

Would be trained on how individual player gate and others particularly moving mechanics they have

Yeah something like that. There will be specifically made AI rendering engines and models. If you provide vector data like games with proper DLSS implementations it will work a lot better. I would be surprsied if NVIDIA doesn't already have something in the works. There are already companies trying to skip the whole real engine thing and make games purely AI video generated but I think it's going to be slop for quite a while and having some sort of game engine will be the way it's done first.

 

[sleepeeg3]

Back in my day, I was happy with 40fps+ with AA felt pretty smooth.

Now in many games today, you see 120+fps using native processing, before DLSS?
Unless you're playing an FPS - why bother? Even then, there is added input lag?

1. Did they just watch the monitors or actually played the games?

2. Public generally has shit taste in picture quality. Overprocessed, oversharpened, oversaturated, dynamic contrast visual garbage will always win in votes as "looks better".

That's kind of what I'm thinking, without having any experience.
There was that recent Cyberpunk video of it running on an Android phone where it looked noticeably worse, compared to the native renderer. That could just be because it was running at low fps.

 

[LukeTbk]

Unless you're playing an FPS - why bother? Even then, there is added input lag?

with regular DLSS usually no, because it boost FPS so it tend to reduce it, with frame generation you add latency with the current way to do it, the future reflex 2 frame warp could be an in between lower latency than with no frame generation for camera movement but higher for other things.

There was that recent Cyberpunk video of it running on an Android phone where it looked noticeably worse, compared to the native renderer. That could just be because it was running at low fps.

Android phone does not tend to have nvidia gpu to run game with DLSS

 

[WilyKit]

Back in my day, I was happy with 40fps+ with AA felt pretty smooth.

Now in many games today, you see 120+fps using native processing, before DLSS?
Unless you're playing an FPS - why bother? Even then, there is added input lag?

That's kind of what I'm thinking, without having any experience.
There was that recent Cyberpunk video of it running on an Android phone where it looked noticeably worse, compared to the native renderer. That could just be because it was running at low fps.

Evolution. Back then, the human eye couldn't see past 60fps

 

[Sycraft]

Now in many games today, you see 120+fps using native processing, before DLSS?

I mean, depends on the game but there's a whole lot out there that no, you can't get those kinds of frame rates. There are heavy hitters like Cyberpunk or Alan Wake 2, of course, but even plenty of lesser games. Satisfactory only gets about 90-110fps on my system with DLSS with the graphics turned up. That's on a 4090 as well.

That's the other thing to remember: Most people don't have a big honkin' GPU. While the 90 series are a lot of fun, most people can't afford them. If you are playing on a 60 series, your FPS will be more limited.

Unless you're playing an FPS - why bother? Even then, there is added input lag?

Because it is smoother. While each doubling in FPS matters less than the one before it, it is still nice. The difference between 100ish and 200ish FPS isn't major, but you can see the extra fluidity. So, if there's a way to get it, why not?

Frame gen doesn't work well in all games, but when it does work it is a nice bonus to smoothness. Particularly if you have 240Hz+ monitor, which are getting more common these days. The chances you can actually get those speeds native are pretty low, hell the CPU often is the limiting factor, but frame gen will let you take advantage of more of that smoothness.

 

[LukeTbk]

Evolution. Back then, the human eye couldn't see past 60fps

25-35 fps the first time you played with an 3d accelerator at 800x600 (or even 1024x768) felt smooth because we were used to play 15-18fps on the cpu (at 480p even was the top of the line when 1997 started pentium 200MMX would struggle to reach 20fps in quake 2 back in the days), benchmark in the late 90s would say stuff like this:
Now here the discussion comes in. What is acceptable? Many Q2 players will answer '40-50+ fps is what it takes'. This is of course incorrect. A sustained frame rate of 25-30 fps would actually do,
https://www.tomshardware.com/reviews/nvidia,87-6.html

 

[Zarathustra[H]]

by fake frame here they mean regular dlss upscaling which do not change latency like the current added interpolating frames tech.


that not what blindtest (with people playing on a computer, the only way to test that tech) shows, that were motion smoothness of the tech help the most has well, lot of people saying 35fps->60 fps using frame generation feel terrible, also think that playing at 35fps feel terrible, if they would test between the 2, many still prefer the added motion clarity.

the question is not just about 35->60 being a great experience, it is also is it better than playing at 35 fps... for many it is

I would argue that 35fps native, and 60fps with DLSS based off of 35FPS native are equally unplayable.

It's only once you get to native framerates where your 0.1% minimums exceed 60fps when things start becoming truly playable, and if you have those framerates already, there really is no need for DLSS. Sure you can turn it on in order to max out your monitor, but it's not as if it makes some sort of transformative difference.

 

[LukeTbk]

I would argue that 35fps native, and 60fps with DLSS based off of 35FPS native are equally unplayable.

you are here obviously talking for yourself (which is fine, but a bit of a different conversation), a lot of the world has been playing at 30fps for a large part of their life (proving without a doubt that it is playable, as they did enjoy games at those pace, I do remember having some fun playing quake / quake 2 at much lower fps than that or Gran turismo 2 with my friends at 20 something fps )

as for being equally unplayable that would require some blind test to try it (i never even ever tryed nvidia frame gen, so would not comment on it myself), but blind test did show that it was more playable to many (and being 100% subjective experience, that something that really possible to argue one way or an other, outside blind test subjective experience, that only thing we can do and rely on to judge this).

 

[Zarathustra[H]]

you are here obviously talking for yourself (which is fine, but a bit of a different conversation), a lot of the world has been playing at 30fps for a large part of their life (proving without a doubt that it is playable, as they did)

On consoles maybe?

I tend to find that if you use controllers - which are inherently less precise control devices than a mouse - input lag is less noticeable, and as such less of a problem. So while I have a difficult time picturing most people being OK with 30fps play in a first person title using their mouse, other types of games I suppose could be acceptable at those lower frame rates.

Personally - however - I won't even bother playing a title if I cant get minimum frame rates above 60 fps. It's just not a pleasurable experience.

Don't get me wrong. I'm not some 165hz+ frame rate snob. I tend to think super high frame rates are somewhat ridiculous, but I have been doing this for a long time now, and have noticed that there is a pretty huge difference between 30fps and 60fps on the input lag side of things, and a much much smaller difference between - say - 60fps and 120fps. As the framerates increase that difference becomes smaller and smaller as you have more and more diminishing returns. I don't know if there is ever really a cap where higher frame rate stops to offer a benefit, but there is definitely a point past where "the juice isn't worth the squeeze". For me that happens in the 90-120fps range somewhere.

60fps - however (and I am talking minimums, not 60fps average) is where the input lag stops being so bad that it bothers me. Having slightly higher framerate is nice, but not a must for me. But if the minimums are under 60fps, I'd rather just put the title in my gaming backlog and revisit it at a later time when I have faster hardware.

And this isn't just a talking point for me. I live it.

I have been a fan of the S.T.A.L.K.E.R series ever since it first came out, and when S.T.A.L.K.E.R. 2 launched in late 2024 I had been looking forward to a sequel for 15 years. This was the "it" title for me that I was excited about and looking forward to. When I got it - however - it was immediately obvious that my hardware (Threadripper 3960x + RTX 4090) was not sufficient for me to meet my performance expectations, so I set it aside, and embarked on a system upgrade (which wound up being a huge project and still isn't done)

as for being equally unplayable that would require some blind test to try it (i never even ever tryed nvidia frame gen, so would not comment on it myself), but blind test did show that it was more playable to many (and being 100% subjective experience, that something that really possible to argue one way or an other, outside blind test subjective experience, that only thing we can do and rely on to judge this).

Do I have blinded A/B testing at my disposal? No. But I have some personal practical experience.

The comparison that really does it for me is the original Half Life. Back in 1998 on my Pentium 150 (Pre-MMX) overclocked to 200Mhz, with a 6MB Voodoo 1 board that game was hard. You had to time jumps or fall to your death, and at 30 or so FPS on that hardware, this was very difficult.

Going back to the title years later on hardware that could handle 60+fps I was struck by how easy it seemed, likely in part due to the input lag making the timing so much easier. Even on the hard setting, I was able to play through the title in a few hours, whereas the first time around it took me a couple of months.

Now, I don't necessarily want my hardware to make gaming easy. I like a good challenge out of my games. But I want the challenge to be because the title actually is difficult, not because my hardware can't keep up.

 

[Zarathustra[H]]

for game that want photo-realistic results with huge training data easily available we can see this happening soon, they could also probably make an pixar type rendering engine that not use in realtime-not shipped with the game, only used to train a model for stylised/non real data to train on look and do something similar, stick model for the characther with others data and so on.

Say an photo realistic NBA game, could maybe even be stick skeleton, motion vector, torque on joints and list of non we usually see in the usual rendering input that the rendering generator would receive a bit like this:

View: https://youtu.be/0WkixvqnPXw?t=84

Would be trained on how individual player gate and others particularly moving mechanics they have

I'm not necessarily opposed to a complete AI rendering pathway. It could result in some interesting results.

I think the hybrid state we are in - however - where we are dependent on some traditionally rendered frames (lets call them keyframes, to borrow a video compression term) which are the only frames to display response to input data (mouse/keyboard/whatever) has some inherent drawbacks when it comes to input lag.

Nvidias "Reflex 2" tech is supposed to help here, but I haven't had an opportunity to test it yet, so I can't comment on it from an educated position. In theory it should help with input lag as it "warps" each generated frame based on control inputs, but I have my doubts about how well it actually works when "surprises" jump on screen tyhat were previously off-screen in the previous keyframe, so the model can't anticipate them and "warp" them in. There is also the fact that title support still is somewhat limited.

If an AI render pathway can completely do away with the keyframes and continuously monitor input sources and reflect it on the screen in each frame, input lag could actually be lower than traditional rendering. As long as it was good, I'd be all for that.

 

[THRESHIN]

Back in my day, I was happy with 40fps+ with AA felt pretty smooth.

Now in many games today, you see 120+fps using native processing, before DLSS?
Unless you're playing an FPS - why bother? Even then, there is added input lag?

That's kind of what I'm thinking, without having any experience.
There was that recent Cyberpunk video of it running on an Android phone where it looked noticeably worse, compared to the native renderer. That could just be because it was running at low fps.

I too remember thinking 40-45 fps was plenty fast. Provided there's no stuttering (or almost none) I still kinda think it is despite having a higher refresh rate monitor.

 

[Verado]

the only problem I have with DLSS/FG is that game developers are obviously getting lazy with optimization because of all this "free horsepower"

 

[M76]

80%+ of RTX owners do use DLSS, there is not necessarily an disconnect between expressed opinion (putting it to on being the ultimate expression of it),

Because there is no other choice but to use it. If your only options are to die of thirst or drink the pisswasser, you drink the pisswasser. This does not mean we like the pisswasser.

It does well in blind test and it is used by almost everyone when available (and been used by console gamers since HD tv became populars).

It wins on blind tests because it smears the shit out of the image obfuscating graphical artifacts caused by other bad practices. I assume all such blind tests where DLSS wins are carried out in a way that DLSS off has no proper anti-aliasing enabled.

 

[LukeTbk]

Because there is no other choice but to use it.

How many game force DLSS on ?

 

[sharknice]

the only problem I have with DLSS/FG is that game developers are obviously getting lazy with optimization because of all this "free horsepower"

Developers aren't getting lazy with optimization. They always have been lazy with optimization.

And I wouldn't call it lazy, it's just priorities. Very few games actually prioritize optimization and that has always been true.

 

[M76]

How many game force DLSS on ?

In most games it is on by default, but that's not what I mean. You have to use it to get playable FPS.

 

[LukeTbk]

On consoles maybe?

yes 30 fps lived much longer on consoles (and still a thing, GTA 6 will that mode and tens of millions will find it playable), controller and sitting distance/screen also could help, you watched 24fps the other day in the same setup.

But would we play a full year with 1% low at 45 fps on a monitor that has good VRR at that frame rate, we would probably find it acceptable again like we would have in the 90s/earlys 00s, it is being used to much higher that make trying at lower feel strange at first, that would be my guess.


That what more the tech goals here, very fast monitor of course (240hz-4k for example) but also for people that do play games in 30hz-40hz mode, having a better subjective experience with the frame generation on vs off, pundit that try and test it just tend to be people that would never play games at those framerate to start with, which of coruse it is irrelevant for them, but hundreds of millions of gamers in the world still play at 30-40 fps.

Nvidias "Reflex 2" tech is supposed to help here, but I haven't had an opportunity to test it yet, so I can't comment on it from an educated position. In theory it should help with input lag as it "warps" each generated frame based on control inputs, but I have my doubts about how well it actually works when "surprises" jump on screen tyhat were previously off-screen in the previous keyframe, so the model can't anticipate them and "warp" them in. There is also the fact that title support still is somewhat limited.

if you try it you will see why it is not out yet, under very high fps it has issues at least at the state the dlls leaked were back then (and why they are going for the under 15 ms high fps crowd for this):


View: https://www.youtube.com/watch?v=tENebDPWgO4
it was on github (or maybe someone google drive finding it, https://www.patreon.com/PureDark), i really do not have the I mind latency to judge the result when I did try it (same for previous attempt in the past of similar tech), at high fps it work well enough, at 30 issh you see artifacts.

One way that could make it work better would be to render a bit of an higher resolution than what we see so when you move it has something to build the estimated image, but obviously you rapidly come in the may has have higher native frame rate instead

 

[Sycraft]

And I wouldn't call it lazy, it's just priorities. Very few games actually prioritize optimization and that has always been true.

Also a lot of "optimization" is "making things worse in the least-worst way possible". It isn't a case of just shake a magic tree and frames fall out, often it means cutting things down. Now that's fine... but maybe not everyone wants that and things like DLSS and frame gen mean that not as much of it need to be done. One of the nice things about PC gaming is we often get to choose. You can choose to turn down graphics settings, that is optimizing the game to run with lower hardware. However we have other options now as well.

A game that just flat out doesn't run well on anything, that's where you can say "This is poorly optimized," because ya, maybe they did NEED to cut some things out, or rework how it was done. If it stutters and hitches even on top spec hardware, even with settings turned down, there's a problem. However if you take something like Alan Wake 2, which just hits really hard when you crank you the settings but also looks stellar, that is just a game using new technology. Yes, it means you need DLSS to get a good frame rate, it is too heavy for 4k native. So what? that's not a lack of optimization, that is a choice to make a really good looking game, which you can also turn down the detail on if you do demand 4k native at higher FPS.