• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Sony Patents: Improved Facial Animations in Games, Interactive 'What-If' Gameplay System, Minimize Frame Rate Dips

IbizaPocholo

NeoGAFs Kent Brockman

Filed on April 1, 2022, by Sony Interactive Entertainment LLC, the patented system takes mesh data from a human actor captured by a 3D camera to record the depth of the face. This polygonal mesh is used in game development to build accurate human models and is typically handcrafted by animators using the vertices dots on a mocap actor’s face as reference. This method is currently the standard across most triple-A videogame development, including the Insomniac's upcoming Spider-Man 2. This system would automatically generate mesh data, however, using a mix of 3D and 2D cameras to train an Artificial Intelligence program that can accurately capture facial expressions and build mesh data with little need for human intervention.

This patent follows an industry trend in its development of AI to produce work that previously took a great deal of human handiwork. The video game industry has seen a huge rise in the use of AI to do many things, from upscaling the resolution of textures to interpolating frames for smoother games. NVIDIA specifically have patented RTX Remix and their Deep Learning Super Sampling systems that make remastering classic games a much easier feat for modders and developers.

sony-patent-ai-facial-animation-learning-system.jpg


--------------------------------------------------------------------------------------------------------------------------------------------------


The latest in a line of speedy patent submissions, Sony's latest listing refers to a unique "what if" game replay system that would identify and display hypothetical game scenarios that are contextually relevant to the player at a given moment in time. Conceptually similar to Sony's recent player takeover patent, this feature would leverage a special machine-learning algorithm to keep the player engaged by showing them what gameplay outcomes their choices might lead to. This could also prevent frustration at coming to terms with the game's more advanced and complex features, as the player would quickly and easily learn whether they wish to pursue the displayed outcome in the first place.

GINVF0Lf0IbKvcbT_mTmXP4JYdR8yTdjQKSOT_WU0gi5h-HMcMSeMmXkeR5rVXLoHgj0SYpGnuXze2cCu-CnQwNoMok0RkHiQocf-8W-W3tYSm9KCWjfL4075wzemTcyt_ktSmIdUWom1eH9VQLuhEF-6F7nLfwT00TquwKQPVIzzoDOKYWldHB5ePekRzk9.jpg
Uda1nY2zMcD0gUdx9Bj30lVeIvJVMcEhYtJO-I7sln69hBz6cSEcB-Mb49RRQ7zBIvGAtiTrwPyPgmb0iedfvjEOI5C20KNNt5EaxZX40NPEoAN165UPdzV4ndP6CoCehh1-F2XFBn4GzkO9LXRh8mxZZH9jGlKbGfEJXSZhqa9vM9pCoxNxCKleMXbtpq33-(1).jpg


----------------------------------------------------------------------------------------------------------------------------------------------------


According to a new patent filing from Sony, PlayStation 5's VRR support could be due for an upgrade sometime down the line. One of the company's latest patent applications describes a special frame rate compensation system that leverages scanouts to smooth out any potential frame dips in taxing gameplay sections. The process includes scanning out the GPU's rendered frames and compensating for fluctuations from the desired target frame rate. And since the described process uses frame buffers to accomplish this job, it's not entirely dissimilar to Nvidia's proprietary G-Sync monitor technology, for example.

In some instances, as the patent describes, the entire frame scanout and compensation process could even be offloaded to a dedicated coprocessor of some sort, which could tie in with Sony's recent multi-GPU hardware patent. Of course, the two listings describe different use cases and situations, but it's unlikely that Sony's engineers aren't thinking ahead and wondering if these technological solutions could work hand-in-hand.

1Fn18lsm1Gp0mOl5wvCjVrFmEWMSn4KBI3OmKJiiCDACGXYoVu6DmNHXovjCBvhXr2NOhFlkLXLZaWlb2L7TaKbG2l4i1epcD4PkNZkmA25JqKy09kyFQReWYLXOkluoErBT7U9HUFNwB1b3IUFumkbuOYG8lLUGa4WDN62tk9N4O0zWnZvHF_BXVjssxFrY.jpg
 

Fafalada

Fafracer forever
What's actually odd is how these web-sites 'reporting' always try to cover their tracks completely. No links, no title of the patent, no authors - but hey - here's a single screenshot from the document and a bunch of nonsense we interpreted about it.

Almost like they don't want anyone actually verifying their article claims by - I dunno - reading the damn patent.
 

CamHostage

Member
(Patent leaks) also hints for the next PlayStation consoles.Advances In gaming is always good.

Patents, eh? Sort of, some times? Not usually.

Sometimes you can find a patent which really does explore some wild new concept (even sometimes for hardware or software that you never actually get; Sony has patents for brand new portable game systems and other technology which it has not acted upon and shown any movement towards in years,) but 99% of the time, nope. And neither of these three feel like 1% cases...

The first two of these patents, for example, are super vague, incredibly basic ideas with no specific tech outlined in them (or at least not in the cited text, though the illustrations would have had more citations of technical implementation if they had actual tech powering the patents.) They've patented the idea of facial mocap with depth data and also branching story paths... huh, did Sony just invent Until Dawn again? The third patent, with midframe reinterpretation, is vague in how they actually plan on accomplishing frame replication (which it would be in a patent, but also the article has no actual quotes or breakdown of info, nor links, so it's just the one image showing the remixing of front and back frame buffer data) but it's nowhere as complex in description as the Optical Multi Frame Generation system of DLSS3's use of motion vector data to have AI interpret upcoming frames before the actual hardware can render one. None of these patents say that Sony has actual new tech to get excited for, as far as I can read from what's provided.

The reason why these patents are so vague isn't because they're protecting special secrets to be unveiled in upcoming games and they don't want the patent to give the good stuff away. It's because these are just patents, and most patents are vague and vapid by design. In fact, a huge percentage of these patents by tech companies are not being made in advance of major new technology coming down the pike. (Those are product registrations, which are different from patents and are not as publicly gettable; patents tend to be very general and mechanical, so a lot of video game tech is tough to patent unless they can physicalize it, or at least illustrate it in an active product. The PS5 has some published patents, for instance, but they are primarily blueprints and mainly address the airflow and shell attachment system.) Most of these patents are instead pre-squatting on ideas that cybersquatters have been taking advantage of to hijack major companies over the years. So like the infamous rumble lawsuits or the podcast lawsuit, industries never want to be endangered for basic ideas like these, so they're filing every basic idea (even if the idea is already applicable to products on the market) and taking hold of what they can. There's sort of a cold war truce in the business that these massive companies are stockpiling patents so that the 'bad guys' can't use them against everybody and that they won't actually use them (that was EA's story on a recent big multiplayer patent, and Sony and MS and others have said similar things,) and so far, so good. But if the company really believes it invented and owns the idea, they can and will go to war (Apple had a ton of patents on the original iPhone, and when it went after Samsung for ripping off iPhone, which really should have been against Google but whatever, those patents were part of the legal battle.)

You can read a lot of Sony's patents (as you can with any other company) and be bored off your ass by all the legal paperwork corporations need to protect basic, often commonplace ideas and designs.
https://patents.justia.com/assignee/sony-interactive-entertainment-inc

(Also, they're not "patent leaks". They're patents. If you apply for a patent, part of the point of the patent is so that inventors out there can know that you have taken ownership of this idea and design. They are by nature public. You just have to find them.)

GameRant is reading a bunch of whatever into this (including mentioning Spider-Man 2 for no reason, because it's a game which has characters which have faces?) so that they have a story, but unless there's actually more in these patents than what's described and shown in the figures of illustration, most likely there is no story here.

What's actually odd is how these web-sites 'reporting' always try to cover their tracks completely. No links, no title of the patent, no authors - but hey - here's a single screenshot from the document and a bunch of nonsense we interpreted about it.

Almost like they don't want anyone actually verifying their article claims by - I dunno - reading the damn patent.

Yup.
 
Last edited:
Facial animations really need work, very few games actually depict the fact that when you speak your whole face and often your body can and often does move. Nobody is still and only the lower face moves for an entire conversation.
 

Robb

Gold Member
The ‘what if’ thing sounds like it could be a fun mechanic. Although I’d guess these will fade into oblivion as most patents do.
 
Last edited:

Paperboy

Member
The ‘what if’ thing sounds like it could be a fun mechanic. Although I’d guess these will fade into oblivion as most patents do.

I have a hard time to see the use for it. Wouldn't you want to go back and take the alternative path yourself? It feels like this feature would potentially obliterate the replay value in story based games.
 
Top Bottom