HL2 + Johnny Lee head tracking effect + faceAPI = OMG

G

GeneralIroh

Member
Check out this awesome demonstration

Headtracking has been around sometime now. We've seen the eyetoy for the Playstation 2, and soon Project Natal. What about that old web camera you have? It isn't about the camera itself, just the software that drives it, even this can make Project Natal feel dumb.


http://www.moddb.com/mods/city-17-episode-1/news/city-17-episode-1-3-day-update-head-tracking

http://www.youtube.com/watch?v=qWkpdtFZoBE&feature=player_embedded

Its all about the software and a standard web camera. Who needs the over priced gear to experience 4d.

http://www.seeingmachines.com/product/faceapi/
 
I can't wait until software similar to this makes it into full retail games. Just the ability to have the camera rotate by me moving my head would be mindblowingly awesome. I'd love to see it in games like STALKER and Battlefield.
 
Yeah it would be great because you wont have to wear anything goofy and it would take the load off the controls so things aren't so complicated.

I hope episode 3 is designed with this in mind from the get go. I hope they improve all the gun models and first person body parts so everything looks natural. L4d already has the full first person model thing going on where you can see your feet.

Imagine counterstrike with the lean feature. :D
 
this tech is impressive... combine this with 3D capability tv's that DON'T require glasses and what you have is a complete virtual reality experience in retail games... this industry is about to explode
 
Karma Kramer said:
this tech is impressive... combine this with 3D capability tv's that DON'T require glasses and what you have is a complete virtual reality experience in retail games... this industry is about to explode
Click to expand...

No.

It will be awesome, but it isn't VR.
 
Peeking around corners looked really cool and natural.

I'm not sure if I'd like to control a game entirely through head-tracking, but a "hybrid" game (traditional controls with headtracking for a few things like the cover system) would be welcome.
 
I wanted to see Half Life with the 3D effect that gives the technique, but it's all about gesture control. It can be good but i feel frustrated !
 
* Enter your email below and click on the '>'
* Open the email you receive and download the file from the link provided
* Unzip the file
* Shut down Steam
* Run 'full_setup.exe'
* If the Face API installer does not run automatically, run 'faceapi_setup.exe'
* If the IPP Runtime installer does not run automatically, run 'IPP5.3.3/setup.exe'
* Connect your web camera and place it under your monitor, facing you
* Open Steam, click on 'My Games' and run 'Head Tracking'
* For the best performance, sit about half a meter away from your web camera
Click to expand...
Faaaarkkkk I need a desktop. Stupid laptop.
For webcam, all I got is the 360 Vision Cam I got for free with my Live Gold. Guesing that should work if I can get the placement right? I could always go VGA to TV I guess.
 
This should be offered as an option in 360 and PS3 shooters, since both systems have cameras. It doesn't even need something like Natal.
 
Suavo said:
Natal+controller use for shooters (especially games like rainbow six) would be awesome.
Click to expand...
I agree, but I also thing that it should be an option not forced. I like the idea and I will use it but sometimes I won't be in the mood.
 
Suavo said:
Natal+controller use for shooters (especially games like rainbow six) would be awesome.
Click to expand...
This is basically why I'm interested in Natal. Not as a standalone product, but as a supplement.
 
And people that are all over excited about this kind of shit need to do themselves a favor and consciously keep track of their body language when they play shooters, especially if you're playing for long stretches of time.

Then realize that for as "neat" as this stuff seems, it's practical application in traditional shooters is pretty much worthless.
 
GeneralIroh said:
Headtracking has been around sometime now. We've seen the eyetoy for the Playstation 2, and soon Project Natal. What about that old web camera you have? It isn't about the camera itself, just the software that drives it, even this can make Project Natal feel dumb.
Click to expand...
It's good, but Natal can do this and will likely be better. Not that that's a shock - the system is *designed* to track Human motions.
 
* Enter your email below and click on the '>'
* Open the email you receive and download the file from the link provided
* Unzip the file
* Shut down Steam
* Run 'full_setup.exe'
* If the Face API installer does not run automatically, run 'faceapi_setup.exe'
* If the IPP Runtime installer does not run automatically, run 'IPP5.3.3/setup.exe'
* Connect your web camera and place it under your monitor, facing you
* Open Steam, click on 'My Games' and run 'Head Tracking'
* For the best performance, sit about half a meter away from your web camera
Click to expand...

Where is this coming from? I don't see a download link? I'd love to give this a shot.
 
Wow. If it can be done this well on a regular cam imagine what Natal will do with better software :D

I want to try this out. Gonna see if I can get it going.

(EDIT 1) Meh, only the API is available.
(EDIT 2) Botolf to the rescue!
 
acm2000 said:
all the things shown in that video in the OP are just as bad as adding waggle to wii games for the sake of waggle
Click to expand...
wtf

Brandon F said:
So now head waggle is the new future?
Click to expand...
It's been the future since I saw that video by Johnny Lee on YouTube in December 2007.
 
Pretty cool stuff, and though I agree with this being a nice concept for console shooters I'm not seeing how Natal would improve it. You only need a normal webcamera for this to work right, so what extra would Natal bring to it that the 360camera or PS3eye wouldn't be able to?
 
Lord Phol said:
You only need a normal webcamera for this to work right, so what extra would Natal bring to it that the 360camera or PS3eye wouldn't be able to?
Click to expand...
Oh, but that's not how the console manufacturers see stuff like this.

$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$.
 
Lord Phol said:
Pretty cool stuff, and though I agree with this being a nice concept for console shooters I'm not seeing how Natal would improve it. You only need a normal webcamera for this to work right, so what extra would Natal bring to it that the 360camera or PS3eye wouldn't be able to?
Click to expand...
Natal is 3D, these are 2D. What does that mean? I have no clue ^_^
 
soultron said:
Oh, but that's not how the console manufacturers see stuff like this.

$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$.
Click to expand...

well natal does 3d scanning, could potentially put yourself in a game in 3d, actually, imagine that, with MK9... :lol id love to rip my own head off
 
This could be good in a cover TPS. Really you shouldn't be able to see over cover when in it, but lifting ur head slightly to peak could make an interesting dynamic with the right camera movement
 
acm2000 said:
well natal does 3d scanning, could potentially put yourself in a game in 3d, actually, imagine that, with MK9... :lol id love to rip my own head off
Click to expand...

I can't believe that people actually buy that argument. It does not. So many people have too high expectations, they are all going to be disappointed.
 
Suavo said:
Natal+controller use for shooters (especially games like rainbow six) would be awesome.
Click to expand...

Sounds like sony wands will be awesome then.. ;)

It is indeed amazing how out of the loop people have been on cam tech. It's like everything's new again.
 
The only one worth much beyond a gimmick IMO is the perspective parallax effect that is commonly credited to Johnny Lee.

As Mickeyknox says; you're going to get tired and fidget, and when you're doing that, you probably don't want your character doing all kinds of crazy stuff. The only one where continued headtracking is acceptable is the perspective change effect; properly calibrated, the effect is as natural as looking through a window. Your head might move around, but the perspective change is intuitive and not frustrating, especially because it's not a huge change in the playing circumstances (i.e. you don't find yourself zooming in and out and flipping around if you suddenly look down at your mobile phone, or find yourself slumping in your chair).

I always find it amusing that people think that true 3D is achieved via one sort of technology... stereoscopic vision is but one aspect of how humans percieve 3D spaces. There's also all the things that conventional 3D games on 2D displays are already doing... but the other part that they've been missing out on is the parallax perspective shift, that this sort of head tracking enables. It's probably an even more convincing form of 3D than stereoscopic vision. But there can be no doubt that once combined with the up and coming stereoscopic tech that this will help us achieve the best possible 3D effect that can be achieved on a 2D display.

Even if it doesn't really change gameplay all that much, it certainly enhances the immersiveness of the 3D game environment, moreso than higher frames per second, or higher resolution, or better shaders could hope to do at this point.
 
Stink said:
http://www.naturalpoint.com/trackir/products/trackir5/
Click to expand...
I own a TrackIR unit. Image recognition software like faceAPI and the software supporting Natal is going to make TrackIR obsolete. Why use the infrared spectrum and wearable reflectors when you can use the visible spectrum and not have to wear anything?

Though it makes me wonder why no one has coded these types of gestures and behavior already using the TrackIR API. faceAPI and TrackIR already have the same amount of head tracking data available to the programmer: pitch, yaw, and roll. Though, the gesture recognition capabilities of faceAPI blew my mind. I'm not sure if that's gonna be as relevant as head tracking though.

EDIT: There are numerous TrackIR enabled games with head tracking features out on the PC already. Head tracking is pretty cool and definitely vital in games like ARMA II and various flight simulators. However, TrackIR is freaking expensive. For those who want to try out head tracking in PC games, there are other alternatives. One is FreeTrack which uses a regular webcam and IR reflectors to make a "poor man's" TrackIR.

But with faceAPI, you just need a webcam. Here's a video of this guy playing a TrackIR enabled game by coding faceAPI into PPJoy which in turn emulates the TrackIR drivers.

http://www.youtube.com/watch?v=eNE9FfFMeh0

He put the source code in the description. I might have some fun tinkering with it in Visual Studio. :D

If you tilt your head or move back, the screen stays in the same position. Not immersive or natural at all. It's weird and I'm pretty sure good old keyboard / controller buttons would feel more natural.
Click to expand...
You have peripheral vision and you have eyeballs that can change direction. This isn't a problem at all when I play PC games with head tracking enabled.
 
Only interesting with VR / LCD glasses.

If you tilt your head or move back, the screen stays in the same position. Not immersive or natural at all. It's weird and I'm pretty sure good old keyboard / controller buttons would feel more natural.
 
i like the idea, but it needs a lot of refinement.

all that head movement could get really tiresome and tedious when playing for 2-3 hours at a time.

i can really only see it as ancillary to gameplay, rather than the core gameplay mechanic.
 
Zaptruder said:
The only one worth much beyond a gimmick IMO is the perspective parallax effect that is commonly credited to Johnny Lee.

As Mickeyknox says; you're going to get tired and fidget, and when you're doing that, you probably don't want your character doing all kinds of crazy stuff. The only one where continued headtracking is acceptable is the perspective change effect; properly calibrated, the effect is as natural as looking through a window. Your head might move around, but the perspective change is intuitive and not frustrating, especially because it's not a huge change in the playing circumstances (i.e. you don't find yourself zooming in and out and flipping around if you suddenly look down at your mobile phone, or find yourself slumping in your chair).

I always find it amusing that people think that true 3D is achieved via one sort of technology... stereoscopic vision is but one aspect of how humans percieve 3D spaces. There's also all the things that conventional 3D games on 2D displays are already doing... but the other part that they've been missing out on is the parallax perspective shift, that this sort of head tracking enables. It's probably an even more convincing form of 3D than stereoscopic vision. But there can be no doubt that once combined with the up and coming stereoscopic tech that this will help us achieve the best possible 3D effect that can be achieved on a 2D display.

Even if it doesn't really change gameplay all that much, it certainly enhances the immersiveness of the 3D game environment, moreso than higher frames per second, or higher resolution, or better shaders could hope to do at this point.
Click to expand...

QFT
 
KevinCow said:
This should be offered as an option in 360 and PS3 shooters, since both systems have cameras. It doesn't even need something like Natal.
Click to expand...

I would think something like this would only work reasonably well in front of a monitor because then your head would be relatively close to the camera. If you set this out infront of your HDTV while you sit on the couch I doubt it would be able to pick you up. Let alone discern precise head movements from the surrounding environment. Natal is absolutely vital for something like this to be seamless and work in all types of environments.
 
No offense Gaffers, but you're doing a lot of talking without actually playing. This little gizmo deserves a little bit of your time, it's pretty cool.

--

Not ideal lighting conditions here at the moment (2 AM, face is mostly masked in shadow to the webcam). Some of the gestures work well enough in these conditions, so I'll comment on those. I'll give it a fuller testing tomorrow.

Iron-sights - I really like the implantation on this gesture. Tilt your head to the right to go into ironsights mode, which will point your gun dead ahead and line the sights up. Something feels right about aiming/firing a weapon with this gesture. Especially when you're carrying a shotgun.

Zoom - This one is easy enough to do, move your face closer/farther. Works fine with the HL2 zoom feature.

Cover lean - Leaning left to right will lean out in the respective direction. Appeared to work fine.

Quickspin - Does as advertised, spins you around in a heartbeat if you jerk around. Also works in the seat of the buggy, allowing you to see more than you could normally.

--

Half-Life 2 maps - I dropped a HL2 bsp into the head tracking mod's map folder, and loaded it up fine. The gestures and tracking seem to work perfectly fine. Tested the zoom, the ironsights, the quickspin, and the cover lean here.

What would really complete this would be sensitivity settings, so gamers could tune this to their exact preferences. As it is, though, it's a fantastic piece of work, and I really wouldn't mind playing through some Source games with it on.
 
You must log in or register to reply here.

Similar threads

Mortal Kombat 2 movie coming soon. Karl Urban as Johnny Cage.
2
75
4K
lmimmfn replied
Phil Spencer removed Head of Xbox from X profile, now Head of MS Gaming
2 3
105
7K
iHaunter replied
The Mandela effect in videogames
Community  2
86
5K
gtabro replied
Does Switch 2 have hall effect joysticks or not?
30
1K
KungFucius replied
Audeze's Maxwell gets Atmos head tracking rendering.
1
1K
S0ULZB0URNE replied
Top Bottom