IbizaPocholo
NeoGAFs Kent Brockman
Vid2Player3D
We present a system that learns diverse, physically simulated tennis skills from large-scale demonstrations of tennis play harvested from broadcast videos. Our approach is built upon hierarchical models, combining a low-level imitation policy and a high-level motion planning policy to steer the...
research.nvidia.com
- A system that learns diverse, physically simulated tennis skills from large-scale demonstrations of tennis play harvested from broadcast videos is presented.
- The approach combines a low-level imitation policy and a high-level motion planning policy to steer the character in a motion embedding learned from broadcast videos.
- The system can learn complex tennis shotmaking skills and realistically chain together multiple shots into extended rallies, using only simple rewards and without explicit annotations of stroke types.
- The estimated motion from broadcast videos is corrected with physics-based imitation, and a hybrid control policy is used to override erroneous aspects of the learned motion embedding with corrections predicted by the high-level policy.
- The system can produce controllers for physically-simulated tennis players that can hit the incoming ball to target positions accurately using a diverse array of strokes, spins, and playing styles.
- Two physically simulated characters playing extended tennis rallies with simulated racket and ball dynamics can be synthesized by the system.
The paper presents a system that enables physically simulated characters to learn complex tennis skills from broadcast videos. The system uses a combination of kinematic motion estimation, low-level imitation policy training, correction of motion artifacts, and high-level motion planning policy training to generate natural and precise tennis motions. The resulting simulated characters can hit incoming tennis balls with diverse and complex skills, and adopt different playing styles by learning from different player motion data. The system’s hybrid control structure and motion embedding help smooth emotions and mitigate jittering artifacts, while the correction of wrist motion allows for precise control of the tennis racket. The system can also generate novel animations of tennis rallies between different players.