Motion Tracking and Image Recognition of Hand Gestures to Animate a Digital Puppet, Synchronized with Recorded Audio

Number: 20210279935
Published: 2021-09-09
Filed: 2021-05-24
Assignee: DISNEY ENTERPRISES, INC.
Inventors: Murdock; Malcolm, Hy; Michael, Mkrtchyan; Armen
CPC: G06T13/40; G06F3/011; G06F3/0304
Verdict: Low Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Hand-gesture motion tracking driving digital puppet/character animation (pre-grant).

Abstract

There is provided a system and method for creating a digital puppet show by animating a digital puppet using gestures and sound. The method includes presenting the digital puppet to a user on a display. The method further includes receiving motion data corresponding to a gesture, using a camera, from the user, translating the motion data into digital data using a motion tracking algorithm, and animating the digital puppet, on the display, using the digital data. The method can further include receiving audio data from the user using a microphone and playing the audio data, using a speaker, while animating the digital puppet on the display to create the digital puppet show.

Background

BACKGROUND

Real life puppet shows are a popular source of entertainment for children and can be performed by placing a sock over a person's hand to impersonate a character. Puppet shows give children the ability to dress socks up to look like their favorite characters, and using their fingers and thumb, children are able to move the mouth of the sock character while giving a narrative in order to create the puppet show. However, this form of entertainment has started to become less popular for children, who are now looking to the digital world for new forms of entertainment.

Interactive digital storybook applications for phones and computers are also very popular with children and allow children to create puppet shows using their favorite characters. Children are able to create and record puppet shows by moving 2-D cutouts of their favorite characters around a scene, while at the same time narrating the characters to give the characters a voice. When a child is finished recording the puppet show, the child can playback the puppet show to entertain friends and family. This gives children the experience of creating their very own animated story. However, these interactive digital storybook applications can be very limited by their simplicity in that they only allow children to move 2-D characters around a scene while narrating the puppet show. As such, any gestures made by the children, like sock puppets in the physical space, have no impact on the animation of th

Claims

21: A computer-readable non-transitory medium having stored thereon a digital puppet application, which when executed by a hardware processor, performs a method comprising: presenting a three-dimensional (3-D) digital puppet to a user on a display; receiving motion data corresponding to a gesture made by the user; receiving audio data corresponding to voice of the user for use in a 3-D digital puppet show; translating the motion data into digital data using a motion tracking algorithm; and creating the 3-D digital puppet show using the 3-D digital puppet by: animating the 3-D digital puppet, on the display, using the digital data; and playing the audio data at the same time as the animating of the 3-D digital puppet on the display, wherein playing the audio data includes synchronizing the audio data with the animating of the 3-D digital puppet. || 29: A method comprising: presenting a three-dimensional (3-D) digital puppet to a user on a display; receiving motion data corresponding to a gesture made by the user; receiving audio data corresponding to voice of the user for use in a 3-D digital puppet show; translating the motion data into digital data using a motion tracking algorithm; and creating the 3-D digital puppet show using the 3-D digital puppet by: animating the 3-D digital puppet, on the display, using the digital data; and playing the audio data at the same time as the animating of the 3-D digital puppet on the display, wherein playing the audio data includes synchronizing the audio data with the animating of the 3-D digital puppet. || 37: A system for animating a three-dimensional (3-D) digital puppet, the system comprising: a display; and a processor configured to: present a three-dimensional (3-D) digital puppet to a user on a display; receive motion data corresponding to a gesture made by the user, receive audio data corresponding to voice of the user for use in a 3-D digital puppet show; translate the motion data into digital data using a motion tracking algorithm; and create the 3-D digital puppet show using the 3-D digital puppet by: animating the 3-D digital puppet, on the display, using the digital data; and playing the audio data at the same time as the animating of the 3-D digital puppet on the display, wherein playing the audio data includes synchronizing the audio data with the animating of the 3-D digital puppet.