SYSTEMS AND METHODS FOR SPEECH ANIMATION USING VISEMES WITH PHONETIC BOUNDARY CONTEXT

Number: 20170154457
Published: 2017-06-01
Filed: 2015-12-01
Assignee: Disney Enterprises, Inc.
Inventors: Theobald; Barry-John, Meyerhofer; Margaret, Matthews; Iain, Taylor; Sarah
CPC: G10L15/187; G10L21/10; G06T13/80; G06T13/205; G06T13/40
Verdict: Medium Notable software
Source: Google Patents · FreePatentsOnline

The keeper's note

Speech-driven facial animation using visemes with phonetic boundary context to realistically simulate lip movement matching speech.

Abstract

Speech animation may be performed using visemes with phonetic boundary context. A viseme unit may comprise an animation that simulates lipmovement of an animated entity. Individual ones of the viseme units may correspond to oneor more complete phonemes and phoneme context of the one or more complete phonemes. Phoneme context may include a phoneme that is adjacent to the one or more complete phonemes that correspond to a given viseme unit. Potential sets of viseme units that correspond with individual phoneme string portions may be determined. One of the potential sets of viseme units may be selected for individual ones of the phoneme string portions based on a fit metric that conveys a match between individual ones of the potential sets and the corresponding phoneme string portion.

Background

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system configured for speech animation using visemes with phonetic boundary context, in accordance with one or more implementations.

FIG. 2 illustrates an exemplary implementation of a server of the system of FIG. 1.

FIG. 3 illustrates an exemplary phonemestring portion of phoneme string.

FIG. 4 illustrates an exemplary implementation ofa set of viseme units.

FIG. 5 illustrates another exemplary implementation of a setof viseme units.

FIG. 6 illustrates a visual representation of a fit metric used toselect one of a plurality of potential sets of viseme units that correspond to a given phoneme sequence, in accordance with one or more implementations.

FIG. 7 illustrates amethod of speech animation using visemes with phonetic boundary context, in accordance with one or more implementations.DETAILED DESCRIPTION

FIG. 1 illustrates a system 100 configured for speech animation using visemes with phonetic boundary context, in accordance with one or more implementations. Visemes, or, more specifically, individual viseme units, may include concatenative units used for generating speech animation. A viseme unit maycorrespond to one or more of one or more complete phonemes; one or more partial phonemes that span a beginning, middle, and/or end of a given phoneme; one or more complete phonemes and phoneme context of the one or more complete phonemes; and/or other information. A vis

Claims

1. A system for speech animation using visemes with phonetic boundary context, the system comprising: one or more physical processors configured by machine-readable instructions to: obtain phoneme strings, the obtained phoneme strings including a first phoneme string, the first phoneme string including a first phoneme string portion; determine potential sets of viseme units that correspond with the first phoneme string portion, a viseme unit comprising an animation that simulates lip movement of an animated entity, individual ones of the viseme units corresponding to one or both of oneor more complete phonemes or one or more phoneme context of one or more complete phonemes, a given phoneme context including a partial phoneme that spans the beginning, middle, orend of a complete phoneme, wherein the individual ones of the potential sets of viseme units that correspond to the first phoneme string portion form different viseme strings thatdefine different animations of lip movement corresponding to the first phoneme string portion, the potential sets of viseme units including a first potential set and a second potential set; and select one of the potential sets of viseme units based on a fit metric thatconveys a match between individual ones of the potential sets and the first phoneme string portion. 14. A method speech animation using visemes with phonetic boundary context, themethod being implemented in a computer system including one or more physical processors and storage media storing machine-readable instructions, the method comprising: obtaining phoneme strings, including obtaining a first phoneme string, the first phoneme string including a first phoneme string portion; determining potential sets of viseme units that correspond with the first phoneme string portion, a viseme unit comprising an animation that simulates lip movement of an animated entity, individual ones of the viseme units corresponding to one or both of one or more complete phonemes or one or more phoneme context of one or more complete phonemes, a given phoneme context including a partial phoneme that spans the beginning, middle, or end of a complete phoneme, wherein the individual ones of the potential sets of viseme units that correspond to the first phoneme string portion form different viseme strings that define different animations of lip movement corresponding to thefirst phoneme string portion, including determining a first potential set and a second potential set; and selecting one of the potential sets of viseme units based on a fit metricthat conveys a match between individual ones of the potential sets and the first phoneme string portion.