- Number
- 10483004
- Published
- 2019-11-19
- Filed
- 2016-09-29
- Assignee
- DISNEY ENTERPRISES, INC.
- Inventors
- Wu; Chenglei, Bradley; Derek, Beeler; Thabo, Gross; Markus
- CPC
- A61C13/0004; G16H70/60; G16H50/50; G16H70/20
- Verdict
- Low Notable software
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Model-based teeth reconstruction (3D face/digital-double modeling).
Abstract
A system and method for non-invasive reconstruction of an entire object-specific or person-specific teeth row from just a set of photographs of the mouth region of an object (e.g., an animal) or a person (e.g., an actor or a patient) are provided. A teeth statistic model defining individual teeth in a teeth row can be developed. The teeth statistical model can jointly describe shape and pose variations per tooth, and as well as placement of the individual teeth in the teeth row. In some embodiments, the teeth statistic model can be trained using teeth information from 3D scan data of different sample subjects. The 3D scan data can be used to establish a database of teeth of various shapes and poses. Geometry information regarding the individual teeth can be extracted from the 3D scan data. The teeth statistic model can be trained using the geometry information regarding the individual teeth.
Background
BACKGROUND OF THE INVENTION(1) Digital models have become widely used in many fields, from digital animation to remote medical diagnosis. Over the past decades, both research and industry have made tremendous progress when it comes to the creation of digital faces, mostly focusing on appearance, shape, and deformation of skin. More recently, some work has emerged which also targets other facial features, such as the eyes and facial hair. However, capturing the mouth cavity in general, and teeth in particular, has only received very little attention so far. Teeth contribute substantially to the appearance of a face, and expressions may convey a very different intent if teeth are not explicitly modeled. Furthermore, teeth can be an invaluable cue for rigid head pose estimation and are essential for physical simulation, where they serve as a collision boundary. In addition, in medical dentistry, digital teeth models have long become a central asset, since they allow to virtually plan a patient's dental procedure.(2) Many efforts to capture teeth have stemmed from the medical dental field. While plaster-cast imprints are widely in the past in dental clinics, more and more intra-oral scanners are making their way into the clinics. While these devices can capture the shape of the teeth at high quality, the capture procedure is very invasive and the devices themselves can be costly and may not be readily available. Moreover, although methods using these devices can yield high-qualit
Claims
1. A method for reconstructing a three-dimensional (3D) model of teeth of an object from one or more images of the object, the method being performed by a computer system and comprising: receiving, by the computer system, sample data of teeth rows of different sample subjects; receiving, by the computer system, tooth templates comprising individual template teeth; mapping, by the computer system, individual teeth in the teeth rows of the different sample subjects to the individual template teeth; for each of the individual template teeth, obtaining, by the computer system, a teeth statistical model, the statistical model encoding a deviation in shape and pose for each of the individual template teeth, wherein the teeth statistical model includes information regarding a rigid transformation and an anisotropic scaling factor for the teeth rows, the anisotropic scaling factor comprising different properties depending on a direction of measurement for the teeth rows; training, by the computer system, each teeth statistical model using the sample data of the teeth rows of the different sample subjects based on the mapping between the individual teeth in the teeth rows of the different sample subjects and the individual template teeth, wherein the training comprises: evaluating the teeth statistics model to provide an instance of a teeth row that represents either an upper or a lower teeth for a sample subject; acquiring, by the computer system, teeth information from the one or more images of the object; and reconstructing, by the computer system, a 3D model of teeth of the object by fitting parameters of the trained teeth statistical model to the teeth information acquired from the one or more images of the object.
2. The method of claim 1, wherein the teeth statistical model encodes an average shape of the tooth, an average pose of the tooth, deviation in shape and pose for the tooth across the different sample subjects, and placement of the tooth in a generic teeth row.
3. The method of claim 1, wherein the received sample data of the teeth rows of the different sample subjects are 3D scans of a mouth region of the different sample subjects.
4. The method of claim 1, wherein training the teeth statistical model comprises performing a principal component analysis to determine a variation in shape for each tooth template.
5. The method of claim 1, wherein mapping individual teeth in the teeth rows of the different sample subject to the individual template teeth includes receiving an indication of a segmentation contour for the individual teeth, the segmentation contour indicating separation of each of the individual teeth and separation of the teeth and a gum region.
6. The method of claim 5, wherein the indication further includes marks representing one or more land marks on the individual template teeth.
7. The method of claim 5, further comprising computing segmentation contour for each tooth based on the received indication.
8. The method of claim 1, wherein a teeth statistical model is obtained for each type of teeth in the teeth row.
9. A system for reconstructing a three-dimensional (3D) model of teeth of an object from one or more images of the object, wherein the system comprises one or more processors configured to execute machine-readable instructions such that when the machine-readable instructions are executed by the one or more processors, the one or more processors are caused to perform: receiving sample data of teeth rows of different sample subjects; receiving tooth templates comprising individual template teeth; mapping individual teeth in the teeth rows of the different sample subject to the individual template teeth; for each of the individual template teeth, obtaining a teeth statistical model, the statistical model encoding a deviation in shape and pose for each of the individual template teeth, wherein the teeth statistical model includes information regarding a rigid transformation and an anisotropic scaling factor for the teeth rows, the anisotropic scaling factor comprising different properties depending on a direction of measurement for the teeth rows; training each teeth statistical model using the sample data of the teeth rows of the different sample subjects based on the mapping between the individual teeth in the teeth rows of the different sample subjects and the individual template teeth, wherein the training comprises: evaluating the teeth statistics model to provide an instance of a teeth row that represents either upper or lower teeth for a sample subject; acquiring teeth information from the one or more images of the object; and reconstructing a 3D model of teeth of the object by fitting parameters of the trained statistical model to the teeth information acquired from the one or more images of the object.
10. The system of claim 9, wherein the teeth statistical model encodes an average shape of the tooth, an average pose of the tooth, deviation in shape and pose for the tooth across the subjects, and placement of the tooth in a generic teeth row.
11. The system of claim 9, wherein the received sample data of the teeth rows of the different sample subjects are 3D scans of a mouth region of the different sample subjects.
12. The system of claim 9, wherein training the teeth statistical model comprises performing a principal component analysis to determine a variation in shape for each tooth template.
13. The system of claim 9, wherein mapping individual teeth in the teeth rows of the different sample subject to the individual template teeth includes receiving an indication of a segmentation contour for the individual teeth, the segmentation contour indicating separation of each of the individual teeth and separation of the teeth and a gum region.
14. The system of claim 13, wherein the indication further includes marks representing one or more land marks on the individual template teeth.
15. The system of claim 13, further comprising computing segmentation contour for each tooth based on the received indication.
16. The system of claim 9, wherein a teeth statistical model is obtained for each type of teeth in the teeth row.