AUTOMATED MACHINE LEARNING TAGGING AND OPTIMIZATION OF REVIEW PROCEDURES

Number: 20210192385
Published: 2021-06-24
Filed: 2019-12-20
Assignee: Disney Enterprises, Inc.
Inventors: FARRÉ GUIU; Miquel Angel, ALFARO VENDRELL; Monica, JUNYENT MARTIN; Marc, ACCARDO; Anthony M.
CPC: G06N3/0464; G06N20/00; G06V10/7784; G06N3/09; G06V20/46; G06V20/41; G06F18/2185
Verdict: Set aside content tagging automation, business
Source: Google Patents · FreePatentsOnline

Abstract

Techniques for machine learning optimization are provided. A video comprising a plurality of segments is received, and a first segment of the plurality of segments is processed with a machine learning (ML) model to generate a plurality of tags, where each of the plurality of tags indicates presence of an element in the first segment. A respective accuracy value is determined for each respective tag of the plurality of tags, where the respective accuracy value is based at least in part on a maturity score for the ML model. The first segment is classified as accurate, based on determining that an aggregate accuracy of tags corresponding to the first segment exceeds a predefined threshold. Upon classifying the first segment as accurate, the first segment is bypassed during a review process.

Background

BACKGROUND

The present disclosure relates to machine learning, and more specifically, to using machine learning to optimize a tagging process and reduce manual review.

Machine learning (ML) algorithms can be used to identify different types of elements in media files at high levels of accuracy. However, to provide high levels of accuracy, the algorithms must be trained based on a training dataset. Preparing an accurate and complete training dataset to train the ML algorithms is difficult due to the amount of data needed, as well as the need to keep the dataset updated (e.g., cleaning the dataset, correcting errors in the dataset, adding more data, and the like). Additionally, existing systems cannot provide transparency to ensure accuracy, nor can they facilitate or expedite review. SUMMARY

According to one embodiment of the present disclosure, a method is provided. The method includes receiving a video comprising a plurality of segments, and processing a first segment of the plurality of segments with a machine learning (ML) model to generate a plurality of tags, wherein each of the plurality of tags indicates presence of an element in the first segment. The method further includes determining, for each respective tag of the plurality of tags, a respective accuracy value, wherein the respective accuracy value is based at least in part on a maturity score for the ML model. Additionally, the method includes classifying the first segment as accurate, based

Claims

1. A method, comprising: receiving a video comprising a plurality of segments; processing a first segment of the plurality of segments with a machine learning (ML) model to generate a plurality of tags, wherein each of the plurality of tags indicates presence of an element in the first segment; determining, for each respective tag of the plurality of tags, a respective accuracy value, wherein the respective accuracy value is based at least in part on a maturity score for the ML model; classifying the first segment as accurate, based on determining that an aggregate accuracy of tags corresponding to the first segment exceeds a predefined threshold; and upon classifying the first segment as accurate, bypassing the first segment during a review process. || 9. A non-transitory computer-readable medium containing computer program code that, when executed by operation of one or more computer processors, performs an operation comprising: receiving a video comprising a plurality of segments; processing a first segment of the plurality of segments with a machine learning (ML) model to generate a plurality of tags, wherein each of the plurality of tags indicates presence of an element in the first segment; determining, for each respective tag of the plurality of tags, a respective accuracy value, wherein the respective accuracy value is based at least in part on a maturity score for the ML model; classifying the first segment as accurate, based on determining that an aggregate accuracy of tags corresponding to the first segment exceeds a predefined threshold; and upon classifying the first segment as accurate, bypassing the first segment during a review process. || 15. A system, comprising: one or more computer processors; and a memory containing a program which when executed by the one or more computer processors performs an operation, the operation comprising: receiving a video comprising a plurality of segments; processing a first segment of the plurality of segments with a machine learning (ML) model to generate a plurality of tags, wherein each of the plurality of tags indicates presence of an element in the first segment; determining, for each respective tag of the plurality of tags, a respective accuracy value, wherein the respective accuracy value is based at least in part on a maturity score for the ML model; classifying the first segment as accurate, based on determining that an aggregate accuracy of tags corresponding to the first segment exceeds a predefined threshold; and upon classifying the first segment as accurate, bypassing the first segment during a review process.