Systems and methods for incremental natural language understanding

Number: 11195533
Published: 2021-12-07
Filed: 2020-03-25
Assignee: Disney Enterprises, Inc.
Inventors: Kumar; Komath Naveen, Kennedy; James R., Lombardo; Salvator D., Gurunath Shivakumar; Prashanth
CPC: G06F3/167; G06V40/20; G06F3/013; G10L15/22; G06V20/40; G10L17/00
Verdict: Set aside generic NLP research, no clear creative hook
Source: Google Patents · FreePatentsOnline

Abstract

A system for incremental natural language understanding includes a media module, a memory storing a software code, and a hardware processor communicatively coupled to the media module. The hardware processor is configured to execute the software code to receive an audio stream including a first utterance, and generate a first and second incremental speech recognition outputs based on first and second portions of the first utterance. In addition, the hardware processor is configured to execute the software code to determine, prior to generating the second incremental speech recognition output, a first intent of the first utterance based on the first incremental speech recognition output. The hardware processor is further configured to execute the software code to retrieve a first resource based on the determined first intent, and incorporate the first resource in the media content to be played by the media module.

Background

BACKGROUND (1) Spoken Language Understanding (SLU) typically comprises of an automatic speech recognition (ASR) followed by a natural language understanding (NLU) module. The two modules process signals in a blocking sequential fashion, i.e., the NLU often has to wait for the ASR to finish processing on an utterance. In a real-time application scenario, the ASR receives a stream of continuous speech signals and outputs corresponding transcriptions. Due to the computational complexity and memory constraints, most ASRs typically operate by chunking and processing the speech in segments. This process is often referred to as end-pointing, and is usually determined based on different heuristics related to duration of inter-pausal units (IPUs), with the goal to minimize disruption during speech. Finally, the ASR outputs a transcript corresponding to each speech segment. As a result, any NLU application operating on the output of the ASR needs to wait at least until end-pointing, which gives rise to a fundamental bottleneck in latency, and potentially renders the spoken interaction less naturally. SUMMARY (2) There are provided systems and methods for incremental natural language understanding, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.

Claims

1. A system comprising: a media module configured to play a video content; a memory storing a software code; a hardware processor communicatively coupled to the media module, and configured to execute the software code to: receive an audio stream including a first utterance; generate a first incremental speech recognition output based on a first portion of the first utterance; generate a second incremental speech recognition output based on a second portion of the first utterance; determine, prior to generating the second incremental speech recognition output, a first intent of the first utterance based on the first incremental speech recognition output; retrieve a first resource from a resource database based on the determined first intent, wherein the resource database includes at least one of a voice over audio, a video or a text; and incorporate the first resource in the video content to be played by the media module. || 11. A method for use by a system including a media module configured to play a video content, a memory storing a software code, and a hardware processor communicatively coupled to the media module, the method comprising: receiving, using the hardware processor, an audio stream including first utterance; generating, using the hardware processor, a first incremental speech recognition output based on a first portion of the first utterance; generating, using the hardware processor, a second incremental speech recognition output based on a second portion of the first utterance; determining, prior to generating the second incremental speech recognition output and using the hardware processor, a first intent of the first utterance based on the first incremental speech recognition output; retrieving, using the hardware processor, a first resource from a resource database based on the determined first intent, wherein the resource database includes at least one of a voice over audio, a video or a text; and incorporating, using the hardware processor, the first resource in the video content to be played by the media module.