Systems and methods for augmenting an appearance of a hilt to simulate a bladed weapon

Number: 9846971
Published: 2017-12-19
Filed: 2016-01-19
Assignee: Disney Enterprises, Inc.
Inventors: Goslin; Michael P., Ryan; Matthew, Olson; Joseph Logan
CPC: G06T11/00; G06F3/012; G06F3/017; G06F3/016; G06T19/006; G06F3/0346; G06F3/0325
Verdict: High Hardware
Source: Google Patents · FreePatentsOnline

The keeper's note

Augments the appearance of a handheld hilt to simulate a bladed weapon (lightsaber-style) by detecting the hilts landmark, tracking its position/orientation, and overlaying a virtual blade image accordingly (granted).

Abstract

This disclosure relates to systems and methods for augmenting an appearance of a hilt to simulate a bladed weapon. A hilt may be augmented with a blade of a bladed weapon by detecting a landmark associated with the hilt, determining a position and/or an orientation of the hilt, determining an overlay image comprising the blade of the bladed weapon, wherein the blade is placed within the overlay image according to the determined position and/or the determined orientation of the hilt, and displaying the overlay image so that the blade of the bladed weapon appears to be attached to the hilt.

Background

BRIEF DESCRIPTION OF THE DRAWINGS(1) FIG. 1 illustrates a system configured to augment an appearance of a hilt to simulate a bladed weapon.(2) FIG. 2 illustrates a method to augment an appearance of a hilt to simulate a bladed weapon.(3) FIGS.3A-3G illustrate examples of landmarks on housings of hilts.(4) FIGS. 4A-4E illustrate examples of blades of bladed weapons augmenting the appearances of hilts.(5) FIG. 5 illustrates an example of a visual effect based on two blades of bladed weapons touching each other.(6) FIGS. 6A-6C illustrate an example of a visual effect based on a blade of a bladed weapon touching a virtual blaster fire.(7) FIGS. 7A-7B illustrate an example of a visual effect based on a blade of a bladed weapon touching a door.DETAILED DESCRIPTION(8) FIG. 1 illustrates a system 10 configured to augment an appearance of a hilt to simulate a bladed weapon. System 10 may include one or more of processor 11, display 12, image sensor 13, first motion and orientation sensor 14, second motion and orientation sensor 15, electronic storage 16, bus 17, a housing of the hilt, and/or other components. To augment an appearanceof the hilt, a landmark may be detected, and the position and/or the orientation of the landmark may be determined. The landmark may indicate a reference point for the hilt that facilitates determination of the position and/or the orientation of the housing of the hilt. The position and/or the orientation of the housing of the hilt may be determined. A b

Claims

1. A system for augmenting an appearance of a hilt to simulate a bladed weapon, the system comprising: a display configuredto display an overlay image; an image sensor configured to generate visual output signalsconveying visual information within a field of view of the image sensor; a housing of thehilt; a landmark carried by the housing, the landmark indicating a reference point for the hilt that facilitates determination of a position and orientation of the housing; a first inertial measurement unit carried by the display, the first inertial measurement unit separate from the image sensor, the first inertial measurement unit configured to generate first motion and/or orientation output signals conveying motion and/or orientation information of the display such that the first motion and/or orientation output signals are separate and independent from the visual output signals, wherein the first motion and/or orientation output signals facilitate determination of the position and orientation of the housing when the landmark is not within the field of view of the image sensor; a second inertialmeasurement unit carried by the housing, the second inertial measurement unit separate from the image sensor, the second inertial measurement unit configured to generate second motion and/or orientation output signals conveying motion and/or orientation information of the housing such that the second motion and/or orientation output signals are separate andindependent from the visual output signals, wherein the second motion and/or orientation output signals facilitate determination of the position and orientation of the housing when the landmark is not within the field of view of the image sensor; and one or more processors configured by machine readable instructions to: when the landmark is within the fieldof view of the image sensor: detect the landmark based on the visual output signals; determine a position and/or an orientation of the landmark; and determine a position and/or anorientation of the housing based on the determined position and/or the determined orientation of the landmark; when the landmark is not within the field of view of the image sensor, determine the position and/or the orientation of the housing with respect to the display based on the first motion and/or orientation output signals and the second motion and/ororientation output signals; determine the overlay image comprising a blade of the bladed weapon, wherein the blade is placed within the overlay image according to the determined position and/or the determined orientation of the housing such that the blade is placed within the overlay image based on the visual output signals when the landmark is within the field of view of the image sensor and the blade is placed within the overlay image based onthe first motion and/or orientation output signals and the second motion and/or orientation output signals when the landmark is not within the field of view of the image sensor; and effectuate displaying of the overlay image on the display so that the blade appears to be attached to the hilt. 12. A method for augmenting an appearance of a hilt to simulate abladed weapon, the method comprising: generating visual output signals conveying visual information within a field of view of an image sensor; generating first motion and/or orientation output signals conveying motion and/or orientation information of a display, the first motion and/or orientation output signals generated by a first inertial measurement unit carried by the display, the first inertial measurement unit separate from the image sensor, wherein the first motion and/or orientation output signals are separate and independent from the visual output signals and the first motion and/or orientation output signals facilitate determination of the position and orientation of the housing when the landmark isnot within the field of view of the image sensor; generating second motion and/or orientation output signals conveying motion and/or orientation information of a housing of the hilt, the second motion and/or orientation output signals generated by a second inertial measurement unit carried by the housing, the second inertial measurement unit separate from the image sensor, wherein the second motion and/or orientation output signals are separate and independent from the visual output signals and the second motion and/or orientation output signals facilitate determination of the position and orientation of the housing when the landmark is not within the field of view of the image sensor; when a landmark is within the field of view of the image sensor: detecting the landmark based on the visual outputsignals, wherein the landmark is carried by the housing and indicates a reference point for the hilt that facilitates determination of a position and orientation of the housing; determining a position and/or an orientation of the landmark; and determining a position and/or an orientation of the housing based on the determined position and/or the determined orientation of the landmark; when the landmark is not within the field of view of the image sensor, determining the position and/or the orientation of the housing with respect to the display based on the first motion and/or orientation output signals and the second motion and/or orientation output signals; determining an overlay image comprising a blade of the bladed weapon, wherein the blade is placed within the overlay image according to the determined position and/or the determined orientation of the housing such that the blade is placed within the overlay image based on the visual output signals when the landmark is within the field of view of the image sensor and the blade is placed within the overlay image based on the first motion and/or orientation output signals and the second motion and/ororientation output signals when the landmark is not within the field of view of the imagesensor; and effectuating displaying of the overlay image on the display so that the bladeappears to be attached to the hilt. 20. A system for augmenting an appearance of a hilt to simulate a bladed weapon, the system comprising: a display configured to display an overlay image; a haptic generator; a speaker; an image sensor configured to generate visual output signals conveying visual information within a field of view of the image sensor; a housing of the hilt; a landmark carried by the housing, the landmark indicating a reference point for the hilt that facilitates determination of a position and orientation of the housing; a first inertial measurement unit carried by the display, the first inertial measurement unit or separate from the image sensor, the first inertial measurement unit configured to generate first motion and/or orientation output signals conveying motion and/or orientation information of the display such that the first motion and/or orientation output signals are separate and independent from the visual output signals, wherein the first motionand/or orientation output signals facilitate determination of the position and orientation of the housing when the landmark is not within the field of view of the image sensor; a second inertial measurement unit carried by the housing, the second inertial measurement unit separate from the image sensor, the second inertial measurement unit configured to generate second motion and/or orientation output signals conveying motion and/or orientation information of the housing such that the second motion and/or orientation output signals are separate and independent from the visual output signals, wherein the second motion and/or orientation output signals facilitate determination of the position and orientation of the housing when the landmark is not within the field of view of the image sensor; and oneor more processors configured by machine readable instructions to: when the landmark is within the field of view of the image sensor: detect the landmark based on the visual output signals; determine a position and/or an orientation of the landmark; and determine a position and/or an orientation of the housing based on the determined position and/or the determined orientation of the landmark; when the landmark is not within the field of view of the image sensor, determine the position and/or the orientation of the housing with respect to the display based on the first motion and/or orientation output signals and the second motion and/or orientation output signals; determine the overlay image comprising a bladeof the bladed weapon, wherein the blade is placed within the overlay image according to the determined position and/or the determined orientation of the housing such that the blade is placed within the overlay image based on the visual output signals when the landmark is within the field of view of the image sensor and the blade is placed within the overlayimage based on the first motion and/or orientation output signals and the second motion and/or orientation output signals when the landmark is not within the field of view of the image sensor; effectuate displaying of the overlay image on the display so that the blade appears to be attached to the hilt; effectuate operation of the haptic generator in response to a movement of the hilt based on one or more of the visual output signals and/or the second motion and/or orientation output signals effectuate operation of the speaker in response to the movement of the hilt based on one or more of the visual output signals and/orthe second motion and/or orientation output signals to simulate one or more sounds of thebladed weapon such that the one or more sounds of the bladed weapon are simulated based on the second motion and/or orientation output signals when the landmark is not within the field of view of the image sensor; detect an object based on the visual output signals; determine when the blade of the bladed weapon touches the object; and determine the overlay image to further include a visual effect based on the blade of the bladed weapon touching the object.