Positional tracking/prediction technique for AR/VR headsets (granted).
Techniques for predicting a virtual camera view in an augmented reality (AR) or virtual reality (VR) application. A first change in position of a user device over a first time period is determined based on analyzing a plurality of frames of image data related to an AR or VR application. A dead reckoning calculation is used to predict a second change in position of the user device over a second time period, based on the first change in position and data received from an Inertial Measurement Unit (IMU) associated with the user device. A plurality of frames of image data are generated for display in the AR or VR application, based on the predicted second change in position of the user device.
BACKGROUNDField of the Invention(1) This disclosure relates in general to virtual reality (VR) and augmented reality (AR), and more particularly, though not exclusively, to dead reckoning positional prediction in an AR or VR application.Description of the Related Art(2) Computer graphics technology has significantly progressed since the first applications were developed. Relatively inexpensive 3D graphics engines now provide nearly photo-realistic interactive applications on hand-held devices and personal computer platforms. These video game systems typically include a hand-held controller, game controller, or, in the case of a hand-held video game platform, an integrated controller. A user interacts with the controller to send commands or other instructions to the video game system to control a video game or other simulation. For example, the controller may include a joystick and buttons operated by the user.(3) While video games allow the user to interact directly with the video game system, such interactions primarily influence the graphical depiction shown on the video game device (or on a connected display), and rarely influence any other objects outside of the virtual world. That is, a user may specify an input to the video game system, indicating that the user's avatar should perform a jump action, and in response the video game system could display the user's avatar jumping. However, such interactions are typically limited to the virtual world, and any interactions ou
1. A method of predicting a virtual camera view in an augmented reality (AR) or virtual reality (VR) application, comprising: determining a first change in position of a user device over a first time period based on analyzing a plurality of frames of image data related to an AR or VR application, comprising: identifying a first position of the user device at a first point in time based on analyzing a first image captured using the user device; identifying a second position of the user device at a second point in time based on analyzing a second image captured using the user device, wherein the second point in time occurs after the first point in time; and identifying the first change in position based on the identified first position of the user device and the identified second position of the user device; predicting a second change in position of the user device over a second time period, using a dead reckoning calculation and based on the first change in position and data received from an Inertial Measurement Unit (IMU) associated with the user device; and generating a plurality of frames of image data for display in the AR or VR application, based on the predicted second change in position of the user device.
9. A computer program product for predicting a virtual camera view in an AR or VR application, the computer program product comprising: a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation, the operation comprising: determining a first change in position of a user device over a first time period based on analyzing a plurality of frames of image data related to an AR or VR application, comprising: identifying a first position of the user device at a first point in time based on analyzing a first image captured using the user device; identifying a second position of the user device at a second point in time based on analyzing a second image captured using the user device, wherein the second point in time occurs after the first point in time; and identifying the first change in position based on the identified first position of the user device and the identified second position of the user device; predicting a second change in position of the user device over a second time period, using a dead reckoning calculation and based on the first change in position and data received from an Inertial Measurement Unit (IMU) associated with the user device; and generating a plurality of frames of image data for display in the AR or VR application, based on the predicted second change in position of the user device.
14. A system, comprising: a processor; and a memory storing a program, which, when executed on the processor, performs an operation, the operation comprising: determining a first change in position of a user device over a first time period based on analyzing a plurality of frames of image data related to an AR or VR application, comprising: identifying a first position of the user device at a first point in time based on analyzing a first image captured using the user device; identifying a second position of the user device at a second point in time based on analyzing a second image captured using the user device, wherein the second point in time occurs after the first point in time; and identifying the first change in position based on the identified first position of the user device and the identified second position of the user device; predicting a second change in position of the user device over a second time period, using a dead reckoning calculation and based on the first change in position and data received from an Inertial Measurement Unit (IMU) associated with the user device; and generating a plurality of frames of image data for display in the AR or VR application, based on the predicted second change in position of the user device.