- Number
- 9619709
- Published
- 2017-04-11
- Filed
- 2015-05-26
- Assignee
- Disney Enterprises, Inc.
- Inventors
- Stark; James Alexander et al.
- CPC
- B64C19/00; B64D47/06; B64U70/50; B64U70/95; G05D1/104; G06K19/06131; G06T7/73; G06V10/245; G06V20/13
- Verdict
- Medium Hardware
- Source
- Google Patents · FreePatentsOnline
The keeper's note
Drone (UAV) visual localization hardware using marker detection for autonomous flight/positioning.
Abstract
A method for locating markers in an image captured by amobile device moving about an operating space. The method includes preprocessing an imageto generate a set of image data, locating fixed features of markers by tracing edges of the fixed features, and extracting variable data payloads of each of the markers associatedwith the located fixed features. The fixed features of each of the markers may include a pair of parallel lines extending along opposite sides of a data area containing the variable data payload, and each of the lines extends a distance beyond each exposed end of the data area to avoid missing data when markers are not arranged orthogonally to the scan direction. The preprocessing involves rotating or skewing the image to provide rotated or skewed versions of the image to facilitate locating markers regardless of their angular orientation in the image.
Background
BRIEF DESCRIPTIONOF THE DRAWINGS(1) FIG. 1 illustrates a functional block diagram of a localization systemincluding a mobile device adapted for performing onboard localization using markers in a mobile device travel space;(2) FIG. 2 is a functional block diagram of a method of generating estimates of a state (position and pose) of a mobile device by combining inertial-based estimates with marker-based estimates of state;(3) FIG. 3 is a schematic of scanning of an image being performed to identify linear features such as bars or lines provided in a localization marker;(4) FIG. 4 illustrates one embodiment or design for a localization marker according to the present description;(5) FIG. 5 illustrates another embodiment or design for a localization marker similar to the marker of FIG. 4 but with the lines or stripes of the fixed pattern extending out a distance from each end of the variable pattern or data area of the marker;(6) FIGS. 6 and 7 provide functional block or schematic drawings use of UAVs in a performance setting showing importance of accurate position determination foruse in controlling movement of UAVs during a task such as a show performance;(7) FIGS. 8Aand 8B are side and top views of an exemplary stage (e.g., parade float) that may be usedto provide launching and landing tubes;(8) FIGS. 9 and 10 illustrate, respectfully, a 2-scan method and a 4-scan method of identifying markers or their locations in an image (e.g., an image from a camera provided onboard a
Claims
1. A system for facilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear bars adjacent a data area, comprising: a mobile device operable to move about the operating space in response to control signals; and a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers, wherein the controller executes a localization module to locate each of the markers, to extract data from the data areas, and to estimate the position of the mobile device based on the extracted data, wherein the pattern of linear bars comprises first and second pairs ofspaced apart and parallel bars with the first and second pairs being disposed on oppositesides of the data area, and wherein each of the linear bars in the first and second pairsof parallel bars has a length greater than a maximum length of the data area such that each of the linear bars extends at each end a distance beyond the data area.
8. A system forfacilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear bars adjacent a data area, comprising: a mobile device operable to move about the operating space in response to control signals; and a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers, wherein the controller executes a localization module to locate each of the markers using a single direction scan, to extract data from the data areas, and to estimate the position of the mobile device based on the extracted data, wherein at least one of the markers is positioned to be oriented in the image such that the pattern of linear bars is non-orthogonal to the single direction scan, and wherein the localization module repeats the single direction scan on a rotated or skewed version ofthe image to locate the markers in the image.
13. A system for facilitating determinationof positions of mobile devices such as unmanned aerial vehicles within an operating spacecontaining a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear bars adjacent a data area, comprising:a mobile device operable to move about the operating space in response to control signals; and a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers, wherein the controller executes a localization module to locate each of the markers, to extract data from the data areas, and to estimate the position of the mobiledevice based on the extracted data, wherein the pattern of linear bars comprises parallelstripes on a pair of opposite sides of the data area, and wherein the parallel stripes oneach of opposite sides of the data area are spaced apart a predefined distance.
17. A system for facilitating determination of positions of mobile devices such as unmanned aerial vehicles within an operating space containing a number of objects and markers placed on exposed surfaces of the objects, with each of the markers including a pattern of linear barsadjacent a data area, comprising: a mobile device operable to move about the operating space in response to control signals; and a controller on the mobile device generating the control signals based on a determination of a position of the mobile device within the operating space relative to locations of the markers, wherein the controller executes a localization module to locate each of the markers, to extract data from the data areas, and to estimate the position of the mobile device based on the extracted data, wherein the patternof linear bars comprises at least a pair of spaced apart and parallel bars disposed on either side of the data area, and wherein the pair of parallel bars has a length greater than a maximum length of the data area such that the pair of parallel bars extends a distancebeyond each exposed end of the pattern in the data area.