IP Library › Granted Patent US 10,937,187
Granted Patent B2
US 10,937,187 · App. 16/371,519 · Granted Mar 2, 2021

Method and system for providing position or movement information for controlling at least one function of an environment

Inventors: Thomas Alt (Saratoga, CA); Lejing Wang (Sunnyvale, CA)
Assignee: Apple Inc.
G06T7/70G06F3/017G06K9/00832G06T7/20G06T7/55B60H1/00792B60Q3/80G06T2207/30244G06T2207/30268
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,937,187
App. No.
16/371,519
Granted
Mar 2, 2021
Kind
B2
Abstract

A method for providing position information for controlling at least one function of an environment provided with an environment coordinate system comprises the steps of receiving image information associated with at least one image of at least one part of the environment captured by a camera of a mobile device, the mobile device being separate from the environment, determining a position associated with the mobile device in the environment coordinate system according to the image information associated with the at least one image, and providing position information indicative of the position associated with the mobile device in the environment coordinate system to control at least one function of the environment according to the position. There is also provided a method and system which provides movement information indicative of a movement associated with the mobile device relative to the environment coordinate system to control at least one function of the environment according to the movement.

Claims (35)

1. A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:

cause a visual content to be displayed on a display device of a vehicle in accordance with a request from a mobile device to control a functionality of the vehicle;

receive a location of a mobile device in a vehicle coordinate system of the vehicle, wherein the mobile device is separate from the vehicle, wherein the location of the mobile device is determined in accordance with the displayed visual content, and wherein an image is received as part of a request from the mobile device to control functionality of the vehicle; and

control functionality of the vehicle for the location of the mobile device in the vehicle.

2. The non-transitory computer readable medium of claim 1 , wherein the image comprises a portion of the vehicle, and wherein the computer readable code to determine the location of the mobile device within the vehicle further comprises computer readable code to:

determine a camera position of a camera of the mobile device relative to the portion of the vehicle in accordance with the displayed visual content, and

determine the location of the mobile device based on the determined camera position relative to the portion of the vehicle.

3. The non-transitory computer readable medium of claim 2 , wherein the camera position of the camera relative to the portion of the vehicle is further determined in accordance with depth information for the display device from a depth sensor of the mobile device.

4. The non-transitory computer readable medium of claim 2 , wherein the camera position of the camera is further determined in accordance with depth information for the portion of the vehicle from a model of the vehicle.

5. The non-transitory computer readable medium of claim 2 , wherein the camera position of the camera is further determined in accordance with an identification of the displayed visual content on the display device,

wherein the location of the mobile device is further determined based on the visual content as it is presented in the image.

6. A system comprising:

one or more processors; and

a computer readable medium comprising computer readable code executable by the one or more processors to:

cause a visual content to be displayed on a display device of a vehicle in accordance with a request from a mobile device to control a functionality of the vehicle;

receive a location of a mobile device in a vehicle coordinate system of the vehicle, wherein the mobile device is separate from the vehicle, wherein the location of the mobile device is determined in accordance with the displayed visual content, and wherein an image is received as part of a request from the mobile device to control functionality of the vehicle; and

control functionality of the vehicle for the location of the mobile device in the vehicle.

7. The system of claim 6 , wherein the image comprises a portion of the vehicle, and wherein the computer readable code to determine the location of the mobile device within the vehicle further comprises computer readable code to:

determine a camera position of a camera of the mobile device relative to the portion of the vehicle in accordance with the displayed visual content, and

determine the location of the mobile device based on the determined camera position relative to the portion of the vehicle.

8. The system of claim 7 , wherein the camera position of the camera relative to the portion of the vehicle is further determined in accordance with depth information for the display device from a depth sensor of the mobile device.

9. The system of claim 7 , wherein the camera position of the camera is further determined in accordance with depth information for the portion of the vehicle from a model of the vehicle.

10. The system of claim 7 , wherein the camera position of the camera is further determined in accordance with an identification of the displayed visual content on the display device,

wherein the location of the mobile device is further determined based on the visual content as it is presented in the image.

11. A method comprising:

causing a visual content to be displayed on a display device of a vehicle in accordance with a request from a mobile device to control a functionality of the vehicle;

receiving a location of a mobile device in a vehicle coordinate system of the vehicle, wherein the mobile device is separate from the vehicle, wherein the location of the mobile device is determined in accordance with the displayed visual content, and wherein an image is received as part of a request from the mobile device to control functionality of the vehicle; and

controlling functionality of the vehicle for the location of the mobile device in the vehicle.

12. The method of claim 11 , wherein the image comprises a portion of the vehicle, and wherein the method further comprises:

determining a camera position of a camera of the mobile device relative to the portion of the vehicle in accordance with the displayed visual content, and

determining the location of the mobile device based on the determined camera position relative to the portion of the vehicle.

13. The method of claim 12 , wherein the camera position is determined in accordance with depth information for the portion of the vehicle from a depth sensor of the mobile device.

14. The method of claim 12 , wherein the camera position is further determined in accordance with depth information for the portion of the vehicle from a model of the vehicle.

15. The method of claim 12 , wherein the camera position of the camera is further determined in accordance with an identification of the displayed visual content on the display device,

wherein the location of the mobile device is further determined based on the visual content as it is presented in the image.

Continuity (3)
Continuation In Part 15093607 · Apr 7, 2016
Continuation In Part PCTEP2013070819 · Oct 7, 2013
Related Publication 20190228539A1 · Jul 25, 2019
Cited By (1)
US 12,588,468