System and method of simultaneous localisation and mapping

1 . A method of environment mapping comprising:

receiving a plurality of images from different viewpoints;

using a simultaneous localisation and mapping (SLAM) process to detect feature points in the plurality of images;

detecting a plurality of objects in the plurality of images using a computer vision technique;

classifying one or more objects of the plurality of objects as objects of interest using a trained machine learning model based on object mobility, wherein classifying comprises:

determining a level of mobility for each object of the plurality of objects,

classifying each object as an object of interest or not an object of interest based on the level of mobility, and

labeling feature points associated with each object using a same classification as the object;

repeatedly determining a position of each object of the plurality of objects using a set of feature points associated with the object, wherein determining a position of an object classified as the object of interest is repeated more frequently than determining a position of an object classified as not the object of interest; and

providing an indication of the position of at least one object of interest to a user of a virtual display based on the determination.

2 . The method of claim 1 , wherein classifying further comprises classifying the one or more objects into one of a plurality of classifications including a static classification.

3 . The method of claim 2 , further comprising

selecting a subset of feature points corresponding to the one or more objects classified in the static classification; and

using the subset of feature points in SLAM processing to generate an environmental map.

4 . The method of claim 1 , wherein classifying further comprises, for an object classified as the object of interest, estimating a degree of interest for the object.

5 . The method of claim 4 , wherein the degree of interest for the object is determined as a function of one or more of:

i. a type of the object;

ii. whether the object is predefined as a hazard;

iii. whether the object is predefined as having inherent motion;

iv. whether the object is predefined as being movable; and

v. a proximity of the object to the user.

6 . The method of claim 5 , wherein the frequency of determining the position of the object that is predefined as the hazard is higher than the frequency for an object that is not a predefined hazard.

7 . The method of claim 4 , wherein the degree of interest for the object is determined as a function of one or more of:

i. a state of the user;

ii. an orientation of the user to the object; and

iii. a motion of the user relative to the object.

8 . The method of claim 4 , wherein

the method is being performed within a context of a game, and

the degree of interest for the object at a given time is determined as a function of one or more of:

i. a type of game content being presented to the user at the given time;

ii. game input requirements at the given time; and

iii. game inputs of the user at the given time.

9 . The method of claim 4 , wherein the frequency of determining the position of the object classified as the object of interest is a function of degree of interest estimated for the object.

10 . The method of claim 4 , wherein the frequency of determining the position of the object is a function of available computational resources unallocated to a primary application.

11 . The method of claim 4 , wherein providing the indication comprises displaying the indication when the degree of interest for the object exceeds a threshold amount.

12 . The method of to claim 1 , wherein providing the indication comprises displaying one or more of:

i. a point cloud of the set of feature points;

ii. image data associated with the object of interest; and

iii. an icon representing the object of interest.

13 . A non-transitory, computer readable storage medium containing a computer program comprising computer executable instructions adapted to cause a computer system to perform a method of environment mapping comprising:

receiving a plurality of images from different viewpoints;

using a simultaneous localisation and mapping (SLAM) process to detect feature points in the plurality of images;

detecting a plurality of objects in the plurality of images using a computer vision technique;

classifying one or more objects of the plurality of objects as objects of interest using a trained machine learning model based on object mobility, wherein classifying comprises:

determining a level of mobility for each object of the plurality of objects,

classifying each object as an object of interest or not an object of interest based on the level of mobility, and

labeling feature points associated with each object using a same classification as the object;

repeatedly determining a position of each object of the plurality of objects using a set of feature points associated with the object, wherein determining a position of an object classified as the object of interest is repeated more frequently than determining a position of an object classified as not the object of interest; and

providing an indication of the position of at least one object of interest to a user of a virtual display based on the determination.

14 . The non-transitory, computer readable storage medium of claim 13 , wherein classifying further comprises, for an object classified as the object of interest, estimating a degree of interest for the object.

15 . The non-transitory, computer readable storage medium of claim 14 , wherein the degree of interest for the object is determined as a function of one or more of:

i. a type of the object;

ii. whether the object is predefined as a hazard;

iii. whether the object is predefined as having inherent motion;

iv. whether the object is predefined as being movable; and

v. a proximity of the object to the user.

16 . The non-transitory, computer readable storage medium of claim 15 , wherein the frequency of determining the position of the object that is predefined as the hazard is higher than the frequency for an object that is not a predefined hazard.

17 . An environment mapping system comprising:

one or more processors; and

one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the environment mapping system to perform operations comprising:

receiving a plurality of images from different viewpoints;

using a simultaneous localisation and mapping (SLAM) process to detect feature points in the plurality of images;

detecting a plurality of objects in the plurality of images using a computer vision technique;

classifying one or more objects of the plurality of objects as objects of interest using a trained machine learning model based on object mobility, wherein classifying comprises:

determining a level of mobility for each object of the plurality of objects,

classifying each object as an object of interest or not an object of interest based on the level of mobility, and

labeling feature points associated with each object using a same classification as the object;

repeatedly determining a position of each object of the plurality of objects using a set of feature points associated with the object, wherein determining a position of an object classified as the object of interest is repeated more frequently than determining a position of an object classified as not the object of interest; and

providing an indication of the position of at least one object of interest to a user of a virtual display based on the determination.

18 . The environment mapping system of claim 17 , wherein classifying further comprises, for an object classified as the object of interest, estimating a degree of interest for the object.

19 . The environment mapping system of claim 18 , wherein the degree of interest for the object is determined as a function of one or more of:

i. a type of the object;

ii. whether the object is predefined as a hazard;

iii. whether the object is predefined as having inherent motion;

iv. whether the object is predefined as being movable; and

v. a proximity of the object to the user.

20 . The environment mapping system of claim 19 , wherein the frequency of determining the position of the object that is predefined as the hazard is higher than the frequency for an object that is not a predefined hazard.