Granted Patent B2

US 12,243,333 · App. 17/714,750 · Granted Mar 4, 2025

Inventor: Camila Dorin (Lund, SE)

Assignee: Inter IKEA Systems B.V.

G06V20/64G06Q30/0631G06T7/11G06T7/62G06V10/764G06V10/82G06V20/70G06Q10/087G06T2207/10028G06V2201/10G06V2201/12

Abstract

Claims (178)

1. A computer implemented method for determining one or more storage boxes for storing objects, the method comprising:

scanning, by an imaging sensor, a plurality of objects, to acquire a scan of the plurality of objects;

segmenting the scan of the plurality of objects into segmented objects;

assigning metadata to a segmented object, wherein to assign the metadata to the segmented object includes to:

compare characteristics of the segmented object with characteristics of objects stored in a database, and

upon a positive match, assign the metadata to the segmented object and storing the metadata as assigned to the segmented object in the database, wherein the metadata comprises information on a 3D measure of an object;

determining a three-dimensional (3D) measure of each of the segmented objects, wherein the 3D measure of the segmented object is determined from the metadata;

grouping the segmented objects into one or more groups of objects; and

determining, for a group of objects of the one or more groups of objects, one or more candidate storage boxes for storing the group of objects by:

calculating a total 3D measure for the group of objects by combining the 3D measure of each object of the group of objects; and

comparing the total 3D measure for the group of objects with inner dimensions of a set of potential storage boxes.

2. The method according to claim 1 , wherein the metadata comprises information on an object category, and grouping the segmented object into the one or more groups of objects is based on the object category.

3. The method according to claim 1 , further comprising:

scanning, by a range sensor, the plurality of objects to be stored thereby adding range information to the scan of the plurality of objects,

wherein determining the 3D measure of a segmented object is further made based on the range information associated with the segmented object.

4. The method according to claim 1 , wherein grouping the objects into the one or more groups of objects further comprises:

categorizing the segmented objects; and

grouping the segmented objects belonging to a same object category into one of the one or more groups of objects.

5. The method according to claim 4 , wherein at least some of the segmented objects are categorized using a neural network trained to categorize objects based on a plurality of images of home environments.

6. The method according to claim 4 , wherein determining the one or more candidate storage boxes further comprises:

selecting the set of potential storage boxes from a group of storage boxes based on the object category of the objects in the group of objects and/or an appearance of boxes of the group of storage boxes.

7. The method according to claim 1 , wherein determining the one or more candidate storage boxes further comprises:

identifying storage boxes comprising objects corresponding to the group of objects from a plurality of images of home environments,

wherein the set of potential storage boxes is selected from a group of storage boxes based on the identified storage boxes.

8. The method according to claim 1 , wherein the one or more candidate storage boxes comprises a set of different storage boxes, each box in the set of different boxes having a different appearance, the method further comprising:

displaying a representation of each of the one or more candidate storage boxes on a display; and

receiving user input selecting at least one storage box among the one or more candidate storage boxes.

9. The method according to claim 1 , the method further comprising:

calculating a total volume for the group of objects;

determining whether the group of objects fits into one storage box based on the total volume for the group of objects; and

splitting the group of objects into a plurality of boxes when it is determined that the group of objects does not fit into the one storage box.

10. The method according to claim 9 , further comprising:

subgrouping the group of objects into subgroups until each of the subgroups fit into an individual one of the one or more candidate storage boxes.

11. The method according to claim 1 , further comprising:

displaying on a display a scheme to organize the group of objects to be stored in a storage box such that they fit into the storage box.

12. The method according to claim 1 , further comprising:

displaying on a display a representation of the one or more candidate storage boxes having objects of the group of objects stored therein.

13. A computing device comprising:

an imaging sensor;

a processor;

a memory device storing instructions which, when executed by the processor cause the computing device to:

scan, by the imaging sensor, a plurality of objects to be stored to acquire a scan of the plurality of objects;

segment the scan of the plurality of objects into segmented objects;

assign metadata to a segmented object, wherein to assign metadata to a segmented object includes to:

compare characteristics of the segmented object with characteristics of objects stored in a database, and

upon a positive match, assign the metadata to the segmented object and store the metadata as assigned to the segmented object in the database, wherein the metadata comprises information on a 3D measure of an object;

determine a three-dimensional (3D) measure of each of the plurality of objects, wherein the 3D measure of the segmented object is determined from the metadata;

calculate a total 3D measure for the plurality of objects; and

compare the total 3D measure for the plurality of objects with inner dimensions of a set of potential storage boxes to determine one or more candidate storage boxes for storing the plurality of objects.

14. The computing device of claim 13 , further comprising:

a range sensor configured to scan the plurality of objects to be stored thereby to add range information to the scan of the plurality of objects, wherein to determine a 3D measure of a segmented object is further made based on the range information associated with the segmented object.

15. The computing device of claim 13 , further comprising:

a display configured to present a scheme to organize a particular group of objects to be stored in a storage box such that they fit into the storage box.

16. The computing device of claim 13 , further comprising:

a display configured to present a representation of the one or more candidate storage boxes having objects of the group of objects stored therein.

17. A computer implemented method for determining one or more storage boxes for storing objects, the method comprising:

scanning, by an imaging sensor, a plurality of objects, to acquire a scan of the plurality of objects;

segmenting the scan of the plurality of objects into segmented objects;

determining a three-dimensional (3D) measure of each of the segmented objects;

grouping the segmented objects into one or more groups of objects, wherein grouping the objects into the one or more groups of objects comprises:

categorizing the segmented objects; and

grouping the segmented objects belonging to a same object category into one of the one or more groups of objects; and

determining, for a group of objects of the one or more groups of objects, one or more candidate storage boxes for storing the group of objects by:

calculating a total 3D measure for the group of objects by combining the 3D measure of each object of the group of objects;

comparing the total 3D measure for the group of objects with inner dimensions of a set of potential storage boxes; and

selecting the set of potential storage boxes from a group of storage boxes based on the object category of the objects in the group of objects and/or an appearance of boxes of the group of storage boxes.

18. The method according to claim 17 , further comprising:

assigning metadata to a segmented object by:

comparing characteristics of the segmented object with characteristics of objects stored in a database, and

upon a positive match, assigning the metadata to the segmented object and storing the metadata as assigned to the segmented object in the database.

19. The method according to claim 18 , wherein the metadata comprises information on an object category, and grouping the segmented object into the one or more groups of objects is based on the object category.

20. The method according to claim 17 , further comprising:

scanning, by a range sensor, the plurality of objects to be stored thereby adding range information to the scan of the plurality of objects,

wherein determining the 3D measure of a segmented object is made based on the range information associated with the segmented object.

21. The method according to claim 17 , wherein at least some of the segmented objects are categorized using a neural network trained to categorize objects based on a plurality of images of home environments.

22. The method according to claim 17 , wherein determining the one or more candidate storage boxes further comprises:

identifying storage boxes comprising objects corresponding to the group of objects from a plurality of images of home environments,

wherein selecting the set of potential storage boxes from the group of storage boxes is further based on the identified storage boxes.

23. The method according to claim 17 , wherein the one or more candidate storage boxes comprises a set of different storage boxes, each box in the set of different boxes having a different appearance, the method further comprising:

displaying a representation of each of the one or more candidate storage boxes on a display; and

receiving user input selecting at least one storage box among the one or more candidate storage boxes.

24. The method according to claim 17 , the method further comprising:

calculating a total volume for the group of objects;

determining whether the group of objects fits into one storage box based on the total volume for the group of objects; and

splitting the group of objects into a plurality of boxes when it is determined that the group of objects does not fit into the one storage box.

25. The method according to claim 24 , further comprising:

subgrouping the group of objects into subgroups until each of the subgroups fit into an individual one of the one or more candidate storage boxes.

26. The method according to claim 17 , further comprising:

displaying on a display a scheme to organize the group of objects to be stored in a storage box such that they fit into the storage box.

27. The method according to claim 17 , further comprising:

displaying on a display a representation of the one or more candidate storage boxes having objects of the group of objects stored therein.

28. A computing device comprising:

an imaging sensor;

a processor;

a memory device storing instructions which, when executed by the processor cause the computing device to:

scan, by the imaging sensor, a plurality of objects to be stored to acquire a scan of the plurality of objects;

segment the scan of the plurality of objects into segmented objects;

determine a three-dimensional (3D) measure of each of the plurality of objects;

group the segmented objects into one or more groups of objects, wherein to group the objects into the one or more groups of objects includes to:

categorize the segmented objects; and

group the segmented objects belonging to a same object category into one of the one or more groups of objects;

calculate a total 3D measure for the plurality of objects;

compare the total 3D measure for the plurality of objects with inner dimensions of a set of potential storage boxes to determine one or more candidate storage boxes for storing the plurality of objects; and

select the one or more candidate storage boxes based on the object category of the objects in the group of objects and/or an appearance of boxes of the one or more candidate storage boxes.

29. The computing device of claim 28 , further comprising:

a range sensor configured to scan the plurality of objects to be stored thereby to add range information to the scan of the plurality of objects, wherein to determine a 3D measure of a segmented object is made based on the range information associated with the segmented object.

30. The computing device of claim 28 , further comprising:

a display configured to present a scheme to organize a particular group of objects to be stored in a storage box such that they fit into the storage box.

31. The computing device of claim 28 , further comprising:

a display configured to present a representation of the one or more candidate storage boxes having objects of the group of objects stored therein.

32. The computing device of claim 28 , wherein the instructions are further configured to cause the computing device to:

assign metadata to a segmented object, wherein to assign the metadata to the segmented object includes to:

compare characteristics of the segmented object with characteristics of objects stored in a database, and

upon a positive match, assign the metadata to the segmented object and store the metadata as assigned to the segmented object in the database.

33. A computer implemented method for determining one or more storage boxes for storing objects, the method comprising:

scanning, by an imaging sensor, a plurality of objects, to acquire a scan of the plurality of objects;

segmenting the scan of the plurality of objects into segmented objects;

determining a three-dimensional (3D) measure of each of the segmented objects;

grouping the segmented objects into one or more groups of objects; and

determining, for a group of objects of the one or more groups of objects, one or more candidate storage boxes for storing the group of objects by:

calculating a total 3D measure for the group of objects by combining the 3D measure of each object of the group of objects;

identifying storage boxes comprising objects corresponding to the group of objects from a plurality of images of home environments;

selecting a set of potential storage boxes from a group of storage boxes based on the identified storage boxes; and

comparing the total 3D measure for the group of objects with inner dimensions of the set of potential storage boxes.

34. The method according to claim 33 , further comprising:

assigning metadata to a segmented object by:

comparing characteristics of the segmented object with characteristics of objects stored in a database, and

upon a positive match, assigning the metadata to the segmented object and storing the metadata as assigned to the segmented object in the database.

35. The method according to claim 34 , wherein the metadata comprises information on an object category, and grouping the segmented object into the one or more groups of objects is based on the object category.

36. The method according to claim 33 , further comprising:

scanning, by a range sensor, the plurality of objects to be stored thereby adding range information to the scan of the plurality of objects,

wherein determining a 3D measure of a segmented object is made based on the range information associated with the segmented object.

37. The method according to claim 33 , wherein grouping the objects into the one or more groups of objects further comprises:

categorizing the segmented objects; and

grouping the segmented objects belonging to a same object category into one of the one or more groups of objects.

38. The method according to claim 37 , wherein at least some of the segmented objects are categorized using a neural network trained to categorize objects based on a plurality of images of home environments.

39. The method according to claim 33 , wherein the one or more candidate storage boxes comprises a set of different storage boxes, each box in the set of different boxes having a different appearance, the method further comprising:

displaying a representation of each of the one or more candidate storage boxes on a display; and

receiving user input selecting at least one storage box among the one or more candidate storage boxes.

40. The method according to claim 33 , the method further comprising:

calculating a total volume for the group of objects;

determining whether the group of objects fits into one storage box based on the total volume for the group of objects; and

splitting the group of objects into a plurality of boxes when it is determined that the group of objects does not fit into the one storage box.

41. The method according to claim 40 , further comprising:

subgrouping the group of objects into subgroups until each of the subgroups fit into an individual one of the one or more candidate storage boxes.

42. The method according to claim 33 , further comprising:

displaying on a display a scheme to organize the group of objects to be stored in a storage box such that they fit into the storage box.

43. The method according to claim 33 , further comprising:

displaying on a display a representation of the one or more candidate storage boxes having objects of the group of objects stored therein.

44. A computing device comprising:

an imaging sensor;

a processor;

a memory device storing instructions which, when executed by the processor cause the computing device to:

scan, by the imaging sensor, a plurality of objects to be stored to acquire a scan of the plurality of objects;

segment the scan of the plurality of objects into segmented objects;

determine a three-dimensional (3D) measure of each of the plurality of objects;

calculate a total 3D measure for the plurality of objects;

identify storage boxes comprising objects corresponding to the plurality of objects from a plurality of images of home environments;

select a set of potential storage boxes from a group of storage boxes based on the identified storage boxes; and

compare the total 3D measure for the plurality of objects with inner dimensions of the set of potential storage boxes to determine one or more candidate storage boxes for storing the plurality of objects.

45. The computing device of claim 44 , further comprising:

a range sensor configured to scan the plurality of objects to be stored thereby to add range information to the scan of the plurality of objects, wherein to determine a 3D measure of a segmented object is made based on the range information associated with the segmented object.

46. The computing device of claim 44 , further comprising:

a display configured to present a scheme to organize a particular group of objects to be stored in a storage box such that they fit into the storage box.

47. The computing device of claim 44 , further comprising:

a display configured to present a representation of the one or more candidate storage boxes having objects of the group of objects stored therein.

48. The computing device of claim 44 , wherein the instructions are further configured to cause the computing device to:

assign metadata to a segmented object, wherein to assign the metadata to the segmented object includes to:

compare characteristics of the segmented object with characteristics of objects stored in a database, and

upon a positive match, assign the metadata to the segmented object and store the metadata as assigned to the segmented object in the database.

49. A non-transitory computer-readable storage medium having stored thereon instructions which, when executed on a device having processing capabilities, cause the device to:

scan, by the imaging sensor, a plurality of objects to be stored to acquire a scan of the plurality of objects;

segment the scan of the plurality of objects into segmented objects;

determine a three-dimensional (3D) measure of each of the plurality of objects;

calculate a total 3D measure for the plurality of objects;

compare the total 3D measure for the plurality of objects with inner dimensions of a set of potential storage boxes to determine one or more candidate storage boxes for storing the plurality of objects, wherein the one or more candidate storage boxes comprises a set of different storage boxes, each box in the set of different boxes having a different appearance;

display a representation of each of the one or more candidate storage boxes on a display; and

receive user input selecting at least one storage box among the one or more candidate storage boxes.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (20)