IP Library › Granted Patent US 12,645,737
Granted Patent B2
US 12,645,737 · App. 18/049,709 · Granted Jun 2, 2026

Far-edge intensive processing for so-maps

Inventors: Werner Spolidoro Freund (Rio de Janeiro, BR); Julia Drummond Noce (Rio de Janeiro, BR); Pablo Nascimento da Silva (Niterói, BR); Vinicius Michel Gottin (Rio de Janeiro, BR); Paulo Abelha Ferreira (Rio de Janeiro, BR)
Assignee: Dell Products L.P.
G06F16/9024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,645,737
App. No.
18/049,709
Filed
Oct 26, 2022
Granted
Jun 2, 2026
Kind
B2
Examiner
LE, MIRANDA
Art Unit
2153
USPC
707/798
Abstract

Systems and methods for decoupling information into multiple levels of detail based on semantics while providing a compact global representation structure. A tree-based graph represents objects in an environment. The top node provides complete environment information at the lowest level of detail. Leaves of the tree-based graph complement the parent nodes with more detailed local information. This achieved efficient processing, communication, and data compaction. Steps for updating the tree-based graph are performed at both a far-edge node and a near-edge node.

Claims (45)

1 . In a computing system including a far-edge node and a near-edge node, a method comprising:

obtaining a set of measurements and interactions at a far-edge node in an environment, wherein the far-edge node is one of a moveable far-edge node or a sensing far-edge node, wherein a plurality of far-edge nodes are present in the environment and each is associated with a different set of measurements, wherein the interactions include relationships between the far-edge node and objects in the environment;

associating the set of measurements and the interactions to a graph node in a local representation of a local graph at the far-edge node, wherein the local graph is a portion of a global graph, wherein each of the far-edge nodes and objects in the environment are represented in the global graph;

updating the graph node in the local representation of the local graph associated with the set of measurements and the interactions with a local update, wherein updating the graph node includes updating graph node relationships based on the interactions, wherein updating graph node relationships may include creating a parent-child relationship in the local graph when the far-edge node begins an interaction with an object or a second far-edge node and pruning a child node in the local graph when the interaction with the object or the second far-edge node ends;

updating other graph nodes in the local representation of the graph not associated with the graph node that was updated, wherein the other graph nodes are associated with other far-edge nodes included in the plurality of far-edge nodes;

generating local environmental information that has been filtered to include information about the far edge nodes included in the local representation of the local graph that have been updated;

sending the local environmental information to the near-edge node to be merged with second local environmental information received the plurality of far-edge nodes and to be used in updating a global representation of a global graph by updating graph nodes in the global graph based on the local environmental information and the second local environment information; and

receiving local update information from the near-edge node at the far-edge node; and

updating the local representation of the local graph at the far-edge node with the local update information received from the near-edge node.

2 . The method of claim 1 , further comprising:

at the far-edge node, updating graph nodes associated with the graph node that was updated at the far-edge node.

3 . The method of claim 1 , wherein the moveable far-edge node undergoes a periodic charging procedure, wherein during the periodic charging procedure processing resources of the far-edge node are integrated into processing resources of the near-edge node so that the far-edge node is able to assist the near-edge node in updating the global representation of the global graph.

4 . The method of claim 1 , wherein each of the graph nodes is represented in its minimal form by a sextuplet including a map, a frame of reference, a parent node, a set of child nodes, a label, and an update timestamp.

5 . The method of claim 1 , further comprising pruning, at the far-edge node, graph nodes in the local representation of the local graph whose likelihood of existing are below a threshold value or based on the interactions.

6 . The method of claim 1 , wherein the local update information includes a region of interest.

7 . The method of claim 1 , further comprising creating, at the far-edge node, a graph node in the local representation of the graph when the set of measurements are not associated with an existing graph node in the local representation of the local graph based on the interactions, wherein the interactions are represented at least semantically.

8 . The method of claim 1 , wherein updating the graph node includes verifying that the set of measurements are from a particular semantic class and determining whether the set of measurements are compatible with an existing graph node in local representation of the local graph.

9 . The method of claim 1 , further comprising updating parent nodes in a bottom-up approach, updating other nodes in a top-down approach, wherein updating other nodes includes updating freespace voxels at the far-edge node.

10 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:

Obtaining a set of measurements and interactions at a far-edge node in an environment, wherein the far-edge node is one of a moveable far-edge node or a sensing far-edge node, wherein a plurality of far-edge nodes are present in the environment and each is associated with a different set of measurements, wherein the interactions include relationships between the far-edge node and objects in the environment;

associating the set of measurements and the interactions to a graph node in a local representation of a local graph at the far-edge node, wherein the local graph is a portion of a global graph, wherein each of the far-edge nodes and objects in the environment are represented in the global graph;

updating the graph node in the local representation of the local graph associated with the set of measurements and the interactions with a local update, wherein updating the graph node includes updating graph node relationships based on the interactions, wherein updating graph node relationships may include creating a parent-child relationship in the local graph when the far-edge node begins an interaction with an object or a second far-edge node and pruning a child node in the local graph when the interaction with the object or the second far-edge node ends;

updating other graph nodes in the local representation of the graph not associated with the graph node that was updated, wherein the other graph nodes are associated with other far-edge nodes included in the plurality of far-edge nodes;

generating local environmental information that has been filtered to include information about the far edge nodes included in the local representation of the local graph that have been updated;

sending the local environmental information to the near-edge node to be merged with second local environmental information received the plurality of far-edge nodes and to be used in updating a global representation of a global graph by updating graph nodes in the global graph based on the local environmental information and the second local environment information; and

receiving local update information from the near-edge node at the far-edge node; and

updating the local representation of the local graph at the far-edge node with the local update information received from the near-edge node.

11 . The non-transitory storage medium of claim 10 , further comprising:

at the far-edge node, updating graph nodes associated with the graph node that was updated at the far-edge node.

12 . The non-transitory storage medium of claim 10 , wherein the moveable far-edge node undergoes a periodic charging procedure, wherein during the periodic charging procedure processing resources of the far-edge node are integrated into processing resources of the near-edge node so that the far-edge node is able to assist the near-edge node in updating the global representation of the global graph.

13 . The non-transitory storage medium of claim 10 , wherein each of the graph nodes is represented in its minimal form by a sextuplet including a map, a frame of reference, a parent node, a set of child nodes, a label, and an update timestamp.

14 . The non-transitory storage medium of claim 10 , further comprising pruning, at the far-edge node, graph nodes in the local representation of the local graph whose likelihood of existing are below a threshold value or based on the interactions.

15 . The non-transitory storage medium of claim 10 , further comprising creating at the far-edge node a node in the local representation of the graph when the set of measurements are not associated with an existing node in the local representation of the graph based on the interactions, wherein the interactions are represented at least semantically.

16 . The non-transitory storage medium of claim 10 , wherein the local update information includes a region of interest.

17 . The non-transitory storage medium of claim 10 , further comprising updating parent nodes in a bottom-up approach, updating other nodes in a top-down approach, wherein updating other nodes includes updating freespace voxels at the far-edge node.

18 . In a computing system including a far-edge node and a near-edge node, a method comprising:

at the far-edge node:

obtaining a set of measurements and interactions from the far-edge node in an environment, wherein the far-edge node is one of a moveable far-edge node or a sensing far-edge node, wherein the interactions include relationships between the far-edge node and objects in the environment, wherein a plurality of far-edge nodes are present in the environment and each is associated with a set of measurements;

associating the set of measurements and the interactions to a graph node in a local representation of a local graph, wherein each of the far-edge nodes and objects in the environment are represented in the global graph;

updating the graph node in the local representation of the local graph associated with the set of measurements and the interactions with a local update, wherein updating the graph node includes updating graph node relationships based on the interactions, wherein updating graph node relationships may include creating a parent-child relationship in the local graph when the far-edge node begins an interaction with an object or a second far-edge node and pruning a child node in the local graph when the interaction with the object or the second far-edge node ends;

updating graph nodes in the local representation of the local graph not associated with the graph node in the local graph that was updated; and

generating local environmental information that has been filtered to include information about the graph nodes in the local representation of the local graph that have been updated; and

at the near-edge node:

merging the local environmental information received from the far-edge node with second local environmental information received other far-edge nodes or generated at the near-edge node; and

updating a global representation of a global graph by updating graph based on the local environmental information and the second local environmental information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2022
From: FREUND, WERNER SPOLIDORO; NOCE, JULIA DRUMMOND; DA SILVA, PABLO NASCIMENTO; GOTTIN, VINICIUS MICHEL; FERREIRA, PAULO ABELHA
To: DELL PRODUCTS L.P.
Reel/Frame 061539/0981 →
Continuity (1)
Related Publication 20240144174A1 · May 2, 2024
References Cited (99)
US 7895021B1 · Andrews, Jr. · 2011 [cited by applicant]
US 8261033B1 · Slik et al. · 2012 [cited by applicant]
US 10142353B2 · Yadav et al. · 2018 [cited by applicant]
US 10346654B2 · Hochhalter · 2019 [cited by examiner]
US 10429197B1 · Carrino et al. · 2019 [cited by applicant]
US 10549928B1 · Chavez · 2020 [cited by examiner]
US 10584971B1 · Askeland · 2020 [cited by applicant]
US 10878386B2 · Hoofard · 2020 [cited by examiner]
US 10964097B2 · Teply et al. · 2021 [cited by applicant]
US 11077548B2 · Stilwell · 2021 [cited by examiner]
US 11157527B2 · Wang · 2021 [cited by examiner]
US 11227401B1 · Mahieu · 2022 [cited by examiner]
US 11312379B2 · Taylor · 2022 [cited by examiner]
US 11314254B2 · Macias et al. · 2022 [cited by applicant]
US 11402830B2 · Sullivan · 2022 [cited by examiner]
US 11436504B1 · Lukarski et al. · 2022 [cited by applicant]
US 11533234B2 · Cencini · 2022 [cited by examiner]
US 11595269B1 · Ghosh et al. · 2023 [cited by applicant]
US 11792262B1 · Chung et al. · 2023 [cited by applicant]
US 11819734B2 · Lee et al. · 2023 [cited by applicant]
US 11836563B2 · Khoche · 2023 [cited by examiner]
US 20020152318A1 · Menon et al. · 2002 [cited by applicant]
US 20100045701A1 · Scott et al. · 2010 [cited by applicant]
US 20120090667A1 · Cap et al. · 2012 [cited by applicant]
US 20120323431A1 · Wong · 2012 [cited by examiner]
US 20130307720A1 · Lilburn · 2013 [cited by applicant]
US 20140074342A1 · Wong · 2014 [cited by examiner]
US 20140278517A1 · Patel et al. · 2014 [cited by applicant]
US 20140309841A1 · Hara · 2014 [cited by examiner]
US 20160071278A1 · Leonard · 2016 [cited by examiner]
US 20160292908A1 · Obert · 2016 [cited by examiner]
US 20160314149A1 · Lection et al. · 2016 [cited by applicant]
US 20160359872A1 · Yadav et al. · 2016 [cited by applicant]
US 20170248963A1 · Levinson et al. · 2017 [cited by applicant]
US 20170317920A1 · Rocquelay · 2017 [cited by examiner]
US 20180137390A1 · Brundage et al. · 2018 [cited by applicant]
US 20180137675A1 · Kwant et al. · 2018 [cited by applicant]
US 20180173239A1 · Yoon · 2018 [cited by examiner]
US 20180222043A1 · Trovero et al. · 2018 [cited by applicant]
US 20190043246A1 · Teply et al. · 2019 [cited by applicant]
US 20190163191A1 · Sorin et al. · 2019 [cited by applicant]
US 20190285396A1 · Yao et al. · 2019 [cited by applicant]
US 20200026292A1 · Douillard · 2020 [cited by examiner]
US 20200030965A1 · Stilwell · 2020 [cited by applicant]
US 20200036595A1 · Wallerstein et al. · 2020 [cited by applicant]
US 20200370920A1 · Ahmed et al. · 2020 [cited by applicant]
US 20210021485A1 · Guim et al. · 2021 [cited by applicant]
US 20210107153A1 · Poornachandran · 2021 [cited by examiner]
US 20210140773A1 · Ondruska et al. · 2021 [cited by applicant]
US 20210144517A1 · Guim et al. · 2021 [cited by applicant]
US 20210150771A1 · Huang et al. · 2021 [cited by applicant]
US 20210169417A1 · Burton · 2021 [cited by applicant]
US 20210187391A1 · Ekkati et al. · 2021 [cited by applicant]
US 20210233390A1 · Georgiou et al. · 2021 [cited by applicant]
US 20210302260A1 · Baggs et al. · 2021 [cited by applicant]
US 20210389817A1 · Spinelli · 2021 [cited by examiner]
US 20220082408A1 · Montemerlo et al. · 2022 [cited by applicant]
US 20220129426A1 · Sohail et al. · 2022 [cited by applicant]
US 20220131934A1 · Oku et al. · 2022 [cited by applicant]
US 20220138966A1 · Sung et al. · 2022 [cited by applicant]
US 20220147059A1 · Borne-Pons · 2022 [cited by examiner]
US 20220147407A1 · Asgar et al. · 2022 [cited by applicant]
US 20220187841A1 · Ebrahimi Afrouzi · 2022 [cited by examiner]
US 20220200917A1 · Mortensen et al. · 2022 [cited by applicant]
US 20220203165A1 · Lee et al. · 2022 [cited by applicant]
US 20220204019A1 · Lauterbach et al. · 2022 [cited by applicant]
US 20220245111A1 · Harrison · 2022 [cited by examiner]
US 20220292068A1 · Lin et al. · 2022 [cited by applicant]
US 20220329650A1 · Zhang et al. · 2022 [cited by applicant]
US 20220413989A1 · Karri et al. · 2022 [cited by applicant]
US 20230005217A1 · Chen · 2023 [cited by examiner]
US 20230071442A1 · Tripathy et al. · 2023 [cited by applicant]
US 20230106877A1 · Simeonov et al. · 2023 [cited by applicant]
US 20230117081A1 · Hunter et al. · 2023 [cited by applicant]
US 20230156074A1 · Kim et al. · 2023 [cited by applicant]
US 20230161041A1 · Schindler et al. · 2023 [cited by applicant]
US 20230213494A1 · Sanden et al. · 2023 [cited by applicant]
US 20230231903A1 · Zeng · 2023 [cited by applicant]
US 20230237064A1 · Bao · 2023 [cited by applicant]
US 20230275834A1 · Huang · 2023 [cited by applicant]
US 20230291794A1 · Bartholomew et al. · 2023 [cited by applicant]
US 20230297356A1 · Hudson · 2023 [cited by applicant]
US 20230376558A1 · Seyfi et al. · 2023 [cited by applicant]
US 20240050803A1 · Lee et al. · 2024 [cited by applicant]
US 20240362808A1 · Del et al. · 2024 [cited by applicant]
CN 110537078A · 2019 [cited by applicant]
CN 116368355A · 2023 [cited by applicant]
WO 2020112827A2 · 2020 [cited by applicant]
Hornung, A. et al. (2013) ‘OctoMap: An efficient probabilistic 3D mapping framework based on octrees’, Autonomous Robots, 34(3), pp. 189-206. doi:10.1007/s10514-012-9321-0. [cited by applicant]
Wurm, K.M. et al. (2011) ‘Hierarchies of Octrees for Efficient 3D Mapping’, (September). doi:10.1109/IROS.2011.6048189. [cited by applicant]
N. Hubel, A. Gusrialdi, H. Fujita, and O. Sawodny, “Coverage Control with Information Decay in Dynamic Environments,” Jan. 2008. [cited by applicant]
Dahmen et al, “Verification and validation of Digital twins and virtual testbed,” International Journal of Advances in Applied sciences, vol. 11, nº 1, 2022. [cited by applicant]
Errandonea, Itxaro et al., “Digital Twin for Maintenance: A literature review”, Computers in Industry 123 103316, Elsevier, Oct. 5, 2020. [cited by applicant]
Hubel et al., Coverage Control with Information Decay in Dynamic Environments, 2008. [cited by applicant]
McMahan, Communication-Efficient Learning of Deep Networks from Decentralized Data, version 3, Feb. 28, 2017, Artificial Intelligence and Statistics, pp. 1273-1282. [cited by applicant]
Tao, Fetal., “Chapter 2—Applications of Digital Twin”, Digital Driven Smart Manufacturing, Feb. 15, 2019, pp. 29-62, https://doi.org/10.1016/B978-0-12-817630-6.00002-3. [cited by applicant]
U.S. Appl. No. 18/049,700, filed Oct. 26, 2022, titled SO-MAP: a Semantic-Aware Algorithm for Optimizing the Representation Structure of Octomaps. [cited by applicant]
U.S. Appl. No. 18/050,274, filed Oct. 27, 2022, titled Orchestration of Action-Input Representations for Decision Making in Edge Environments. [cited by applicant]
Seong Jun Kim and Sung Ha Kang and Haomin Zhou, Optimal Sensor Positioning (Osp); A Probability Perspective Study, Apr. 19, 2016, arXiv: 1604.05391 (Year: 2016). [cited by applicant]