Granted Patent B1

US 12,524,254 · App. 18/957,719 · Granted Jan 13, 2026

Inventors: Blake Browder (Dallas, TX); Joy Figarsky (Little Rock, AR)

Assignee: BH Operations, LLC

G06F9/451G06V10/764G06V10/82G06V20/50

Abstract

Claims (77)

1 . An apparatus comprising a graphical user interface (GUI) for adjusting its display based on image data, the apparatus comprising:

a machine vision system, comprising a camera, wherein the camera is configured to capture images of real-world objects within a facility;

at least a processor; and

a memory coupled to the at least a processor, wherein the memory contains instructions configuring the at least a processor to:

receive input data objects using the machine vision system, wherein the input data objects comprise;

the images of the real-world objects within a facility; and

geospatial coordinates, derived from GPS signals, comprising a user's location;

and wherein receiving the input data objects further comprises:

cross-referencing the image data with applicable regulatory mandates by performing real-time safety assessments and compliance checks based on visual inputs of the image data, wherein cross-referencing the image data with applicable regulatory mandates comprises:

mapping, using the geospatial coordinates, a location of a facility to a jurisdiction; and

locating, in a regulatory database, one or more regulations as a function of the jurisdiction using a look up algorithm;

process the image data using the machine vision system to extract and classify components of the visual inputs to generate structured environmental data;

generate structured environmental data based on the processed image data; and

modify a GUI based on the input data objects and the structured environmental data, wherein the GUI modification comprises automatically generating context-sensitive command inputs that trigger analysis routines within the GUI to execute a simulation module to model hypothetical modifications based on the input data objects and structured environmental data.

2 . The apparatus of claim 1 , wherein modifying the GUI further comprises displaying a heatmap visualization of a facility, using color-coded indicators to represent areas of compliance and non-compliance, based on a safety analysis, wherein generating the heatmap comprises:

calculating a plurality of compliance scores based on the structured environmental data; and

employing a grid-based layout algorithm to map a layout of the facility layout into a structured grid format, wherein each cell in the grid is assigned a compliance score.

3 . The apparatus of claim 1 , wherein processing the image data using the machine vision system to extract and classify components of the visual inputs to generate the structured environmental data, comprising:

employing a feature extraction algorithm to identify and isolate visual features within the image data;

training a classification model configured to receive at least the image data and the visual features and classify each visual input of the image data to a predefined class of compliant materials or non-compliant; and

converting the classified visual inputs into the structured environmental data by assigning the classified visual inputs to a set of attributes organized into a structured format.

4 . The apparatus of claim 3 , wherein the feature extraction algorithm comprises a convolutional neural network configured to apply convolutional filters to detect low-level visual features within the image data.

5 . The apparatus of claim 1 , wherein processing the image data comprises employing a post-processing algorithm configured to delineate boundaries of objects in the image data.

6 . The apparatus of claim 1 , wherein processing the image data comprises filtering noise by employing an autoencoder trained on a plurality of datasets encompassing high-quality and noisy images of facility environments annotated to highlight safety elements.

7 . The apparatus of claim 1 , wherein the at least a processor is further configured to implement a geographical mapping module configured to:

receive the input data objects;

employ a geospatial analysis algorithm to map a facility's location based on the input data objects;

classifying the facility's location to regulatory requirements; and

output regulatory requirements based on the facility's location to the simulation module.

8 . The apparatus of claim 1 , wherein the at least a processor is further configured to receive additional input data objects by implementing Application Programming Interfaces (APIs) configured to:

send a request to an external endpoint based on a geographic location of a facility; and

obtain structured data from third-party platforms for further processing in execution of the simulation module.

9 . The apparatus of claim 1 , wherein cross-referencing the image data with applicable regulatory mandates further comprises cross-referencing a material of the input data objects against the one or more regulations from the regulatory database using a regulation-to-material matching module.

10 . The apparatus of claim 1 , wherein the memory contains instructions further configuring the at least a processor to collect regulation data using a data crawler, wherein collecting the regulation data comprises:

automatically retrieving the regulation data from a plurality of websites;

processing the regulation data, wherein processing the regulation data comprises:

extracting a first set of structured data of the regulation data; and

processing unstructured data using at least an optical character recognition module to generate a second set of structured data, wherein the optical character recognition module is configured to generate machine-readable text from an image format; and

storing the first and second sets of structured data in a database.

11 . A method for adjusting a display of a graphical user interface (GUI) based on image data, the method comprising:

receiving, by a computing device, input data objects using a machine vision system, wherein:

the machine vision system comprises a camera, wherein the camera is configured to capture images of real-world objects within a facility; and

the input data objects comprise:

the images of the real-world objects within a facility; and

geospatial coordinates, derived from GPS signals, comprising a user's location;

and wherein receiving the input data objects further comprises:

cross-referencing the image data with applicable regulatory mandates by performing real-time safety assessments and compliance checks based on visual inputs of the image data, wherein cross-referencing the image data with applicable regulatory mandates comprises:

mapping, using the geospatial coordinates, a location of a facility to a jurisdiction; and

locating, in a regulatory database, one or more regulations as a function of the jurisdiction using a look up algorithm;

processing, by the computing device, the image data using the machine vision system to extract and classify components of the visual inputs to generate structured environmental data;

generating, by the computing device, structured environmental data based on the processed image data; and

modifying, by the computing device, a GUI based on the input data objects and the structured environmental data, wherein modifying the GUI comprises automatically generating context-sensitive command inputs that trigger analysis routines within the GUI to execute a simulation module to model hypothetical modifications based on the input data objects and structured environmental data.

12 . The method of claim 11 , wherein cross-referencing the image data with applicable regulatory mandates further comprises cross-referencing a material of the input data objects against the one or more regulations from the regulatory database using a regulation-to-material matching module.

13 . The method of claim 12 , wherein modifying the GUI further comprises displaying a heatmap visualization of a facility, using color-coded indicators to represent areas of compliance and non-compliance based on a safety analysis, wherein generating the heatmap comprises:

calculating a plurality of compliance scores based on the structured environmental data; and

employing a grid-based layout algorithm to map a layout of the facility layout into a structured grid format, wherein each cell in the grid is assigned a compliance score.

14 . The method of claim 12 , wherein processing the image data using the machine vision system to extract and classify components of the visual inputs to generate the structured environmental data comprises:

employing a feature extraction algorithm to identify and isolate visual features within the image data;

training a classification model configured to receive at least the image data and the visual features and classify each visual input of the image data to a predefined class of compliant materials or non-compliant; and

converting the classified visual inputs into the structured environmental data by assigning the classified visual inputs to a set of attributes organized into a structured format.

15 . The method of claim 14 , wherein the feature extraction algorithm comprises a convolutional neural network configured to apply convolutional filters to detect low-level visual features within the image data.

16 . The method of claim 12 , wherein processing the image data comprises employing a post-processing algorithm configured to delineate boundaries of objects in the image data.

17 . The method of claim 12 , wherein processing the image data comprises filtering noise by employing an autoencoder trained on a plurality of datasets encompassing high-quality and noisy images of facility environments annotated to highlight safety elements.

18 . The method of claim 12 , wherein modifying the GUI further comprises implementing, by the computing device, a geographical mapping module configured to:

receive the input data objects;

employ a geospatial analysis algorithm to map a facility's location based on the input data objects;

classifying the facility's location to regulatory requirements; and

output regulatory requirements based on the facility's location to the simulation module.

19 . The method of claim 12 , wherein receiving the input data objects further comprises receiving additional input data objects by implementing Application Programming Interfaces (APIs) configured to:

send a request to an external endpoint based on a geographic location of a facility; and

obtain structured data from third-party platforms for further processing in execution of the simulation module.

20 . The method of claim 12 , further comprising collecting, by the at least a processor, regulation data using a data crawler, wherein collecting the regulation data comprises:

automatically retrieving the regulation data from a plurality of websites; and

processing the regulation data, wherein processing the regulation data comprises:

extracting a first set of structured data of the regulation data;

processing unstructured data using at least an optical character recognition module to generate a second set of structured data, wherein the optical character recognition module is configured to generate machine-readable text from an image format; and

storing the first and second sets of structured data in a database.

Assignments (3)

References Cited (15)