Operation of a self-check out surface area of a retail store

1 . A method of operating a plurality of self-checkout (SCO) terminals of a SCO environment, the method comprising:

receiving, by a control unit, a plurality of video frames captured using one or more video sensors installed in a pre-defined location from each of the plurality of SCO terminals;

obtaining, by the control unit, status data of each of the plurality of SCO terminals using an SCO unit communicably coupled to each of the plurality of SCO terminals,

wherein the status data identifies each of the plurality of SCO terminals as one of a locked or an active,

wherein the control unit is communicably coupled to the one or more video sensors and the SCO unit, wherein the control unit comprises a processing unit connected to a memory, wherein the memory comprises a set of instructions executed by the processing unit for:

determining one or more frames of interest from the plurality of video frames using a machine learning model, wherein the determination of the one or more frames of interest comprises:

detecting a primary subject of interest using a human classification module, wherein the human classification module is trained based on training data determined based on the plurality of video frames captured using the one or more video sensors;

classifying the detected primary subject of interest based on an age group of the primary subject of interest;

detecting one or more secondary subject of interest using an object detection module, post the detection of the primary subject of interest within a pre-defined distance from each of the plurality of SCO terminals;

determining location and time of detection of the primary subject of interest and the secondary subject of interest;

generating a motion trigger based on detection of change in the location of the primary subject of interest and the secondary subject of interest within the pre-defined distance of one of the plurality of SCO terminals using a motion detection unit; and

based on the generated motion trigger:

receiving transaction data from the SCO unit, wherein the transaction data comprises transactions received by scanning one or more objects at the respective one of the plurality of SCO terminals;

comparing the transaction data from the SCO unit with the detected one or more secondary subject of interest;

generating a non-scan event alert based on a mismatch in the comparison of the transaction data and the detected one or more secondary subject of interest; and

initiating a terminal lockdown procedure by identifying a predefined threshold of concurrently locked terminals and, in response to the processing unit determining that a number of concurrently locked terminal is less than said threshold, the processing unit generating an SCO LOCK signal, and sending said SCO LOCK signal to the specific SCO unit of the SCO terminal that is to be locked, and thereby interrupting continuous operation of the SCO terminal coupled to the SCO unit.

2 . The method of claim 1 , wherein the classification of the detected primary subject of interest based on the age group of the primary subject of interest comprises classifying the primary subject of interest as a SCO supervisor, a child, or an adult; and

wherein the secondary subject of interest is associated with the detected primary subject of interest, and wherein the secondary subject of interest comprises a shopping trolley, or stacked goods.

3 . The method of claim 1 further comprising:

identifying physical features of the primary subject of interest in one video frame of the plurality of video frames using a human tracking module;

forming, by the human tracking module, a person identification data based on the identified physical features of the primary subject of interest;

linking, by the human tracking module, a biometric signature of the primary subject of interest with the person identification data; and

storing, by the human tracking module, the person identification data in an internal repository of the human tracking module.

4 . The method of claim 3 further comprising:

forming, by the human tracking module, a query identification data of the primary subject of interest in a successive frame of the plurality of video frames;

comparing, by the human tracking module, the query identification data of the primary subject of interest with the stored person identification data in the internal repository; and

based on the comparison:

assigning, by the human tracking module, a new person identification data to the primary subject of interest in another video frame, when the query identification data fails to match with the person identification data.

5 . The method of claim 1 further comprising:

detecting, using an object recognition module, an object as the secondary subject of interest; and

forming, by the object recognition module, an object detection data associated with the detected object; and

processing, using a cropping module, the plurality of video frames based on the object detection data to crop a region in the plurality of video frames to form a product cropped region.

6 . The method of claim 5 , wherein the cropping module is further connected to an embedding unit, wherein the embedding unit is configured for:

forming an embedding data of the object present in an inventory of the SCO environment;

receiving the product cropped region;

creating a query embedding data in response to the received product cropped region; and

comparing the query embedding data and the embedding data; and

determining a match of the query embedding data with the embedding data if a similarity between the query embedding data with the embedding data exceeds a pre-defined threshold.

7 . The method of claim 1 , wherein the control unit further comprises an SCO supervisor locator module configured for:

receiving the non-scan event alert;

determining a distance of an SCO supervisor within the pre-defined distance from the SCO terminal for which the non-scan event alert is generated; and

locking the SCO terminal when the distance of the SCO supervisor is greater than the pre-defined distance for a pre-defined time interval.

8 . A system for operating a plurality of self-checkout (SCO) terminals of a SCO environment, the system comprising:

one or more video sensors installed in a pre-defined location from each of the plurality of SCO terminals configured to capture a plurality of video frames;

an SCO unit communicably coupled to each of the plurality of SCO terminals and configured to obtain status data of each of the plurality of SCO terminals,

wherein the status data identifies each of the plurality of SCO terminals as one of a locked or an active;

a control unit coupled to the one or more video sensors and the SCO unit, wherein the control unit comprises a processing unit connected to a memory, wherein the memory comprises a set of instructions executed by the processing unit configured to:

determine one or more frames of interest from the plurality of video frames using a machine learning model, wherein the determination of the one or more frames of interest comprises:

detect a primary subject of interest using a human classification module, wherein the human classification module is trained based on training data determined based on the plurality of video frames captured using the one or more video sensors;

classify the detected primary subject of interest based on an age group of the primary subject of interest; and

detect one or more secondary subject of interest using an object detection module, post appearance of the primary subject of interest within a pre-defined distance from each of the plurality of SCO terminals;

determine location and time of appearance of the primary subject of interest and the secondary subject of interest;

generate a motion trigger based on detection of change in location data of the primary subject of interest and the secondary subject of interest within the pre-defined distance of one of the plurality of SCO terminals, using a motion detection unit;

based on the generated motion trigger:

receive transaction data from the SCO unit, wherein the transaction data comprises transactions received by scanning one or more objects at the respective one of the plurality of SCO terminals;

compare the transaction data from the SCO unit with the detected secondary subject of interest using a non-scan event detector;

generate a non-scan event alert based on a mismatch in the comparison of the transaction data and the detected one or more secondary subject of interest;

the processing unit further configured to initiate a terminal lockdown procedure by identifying a predefined threshold of concurrently locked terminals and, in response to the processing unit determining that a number of concurrently locked terminal is less than said threshold, the processing unit generating an SCO LOCK signal, and sending said SCO LOCK signal to the specific SCO unit of the SCO terminal that is to be locked, and thereby interrupting continuous operation of the SCO terminal coupled to the SCO unit.

9 . The system of claim 8 , wherein the classification of the detected primary subject of interest based on the age group of the primary subject of interest comprises classifying the primary subject of interest as a SCO supervisor, a child, or an adult; and

wherein the secondary subject of interest is associated with the detected primary subject of interest, and wherein the secondary subject of interest comprises any one of a shopping trolley, or stacked goods.

10 . The system of claim 8 further comprising a human tracking module configured to:

identify physical features of the primary subject of interest in one video frame of the plurality of video frames;

form a person identification data based on the identified physical features of the primary subject of interest;

link a biometric signature of a primary subject of interest with the person identification data; and

store the person identification data in an internal repository of the human tracking module.

11 . The system of claim 10 , wherein the human tracking module is further configured to:

form a query identification data of the primary subject of interest in a successive frame of the plurality of video frames;

compare the query identification data of the primary subject of interest with stored person identification data in an internal repository; and

based on the comparison:

assign a new person identification data to the primary subject of interest in another video frame, when the query identification data fails to match with the person identification data.

12 . The system of claim 8 further comprising:

an object recognition module configured to:

detect an object as the secondary subject of interest; and

form an object detection data associated with the detected object; and

a cropping module configured to:

process the plurality of video frames based on the object detection data to crop a region in the plurality of video frames to form a product cropped region.

13 . The system of claim 12 , wherein the cropping module is further connected to an embedding unit, wherein the embedding unit is configured to:

form an embedding data of the object present in an inventory of the SCO environment;

receive the product cropped region;

create a query embedding data in response to the received product cropped region; and

compare the query embedding data and the embedding data; and

determine a match of the query embedding data with the embedding data if a similarity between the query embedding data with the embedding data exceeds a pre-defined threshold.

14 . The system of claim 8 , wherein the control unit further comprises an SCO supervisor locator module configured to:

receive the non-scan event alert;

determine a distance of an SCO supervisor within the pre-defined distance from the SCO terminal for which the non-scan event alert is generated; and

lock the SCO terminal when the distance of the SCO supervisor is greater than the pre-defined distance for a pre-defined time interval.

15 . A non-transitory computer-readable medium having stored thereon a set of instructions which when executed by a computer causes the computer to perform operations for operating a plurality of self-checkout (SCO) terminals of an SCO environment, comprising:

receiving a plurality of video frames captured using one or more video sensors installed in a pre-defined location from each of the plurality of SCO terminals;

obtaining status data of each of the plurality of SCO terminals using an SCO unit communicably coupled to each of the plurality of SCO terminals,

wherein the status data identifies each of the plurality of SCO terminals as one of a locked or an active,

wherein a control unit is communicably coupled to the one or more video sensors and the SCO unit, wherein the control unit comprises a processing unit connected to a memory, wherein the memory comprises a set of instructions executed by the processing unit for:

determining one or more frames of interest from the plurality of video frames using a machine learning model, wherein the determination of the one or more frames of interest comprises:

detecting a primary subject of interest using a human classification module, wherein the human classification module is trained based on training data determined based on the plurality of video frames captured using the one or more video sensors;

classifying the detected primary subject of interest based on an age group of the primary subject of interest;

detecting one or more secondary subject of interest using an object detection module, post the detection of the primary subject of interest and within a pre-defined distance from each of the plurality of SCO terminals;

determining location and time of detection of the primary subject of interest and the secondary subject of interest;

generating a motion trigger based on detection of change in the location of the primary subject of interest and the secondary subject of interest within the pre-defined distance of one of the plurality of SCO terminals using a motion detection unit; and

based on the generated motion trigger:

receiving transaction data from the SCO unit, wherein the transaction data comprises transactions received by scanning one or more objects at the respective one of the plurality of SCO terminals;

comparing the transaction data from the SCO unit with the detected one or more secondary subject of interest;

generating a non-scan event alert based on a mismatch in the comparison of the transaction data and the detected one or more secondary subject of interest; and

the processing unit further configured to initiate a terminal lockdown procedure by identifying a predefined threshold of concurrently locked terminals and, in response to the processing unit determining that a number of concurrently locked terminal is less than said threshold, the processing unit generating an SCO LOCK signal, and sending said SCO LOCK signal to the specific SCO unit of the SCO terminal that is to be locked, and thereby interrupting continuous operation of the SCO terminal coupled to the SCO unit.

16 . The non-transitory computer-readable medium of claim 15 further comprising:

identifying, using a human tracking module, physical features of the primary subject of interest in one video frame of the plurality of video frames;

forming, by the human tracking module, a person identification data based on the identified physical features of the primary subject of interest;

linking, by the human tracking module, a biometric signature of the primary subject of interest with the person identification data; and

storing, by the human tracking module, the person identification data in an internal repository of the human tracking module.

17 . The non-transitory computer-readable medium of claim 16 , wherein the human tracking module is further configured for:

forming a query identification data of the primary subject of interest in a successive frame of the plurality of video frames;

comparing the query identification data of the primary subject of interest with the stored person identification data in the internal repository; and

based on the comparison:

assigning a new person identification data to the primary subject of interest in another video frame, when the query identification data fails to match with the person identification data.

18 . The non-transitory computer-readable medium of claim 15 , wherein:

an object recognition module is configured for:

detecting an object as the secondary subject of interest; and

forming an object detection data associated with the detected object; and

a cropping module is configured for:

processing the plurality of video frames based on the object detection data to crop a region in the plurality of video frames to form a product cropped region.

19 . The non-transitory computer-readable medium of claim 18 , wherein the cropping module is further connected to an embedding unit, wherein the embedding unit is configured for:

forming an embedding data of the object present in an inventory of the SCO environment;

receiving the product cropped region;

creating a query embedding data in response to the received product cropped region; and

comparing the query embedding data and the embedding data; and

determining a match of the query embedding data with the embedding data if a similarity between the query embedding data with the embedding data exceeds a pre-defined threshold.

20 . The non-transitory computer-readable medium of claim 15 , wherein the control unit further comprises an SCO supervisor locator module configured for:

receiving the non-scan event alert;

determining a distance of an SCO supervisor within a pre-defined distance from the SCO terminal for which the non-scan event alert is generated; and

locking the SCO terminal when the distance of the SCO supervisor is greater than the pre-defined distance for a pre-defined time interval.