Method and apparatus for auditing transaction activity in retail and other environments using visual recognition

1. A method of detecting fraudulent transactions, the method comprising:

obtaining video data originating from at least one video camera that captures video of a specified area;

obtaining transaction data of at least one transaction occurring in the specified area;

correlating the video data originating from the at least one video camera to the transaction data to identify specific video data captured during occurrence of the at least one transaction occurring in the specified area;

analyzing the specific video data to detect whether a customer is present in the specified area during occurrence of the at least one transaction; and

in response to identifying absence of the customer in the specified area, flagging the at least one transaction as suspicious of fraud.

2. The method of claim 1 , wherein correlating the video data includes performing a video extraction process that produces a video clip by extracting from the video data a corresponding segment of video associated with a time range of the at least one transaction occurring in the specified area.

3. The method of claim 2 , wherein analyzing the specific video data comprises:

transmitting the video clip over a network;

displaying the video clip on a graphical user interface; and

receiving input, via the graphical user interface, indicating suspicious activity.

4. The method of claim 3 , wherein receiving input, via the graphical user interface, indicating suspicious activity includes receiving input, via the graphical user interface, indicating absence of the customer in the specified area.

5. The method of claim 1 , wherein correlating the video data includes performing a video extraction process that produces an individual image of the specified area, the individual image representative of the occurrence of the at least one transaction.

6. The method of claim 5 , wherein performing the video extraction process that produces the individual image of the specified area includes producing the individual image as a time-composite image, the time-composite image being produced by overlapping images of objects extracted from periodically sampled frames captured during the occurrence of the at least one transaction.

7. The method of claim 5 , wherein performing the video extraction process that produces the individual image of the specified area includes producing the individual image as a time-composite image, the time-composite image being produced by positioning multiple frames adjacent to each other, the multiple frames captured during the occurrence of the at least one transaction.

8. The method of claim 5 , wherein analyzing the specific video data comprises:

transmitting the individual image over a network;

displaying the individual image on a graphical user interface; and

receiving input, via the graphical user interface, indicating absence of the customer in the specified area.

9. The method of claim 1 , wherein analyzing the specific video data to detect whether the customer is present in the specified area during occurrence of the at least one transaction includes using an automated object detection process that automatically identifies absence of the customer in the specified area based on image analysis.

10. The method of claim 1 , wherein analyzing the specific video data comprises:

transmitting the specific video data over a network;

displaying the specific video data on a graphical user interface; and

receiving manual input, via the graphical user interface, indicating absence of the customer in the specified area.

11. The method of claim 10 , wherein obtaining video data originating from the at least one video camera that captures video of the specified area includes identifying a region of interest indicating an area relative to a transaction counter where the customer is expected to be located.

12. The method of claim 1 , further comprising:

obtaining no-sale transaction data of a no-sale transaction occurring in the specified area, the no-sale transaction data indicating a cash drawer of a retail register being opened;

correlating the video data originating from the at least one video camera to the no-sale transaction data to identify specific video data captured during occurrence of the no-sale transaction occurring in the specified area;

analyzing the specific video data to detect whether a customer is present in the specified area during occurrence of the no-sale transaction; and

in response to identifying absence of the customer in the specified area, flagging the no-sale transaction as suspicious of fraud.

13. The method of claim 1 , further comprising:

wherein obtaining transaction data includes obtaining transaction data of at least one refund transaction occurring in the specified area;

wherein correlating the video data includes identifying specific video data captured during occurrence of the at least one refund transaction occurring in the specified area;

wherein analyzing the specific video data includes detecting whether a customer is present in the specified area during occurrence of the at least one refund transaction; and

wherein the response to identifying absence of the customer in the specified area includes flagging the at least one refund transaction as suspicious of fraud.

14. A method of detecting fraudulent transactions, the method comprising:

obtaining video data originating from at least one video camera that captures video of a specified area;

obtaining transaction data of a voided transaction occurring in the specified area;

correlating the video data originating from the at least one video camera to the transaction data to identify specific video data captured during occurrence of the voided transaction occurring in the specified area;

analyzing the specific video data to detect whether the customer leaves the specified area with merchandise after the occurrence of the voided transaction; and

in response to identifying the customer leaving the specified area with merchandise after the voided transaction and identifying an absence of a subsequent corresponding purchase transaction, flagging the voided transaction as suspicious of fraud.

15. A method of detecting fraudulent refund transactions, the method comprising:

obtaining video data originating from at least one video camera that captures video of a specified area;

obtaining refund transaction data of at least one refund transaction occurring in the specified area;

correlating the video data originating from the at least one video camera to the refund transaction data to identify specific video data captured during occurrence of the at least one refund transaction occurring in the specified area;

analyzing the specific video data to detect whether merchandise is present in the specified area during occurrence of the at least one refund transaction; and

in response to identifying absence of merchandise in the specified area, flagging the at least one refund transaction as suspicious of fraud.

16. The method of claim 15 , wherein correlating the video data includes performing a video extraction process that produces a video clip by extracting from the video data corresponding segments of video associated with a time range of the at least one refund transaction occurring in the specified area.

17. The method of claim 16 , wherein analyzing the specific video data comprises:

transmitting the video clip over a network;

displaying the video clip on a graphical user interface; and

receiving manual input, via the graphical user interface, indicating absence of the merchandise in the specified area.

18. The method of claim 15 , wherein correlating the video data includes performing a video extraction process that produces an individual image of the specified area, the individual image representative of the occurrence of the at least one refund transaction.

19. The method of claim 18 , wherein analyzing the specific video data comprises:

transmitting the individual image over a network;

displaying the individual image on a graphical user interface; and

receiving input, via the graphical user interface, indicating absence of the merchandise in the specified area.

20. The method of claim 15 , wherein analyzing the specific video data to detect whether the merchandise is present in the specified area during occurrence of the at least one refund transaction includes using an automated object detection process that automatically identifies absence of the merchandise in the specified area based on image analysis.

21. The method of claim 15 , wherein analyzing the specific video data comprises:

transmitting the specific video data over a network;

displaying the specific video data on a graphical user interface; and

receiving input, via the graphical user interface, indicating absence of the merchandise in the specified area.

22. A computer system for detecting fraudulent refund transactions, the computer system comprising:

a processor; and

a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the system to perform the operations of:

obtaining video data originating from at least one video camera that captures video of a specified area;

obtaining transaction data of at least one refund transaction occurring in the specified area;

correlating the video data originating from the at least one video camera to the transaction data to identify specific video data captured during occurrence of the at least one refund transaction occurring in the specified area;

analyzing the specific video data to detect whether a customer is present in the specified area during occurrence of the at least one refund transaction; and

in response to identifying absence of the customer in the specified area, flagging the at least one refund transaction as suspicious of fraud.