Controlled-environment facility video communications monitoring system
Video communications of residents of a controlled-environment facility are monitored to detect instances of non-verbal communications within the video, such as the display of written messages and hand signals. Residents may participate in live video visitation sessions with non-residents. Video recordings are generated for each video feed of the live video visitation. During the live video visitation, indications of non-verbal communications displayed in the live video are detected. If indications of non-verbal communications are detected in the live video, the recorded video is annotated to specify the locations of the detected indications in the recording. Upon completion of the video visitation session, post-processing of the recorded video detects additional indications of displayed non-verbal communication. The additional indications may be detected at proximal locations to the annotations and may be detected based on sampling frames of the recording to detect changes in the color balance indicating the display of a message surface.
1. A system for monitoring video communications provided to a resident of a controlled-environment facility, the system comprising:
a communication system configured to:
host a live video visitation between the resident and one or more non-residents, wherein the video visitation comprises a plurality of live video feeds, and wherein the resident is associated with a privilege classification by the controlled-environment facility; and
a video feed recording system configured to:
generate a plurality of video recordings, each video recording corresponding to a video feed of the plurality of live video feeds; and
a video processing system configured to:
sample a plurality of frames of a first video feed of the plurality of video feeds to detect indications of a non-verbal communication displayed in the first video feed, wherein the frames are sampled at a frequency determined based on the privilege classification of the resident; and
annotate a first video recording corresponding to the first video feed, wherein the annotations specify the locations of the detected indications of the non-verbal communication displayed in the first video recording.
2. The system of claim 1 , wherein a first sampling frequency is utilized when the resident is associated with a lower-risk privilege classification and wherein a second sampling frequency is utilized when the resident is associated with a higher-risk privilege classification, and wherein the second sampling frequency is higher than the first sampling frequency.
3. The system of claim 1 , wherein the sampling frequency is further determined based on a state of resource exhaustion by the video processing system.
4. The system of claim 1 , wherein the video processing system is further configured to determine a color balance for a plurality of the frames sampled from the first video feed and further configured to determine a color balance baseline for the first video feed based on the color balances determined for the plurality of the frames.
5. The system of claim 4 , wherein the video processing system is further configured to detect sampled frames with a color balance that deviates from the color balance baseline.
6. The system of claim 5 , wherein the video processing system is further configured to detect indications of the non-verbal communication in the first live video feed based on the detected deviations from the color balance baseline.
7. The system of claim 5 , wherein the video processing system is further configured to modify the sampling rate based on the detected deviations from the color balance baseline.
8. The system of claim 6 , wherein the indications of the non-verbal communication are detected based on a rate of the deviation from the color balance baseline.
9. A method for monitoring video communications of a resident of a controlled-environment facility, the method comprising:
hosting a live video visitation between the resident and one or more non-residents, wherein the video visitation comprises a plurality of live video feeds, and wherein the resident is associated with a privilege classification by the controlled-environment facility;
generating a plurality of video recordings, each video recording corresponding to a live video feed of the plurality of live video feeds;
sampling a plurality of frames of a first recorded video feed to detect indications of a non-verbal communication displayed in the first video feed, wherein the frames are sampled at a frequency determined based on the privilege classification of the resident; and
annotating the first video recording, wherein the annotations specify the locations of the detected indications of the non-verbal communication displayed in the first video recording.
10. The method of claim 9 , wherein a first sampling frequency is utilized when the resident is associated with a lower-risk privilege classification and wherein a second sampling frequency is utilized when the resident is associated with a higher-risk privilege classification, and wherein the second sampling frequency is higher than the first sampling frequency.
11. The method of claim 9 , wherein the sampling frequency is further determined based on a state of resource exhaustion by a video processing system.
12. The method of claim 9 , further comprising:
determining a color balance for a plurality of the frames sampled from the first recorded video feed; and
determining a color balance baseline for the first recorded video feed based on the color balances determined for the plurality of the frames.
13. The method of claim 12 , further comprising detecting sampled frames with a color balance that deviates from the color balance baseline.
14. The method of claim 13 , wherein the indications of the non-verbal communication in the first live video feed are detected based on the detected deviations from the color balance baseline.
15. The method of claim 13 , further comprising:
modifying the sampling rate based on the detected deviations from the color balance baseline.
16. The method of claim 14 , wherein the indications of the non-verbal communication are detected based on a rate of the deviation from the color balance baseline.
17. A computer-readable storage device having program instructions stored thereon for monitoring video communications of a resident of a controlled-environment facility, upon execution by one or more processors, the program instructions cause the one or more processors to:
host a live video visitation between the resident and one or more non-residents, wherein the video visitation comprises a plurality of live video feeds, and wherein the resident is associated with a privilege classification by the controlled-environment facility;
generate a plurality of video recordings, each video recording corresponding to a live video feed of the plurality of live video feeds;
sample a plurality of frames of a first video feed of the plurality of video feeds to detect indications of a non-verbal communication displayed in the first live video feed, wherein the frames are sampled at a frequency determined based on the privilege classification of the resident; and
annotate a first video recording corresponding to the first live video feed, wherein the annotations specify the locations of the detected indications of the non-verbal communication displayed in the first video recording.
18. The computer-readable storage device of claim 17 , wherein a first sampling frequency is utilized when the resident is associated with a lower-risk privilege classification and wherein a second sampling frequency is utilized when the resident is associated with a higher-risk privilege classification, and wherein the second sampling frequency is higher than the first sampling frequency.
19. The computer-readable storage device of claim 18 , the program instructions further causing the one or more processors to: determine a color balance for a plurality of the frames sampled from the first live video feed and determine a color balance baseline for the first live video feed based on the color balances determined for the plurality of the frames.
20. The computer-readable storage device of claim 17 , wherein the indications of the non-verbal communication in the first live video feed are detected based on detected deviations in the color balances of the plurality of sampled frames from the color balance baseline.