IP Library Granted Patent US 11,937,008
Granted Patent B2
US 11,937,008 · App. 17/184,583 · Granted Mar 19, 2024

Method and system for content aware dynamic image framing

Inventors: Naveed Alam (Cupertino, CA); John Zhang (San Jose, CA); Aurangzeb Khan (Portola Valley, CA)
Assignee: GN AUDIO A/S
H04N5/272G06V20/40G06V30/32G06V40/107G09B5/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,937,008
App. No.
17/184,583
Granted
Mar 19, 2024
Kind
B2
Abstract

Embodiments of the present invention disclose techniques for outputting content aware video based on at least one a video application use case. The technique recognizes objects associated with the use case and performs enhancement of the objects based on content-aware rules, and composes at least some of the objects in an output frame based on on content-aware frame composition templates. Embodiments of the present invention also disclose systems for implementing the above techniques.

Claims (39)

1. A method for framing video content, comprising:

receiving at least one input video stream from at least one source;

applying at least one image analysis technique to recognize objects in each input video stream;

extracting at least some of the recognized objects;

composing an output frame comprising the at least some of the recognized objects, wherein composing the output frame comprises selecting a content aware framing template that includes a plurality of dedicated zones to place the at least some of the recognized objects, wherein each object of the at least some of the recognized objects is associated with a respective dedicated zone of the plurality of dedicated zones in the selected content aware framing template;

placing each object into the respective dedicated zone in the selected content aware framing template; and

outputting the output frame to video client device.

2. The method of claim 1 , wherein each stream is generated by a camera configured to capture dedicated aspects of a video application use case.

3. The method of claim 1 , comprising receiving a plurality of input video streams, each orientated to capture a different aspect of a video application use case.

4. The method of claim 1 , wherein at least one video analysis technique is selected from a group consisting of artificial intelligence (AI), machine learning (ML), and deep learning.

5. The method of claim 1 , further comprising applying at least one content-aware modification to at least some of the recognized objects.

6. The method of claim 4 , wherein for a video application use case in a form of a remote training session by a training instructor, further comprising:

tuning the at least one video analysis technique to recognize a instructor's hand, and any objects held in therein; and

modifying any recognized object in the instructor's hand based on a content-aware rule.

7. The method of claim 6 , wherein the modification is selected from a group consisting of handwriting sharpening, object contrast enhancement, image straightening; image magnification; white board sharpening; and object extraction and placement in the output frame, independently of the instructor.

8. The method of claim 1 , wherein, for a video application use case in a form of a teacher-student remote teaching session with notebook-based teaching, further comprising:

tuning at least one video analysis technique to recognize objects on a desk of a student; and

modifying any recognized object on the desk based on a content-rule.

9. The method of claim 8 , the modification is selected from a group comprising extracting a notebook on the desk for presentation in the output frame independently of said desk; and at least one image enhancement technique to the notebook prior to presentation.

10. The method of claim 8 , wherein the image enhancement technique comprises handwriting recognition to recognize the handwriting of the student.

11. A system for framing video content, comprising:

a mechanism for receiving at least one input video stream from at least one source;

an object detection module configured to apply at least one image analysis technique to recognize objects in each input video stream;

extracting at least some of the recognized objects;

a frame composition module configured to compose an output frame comprising the at least some of the recognized objects, wherein composing the output frame comprises selecting a content aware framing template that includes a plurality of dedicated zones to place the at least some of the recognized objects, wherein each object of the at least some of the recognized objects is associated with a respective dedicated zone of the plurality of dedicated zones in the selected content aware framing template;

placing each object into the respective dedicated zone in the selected content aware framing template; and

a mechanism configured to output the output frame to video client device.

12. The system of claim 11 , further comprising at least one camera configured to capture a dedicated aspect of a video application use case.

13. The system of claim 11 , comprising at least one video analysis technique provisioned in memory and selected from a group consisting of artificial intelligence (AD, machine learning (ML), and deep learning.

14. The system of claim 11 , further comprising a mechanism to apply at least one content-aware modification to at least some of the recognized objects.

15. The system of claim 13 , wherein for a video application use case in a form of a remote training session by a training instructor, further comprising:

the at least one video analysis technique tuned to recognize a instructor's hand, and any objects held in therein; and

a mechanism to modify any recognized object in the instructor's hand based on a content-aware rule.

16. The system of claim 15 , comprising a plurality of modifications is selected from a group consisting of handwriting sharpening, object contrast enhancement, image straightening; image magnification; white board sharpening; and object extraction and placement in the output frame, independently of the instructor.

17. The system of claim 11 , wherein, for a video application use case in a form of a teacher-student remote teaching session with notebook-based teaching, further comprising:

the at least one video analysis techniques tuned to recognize objects on a desk of a student; and

a mechanism to modify any recognized object on the desk based on a content-rule.

18. The system of claim 17 , comprising at least one modification selected from a group comprising extracting a notebook on the desk for presentation in the output frame independently of said desk; and at least one image enhancement technique to the notebook prior to presentation.

19. The system of claim 17 , wherein the image enhancement technique comprises handwriting recognition to recognize the handwriting of the student.

Assignments (3)
MERGER Recorded Mar 30, 2026
From: GN AUDIO A/S
To: GN HEARING A/S
Reel/Frame 075299/0225 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2022
From: ALTIA SYSTEMS, INC.
To: GN AUDIO A/S
Reel/Frame 058935/0768 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2021
From: ALAM, NAVEED; ZHANG, JOHN; KHAN, AURANGZEB
To: ALTIA SYSTEMS, INC.
Reel/Frame 055463/0856 →
Cited By (1)
US 12,323,727