IP Library › Granted Patent US 12,568,271
Granted Patent B2
US 12,568,271 · App. 18/560,293 · Granted Mar 3, 2026

User aware automated content capture and augmentation

Inventors: Lu Liu (Conshohocken, PA); Dale Seed (Allentown, PA); Paul Dougherty (Springfield, PA); Sanghoon Kim (Chadds Ford, PA)
Assignee: INTERDIGITAL MADISON PATENT HOLDINGS, SAS
H04N21/4334H04N21/4312H04N21/44218H04N21/8547
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,568,271
App. No.
18/560,293
Granted
Mar 3, 2026
Kind
B2
Abstract

A method performed by a first electronic device includes receiving from a second electronic device an indication of an occurrence of one or more trigger conditions for content capture of multimedia content displayed on the first device, starting capture of the multimedia content in response to the trigger condition indication, collecting information from a third device, wherein the information from the third device is response information that is related to the multimedia content displayed on the first device, augmenting the captured multimedia content with the collected information from the third device for simultaneous display, and displaying the augmented multimedia content.

Claims (30)

1 . A method performed by a first electronic device, the method comprising:

receiving from a second electronic device an indication of an occurrence of one or more conditions for content capture of multimedia content displayed on the first electronic device;

starting capture of the multimedia content in response to the indication;

collecting response information from a third electronic device, wherein the response information is generated at the third electronic device as a reaction to the multimedia content displayed on the first electronic device;

augmenting the captured multimedia content with the collected response information from the third device for simultaneous display; and

providing for display of the augmented multimedia content.

2 . The method of claim 1 , wherein collecting response information from a third electronic device further comprises collecting information of a user's response to viewing the multimedia content displayed on the first electronic device.

3 . The method of claim 2 , wherein collecting information from the third device of response information comprises collecting any of notes, sensor readings, motions, video, or audio in response to the multimedia content displayed on the first electronic device.

4 . The method of claim 3 , wherein collecting information from the third device further comprises receiving timestamp information to align the response information with the captured multimedia content.

5 . The method of claim 1 , wherein receiving from a second electronic device an indication of an occurrence of one or more conditions comprises receiving any of: a user command, a user physical absence from a media consumption device (MCD), a user's status or activity, a user's input to the first or second electronic device, a user's predicted physical absence from the MCD, a user's environment, and a user's specified content occurrence.

6 . The method of claim 1 , wherein augmenting the captured multimedia content with the collected response information is performed according to an augmentation setting that defines a combination of the response information with the multimedia content.

7 . The method of claim 6 , wherein the combination of the response information with the multimedia content comprises an overlay of any of video and audio with the multimedia content.

8 . The method of claim 1 , wherein providing for display of the augmented multimedia content comprises providing a rendering on either the first electronic device or the third device for playback immediately or at a later time.

9 . The method of claim 1 , further comprising collecting feedback information from a user and updating the conditions based on the collected feedback.

10 . An electronic device, comprising at least one processor and circuitry configured to:

receive from a second electronic device an indication of an occurrence of one or more conditions for content capture of multimedia content displayed on the electronic device;

start capture of the multimedia content in response to the indication;

collect response information from a third electronic device, wherein the response information is generated at the third electronic device as a reaction to the multimedia content displayed on the electronic device;

augment the captured multimedia content with the collected response information from the third device for simultaneous display; and

provide for display of the augmented multimedia content.

11 . The electronic device of claim 10 , wherein the electronic device collects response information of a user's response to viewing the multimedia content displayed on the electronic device.

12 . The electronic device of claim 10 comprising one or more a digital television, head-mounted device, personal computer, a computer tablet device, and a smart phone.

13 . The electronic device of claim 10 , wherein the electronic device receives an indication of one or more conditions for content capture via any of a user command, a user physical absence from a media consumption device (MCD), a user's status or activity, a user's input to the electronic device or second electronic device, a user's predicted physical absence from the MCD, a user's environment, and a user's specified content occurrence.

14 . The electronic device of claim 10 , wherein the electronic device collects, from the third electronic device, response information comprising any of notes, sensor readings, motions, video, or audio in response to the multimedia content displayed on the electronic device.

15 . The electronic device of claim 14 , wherein the electronic device augments the captured multimedia content with the response information according to an augmentation setting that defines a combination of the response information with the multimedia content.

16 . The electronic device of claim 15 , wherein the augmentation setting comprises information of an overlay of any of video and audio with the multimedia content.

17 . The electronic device of claim 14 , wherein the electronic device further collects timestamp information to align the response information with the captured multimedia content.

18 . The electronic device of claim 10 , wherein the electronic device provides for display of the augmented multimedia content for rendering on either the electronic device or the third electronic device for playback in real-time or at a later time.

19 . The electronic device of claim 10 , wherein the electronic device is further configured to collect feedback information from a user and update the conditions based on the collected feedback.

20 . The electronic device of claim 10 , wherein the electronic device is configured such that the second or the third electronic device could be a sensor or sensing device, and the electronic device and the second electronic device are combined into a single device or the second electronic device and the third electronic device are combined into a single device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 16, 2024
From: INTERDIGITAL PATENT HOLDINGS, INC.
To: INTERDIGITAL MADISON PATENT HOLDINGS, SAS
Reel/Frame 068916/0709 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2023
From: LIU, LU; SEED, DALE; DOUGHERTY, PAUL; KIM, SANGHOON
To: INTERDIGITAL PATENT HOLDINGS, INC.
Reel/Frame 065553/0539 →
Continuity (2)
Provisional Application 63190518 · May 19, 2021
Related Publication 20240236404A1 · Jul 11, 2024
References Cited (22)
US 9336268B1 · Moudy · 2016 [cited by examiner]
US 20030206710A1 · Ferman · 2003 [cited by examiner]
US 20090150939A1 · Drucker · 2009 [cited by examiner]
US 20100171846A1 · Wood et al. · 2010 [cited by applicant]
US 20150220537A1 · Cosgrove et al. · 2015 [cited by applicant]
US 20150223731A1 · Sahin · 2015 [cited by examiner]
US 20160055879A1 · Arai et al. · 2016 [cited by applicant]
US 20170142481A1 · Caruana · 2017 [cited by examiner]
US 20170195435A1 · Sarukkai et al. · 2017 [cited by applicant]
US 20170339020A1 · Khanna · 2017 [cited by examiner]
US 20180315076A1 · Andreou · 2018 [cited by examiner]
US 20200082436A1 · Garcia · 2020 [cited by applicant]
US 20210014579A1 · Newell et al. · 2021 [cited by applicant]
US 20210150219A1 · Karoui et al. · 2021 [cited by applicant]
US 20210274244A1 · Wondra · 2021 [cited by examiner]
US 20220351218A1 · Karri · 2022 [cited by examiner]
EP 3726344A1 · 2020 [cited by applicant]
WO WO2021046324A1 · 2021 [cited by applicant]
Steve Hodges et al, “SenseCam: a Retrospective Memory Aid”, Jan. 1, 2006, arxiv.org, Cornell University Library, 201 Olin Library Cornell University, Ithaca, NY 14853, pp. 177-193. [cited by applicant]
Kyungmo Park: “The proposed update requirement of NBMP”, 122. MPEG Meeting; Apr. 16, 2018-Apr. 20, 2018; San Diego; Motion Picture Expert Group or ISO/IEC JTC1/SC29/WH11), No. m42784, pp. 1, 5, 10, 11, 16, 17, 27, 28. [cited by applicant]
“Text on ISO/IEC FDIS 23001-Part 13: Media Orchestration”, 121. MGEG Meeting; Jan. 22, 2018-Jan. 26, 208; Gwangju; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG110, No. n17415 Apr. 21, 2018, pp. 12, 14, 15, 21, 2… [cited by applicant]
J. Sedmidubsky and P. Zezula, “Augmenting Spatio-Temporal Human Motion Data for Effective 3D Action Recognition,” 2019 IEEE International Symposium on Multimedia (ISM), 2019, pp. 204-207. [cited by applicant]