IP Library Granted Patent US 11,949,948
Granted Patent B2
US 11,949,948 · App. 17/317,079 · Granted Apr 2, 2024

Playback control based on image capture

Inventor: Brant Candelore (San Diego, CA)
Assignee: SONY GROUP CORPORATION
H04N21/44218H04N21/42203H04N21/4223H04N21/47217H04N21/6587
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,949,948
App. No.
17/317,079
Granted
Apr 2, 2024
Kind
B2
Abstract

An electronic device is provided for control of playback based on image capture. The electronic device includes circuitry. The circuitry is communicatively coupled to an imaging apparatus and a rendering device that plays content. The circuitry acquires, from the imaging apparatus, one or more images of a user of the electronic device and a physical space associated with the user. The circuitry detects a lip movement of the user based on the acquired one or more images. The circuitry determines whether the user is in a conversation based on the detected lip movement of the user. The circuitry controls playback of the content on the rendering device based on the determination.

Claims (99)

1. An electronic device, comprising:

circuitry communicatively coupled to an imaging apparatus and a rendering device that plays content, wherein the circuitry is configured to:

acquire, from the imaging apparatus, one or more images of a first user of the electronic device and a physical space associated with the first user;

receive a user input that includes a settling time of the first user, wherein the settling time indicates a time required by the first user to settle after an entry into the physical space;

detect a lip movement of the first user based on the acquired one or more images;

determine whether the first user is in a conversation based on the detected lip movement of the first user;

control playback of the content on the rendering device based on the determination;

detect a movement of the first user;

compare the detected movement of the first user with a threshold movement, wherein the threshold movement is a specific amount of movement in the physical space;

pause the playback of the content on the rendering device based on a result of the comparison that the detected movement of the first user is larger than the threshold movement;

detect, based on the acquired one or more images, the entry of the first user into the physical space;

resume the playback of the content on the rendering device after the settling time elapses from the detection of the entry of the first user;

receive, from a user device, a trigger signal based on an incoming notification at the user device;

pause, in response to the trigger signal, the playback of the content on the rendering device for a specific period;

detect, in response to the trigger signal, a posture of the first user based on the acquired one or more images; and

determine, based on the detected posture, whether to resume or continue to pause after the specific period, the playback of the content on the rendering device.

2. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect the posture of the first user based on the acquired one or more images;

determine that the first user is in the conversation with a second user in the physical space, based on the detected posture of the first user; and

pause the playback of the content on the rendering device based on the determination that the first user is in the conversation with the second user.

3. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect the posture of the first user based on the acquired one or more images;

determine that the first user is in the conversation on the user device, based on the detected posture of the first user; and

pause the playback of the content on the rendering device based on the determination that the first user is in the conversation on the user device.

4. The electronic device according to claim 1 , wherein the circuitry is further configured to:

determine an end of the conversation of the first user based on the lip movement of the first user; and

resume the playback of the content on the rendering device based on the end of the conversation of the first user.

5. The electronic device according to claim 4 , wherein the circuitry is further configured to resume the playback of the content after a specific time from the end of the conversation of the first user.

6. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect a second user in the physical space based on the acquired one or more images; and

control the playback of the content on the rendering device based on the detected second user after a specific time from the detection of the second user.

7. The electronic device according to claim 6 , wherein the circuitry is further configured to:

detect the lip movement of the second user based on the acquired one or more images; and

pause the playback of the content on the rendering device based on the lip movement of the second user.

8. The electronic device according to claim 1 , wherein

the detected posture indicates one of:

the user device is held close to a head of the first user, or

the user device is held in a hand of the first user.

9. The electronic device according to claim 8 , wherein the incoming notification corresponds to at least one of an incoming call, an incoming text message, or an incoming email on the user device.

10. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect one of an infant, an infant bed, infant playpen, or an infant carrier in the physical space based on the acquired one or more images;

receive audio information associated with the infant; and

pause the playback of the content on the rendering device based on the audio information associated with the infant.

11. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect a gaze of the first user;

determine the first user pays attention to the content on the rendering device based on the detected gaze of the first user; and

resume the playback of the content on the rendering device based on the determination the first user pays attention to the content.

12. The electronic device according to claim 1 , wherein the content is at least one of a recorded content, a streaming content, an audio content, or a video content.

13. The electronic device according to claim 1 , wherein the circuitry is further configured to:

identify the first user based on the acquired one or more images;

determine a viewing pattern associated with the identified first user based on a user history associated with the identified first user; and

pause or resume the playback of the content on the rendering device based on the determined viewing pattern.

14. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect the posture of the first user based on the acquired one or more images;

determine a state of the first user based on the detected posture of the first user; and

control one of a volume of the content or the playback of the content based on the state of the first user.

15. The electronic device according to claim 1 , wherein the circuitry is further configured to:

determine an exit of the first user from the physical space; and

pause the playback of the content on the rendering device based on the determined exit of the first user from the physical space.

16. An electronic device, comprising:

circuitry communicatively coupled to an imaging apparatus and a rendering device that plays content, wherein the circuitry is configured to:

acquire, from the imaging apparatus, one or more images of a first user of the electronic device and a physical space associated with the first user;

receive a user input that includes a settling time of the first user, wherein the settling time indicates a time required by the first user to settle after an entry into the physical space;

detect a lip movement of the first user and a posture of the first user based on the acquired one or more images;

determine whether the first user is in a conversation based on the detected lip movement and the detected posture of the first user;

control playback of the content on the rendering device based on the determination;

detect a movement of the first user;

compare the detected movement of the first user with a threshold movement, wherein the threshold movement is a specific amount of movement in the physical space;

pause the playback of the content on the rendering device based on a result of the comparison that the detected movement of the first user is larger than the threshold movement;

detect, based on the acquired one or more images, the entry of the first user into the physical space;

resume the playback of the content on the rendering device after the settling time elapses from the detection of the entry of the first user;

receive, from a user device, a trigger signal based on an incoming notification at the user device;

pause, in response to the trigger signal, the playback of the content on the rendering device for a specific period;

detect, in response to the trigger signal, a posture of the first user based on the acquired one or more images; and

determine, based on the detected posture, whether to resume or continue to pause after the specific period, the playback of the content on the rendering device.

17. A method, comprising:

in an electronic device comprising circuitry communicatively coupled to an imaging apparatus and a rendering device that plays content:

acquiring, from the imaging apparatus, one or more images of a first user of the electronic device and a physical space associated with the first user;

receiving, by the circuitry, a user input that includes a settling time of the first user, wherein the settling time indicates a time required by the first user to settle after an entry into the physical space;

detecting, by the circuitry, a lip movement of the first user based on the acquired one or more images;

determining, by the circuitry, whether the first user is in a conversation based on the detected lip movement of the first user;

controlling, by the circuitry, playback of the content on the rendering device based on the determination;

detecting, by the circuitry, a movement of the first user;

comparing, by the circuitry, the detected movement of the first user with a threshold movement, wherein the threshold movement is a specific amount of movement in the physical space;

pausing, by the circuitry, the playback of the content on the rendering device based on a result of the comparison that the detected movement of the first user is larger than the threshold movement;

detecting, by the circuitry, based on the acquired one or more images, the entry of the first user into the physical space;

resuming, by the circuitry, the playback of the content on the rendering device after the settling time elapses from the detection of the entry of the first user;

receiving, by the circuitry, from a user device, a trigger signal based on an incoming notification at the user device;

pausing, by the circuitry, in response to the trigger signal, the playback of the content on the rendering device for a specific period;

detecting, by the circuitry, in response to the trigger signal, a posture of the first user based on the acquired one or more images; and

determining, by the circuitry, based on the detected posture, whether to resume or continue to pause after the specific period, the playback of the content on the rendering device.

18. The method according to claim 17 , further comprising:

detecting the posture of the first user based on the acquired one or more images;

determining that the first user is in the conversation with a second user in the physical space, based on the detected posture of the first user; and

pausing the playback of the content on the rendering device based on the determination that the first user is in the conversation with the second user.

19. The method according to claim 17 , further comprising:

detecting a second user in the physical space based on the acquired one or more images;

detecting a lip movement of the second user based on the acquired one or more images; and

pausing the playback of the content on the rendering device based on the lip movement of the second user.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 4, 2024
From: SONY GROUP CORPORATION
To: SATURN LICENSING LLC
Reel/Frame 069492/0943 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 10, 2021
From: CANDELORE, BRANT
To: SONY GROUP CORPORATION
Reel/Frame 057137/0737 →
Continuity (1)
Related Publication 20220368984A1 · Nov 17, 2022
Cited By (1)
US 12,731,569