IP Library › Granted Patent US 11,853,635
Granted Patent B2
US 11,853,635 · App. 15/283,005 · Granted Dec 26, 2023

Configuration and operation of display devices including content curation

Inventors: Arun Rakesh Yoganandan (San Francisco, CA); Kumi Akiyoshi (San Francisco, CA); Melinda Yang (Mountain View, CA); Yuchang Hu (Santa Clara, CA); Mark Schlager (San Carlos, CA)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06F3/1446G06F3/0482G06F3/04845G06F3/147G06F3/1423G06F3/1438G06F3/1454G06F3/1462G06T7/70G06T7/90G06T11/001G06T11/40G09G5/14H04N23/63H04N23/80G06T3/40G06T2207/10004G06T2207/20024G06T2210/22G09G5/377G09G2360/144G09G2370/02G09G2370/042G09G2370/06G09G2380/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,853,635
App. No.
15/283,005
Granted
Dec 26, 2023
Kind
B2
Abstract

Content management includes receiving an image of one or more display devices with respective first content displayed. For at least one display device, a display device identification (ID) is retrieved. A change to the first content displayed at the at least one display device is caused, based on the display device ID of the at least one display device, upon interaction with the image.

Claims (103)

1. A method for content management, comprising:

receiving, by an electronic device, a captured photo including images of an environment and one or more physical display devices with respective first content currently displayed;

on a user interface, overlaying each of the images of the one or more physical display devices in the captured photo with an outline region;

recognizing a displayed feature of the first content displayed on at least one physical display device;

identifying a display device of the at least one physical display device based on the recognized displayed feature of the first content, wherein the display device is identified using a plurality of physical markers disposed on a border of the one or more physical display devices;

controlling the first content displayed on the at least one physical display device comprising: selecting content matching one or more patterns in a pattern matching database in two or more portions of the first content; automatically overlaying box regions, by the electronic device, around the two or more portions; causing only the content within the box regions to currently be displayed on a selected one of the at least one physical display device and at least one other selected physical display device, upon interaction with the captured photo via the electronic device, wherein each of the two or more portions having less content than the first content.

2. The method of claim 1 , further comprising:

delineating regions, on the captured photo, via a computer vision technique, each region corresponding to a distinctive display device of the one or more physical display devices, wherein interaction with the captured photo including interaction with at least one of the one or more regions, and a display device identification (ID) is retrieved from a look up table that includes a relationship between the display device ID and currently displayed image.

3. The method of claim 1 , further comprising:

causing a second content to be sent to at least two display devices of the one or more physical display devices for display;

wherein the second content is displayed across the at least two display devices maintaining spatial relationship of the second content, and the outline region is on top of and encompasses each of the images of the one or more physical display devices in the captured photo.

4. The method of claim 2 , further comprising:

performing content recognition based on display device ID using computer vision, wherein an image identifier is stored for the displayed feature of the first content, and the plurality of physical markers provides for distinct identification of each display device of the one or more display devices using computer vision

wherein upon a particular box region changing by its corners being dragged, moved or scaled, the changes are tracked and a new position of the box region is recorded without losing display device association.

5. The method of claim 1 , wherein:

the one or more physical display devices are linked to a network;

recognizing the displayed feature comprises performing content recognition on the displayed feature based on pattern matching and accessing a pattern library; and

an image identifier is stored for the displayed feature of the first content prior to recognizing the displayed feature.

6. The method of claim 1 , further comprising:

analyzing features of the environment in the captured photo; and

mimicking the features of the environment for changing features of digital mat regions of the one or more physical display devices;

wherein the interaction with the captured photo is via the user interface supporting selection of display devices and interaction with the selected display devices, and recognizing the displayed feature comprises performing character recognition on the displayed feature.

7. The method of claim 1 , wherein

the user interface enables various gestures for managing content displayed on the one or more physical display devices, the displayed feature of the first content comprises a watermark marker, and an aspect ratio of each box region is always locked to that of a display device it is assigned to even when its scale changes.

8. The method of claim 1 , further comprising:

receiving a selection of a portion of an image of the first content;

receiving instructions to crop the portion; and

causing the cropped portion to be displayed in a selected display device of the one or more physical display devices;

wherein only the selected content within the box regions is displayed.

9. The method of claim 1 , further comprising:

receiving a selection of one or more portions of an image of the second content;

receiving cropping instructions to crop the one or more portions; and

causing the cropped one or more portions to be displayed across the at least two display devices;

wherein the box regions are user selectable for portions of the first content.

10. A device for content management comprising:

a memory storing instructions; and

at least one processor executing the instructions including a process configured to:

receive, by the device, a captured photo including images of an environment and one or more physical display devices with respective first content currently displayed;

on a user interface, overlay each of the images of the one or more physical display devices in the captured photo with an outline region;

recognize a displayed feature of the first content displayed on at least one physical display device;

identify a display device of the at least one physical display device based on the recognized displayed feature of the first content, wherein the display device is identified using a plurality of physical markers disposed on a border of the one or more physical display devices; and

control the first content currently displayed on the at least one physical display device comprising: selecting content matching one or more patterns in a pattern matching database in two or more portions of the first content; automatically overlaying box regions, by the device, around the two or more portions; causing only the content within the box regions to currently be displayed on a selected one of the at least one physical display device and at least one other selected physical display device, upon interaction with the captured photo via the device, wherein each of the two or more portions having less content than the first content.

11. The device of claim 10 , wherein the process is further configured to:

delineate regions, on the captured photo, via a computer vision technique, each region corresponding to a distinctive display device of the one or more physical display devices, wherein interaction with the captured photo including interaction with at least one of the one or more regions, a display device identification (ID) is retrieved from a look up table that includes a relationship between the display device ID and currently displayed image, and an image identifier is stored for the displayed feature of the first content.

12. The device of claim 10 , wherein the process is further configured to:

cause a second content to be sent to at least two display devices of the one or more physical display devices for display;

wherein:

the second content is displayed across the at least two display devices maintaining the spatial relationship of the second content;

the plurality of physical markers provides for distinct identification of each display device of the one or more display devices using computer vision;

an image identifier is stored for the displayed feature of the first content prior to recognition of the displayed feature;

the outline region is on top of and encompasses each of the one or more physical display devices in the captured photo;

an aspect ratio of each box region is always locked to that of a display device it is assigned to even when its scale changes; and

upon a particular box region changing by its corners being dragged, moved or scaled, the changes are tracked and a new position of the box region is recorded without losing display device association.

13. The device of claim 10 , further comprising:

providing selection of display devices and interaction with the selected display devices via the user interface;

analyzing features of the environment in the captured photo; and

using the features of the environment for changing features of digital mat regions of the one or more physical display devices;

wherein:

recognizing the displayed feature of the first content comprises at least one of:

performing content recognition on the displayed feature based on pattern matching and accessing a pattern library;

performing character recognition on the displayed feature; or

identifying a watermark marker of the first content; and

only the selected content within the box regions is displayed.

14. The device of claim 10 , wherein:

the process is further configured to:

receive a selection for a portion of an image of the first content;

receive crop instructions to crop the portion; and

display the cropped portion in a selected display device of the one or more physical display devices; and

the box regions are user selectable for portions of the first content.

15. A non-transitory processor-readable medium that includes a program that when executed by a processor performing a method comprising:

receiving, by an electronic device, a captured photo including images of an environment and one or more physical display devices with respective first content currently displayed;

on a user interface, overlaying each of the images of the one or more physical display devices in the captured photo with an outline region;

recognizing a displayed feature of the first content displayed on at least one physical display device;

identifying a display device of the at least one physical display device based on the recognized displayed feature of the first content, wherein the display device is identified using a plurality of physical markers disposed on a border of the one or more physical display devices; and

controlling the first content displayed on the at least one physical display device comprising: selecting content matching one or more patterns in a pattern matching database in two or more portions of the first content; automatically overlaying box regions, by the electronic device, around the two or more portions; causing only the content within the box regions to currently be displayed on a selected one of the at least one physical display device and at least one other selected physical display device, upon interaction with the captured photo via the electronic device, wherein each of the two or more portions having less content than the first content.

16. The non-transitory processor-readable medium of claim 15 , further comprising:

delineating regions, on the captured photo, via a computer vision technique, each region corresponding to a distinctive display device of the one or more physical display devices, wherein interaction with the captured photo including interaction with at least one of the one or more regions, a display device identification (ID) is retrieved from a look up table that includes a relationship between the display device ID and currently displayed image, and an image identifier is stored for the displayed feature of the first content.

17. The non-transitory processor-readable medium of claim 15 , further comprising:

causing second content to be sent to at least two display devices of the one or more physical display devices for display;

wherein:

the second content is displayed across the at least two display devices maintaining the spatial relationship of the second content;

the plurality of physical markers provides for distinct identification of each display device of the one or more display devices using computer vision;

an image identifier is stored for the displayed feature of the first content prior to recognizing the displayed feature;

the outline region is on top of and encompasses each of the one or more physical display devices in the captured photo;

an aspect ratio of each box region is always locked to that of a display device it is assigned to even when its scale changes; and

upon a particular box region changing by its corners being dragged, moved or scaled, the changes are tracked and a new position of the box region is recorded without losing display device association.

18. The non-transitory processor-readable medium of claim 15 , wherein:

the one or more physical display devices are linked to a network;

recognizing the displayed feature of the first content comprises at least one of:

performing content recognition on the displayed feature based on pattern matching and accessing a pattern library;

performing character recognition on the displayed feature; or

identifying a watermark marker of the first content; and

only the selected content within the box regions is displayed.

19. The non-transitory processor-readable medium of claim 15 , further comprising:

analyzing features of the environment in the captured photo; and

using the features of the environment for changing features of digital mat regions of the one or more physical display devices;

wherein:

the user interface enables various gestures for managing content displayed on the one or more physical display devices; and

the box regions are user selectable for portions of the first content.

20. The non-transitory processor-readable medium of claim 15 , further comprising:

receiving a selection of a portion of an image of the first content;

receiving instructions to crop the portion; and

causing the cropped portion to be displayed in a selected display device of the one or more physical display devices.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 28, 2017
From: YOGANANDAN, ARUN RAKESH
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 041771/0795 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2016
From: YOGANANDAN, ARUN; AKIYOSHI, KUMI; YANG, MELINDA; HU, YUCHANG; SCHLAGER, MARK
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 039915/0785 →
Continuity (2)
Provisional Application 62305938 · Mar 9, 2016
Related Publication 20170262247A1 · Sep 14, 2017