IP Library Granted Patent US 10,216,996
Granted Patent B2
US 10,216,996 · App. 14/565,236 · Granted Feb 26, 2019

Schemes for retrieving and associating content items with real-world objects using augmented reality and object recognition

Inventor: Glenn Black (San Mateo, CA)
Assignee: Sony Interactive Entertainment Inc.
G06K9/00624G06K9/00214G06K9/6201G06T11/60G06T19/006G06K9/00664G06K9/4604G06K9/4619G06K9/4647G06T2219/004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,216,996
App. No.
14/565,236
Granted
Feb 26, 2019
Kind
B2
Abstract

A method includes identifying a real-world object in a scene viewed by a camera of a user device, matching the real-world object with a tagged object based at least in part on image recognition and a sharing setting of the tagged object, the tagged object having been tagged with a content item, providing a notification to a user of the user device that the content item is associated with the real-world object, receiving a request from the user for the content item, and providing the content item to the user. A computer readable storage medium stores one or more computer programs, and an apparatus includes a processor-based device.

Claims (50)

1. A method comprising:

identifying a real-world object in a scene that is being viewed by a camera of a user device;

while the scene is being viewed by the camera of the user device, matching the real-world object with a tagged object based at least in part on image recognition and a sharing setting of the tagged object, the tagged object having been tagged with a content item;

while the scene is being viewed by the camera of the user device, providing a notification to a user of the user device that the content item is associated with the real-world object;

receiving a request from the user for the content item; and

providing the content item to the user;

wherein the tagged object has been tagged with the content item and assigned the sharing setting before the step of identifying a real-world object is performed.

2. The method of claim 1 , wherein the sharing setting comprises whether the tagged object will be matched only with an image of the tagged object or with an image of any object sharing one or more common attributes with the tagged object.

3. The method of claim 1 , wherein the matching of the real-world object with the tagged object is further based on one or more of a location of the user device and a social network connection between the user and an author of the content item.

4. The method of claim 1 , wherein the notification comprises one or more of a sound notification, a pop-up notification, a vibration, and an icon in a display of the scene viewed by the camera of the user device.

5. The method of claim 1 , wherein the content item comprises one or more of a text comment, an image, an audio clip, and a video clip.

6. An apparatus comprising:

a processor-based device; and

a non-transitory storage medium storing a set of computer readable instructions configured to cause the processor-based device to perform steps comprising:

identifying a real-world object in a scene that is being viewed by a camera of a user device;

while the scene is being viewed by the camera of the user device, matching the real-world object with a tagged object based at least in part on image recognition and a sharing setting of the tagged object, the tagged object having been tagged with a content item;

while the scene is being viewed by the camera of the user device, providing a notification to a user of the user device that the content item is associated with the real-world object;

receiving a request from the user for the content item; and

providing the content item to the user;

wherein the tagged object has been tagged with the content item and assigned the sharing setting before the step of identifying a real-world object is performed.

7. The apparatus of claim 6 , wherein the sharing setting comprises whether the tagged object will be matched only with an image of the tagged object or with an image of any object sharing one or more common attributes with the tagged object.

8. The apparatus of claim 6 , wherein the matching of the identified real-world object with the tagged object is further based on one or more of a location of the user device and a social network connection between the user and an author of the content item.

9. The apparatus of claim 6 , wherein the notification comprises one or more of a sound notification, a pop-up notification, a vibration, and an icon in a display of the scene viewed by the camera of the user device.

10. The apparatus of claim 6 , wherein the content item comprises one or more of a text comment, an image, an audio clip, and a video clip.

11. A method comprising:

receiving a selection of a first real-world object while the first real-world object is being viewed by a camera of a first user device from a user;

while the first real-world object is being displayed on a display of the first user device, receiving a content item to tag to the first real-world object;

capturing one or more images of the first real-world object with the camera;

while the first real-world object is being displayed on the display of the first user device, receiving a sharing setting from the user, wherein the sharing setting is set by the user and comprises whether the user wants the first real-world object to be matched only with an image of the first real-world object or with an image of any object sharing one or more common attributes with the first real-world object;

storing the content item, the one or more images of the first real-world object, and the sharing setting on a server, wherein the content item is configured to be retrieved by a second user device viewing an object matching the first real-world object;

after the content item and the sharing setting for the first real-world object have been received and stored on the server, identifying a second real-world object in a scene viewed by a camera of the second user device; and

matching the second real-world object with the first real-world object having the previously tagged content item and associated sharing setting based at least in part on image recognition and the sharing setting.

12. The method of claim 11 , wherein the content item comprises one or more of a text comment, an image, an audio clip, and a video clip.

13. The method of claim 11 , wherein the sharing setting further comprises restrictions on who can view the content item.

14. The method of claim 11 , wherein the sharing setting is entered by the user selecting from a list of attributes corresponding to the first real-world object.

15. The method of claim 14 , wherein the list of attributes is generated through object recognition performed on the one or more images of the first real-world object.

16. An apparatus comprising:

a processor-based device; and

a non-transitory storage medium storing a set of computer readable instructions configured to cause the processor-based device to perform steps comprising:

receiving a selection of a first real-world object while the first real-world object is being viewed by a camera of a first user device from a user;

while the first real-world object is being displayed on a display of the first user device, receiving a content item to tag to the first real-world object;

capturing one or more images of the first real-world object with the camera;

while the first real-world object is being displayed on the display of the first user device, receiving a sharing setting from the user, wherein the sharing setting is set by the user and comprises whether the user wants the first real-world object to be matched only with an image of the first real-world object or with an image of any object sharing one or more common attributes with the first real-world object;

storing the content item, the one or more images of the first real-world object, and the sharing setting on a server, wherein the content item is configured to be retrieved by a second user device viewing an object matching the first real-world object;

after the content item and the sharing setting for the first real-world object have been received and stored on the server, identifying a second real-world object in a scene viewed by a camera of the second user device; and

matching the second real-world object with the first real-world object having the previously tagged content item and associated sharing setting based at least in part on image recognition and the sharing setting.

17. The apparatus of claim 16 , wherein the content item comprises one or more of a text comment, an image, an audio clip, and a video clip.

18. The apparatus of claim 16 , wherein the sharing setting further comprises restrictions on who can view the content item.

19. The apparatus of claim 16 , wherein the sharing setting is entered by the user selecting from a list of attributes corresponding to the first real-world object.

20. The apparatus of claim 19 , wherein the list of attributes is generated through automatic object recognition performed on the one or more images of the first real-world object.

Assignments (2)
CHANGE OF NAME Recorded Jul 6, 2016
From: SONY COMPUTER ENTERTAINMENT INC.
To: SONY INTERACTIVE ENTERTAINMENT INC.
Reel/Frame 039276/0023 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: BLACK, GLENN
To: SONY COMPUTER ENTERTAINMENT INC.
Reel/Frame 034443/0572 →
Continuity (3)
Provisional Application 62057219 · Sep 29, 2014
Provisional Application 62057223 · Sep 29, 2014
Related Publication 20160093106A1 · Mar 31, 2016
Cited By (2)
US 12,387,384 US 12,436,601