IP Library Granted Patent US 12,386,497
Granted Patent B2
US 12,386,497 · App. 18/516,893 · Granted Aug 12, 2025

Shared real-time content editing activated by an image

Inventor: Patrick Soon-Shiong (Los Angeles, CA)
Assignee: Nant Holdings IP, LLC
G06F3/04847G06F3/03G06F3/048H04L65/4015H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,386,497
App. No.
18/516,893
Granted
Aug 12, 2025
Kind
B2
Abstract

Systems and methods for instantiating collaboration interfaces via object recognition are disclosed. An object recognition engine communicatively coupled to a database and an electronic device receives sensor data related to an object and identifies an object characteristic. A component is selected having criteria satisfied by the object characteristic and a collaboration interface is instantiated on the electronic device. This can allow a user to edit content related to an object. A substantially identical collaboration interface can be instantiated on a second electronic device to allow collaboration among user with respect to content related to the object.

Claims (41)

1. A collaborative augmented reality (AR) content management system comprising:

at least one computer readable, non-transitory memory storing software instructions;

at least one collaboration database capable of storing AR-related components indexed at least by location characteristics; and

at least one processor coupled with the memory and with the collaboration database and, upon execution of the software instructions, performs the operations of:

determining at least one object characteristic including an object location of at least one recognizable object associated with a first device location;

receiving a selected AR-related component from a user and associated with the at least one recognizable object, the AR-related component comprising a reconciliation control over the at least one recognizable object, wherein the reconciliation control is based on a priority;

storing the selected AR-related component in the collaboration database according to at least the object location as a relevant AR-related component;

enabling a second device to obtain the relevant AR-related component from the collaboration database upon satisfaction of selection criteria satisfied by a second device location of the second device and based on the object location; and

enabling the second device to superimpose the relevant AR-related component on a display of the second device according to the reconciliation control.

2. The system of claim 1 , wherein the AR-related component comprises a command.

3. The system of claim 2 , wherein the command comprises a superimpose command.

4. The system of claim 2 , wherein the command comprises at least one of the following commands related to content associated with the at least one recognizable object: an add command, a delete command, a modify command, an insert command, a review command, a proofread command, a crop command, a clip command, a speed up command, a highlight command, an enlarge command, a shrink command, a slowdown command, a distort command, a skew command, a rotate command, a reverse command, a mirror command, a comment upon command, a manipulate command, a replace command, or a create command.

5. The system of claim 1 , wherein the AR-related component allows an addition of content or a modification of content associated with the at least one recognizable object.

6. The system of claim 1 , wherein the AR-related component comprises content related to the at least one recognizable object.

7. The system of claim 6 , wherein the content related to the at least one recognizable object includes at least one of the following: an image, a video, audio, text, a game, or a software application.

8. The system of claim 1 , wherein the AR-related component comprises a version control over the at least one recognizable object.

9. The system of claim 1 , wherein the at least one object characteristic further includes one or more of the following: a characteristic obtained from an object recognition algorithm, a sonic feature, an acceleration, text from an optical character recognition algorithm, and an image attribute.

10. The system of claim 1 , wherein the at least one recognizable object comprises one or more of the following: a building, a poster, a billboard, a sign, a television show, a move an advertisement, a document, a photograph, a sculpture, or a magazine.

11. The system of claim 1 , wherein the AR-related component includes one or more of the following: an image editor, a video editor, a text editor, a game editor, an audio editor, an application program interface, a remote procedure call, or a content hosting function.

12. The system of claim 1 , wherein the object location is based at least in part on global positioning system data.

13. The system of claim 1 , wherein the AR-related component comprises a conflicting command relative to other AR-related components related to the at least one recognizable object at the object location.

14. The system of claim 13 , wherein the operations further include reconciling the conflicting command according to reconciliation rules comprising the reconciliation control.

15. The system of claim 14 , wherein the reconciliation rules are based on the priority.

16. The system of claim 1 , wherein the AR-related component comprises a collaboration interface component.

17. The system of claim 1 , wherein the operations further include generating an alert on the first device or the second device regarding collaboration with respect to content related to the at least one recognizable object.

18. The system of claim 17 , wherein the alert indicates a conflict associated with the collaboration interface component.

19. The system of claim 17 , wherein the alert comprises a text message.

20. The system of claim 17 , wherein the alert indicates that a collaboration is about to begin.

21. The system of claim 1 , wherein the relevant AR-related component is part of an interactive game.

22. A non-transitory computer-readable medium having computer instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to perform the operations of:

determining at least one object characteristic including an object location of at least one recognizable object associated with a first device location;

receiving a selected AR-related component from a user and associated with the at least one recognizable object, the AR-related component comprising a reconciliation control over the at least one recognizable object, wherein the reconciliation control is based on a priority;

storing the selected AR-related component in the collaboration database according to at least the object location as a relevant AR-related component;

enabling a second device to obtain the relevant AR-related component from the collaboration database upon satisfaction of selection criteria satisfied by a second device location of the second device and based on the object location; and

enabling the second device to superimpose the relevant AR-related component on a display of the second device according to the reconciliation control.

23. A method of collaborative augmented reality (AR) content management, the method comprising:

determining at least one object characteristic including an object location of at least one recognizable object associated with a first device location;

receiving a selected AR-related component from a user and associated with the at least one recognizable object, the AR-related component comprising a reconciliation control over the at least one recognizable object, wherein the reconciliation control is based on a priority;

storing the selected AR-related component in the collaboration database according to at least the object location as a relevant AR-related component;

enabling a second device to obtain the relevant AR-related component from the collaboration database upon satisfaction of selection criteria satisfied by a second device location of the second device and based on the object location; and

enabling the second device to superimpose the relevant AR-related component on a display of the second device according to the reconciliation control.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 12, 2024
From: SOON-SHIONG, PATRICK
To: NANT HOLDINGS IP, LLC
Reel/Frame 068575/0424 →
Continuity (8)
Continuation 17990643 · Nov 18, 2022
Continuation 17556538 · Dec 20, 2021
Continuation 17112830 · Dec 4, 2020
Continuation 15843918 · Dec 15, 2017
Continuation 15591017 · May 9, 2017
Continuation 14155252 · Jan 14, 2014
Provisional Application 61752091 · Jan 14, 2013
Related Publication 20240094892A1 · Mar 21, 2024
References Cited (90)
US 7016532B2 · Boncyk et al. · 2006 [cited by applicant]
US 7162690B2 · Gupta · 2007 [cited by examiner]
US 7242988B1 · Hoffberg et al. · 2007 [cited by applicant]
US 7379047B2 · Drucker et al. · 2008 [cited by applicant]
US 7403652B2 · Boncyk et al. · 2008 [cited by applicant]
US 7477780B2 · Boncyk et al. · 2009 [cited by applicant]
US 7493243B2 · Choi et al. · 2009 [cited by applicant]
US 7564469B2 · Cohen · 2009 [cited by applicant]
US 7565008B2 · Boncyk et al. · 2009 [cited by applicant]
US 7613630B2 · Wolinsky · 2009 [cited by examiner]
US 7680324B2 · Boncyk et al. · 2010 [cited by applicant]
US 7739599B2 · Patten et al. · 2010 [cited by applicant]
US 7775437B2 · Cohen · 2010 [cited by applicant]
US 7831246B1 · Smith et al. · 2010 [cited by applicant]
US 7881529B2 · Boncyk et al. · 2011 [cited by applicant]
US 7899243B2 · Boncyk et al. · 2011 [cited by applicant]
US 7899252B2 · Boncyk et al. · 2011 [cited by applicant]
US 8229715B1 · Teller et al. · 2012 [cited by applicant]
US 8280949B2 · Bergquist et al. · 2012 [cited by applicant]
US 8332470B2 · Arun et al. · 2012 [cited by applicant]
US 8340651B1 · Gailloux et al. · 2012 [cited by applicant]
US 8484561B1 · Lemonik · 2013 [cited by examiner]
US 8523682B2 · Bickley · 2013 [cited by applicant]
US 8684839B2 · Mattice et al. · 2014 [cited by applicant]
US 8819536B1 · Lucovsky et al. · 2014 [cited by applicant]
US 9498720B2 · Geisner · 2016 [cited by examiner]
US 9678617B2 · Soon-Shiong · 2017 [cited by applicant]
US 9857964B2 · Soon-Shiong · 2018 [cited by applicant]
US 10740974B1 · Cowburn et al. · 2020 [cited by applicant]
US 10891039B2 · Soon-Shiong · 2021 [cited by applicant]
US 11237715B2 · Soon-Shiong · 2022 [cited by applicant]
US 11543953B2 · Soon-Shiong · 2023 [cited by applicant]
US 11861154B2 · Soon-Shiong · 2024 [cited by applicant]
US 20020010734A1 · Ebersole · 2002 [cited by examiner]
US 20020164858A1 · Sayama et al. · 2002 [cited by applicant]
US 20030177187A1 · Levine · 2003 [cited by examiner]
US 20040121786A1 · Radcliffe · 2004 [cited by examiner]
US 20040260669A1 · Fernandez · 2004 [cited by applicant]
US 20060252530A1 · Oberberger et al. · 2006 [cited by applicant]
US 20070021198A1 · Muir et al. · 2007 [cited by applicant]
US 20070043843A1 · Devereaux · 2007 [cited by examiner]
US 20070124374A1 · Arun et al. · 2007 [cited by applicant]
US 20070250840A1 · Coker et al. · 2007 [cited by applicant]
US 20080207296A1 · Lutnick et al. · 2008 [cited by applicant]
US 20090029766A1 · Lutnick et al. · 2009 [cited by applicant]
US 20090054123A1 · Mityagin et al. · 2009 [cited by applicant]
US 20090119731A1 · Perlman et al. · 2009 [cited by applicant]
US 20090156303A1 · Kiely et al. · 2009 [cited by applicant]
US 20090305218A1 · Dozier et al. · 2009 [cited by applicant]
US 20100124983A1 · Gowin et al. · 2010 [cited by applicant]
US 20100141554A1 · Devereaux · 2010 [cited by examiner]
US 20100162131A1 · Gaillard et al. · 2010 [cited by applicant]
US 20100169269A1 · Chandrasekaran · 2010 [cited by applicant]
US 20110060729A1 · Marquez et al. · 2011 [cited by applicant]
US 20110105226A1 · Perlman · 2011 [cited by applicant]
US 20110134204A1 · Rodriguez · 2011 [cited by examiner]
US 20110170004A1 · Nunes · 2011 [cited by examiner]
US 20110225200A1 · Danis et al. · 2011 [cited by applicant]
US 20110239135A1 · Spataro et al. · 2011 [cited by applicant]
US 20120058814A1 · Lutnick et al. · 2012 [cited by applicant]
US 20120078802A1 · Kieselbach · 2012 [cited by applicant]
US 20120086727A1 · Korah · 2012 [cited by examiner]
US 20120254040A1 · Dixon et al. · 2012 [cited by applicant]
US 20120295717A1 · Belz et al. · 2012 [cited by applicant]
US 20120324353A1 · Torbey et al. · 2012 [cited by applicant]
US 20130027404A1 · Sarnoff · 2013 [cited by applicant]
US 20130035158A1 · Osvald et al. · 2013 [cited by applicant]
US 20130065668A1 · Lemay et al. · 2013 [cited by applicant]
US 20130079128A1 · Thomas et al. · 2013 [cited by applicant]
US 20130116044A1 · Schwartz · 2013 [cited by applicant]
US 20130144950A1 · Sanakaranarasimhan et al. · 2013 [cited by applicant]
US 20130165215A1 · Arezina · 2013 [cited by examiner]
US 20130201215A1 · Martellaro · 2013 [cited by examiner]
US 20130278631A1 · Border et al. · 2013 [cited by applicant]
US 20130326376A1 · Stachniak et al. · 2013 [cited by applicant]
US 20140006660A1 · Frei · 2014 [cited by examiner]
US 20140173457A1 · Wang et al. · 2014 [cited by applicant]
CA 2547468A1 · 2005 [cited by applicant]
CA 2881533A1 · 2011 [cited by applicant]
CN 101179629A · 2008 [cited by applicant]
EP 2458537A1 · 2012 [cited by applicant]
WO 0381447A1 · 2003 [cited by applicant]
WO 2010039414A1 · 2010 [cited by applicant]
WO WO2012061760A2 · 2012 [cited by examiner]
“Advantages and Implementation of HP Remote Graphics Software,” Hewlett Packard, http://h20331.www2.hp.com/Hpsub/downloads/hp_remotegraphics.pdf, 11 pages. [cited by applicant]
“Live Online Video Technical Support From LiveTechTM,” LIVETECH, http://livetechsite.com, 1 page. [cited by applicant]
Boring et al., “Multi-User Interaction on Media Facades Through Live Video on Mobile Devices,” Proceedings of the SIGCHI conference on human factors in computing systems, 2011, pp. 2721-2724. [cited by applicant]
“Team Whiteboarding With Twiddla—Painless Team Collaboration for the Web,” Twiddla, http://www.twiddla.com/, 1 page. [cited by applicant]
“VYEW Product Features,” VYEW, http://vyew.com/site/product, 2 pages. [cited by applicant]
Wallace et al., “Virtually Shared Displays and User Input Devices,” USENIX Annual Technical Conference, 2007, vol. 7, pp. 375-380. [cited by applicant]
Cited By (1)
US 12,704,952