IP Library Granted Patent US 12,704,952
Granted Patent B2
US 12,704,952 · App. 19/278,673 · Granted Aug 11, 2026

Shared real-time content editing activated by an image

Inventor: Patrick Soon-Shiong (Los Angeles, CA)
Assignee: Nant Holdings IP, LLC
G06F3/04847G06F3/03G06F3/048H04L65/4015H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,704,952
App. No.
19/278,673
Filed
Jul 23, 2025
Granted
Aug 11, 2026
Kind
B2
Art Unit
2454
USPC
715/753
Abstract

Systems and methods for instantiating collaboration interfaces via object recognition are disclosed. An object recognition engine communicatively coupled to a database and an electronic device receives sensor data related to an object and identifies an object characteristic. A component is selected having criteria satisfied by the object characteristic and a collaboration interface is instantiated on the electronic device. This can allow a user to edit content related to an object. A substantially identical collaboration interface can be instantiated on a second electronic device to allow collaboration among user with respect to content related to the object.

Claims (65)

1 . A computer-based location-based mobile game system comprising:

at least one non-transitory computer-readable memory storing software instructions;

at least one collaboration database storing game interface components; and

at least one processor coupled with the at least one non-transitory computer-readable memory and the at least one collaboration database, and that performs the following operations upon execution of the software instructions:

receiving sensor data from a mobile device, including location data and image data of a physical environment;

identifying multiple game-related objects within the image data;

determining respective object characteristics for each identified game-related object, including corresponding physical locations;

selecting, for each identified game-related object, respective game interface components from the at least one collaboration database based on the object characteristics;

instantiating a game collaboration interface on the mobile device comprising the selected game interface components, wherein different game interface components are associated with different identified game-related objects; and

enabling the mobile device to interact with the identified game-related objects through the game collaboration interface based on a location of the mobile device relative to the physical locations of the identified game-related objects.

2 . The system of claim 1 , wherein the operations further comprise: instantiating the game collaboration interface on a second mobile device; and

enabling multi-user interaction between the mobile device and the second mobile device with respect to the identified game-related objects.

3 . The system of claim 2 , wherein the operations further comprise:

detecting a conflict between a first command from the mobile device and a second command from the second mobile device; and

reconciling the conflict based on a rule associated with at least one of the identified game-related objects.

4 . The system of claim 1 , wherein the selected game interface components include at least one of: a game editor, a video editor, an image editor, a text editor, an audio editor, or an API.

5 . The system of claim 1 , wherein the operations further comprise:

initiating a synchronizing action between the mobile device and a second mobile device prior to enabling interaction with the identified game-related objects.

6 . The system of claim 1 , wherein the operations further comprise:

determining a user hierarchy among multiple users; and

modifying available game interface components based on the user hierarchy.

7 . The system of claim 1 , wherein the operations further comprise:

enforcing content restriction rules associated with the identified game-related objects.

8 . The system of claim 1 , wherein the operations further comprise:

updating the game collaboration interface in real-time based on changes in the location of the mobile device.

9 . The system of claim 1 , wherein the operations further comprise:

storing edited versions of content associated with the identified game-related objects in the at least one memory or the at least one collaboration database.

10 . The system of claim 1 , wherein the operations further comprise: authenticating the mobile device prior to enabling interaction with the identified game-related objects.

11 . The system of claim 1 , wherein the operations further comprise:

identifying different content modalities from the sensor data; and

instantiating different game interface components for each identified content modality.

12 . The system of claim 1 , wherein the operations further comprise:

linking multiple game interface components such that an edit in one game interface component causes a corresponding change in another game interface component.

13 . The system of claim 1 , wherein the operations further comprise:

receiving audio data from the mobile device; and

recognizing game-related sounds within the audio data as a game-related object.

14 . The system of claim 1 , wherein the operations further comprise:

determining device capabilities of the mobile device; and

modifying the game collaboration interface based on the determined device capabilities.

15 . The system of claim 1 , wherein the operations further comprise:

instantiating the game collaboration interface based on a pre-constructed interface template; and

modifying the pre-constructed interface template based on specific game-related objects identified in the sensor data.

16 . The system of claim 1 , wherein the operations further comprise:

enabling the mobile device to interact through the game collaboration interface based on a proximity of a second mobile device.

17 . The system of claim 1 , wherein the operations further include selecting game interface components based on a mapping between object characteristics and collaboration interface components stored in the collaboration database.

18 . The system of claim 1 , wherein the operations further comprise:

identifying an activity or location through integration of acceleration data from the mobile device.

19 . The system of claim 1 , wherein the operations further comprise:

instantiating different versions of the game collaboration interface for different users based on user-specific characteristics.

20 . The system of claim 1 , wherein the operations further comprise:

automatically generating collaboration commands in response to a user input without requiring confirmation.

21 . A non-transitory computer-readable medium having computer instructions stored thereon, which, when executed by at least one processor, cause the at least one processor to perform the operations of:

receiving sensor data from a mobile device, including location data and image data of a physical environment;

identifying multiple game-related objects within the image data;

determining respective object characteristics for each identified game-related object, including corresponding physical locations;

selecting, for each identified game-related object, respective game interface components from the at least one collaboration database based on the object characteristics;

instantiating a game collaboration interface on the mobile device comprising the selected game interface components, wherein different game interface components are associated with different identified game-related objects; and

enabling the mobile device to interact with the identified game-related objects through the game collaboration interface based on a location of the mobile device relative to the physical locations of the identified game-related objects.

22 . A method comprising:

receiving sensor data from a mobile device, including location data and image data of a physical environment;

identifying multiple game-related objects within the image data;

determining respective object characteristics for each identified game-related object, including corresponding physical locations;

selecting, for each identified game-related object, respective game interface components from the at least one collaboration database based on the object characteristics;

instantiating a game collaboration interface on the mobile device comprising the selected game interface components, wherein different game interface components are associated with different identified game-related objects; and

enabling the mobile device to interact with the identified game-related objects through the game collaboration interface based on a location of the mobile device relative to the physical locations of the identified game-related objects.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 1, 2025
From: SOON-SHIONG, PATRICK
To: NANT HOLDINGS IP, LLC
Reel/Frame 071914/0141 →
Continuity (9)
Continuation 18516893 · Nov 21, 2023
Continuation 17990643 · Nov 18, 2022
Continuation 17556538 · Dec 20, 2021
Continuation 17112830 · Dec 4, 2020
Continuation 15843918 · Dec 15, 2017
Continuation 15591017 · May 9, 2017
Continuation 14155252 · Jan 14, 2014
Provisional Application 61752091 · Jan 14, 2013
Related Publication 20250348202A1 · Nov 13, 2025
References Cited (103)
US 7016532B2 · Boncyk et al. · 2006 [cited by applicant]
US 7162690B2 · Gupta et al. · 2007 [cited by applicant]
US 7242988B1 · Hoffberg et al. · 2007 [cited by applicant]
US 7379047B2 · Drucker et al. · 2008 [cited by applicant]
US 7403652B2 · Boncyk et al. · 2008 [cited by applicant]
US 7477780B2 · Boncyk et al. · 2009 [cited by applicant]
US 7493243B2 · Choi et al. · 2009 [cited by applicant]
US 7564469B2 · Cohen · 2009 [cited by applicant]
US 7565008B2 · Boncyk et al. · 2009 [cited by applicant]
US 7613630B2 · Wolinsky et al. · 2009 [cited by applicant]
US 7680324B2 · Boncyk et al. · 2010 [cited by applicant]
US 7739599B2 · Patten et al. · 2010 [cited by applicant]
US 7775437B2 · Cohen · 2010 [cited by applicant]
US 7831246B1 · Smith et al. · 2010 [cited by applicant]
US 7881529B2 · Boncyk et al. · 2011 [cited by applicant]
US 7899243B2 · Boncyk et al. · 2011 [cited by applicant]
US 7899252B2 · Boncyk et al. · 2011 [cited by applicant]
US 8229715B1 · Teller et al. · 2012 [cited by applicant]
US 8280949B2 · Bergquist et al. · 2012 [cited by applicant]
US 8332470B2 · Arun et al. · 2012 [cited by applicant]
US 8340651B1 · Gailloux et al. · 2012 [cited by applicant]
US 8484561B1 · Lemonik et al. · 2013 [cited by applicant]
US 8523682B2 · Bickley · 2013 [cited by applicant]
US 8684839B2 · Mattice et al. · 2014 [cited by applicant]
US 8819536B1 · Lucovsky et al. · 2014 [cited by applicant]
US 9498720B2 · Geisner et al. · 2016 [cited by applicant]
US 9678617B2 · Soon-Shiong · 2017 [cited by applicant]
US 9857964B2 · Soon-Shiong · 2018 [cited by applicant]
US 10740974B1 · Cowburn et al. · 2020 [cited by applicant]
US 10891039B2 · Soon-Shiong · 2021 [cited by applicant]
US 11237715B2 · Soon-Shiong · 2022 [cited by applicant]
US 11543953B2 · Soon-Shiong · 2023 [cited by applicant]
US 11861154B2 · Soon-Shiong · 2024 [cited by applicant]
US 12386497B2 · Soon-Shiong · 2025 [cited by applicant]
US 20020010734A1 · Ebersole et al. · 2002 [cited by applicant]
US 20020164858A1 · Sayama et al. · 2002 [cited by applicant]
US 20030177187A1 · Levine et al. · 2003 [cited by applicant]
US 20040121786A1 · Radcliffe et al. · 2004 [cited by applicant]
US 20040260669A1 · Fernandez · 2004 [cited by applicant]
US 20060252530A1 · Oberberger · 2006 [cited by examiner]
US 20070015494A1 · Sinclair · 2007 [cited by examiner]
US 20070021198A1 · Muir et al. · 2007 [cited by applicant]
US 20070043843A1 · Devereaux et al. · 2007 [cited by applicant]
US 20070106754A1 · Moore · 2007 [cited by examiner]
US 20070124374A1 · Arun et al. · 2007 [cited by applicant]
US 20070250840A1 · Coker et al. · 2007 [cited by applicant]
US 20080207296A1 · Lutnick et al. · 2008 [cited by applicant]
US 20090029766A1 · Lutnick et al. · 2009 [cited by applicant]
US 20090054123A1 · Mityagin et al. · 2009 [cited by applicant]
US 20090119731A1 · Perlman et al. · 2009 [cited by applicant]
US 20090156303A1 · Kiely et al. · 2009 [cited by applicant]
US 20090182813A1 · McCartie · 2009 [cited by examiner]
US 20090275414A1 · Lee · 2009 [cited by examiner]
US 20090305218A1 · Dozier et al. · 2009 [cited by applicant]
US 20100122196A1 · Wetzer · 2010 [cited by examiner]
US 20100124983A1 · Gowin et al. · 2010 [cited by applicant]
US 20100141554A1 · Devereaux et al. · 2010 [cited by applicant]
US 20100162131A1 · Gaillard et al. · 2010 [cited by applicant]
US 20100169269A1 · Chandrasekaran · 2010 [cited by applicant]
US 20110060729A1 · Marquez et al. · 2011 [cited by applicant]
US 20110105226A1 · Perlman · 2011 [cited by applicant]
US 20110134204A1 · Rodriguez et al. · 2011 [cited by applicant]
US 20110170004A1 · Nunes · 2011 [cited by applicant]
US 20110191417A1 · Rathod · 2011 [cited by examiner]
US 20110225200A1 · Danis et al. · 2011 [cited by applicant]
US 20110239135A1 · Spataro et al. · 2011 [cited by applicant]
US 20110257977A1 · Greenberg · 2011 [cited by examiner]
US 20120058814A1 · Lutnick et al. · 2012 [cited by applicant]
US 20120064968A1 · Youm · 2012 [cited by examiner]
US 20120078802A1 · Kieselbach · 2012 [cited by applicant]
US 20120086727A1 · Korah et al. · 2012 [cited by applicant]
US 20120254040A1 · Dixon et al. · 2012 [cited by applicant]
US 20120295717A1 · Belz et al. · 2012 [cited by applicant]
US 20120315993A1 · Dumont · 2012 [cited by examiner]
US 20120324353A1 · Torbey et al. · 2012 [cited by applicant]
US 20130027404A1 · Sarnoff · 2013 [cited by applicant]
US 20130035158A1 · Osvald et al. · 2013 [cited by applicant]
US 20130065668A1 · Lemay et al. · 2013 [cited by applicant]
US 20130079128A1 · Thomas · 2013 [cited by examiner]
US 20130116044A1 · Schwartz · 2013 [cited by examiner]
US 20130144950A1 · Sanakaranarasimhan et al. · 2013 [cited by applicant]
US 20130165215A1 · Arezina et al. · 2013 [cited by applicant]
US 20130201215A1 · Martellaro et al. · 2013 [cited by applicant]
US 20130278631A1 · Border et al. · 2013 [cited by applicant]
US 20130326376A1 · Stachniak et al. · 2013 [cited by applicant]
US 20140006660A1 · Frei et al. · 2014 [cited by applicant]
US 20140082503A1 · Tan · 2014 [cited by examiner]
US 20140149874A1 · Harper · 2014 [cited by examiner]
US 20140173457A1 · Wang et al. · 2014 [cited by applicant]
CA 2547468A1 · 2005 [cited by applicant]
CA 2881533A1 · 2011 [cited by applicant]
CN 101179629A · 2008 [cited by applicant]
EP 2458537A1 · 2012 [cited by applicant]
TW I485613B · 2015 [cited by examiner]
WO 0381447A1 · 2003 [cited by applicant]
WO 2010039414A1 · 2010 [cited by applicant]
WO 2012061760A2 · 2012 [cited by applicant]
Boring et al., “Multi-User Interaction on Media Facades Through Live Video on Mobile Devices,” CHI '11 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, p. 2721-2724. [cited by applicant]
Expat Software, “Team Whiteboarding With Twiddla—Painless Team Collaboration for the Web,” downloaded Dec. 12, 2012 from http://www.twiddla.com/. [cited by applicant]
Hewlett Packard, “Advantages and Implementation of HP Remote Graphics Software”, downloaded Dec. 12, 2012 from http://www8.hp.com/us/en/campaigns/workstations/remote-graphics-software.html. [cited by applicant]
Livetech, “Live Online Video Technical Support From LiveTechTM” , downloaded Dec. 12, 2012 from http://livetechsite.com. [cited by applicant]
Simulat, Inc., “VYEW Product Features”, downloaded Dec. 12, 2012 from http://vyew.com/site/product. [cited by applicant]
Wallace et al., “Virtually Shared Displays and User Input Devices,” 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference, pp. 375-380. [cited by applicant]