IP Library › Granted Patent US 12,711,717
Granted Patent B2
US 12,711,717 · App. 18/731,971 · Granted Aug 18, 2026

System and method for selecting targets in an augmented reality environment

Inventor: Nova T. Spivack (Sherman Oaks, CA)
Assignee: Samsung Electronics Co., Ltd.
G06T19/006G02B27/017G06F3/011G06F3/013G06F3/017G06F3/0488G06Q30/0623G06Q30/0639G06Q30/0641G06Q30/0643G06V20/20G09G5/377G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,711,717
App. No.
18/731,971
Granted
Aug 18, 2026
Kind
B2
Abstract

Techniques are disclosed for facilitating electronic commerce in an augmented reality environment. In some embodiments, a method comprises detecting, by a mobile device, presence of the physical product or the real life service; and presenting, on the mobile device, information to conduct the transaction of a physical product or a real life service via the augmented reality environment. In some embodiments, a method comprises detecting one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by a mobile device; and prompting the user to choose an object of interest from the one or more detected targets.

Claims (49)

1 . A mobile device comprising:

a display;

a camera;

a processor; and

memory storing instructions which, when executed by the processor individually or collectively, cause the mobile device to:

display an image, being captured via the camera, the image including one or more objects, display a selection tool in the image to form a selected area for selecting a target among the one or more objects in the image,

receive a user input for moving or changing a size of the selection tool, the selected area being moved or resized according to the user input for updating the selected area,

select an object being at least partially included in the updated selected area as the target, and

when the target is selected based on the user input, display first information regarding a product corresponding to the target and second information regarding a similar product relating to the product, wherein the target is changed based on the movement or resizing of the selected area according to the user input for moving the selection tool.

2 . The mobile device of claim 1 , wherein the instructions which, when executed by the processor individually or collectively, cause the mobile device to:

prompt, via the display, the user to enter a mode in which the mobile device receives assistance from the user to perform the selection.

3 . The mobile device of claim 2 , wherein the instructions which, when executed by the processor individually or collectively, cause the mobile device to:

provide, via the display, the user with the selection tool in a perspective to form the select area, and

receive a user input for selecting a target corresponding to the selected area by using the selection tool.

4 . The mobile device of claim 3 , wherein the instructions which, when executed by the processor individually or collectively, cause the mobile device to:

receive a user input for moving the selected area, and

change another target based on the user input.

5 . The mobile device of claim 3 , wherein the instructions which, when executed by the processor individually or collectively, cause the mobile device to:

receive a user input for changing a size of the selected area, and

detect the target based on resizing of the selected area.

6 . The mobile device of claim 1 , wherein the displayed information comprises an advertisement regarding the product and another similar product relating to the product.

7 . The mobile device of claim 1 , wherein the displayed information comprises at least one of purchase information, price information or vendor information of the product and another similar product relating to the product.

8 . The mobile device of claim 1 , wherein the instructions which, when executed by the processor individually or collectively, cause the mobile device to:

detect the presence of the target in the selected area based on geolocation information of the mobile device.

9 . The mobile device of claim 1 , wherein the user input includes a gesture from one or more of movements of an eye of the user, locations of a focal point of an eye of the user, movements of an eye lid of the user and movements of part of body of the user.

10 . A method for selecting a target representing a physical object, via a mobile device, the method comprising:

displaying an image, being captured via a camera of the mobile device, the image including one or more objects;

display a selection tool in the image to form a select area for selecting the target among the one or more objects in the image;

receiving a user input for moving or changing a size of the selection tool, the selected area being moved or resized according to the user input for updating the selected area;

selecting an object being at least partially included in the updated selected area as the target; and

when the target is selected based on the user input, displaying first information regarding a product corresponding to the target and second information regarding a product relating to the product,

wherein the target is changed based on the movement or resizing of the selected area according to the user input for moving the selection tool.

11 . The method of claim 10 , wherein the method further comprises:

prompting, via a display of the mobile device, the user to enter a mode in which the mobile device receives assistance from the user to select the target.

12 . The method of claim 11 , wherein the method further comprises:

providing, via the display, the user with the selection tool in a perspective to form a select area; and

receiving a user input for selecting the target corresponding to the selected area by using the selection tool.

13 . The method of claim 12 , wherein the method further comprises:

receiving the user input for moving the selected area; and

changing another target based on the user input.

14 . The method of claim 12 , wherein the method further comprises:

receiving the user input for changing a size of the selected area; and

detecting the target based on resizing of the selected area.

15 . The method of claim 10 , wherein the displayed information comprises an advertisement regarding the product and another similar product relating to the product.

16 . The method of claim 10 , wherein the displayed information comprises at least one of purchase information, price information or vendor information of the product and another similar product relating to the product.

17 . The method of claim 10 , wherein the method further comprises:

detecting the presence of the target in the selected area based on geolocation information of the mobile device.

18 . The method of claim 10 , wherein the user input includes a gesture from one or more of movements of an eye of the user, locations of a focal point of an eye of the user, movements of an eye lid of the user and movements of part of body of the user.

19 . The mobile device of claim 1 , wherein the displayed first information further includes advisory data entered by a third party.

Continuity (7)
Continuation 17887075 · Aug 12, 2022
Continuation 17101429 · Nov 23, 2020
Continuation 16543011 · Aug 16, 2019
Continuation 14939889 · Nov 12, 2015
Continuation 13874421 · Apr 30, 2013
Provisional Application 61640856 · May 1, 2012
Related Publication 20240320939A1 · Sep 26, 2024
References Cited (221)
US 4519490A · White · 1985 [cited by applicant]
US 4829899A · Wiker et al. · 1989 [cited by applicant]
US 5009598A · Bennington · 1991 [cited by applicant]
US 5450590A · Elko et al. · 1995 [cited by applicant]
US 5600777A · Wang et al. · 1997 [cited by applicant]
US 5604907A · Conner et al. · 1997 [cited by applicant]
US 5616030A · Watson · 1997 [cited by applicant]
US 5623657A · Conner et al. · 1997 [cited by applicant]
US 6023270A · Brush et al. · 2000 [cited by applicant]
US 6028593A · Rosenberg et al. · 2000 [cited by applicant]
US 6050898A · Vange et al. · 2000 [cited by applicant]
US 6080063A · Khosla · 2000 [cited by applicant]
US 6241609B1 · Rutgers · 2001 [cited by applicant]
US 6292798B1 · Dockter et al. · 2001 [cited by applicant]
US 6302941B1 · Oya et al. · 2001 [cited by applicant]
US 6314167B1 · Johnson et al. · 2001 [cited by applicant]
US 6549893B1 · Lannert et al. · 2003 [cited by applicant]
US 6572380B1 · Buckley et al. · 2003 [cited by applicant]
US 6680909B1 · Bansal et al. · 2004 [cited by applicant]
US 6842175B1 · Schmalstieg et al. · 2005 [cited by applicant]
US 6983232B2 · Nguyen et al. · 2006 [cited by applicant]
US 7054848B1 · Lannert et al. · 2006 [cited by applicant]
US 7065553B1 · Chesley et al. · 2006 [cited by applicant]
US 7072919B2 · Sexton et al. · 2006 [cited by applicant]
US 7155496B2 · Froyd et al. · 2006 [cited by applicant]
US 7349840B2 · Budzinski · 2008 [cited by applicant]
US 7353160B2 · Voigt · 2008 [cited by applicant]
US 7487177B2 · Kilian-Kehr et al. · 2009 [cited by applicant]
US 7516052B2 · Hatcherson et al. · 2009 [cited by applicant]
US 7543238B2 · Alcazar et al. · 2009 [cited by applicant]
US 7545225B2 · Beccue · 2009 [cited by applicant]
US 7570261B1 · Edecker et al. · 2009 [cited by applicant]
US 7685508B2 · Froyd et al. · 2010 [cited by applicant]
US 7702693B1 · Aiyagari et al. · 2010 [cited by applicant]
US 7739479B2 · Bordes et al. · 2010 [cited by applicant]
US 7797168B2 · Kusumoto et al. · 2010 [cited by applicant]
US 7824268B2 · Harvey et al. · 2010 [cited by applicant]
US 7859551B2 · Bulman et al. · 2010 [cited by applicant]
US 7890419B2 · Jung et al. · 2011 [cited by applicant]
US 7955168B2 · Mendelsohn et al. · 2011 [cited by applicant]
US 7991706B2 · Mattern · 2011 [cited by applicant]
US 7996264B2 · Kusumoto et al. · 2011 [cited by applicant]
US 8046338B2 · Basso et al. · 2011 [cited by applicant]
US 8138930B1 · Heath et al. · 2012 [cited by applicant]
US 8181152B2 · Choi et al. · 2012 [cited by applicant]
US 8191121B2 · Ruppert et al. · 2012 [cited by applicant]
US 8192283B2 · Ruppert et al. · 2012 [cited by applicant]
US 8201229B2 · Ruppert et al. · 2012 [cited by applicant]
US 8205092B2 · Stokes · 2012 [cited by applicant]
US 8246467B2 · Huang et al. · 2012 [cited by applicant]
US 8279862B2 · Sbisa et al. · 2012 [cited by applicant]
US 8287383B1 · Etter et al. · 2012 [cited by applicant]
US 8303387B2 · Spivack · 2012 [cited by applicant]
US 8307273B2 · Pea et al. · 2012 [cited by applicant]
US 8494498B2 · Kwon · 2013 [cited by examiner]
US 8532340B2 · Kruglick · 2013 [cited by applicant]
US 8566786B2 · Choi et al. · 2013 [cited by applicant]
US 8610771B2 · Leung et al. · 2013 [cited by applicant]
US 8630620B2 · Cha et al. · 2014 [cited by applicant]
US 8660679B2 · Kurabayashi et al. · 2014 [cited by applicant]
US 8682879B2 · Kim et al. · 2014 [cited by applicant]
US 8719077B2 · Titus et al. · 2014 [cited by applicant]
US 8732025B2 · Gokturk et al. · 2014 [cited by applicant]
US 8745494B2 · Spivack · 2014 [cited by applicant]
US 8805110B2 · Rhoads et al. · 2014 [cited by applicant]
US 8866847B2 · Bedi et al. · 2014 [cited by applicant]
US 8947455B2 · Friesen · 2015 [cited by applicant]
US 8947456B2 · Chen et al. · 2015 [cited by applicant]
US 8957835B2 · Hoellwarth et al. · 2015 [cited by applicant]
US 9001252B2 · Hannaford · 2015 [cited by applicant]
US 9031809B1 · Kumar et al. · 2015 [cited by applicant]
US 9047700B2 · Ur · 2015 [cited by applicant]
US 9077771B2 · Kruglick · 2015 [cited by applicant]
US 9105207B2 · Leung · 2015 [cited by applicant]
US 9196094B2 · Ur · 2015 [cited by applicant]
US 9215530B2 · Kurabayashi et al. · 2015 [cited by applicant]
US 9257089B2 · Ur · 2016 [cited by applicant]
US 9280853B2 · Ur · 2016 [cited by applicant]
US 9390503B2 · Leung et al. · 2016 [cited by applicant]
US 9486707B2 · Kruglick · 2016 [cited by applicant]
US 10878636B2 · Spivack · 2020 [cited by examiner]
US 11170286B2 · Dolignon et al. · 2021 [cited by applicant]
US 11417066B2 · Spivack · 2022 [cited by examiner]
US 12002169B2 · Spivack · 2024 [cited by examiner]
US 20010053968A1 · Galitsky et al. · 2001 [cited by applicant]
US 20020010734A1 · Ebersole et al. · 2002 [cited by applicant]
US 20020029298A1 · Wilson · 2002 [cited by applicant]
US 20020133325A1 · Hoare, II et al. · 2002 [cited by applicant]
US 20020184516A1 · Hale et al. · 2002 [cited by applicant]
US 20030064712A1 · Gaston et al. · 2003 [cited by applicant]
US 20030217122A1 · Roese et al. · 2003 [cited by applicant]
US 20030221022A1 · Sexton et al. · 2003 [cited by applicant]
US 20030224855A1 · Cunningham · 2003 [cited by applicant]
US 20040002843A1 · Robarts et al. · 2004 [cited by applicant]
US 20040027258A1 · Pechatnikov et al. · 2004 [cited by applicant]
US 20040053686A1 · Pacey et al. · 2004 [cited by applicant]
US 20040056898A1 · Jetha · 2004 [cited by examiner]
US 20040095311A1 · Tarlton et al. · 2004 [cited by applicant]
US 20040158455A1 · Spivack et al. · 2004 [cited by applicant]
US 20040248632A1 · French et al. · 2004 [cited by applicant]
US 20050009608A1 · Robarts et al. · 2005 [cited by applicant]
US 20050091118A1 · Fano · 2005 [cited by applicant]
US 20050172018A1 · Devine et al. · 2005 [cited by applicant]
US 20050246275A1 · Nelson · 2005 [cited by applicant]
US 20050267731A1 · Hatcherson et al. · 2005 [cited by applicant]
US 20050286421A1 · Janacek · 2005 [cited by applicant]
US 20060092170A1 · Bathiche et al. · 2006 [cited by applicant]
US 20060189386A1 · Rosenberg · 2006 [cited by applicant]
US 20060192852A1 · Rosenthal et al. · 2006 [cited by applicant]
US 20060223635A1 · Rosenberg · 2006 [cited by applicant]
US 20060230073A1 · Gopalakrishnan · 2006 [cited by applicant]
US 20060235674A1 · Voigt · 2006 [cited by applicant]
US 20060287815A1 · Gluck · 2006 [cited by applicant]
US 20060293110A1 · Mendelsohn · 2006 [cited by applicant]
US 20070024644A1 · Bailey · 2007 [cited by applicant]
US 20070097832A1 · Koivisto et al. · 2007 [cited by applicant]
US 20070117576A1 · Huston · 2007 [cited by applicant]
US 20070214449A1 · Choi et al. · 2007 [cited by applicant]
US 20070223675A1 · Surin et al. · 2007 [cited by applicant]
US 20070265089A1 · Robarts et al. · 2007 [cited by applicant]
US 20070299559A1 · Janssen et al. · 2007 [cited by applicant]
US 20080026838A1 · Dunstan et al. · 2008 [cited by applicant]
US 20080031234A1 · Sbisa et al. · 2008 [cited by applicant]
US 20080036653A1 · Huston · 2008 [cited by applicant]
US 20080070696A1 · Kim et al. · 2008 [cited by applicant]
US 20080133189A1 · Criswell et al. · 2008 [cited by applicant]
US 20080146342A1 · Harvey et al. · 2008 [cited by applicant]
US 20080162498A1 · Omoigui · 2008 [cited by applicant]
US 20080162707A1 · Beck et al. · 2008 [cited by applicant]
US 20080177650A1 · Jung et al. · 2008 [cited by applicant]
US 20080182592A1 · Cha et al. · 2008 [cited by applicant]
US 20080184138A1 · Krzanowski et al. · 2008 [cited by applicant]
US 20080189360A1 · Kiley et al. · 2008 [cited by applicant]
US 20080220397A1 · Capone et al. · 2008 [cited by applicant]
US 20080247636A1 · Davis et al. · 2008 [cited by applicant]
US 20080261564A1 · Logan · 2008 [cited by applicant]
US 20080320419A1 · Matas et al. · 2008 [cited by applicant]
US 20090005018A1 · Forstall et al. · 2009 [cited by applicant]
US 20090005140A1 · Rose et al. · 2009 [cited by applicant]
US 20090005167A1 · Arrasvuori et al. · 2009 [cited by applicant]
US 20090007229A1 · Stokes · 2009 [cited by applicant]
US 20090024629A1 · Miyauchi · 2009 [cited by applicant]
US 20090036186A1 · Benco et al. · 2009 [cited by applicant]
US 20090061901A1 · Arrasvuori et al. · 2009 [cited by applicant]
US 20090061974A1 · Lutnick et al. · 2009 [cited by applicant]
US 20090069033A1 · Karstens et al. · 2009 [cited by applicant]
US 20090089825A1 · Coldwell · 2009 [cited by applicant]
US 20090102616A1 · Stone et al. · 2009 [cited by applicant]
US 20090125823A1 · Moll et al. · 2009 [cited by applicant]
US 20090150802A1 · Do et al. · 2009 [cited by applicant]
US 20090199302A1 · So et al. · 2009 [cited by applicant]
US 20090237546A1 · Bloebaum et al. · 2009 [cited by applicant]
US 20090265257A1 · Klinger et al. · 2009 [cited by applicant]
US 20090291750A1 · Herrmann · 2009 [cited by applicant]
US 20090293011A1 · Nassar · 2009 [cited by applicant]
US 20090304283A1 · Predovic et al. · 2009 [cited by applicant]
US 20100017820A1 · Thevathasan et al. · 2010 [cited by applicant]
US 20100050100A1 · Dettinger et al. · 2010 [cited by applicant]
US 20100125362A1 · Canora et al. · 2010 [cited by applicant]
US 20100199193A1 · Hamilton, II et al. · 2010 [cited by applicant]
US 20100217573A1 · Hatcherson et al. · 2010 [cited by applicant]
US 20100260426A1 · Huang · 2010 [cited by applicant]
US 20100302143A1 · Spivack · 2010 [cited by applicant]
US 20100304804A1 · Spivack · 2010 [cited by applicant]
US 20100331041A1 · Liao et al. · 2010 [cited by applicant]
US 20110055049A1 · Harper et al. · 2011 [cited by applicant]
US 20110138317A1 · Kang et al. · 2011 [cited by applicant]
US 20110173576A1 · Murphy · 2011 [cited by examiner]
US 20110184972A1 · Ard et al. · 2011 [cited by applicant]
US 20110213664A1 · Osterhout et al. · 2011 [cited by applicant]
US 20110214082A1 · Osterhout et al. · 2011 [cited by applicant]
US 20110225069A1 · Cramer et al. · 2011 [cited by applicant]
US 20110310227A1 · Konertz et al. · 2011 [cited by applicant]
US 20120011142A1 · Baheti et al. · 2012 [cited by applicant]
US 20120038668A1 · Kim · 2012 [cited by examiner]
US 20120038670A1 · Choi et al. · 2012 [cited by applicant]
US 20120050324A1 · Jeong et al. · 2012 [cited by applicant]
US 20120062596A1 · Bedi et al. · 2012 [cited by applicant]
US 20120095853A1 · Von Bose et al. · 2012 [cited by applicant]
US 20120143361A1 · Kurabayashi et al. · 2012 [cited by applicant]
US 20120174062A1 · Choi et al. · 2012 [cited by applicant]
US 20120230538A1 · Calman et al. · 2012 [cited by applicant]
US 20120239469A1 · Steinberg et al. · 2012 [cited by applicant]
US 20120249586A1 · Wither et al. · 2012 [cited by applicant]
US 20120253825A1 · Di Fabbrizio et al. · 2012 [cited by applicant]
US 20120323938A1 · Skeen et al. · 2012 [cited by applicant]
US 20120326966A1 · Rauber · 2012 [cited by applicant]
US 20130030903A1 · Spivack · 2013 [cited by applicant]
US 20130033522A1 · Calman · 2013 [cited by examiner]
US 20130042296A1 · Hastings et al. · 2013 [cited by applicant]
US 20130050258A1 · Liu et al. · 2013 [cited by applicant]
US 20130147836A1 · Small · 2013 [cited by examiner]
US 20130147837A1 · Stroila · 2013 [cited by applicant]
US 20130155105A1 · Boldyrev · 2013 [cited by examiner]
US 20130174268A1 · Wang et al. · 2013 [cited by applicant]
US 20130179303A1 · Petrou et al. · 2013 [cited by applicant]
US 20130239019A1 · Pike et al. · 2013 [cited by applicant]
US 20130293580A1 · Spivack · 2013 [cited by applicant]
US 20130297460A1 · Spivack · 2013 [cited by applicant]
US 20140009494A1 · Kasahara · 2014 [cited by applicant]
US 20140222558A1 · Spivack · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20150015459A1 · Cho et al. · 2015 [cited by applicant]
US 20150302517A1 · Spivack · 2015 [cited by applicant]
US 20150350223A1 · Spivack · 2015 [cited by applicant]
US 20150367233A1 · Hicks et al. · 2015 [cited by applicant]
US 20160071326A1 · Spivack · 2016 [cited by applicant]
US 20160299569A1 · Fisher et al. · 2016 [cited by applicant]
US 20190102946A1 · Spivack et al. · 2019 [cited by applicant]
US 20230153475A1 · Jovanovic · 2023 [cited by applicant]
US 20230154114A1 · Sears et al. · 2023 [cited by applicant]
US 20240320939A1 · Spivack · 2024 [cited by examiner]
EP 0926614A2 · 1999 [cited by applicant]
KR 1020110133655A · 2011 [cited by applicant]
WO 2006024866A1 · 2006 [cited by applicant]
WO 2009002879A1 · 2008 [cited by applicant]
International Search Report PCT/US2010/035282 dated Feb. 1, 2001; pp. 1-3. [cited by applicant]
Written Opinion PCT/US2010/035282 dated Feb. 1, 2011; pp. 1-6. [cited by applicant]
“Object-Oriented Programming,” as shown in http://en.wikipedia.org/wiki/Object-oriented_programming, dated Apr. 22, 2009, last accessed Nov. 4, 2013; pp. 1-9. [cited by applicant]
Hughes, N., “Apple Envisions Interactive, Augmented Reality iPhone Multiplayer Games,” http://appleinsider.com/articles/10/11/2014, Nov. 4, 2010, 6 pages. [cited by applicant]
Kaiser, E. Mutual disambiguation of 3 D multimodal interaction in augmented and virtual reality, Proceedings of the 5th international conference on multimodal interfaces (ICMI'03: Fifth International Conference on Multi… [cited by applicant]