IP Library › Granted Patent US 12,278,936
Granted Patent B2
US 12,278,936 · App. 18/492,088 · Granted Apr 15, 2025

Information processing system, information processing method, and computer program

Inventor: Koji Ohata (Tokyo, JP)
Assignee: Sony Interactive Entertainment Inc.
H04N13/122H04N13/156H04N13/344H04N13/366
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,278,936
App. No.
18/492,088
Granted
Apr 15, 2025
Kind
B2
Abstract

Methods and/or apparatus provide for acquiring a first image obtained by imaging an event taking place in a real space; generating a display image by merging the first image into a second image representing a virtual space that establishes an alternative version by: (i) reproducing real elements that are a focus of the event, (ii) omitting real peripheral elements, and (iii) presenting alternative virtual elements, such that the first image has an inconspicuous border when the display image is presented to a user.

Claims (25)

1. An information processing system comprising:

an acquisition section configured to acquire a first image obtained by imaging an event taking place in a real space, where the real space includes real elements that are a focus of the event and real peripheral elements establishing a venue having an entertainment setting in which the event takes place, where real elements that are the focus of the event are circumscribed by a peripheral boundary and the real peripheral elements of the venue are not within the peripheral boundary;

a generation section configured to generate a display image by merging the first image into a second image representing a virtual space that establishes an alternative version of the entertainment setting of the venue where the event takes place by: (i) reproducing the real elements that are the focus of the event up to the peripheral boundary, (ii) omitting the real peripheral elements of the venue are not within the peripheral boundary, and (iii) presenting alternative virtual elements from the peripheral boundary and outward instead of the real peripheral elements, such that the first image has an inconspicuous border at the peripheral boundary when the display image is presented to a user; and

an output section configured to display the display image generated by the generation section to the user.

2. The information processing system according to claim 1 , wherein the second image includes an object covering at least part of the border of the first image.

3. The information processing system according to claim 2 , wherein the object is an image of an item corresponding to a type of the virtual space represented by the second image.

4. The information processing system according to claim 2 , wherein

the first image is obtained by imaging an event taking place on stage, and

the object is an image representing furnishings on a stage.

5. The information processing system according to claim 1 , wherein the generation section generates a display image in which at least part of a boundary between the first image and the second image is unclear.

6. The information processing system according to claim 5 , wherein the generation section choose, as a color of a portion of the second image that is close to the boundary with the first image, a color closely analogous to a color of a portion of the first image that is close to the boundary.

7. The information processing system according to claim 5 , wherein the generation section sets the portion of the second image that is close to the boundary with the first image, to a relatively low image quality.

8. The information processing system according to claim 1 , the wherein the display image is presented to the user on a head-mounted display.

9. An information processing method causing a computer to perform:

acquiring a first image obtained by imaging an event taking place in a real space, where the real space includes real elements that are a focus of the event and real peripheral elements establishing a venue having an entertainment setting in which the event takes place, where real elements that are the focus of the event are circumscribed by a peripheral boundary and the real peripheral elements of the venue are not within the peripheral boundary;

generating a display image by merging the first image into a second image representing a virtual space that establishes an alternative version of the entertainment setting of the venue where the event takes place by: (i) reproducing the real elements that are the focus of the event up to the peripheral boundary, (ii) omitting the real peripheral elements of the venue are not within the peripheral boundary, and (iii) presenting alternative virtual elements from the peripheral boundary and outward instead of the real peripheral elements, such that the first image has an inconspicuous border at the peripheral boundary when the display image is presented to a user; and

displaying the display image to the user.

10. A non-transitory, computer readable storage medium containing a computer program, which when executed by a computer, causes the computer to perform an information processing method by carrying out actions, comprising:

acquiring a first image obtained by imaging an event taking place in a real space, where the real space includes real elements that are a focus of the event and real peripheral elements establishing a venue having an entertainment setting in which the event takes place, where real elements that are the focus of the event are circumscribed by a peripheral boundary and the real peripheral elements of the venue are not within the peripheral boundary;

generating a display image by merging the first image into a second image representing a virtual space that establishes an alternative version of the entertainment setting of the venue where the event takes place by: (i) reproducing the real elements that are the focus of the event up to the peripheral boundary, (ii) omitting the real peripheral elements of the venue are not within the peripheral boundary, and (iii) presenting alternative virtual elements from the peripheral boundary and outward instead of the real peripheral elements, such that the first image has an inconspicuous border at the peripheral boundary when the display image is presented to a user; and

displaying the display image to the user.

11. A server comprising:

an acquisition section configured to acquire a first image obtained by imaging an event taking place in a real space, where the real space includes real elements that are a focus of the event and real peripheral elements establishing a venue having an entertainment setting in which the event takes place, where real elements that are the focus of the event are circumscribed by a peripheral boundary and the real peripheral elements of the venue are not within the peripheral boundary;

a generation section configured to generate a display image by merging the first image into a second image representing a virtual space that establishes an alternative version of the entertainment setting of the venue where the event takes place by: (i) reproducing the real elements that are the focus of the event up to the peripheral boundary, (ii) omitting the real peripheral elements of the venue are not within the peripheral boundary, and (iii) presenting alternative virtual elements from the peripheral boundary and outward instead of the real peripheral elements, such that the first image has an inconspicuous border at the peripheral boundary when the display image is presented to a user; and

an output section configured to display the display image generated by the generation section.

Continuity (2)
Continuation 17312111
Related Publication 20240048677A1 · Feb 8, 2024
References Cited (39)
US 10636185B2 · Wu · 2020 [cited by applicant]
US 10748341B2 · Ogasawara · 2020 [cited by examiner]
US 10796489B1 · Cordes · 2020 [cited by examiner]
US 11093025B2 · Kakehashi · 2021 [cited by examiner]
US 11263813B2 · Ohashi · 2022 [cited by examiner]
US 11288733B2 · Scholl · 2022 [cited by examiner]
US 11393176B2 · Shriram · 2022 [cited by examiner]
US 20020177481A1 · Kitsutaka · 2002 [cited by examiner]
US 20030020716A1 · Ouchi · 2003 [cited by examiner]
US 20040105579A1 · Ishii · 2004 [cited by examiner]
US 20120044260A1 · Hirai · 2012 [cited by applicant]
US 20120182399A1 · Andersson · 2012 [cited by examiner]
US 20140092221A1 · Nagai · 2014 [cited by examiner]
US 20140362188A1 · Yokokawa · 2014 [cited by examiner]
US 20150294492A1 · Koch · 2015 [cited by examiner]
US 20160379407A1 · Foster · 2016 [cited by examiner]
US 20170053452A1 · Arai · 2017 [cited by examiner]
US 20170193679A1 · Wu · 2017 [cited by applicant]
US 20180321798A1 · Kawamura · 2018 [cited by applicant]
US 20190358547A1 · Mack · 2019 [cited by examiner]
US 20200167996A1 · Watanabe · 2020 [cited by examiner]
US 20200371737A1 · Leppanen · 2020 [cited by examiner]
US 20200410960A1 · Saito · 2020 [cited by examiner]
US 20210204014A1 · Kawakami · 2021 [cited by examiner]
US 20210211632A1 · Kawakami · 2021 [cited by examiner]
US 20210248788A1 · Imao · 2021 [cited by examiner]
US 20210248803A1 · Kojima · 2021 [cited by examiner]
EP 0920198A1 · 1999 [cited by applicant]
JP 07168121A · 1995 [cited by applicant]
JP 10229535A · 1998 [cited by applicant]
JP 2004328241A · 2004 [cited by applicant]
JP 2012044407A · 2012 [cited by applicant]
JP 2015228050A · 2015 [cited by applicant]
JP 2016062486A · 2016 [cited by applicant]
JP 2017138913A · 2017 [cited by applicant]
WO 2017029918A1 · 2017 [cited by applicant]
WO 2017110632A1 · 2017 [cited by applicant]
International Search Report for corresponding PCT Application PCT/JP2018/046288, 4 pages, dated Feb. 26, 2019. [cited by applicant]
International Preliminary Report on Patentability and Written Opinion for corresponding PCT Application PCT/JP2018/046288, 15 pages, dated Jul. 1, 2021. [cited by applicant]