Updating background texture for virtual viewpoint animations
View Patent ↗Image data of an event is provided by updating a textured 3d model of the event. For example, in a sporting event, a model of a stadium can be periodically updated to reflect changes over time in lighting, advertisements, number of spectators in the stands and so forth. Different virtual viewpoints of the event can be depicted in an animation using the textured 3d model and image data from objects at the event such as participants in the sporting event. The same image from which object data is obtained can also be used to update the textured 3d model so that the model is current in the animation, resulting in greater realism. The updating can be based on an operator command or automatic detection of a specified event, such as change in lighting or passage of time. The animation can be provided in a broadcast television signal.
1. A method for providing image data, comprising:
providing a 3d model of a stadium in which an event is held;
obtaining camera images of the stadium, the camera images comprising a first image which is captured at a first time and a second image which is captured at a second time after the first time;
providing a texture of the 3d model of the stadium using the first image, to provide a first textured 3d model of the stadium;
receiving a user command indicating that the second image is to be used in an animation, the second image comprises an image of a player in the stadium;
providing a model of the player;
in response to the user command, texturing the model of the player using the second image to provide a textured model of the player;
automatically detecting whether there is a change in an appearance of the stadium which triggers updating of the texture of the 3d model of the stadium;
if the automatically detecting indicates that there is the change in the appearance of the stadium which triggers automatic updating of the texture of the 3d model of the stadium, automatically updating the texture of the 3d model of the stadium using an image from the camera to provide an updated textured 3d model of the stadium, and causing the animation to depict the textured model of the player and the updated textured 3d model of the stadium in a sequence of virtual viewpoints of the event; and
if the automatically detecting indicates that there is not the change in the appearance of the stadium which triggers the automatic update of the texture of the 3d model of the stadium, causing the animation to depict the textured model of the player and the first textured 3d model of the stadium in the sequence of virtual viewpoints of the event.
2. The method of claim 1 , wherein:
the 3d model of the stadium comprises a playing field and a grandstand.
3. The method of claim 1 , wherein:
the first and second images are from different cameras having different viewpoints of the stadium, the sequence of virtual viewpoints differ from the different viewpoints.
4. The method of claim 1 , wherein:
the 3d model of the stadium comprises a roof; and
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises automatically detecting whether there is a movement of the roof.
5. The method of claim 1 , wherein:
the 3d model of the stadium comprises a sign; and
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises automatically detecting whether there is a change in the sign.
6. The method of claim 1 , wherein:
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises using a timing device to determine if a specified time of day has been reached.
7. The method of claim 1 , wherein:
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises using a timing device to determine if a specified time period is completed since a last update of the texture of the 3d model of the stadium.
8. The method of claim 1 , wherein:
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises using a light sensor to detect a change in an ambient light level at the stadium.
9. The method of claim 1 , wherein:
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises using a sensor to detect whether electric lights have been turned on at the stadium.
10. The method of claim 1 , wherein:
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises automatically detecting a change in a number of fans in stands of the stadium.
11. The method of claim 1 , wherein:
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises automatically detecting a change in an advertising board of a grandstand in the stadium.
12. The method of claim 1 , wherein:
the providing the texture of the 3d model of the stadium using the first image is performed in response to a user command made via a user interface; and
the automatically updating the texture of the 3d model of the stadium is not performed in response to a user command.
13. The method of claim 1 , wherein:
the 3d model of the stadium comprises an advertisement; and
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises detecting whether there is a change in the advertisement.
14. The method of claim 1 , wherein:
the automatically detecting whether there is the change in the appearance of the stadium which triggers updating of the texture of the 3d model of the stadium comprises automatically detecting a reconfiguration of the stadium.
15. A system for providing image data, comprising:
a camera which obtains images of a stadium in which an event is held, the images comprising a first image which is captured at a first time and second image which is captured at a second time which is after the first time; and
a processing facility, the processing facility is configured to:
provide a texture of a 3d model of the stadium using the first image, to provide a first textured 3d model of the stadium,
receive a user command to provide an animation and to use the second image in the animation, the second image comprises an image of a participant in the stadium; and
in response to the user command, texture a model of the participant using the second image to provide a textured model of the participant, and provide the animation, wherein:
the animation depicts the textured model of the participant in a sequence of virtual viewpoints of the event,
if there is a change in an appearance of the stadium which triggers an automatic update of the texture of the 3d model of the stadium, the animation depicts the textured model of the participant with an updated textured 3d model of the stadium in the sequence of virtual viewpoints of the event, and
if there is not the change in the appearance of the stadium which triggers the automatic update of the texture of the 3d model of the stadium, the animation depicts the textured model of the participant with the first textured 3d model of the stadium in the sequence of virtual viewpoints of the event.
16. The system of claim 15 , wherein:
the event comprises a sporting event, and the participant is a player in the sporting event.
17. The system of claim 15 , wherein:
the 3d model of the stadium comprises a playing field, a grandstand and an advertising board of the grandstand; and
the change in the appearance of the stadium which triggers the automatic update of the texture of the 3d model of the stadium comprises a change in an appearance of the advertising board.
18. The system of claim 15 , further comprising:
a light sensor which detects an ambient light level at the stadium, the processing facility is responsive to the light sensor to detect whether there is the change in the appearance of the stadium which triggers the automatic update of the texture of the 3d model of the stadium.
19. The system of claim 15 , further comprising:
a timing device, the processing facility is responsive to the timing device indicating that a specified time of day has been reached to detect whether there is the change in the appearance of the stadium which triggers the automatic update of the texture of the 3d model of the stadium.
20. The system of claim 15 , further comprising:
a sensor which detects whether electric lights have been turned on at the stadium, the processing facility is responsive to the sensor to detect whether there is the change in the appearance of the stadium which triggers the automatic update of the texture of the 3d model of the stadium.
21. The system of claim 15 , further comprising:
a timing device, wherein the automatic update of the texture of the 3d model of the stadium is responsive to the timing device indicating that a specified time period has passed since a last update of the texture of the 3d model of the stadium.
22. A processor readable storage device comprising processor readable code embodied thereon for programming a processor to perform a method for providing image data, the method comprising:
providing a 3d model of a stadium in which an event is held;
obtaining camera images of the stadium, the camera images comprising a first image which is captured at a first time and a second image which is captured at a second time after the first time;
providing a first texture of the 3d model of the stadium using the first image;
receiving a user command to provide an animation and to use the second image in the animation, the second image comprises an image of a player in the stadium;
automatically detecting that there is a change in an appearance of the stadium;
in response to the automatically detecting that there is the change in the appearance of the stadium, providing an updated texture of the 3d model of the stadium in place of the first texture; and
when the user command is made,
if the 3d model of the stadium has the updated texture, causing the animation to depict data from the image of the player and the 3d model of the stadium with the updated textured in a sequence of virtual viewpoints of the event, and
if the 3d model of the stadium has the first texture, causing the animation to depict data from the image of the player and the 3d model of the stadium with the first texture in the sequence of virtual viewpoints of the event.
23. The processor readable storage device of claim 22 , wherein:
the providing the first texture of the 3d model of the stadium using the first image is performed in response to a user command made via a user interface; and
the providing the updated texture of the 3d model of the stadium in place of the first texture is not performed in response to a user command.
24. The processor readable storage device of claim 22 , wherein:
when the user command is made, texturing a model of the player using the second image to provide a textured model of the player, and
the data from the image of the player comprises the textured model of the player.