METHODS AND APPARATUS FOR MAKING ENVIRONMENTAL MEASUREMENTS AND/OR USING SUCH MEASUREMENTS IN 3D IMAGE RENDERING
Methods and apparatus for making and using environmental measurements are described. Environmental information captured using a variety of devices is processed and combined to generate an environmental model which is communicated to customer playback devices. A UV map which is used for applying, e.g., wrapping, images onto the environmental model is also provided to the playback devices. A playback device uses the environmental model and UV map to render images which are then displayed to a viewer as part of providing a 3D viewing experience. In some embodiments updated environmental model is generated based on more recent environmental measurements, e.g., performed during the event. The updated environmental model and/or difference information for updating the existing model, optionally along with updated UV map(s), is communicated to the playback devices for use in rendering and playback of subsequently received image content. By communicating updated environmental information improved 3D simulations are achieved.
1 . A method of operating a playback device, the method comprising:
receiving information communicating a first mesh model of a 3D environment generated based on measurements of a portion of said environment made using a light field camera at a first time;
receiving image content; and
rendering, using said first mesh model at least some of the received image content.
2 . The method of claim 1 , further comprising:
receiving updated mesh model information, said updated mesh model information including at least some updated mesh model information generated based on measurements of said portion of said environment using said light field camera at a second time.
3 . The method of claim 2 , further comprising:
receiving additional image content; and
rendering, using said updated mesh model information at least some of the received additional image content.
4 . The method of claim 3 , wherein said information communicating a first mesh model of the 3D environment includes information defining a complete mesh model.
5 . The method of claim 4 , wherein said updated mesh model information communicates a complete updated mesh model.
6 . The method of claim 5 , wherein said updated mesh model information provides new mesh information for portions of said 3D environment which have changed between said first and second time periods.
7 . The method of claim 6 , wherein said updated mesh model information is difference information indicating a difference between said first mesh model and an updated mesh model.
8 . The method of claim 7 , wherein said first mesh model information includes a first set of coordinate triples, each coordinate triple indicating a coordinate in X, Y, Z space of a node in the first mesh model.
9 . The method of claim 8 , wherein said updated mesh model information includes at least one of: i) new sets of mesh coordinates for at least some nodes in said first mesh model information, said new coordinates being intended to replace coordinates of corresponding nodes in said first mesh model; or ii) a new set of coordinate triples to be used for at least a portion of said first mesh model in place of a previous set of coordinate triples, said new set of coordinate triples including the same or a different number of coordinate triples than the previous set of coordinate triples to be replaced.
10 . The method of claim 9 , further comprising:
receiving a first map mapping a 2D image space to said first mesh model; and
wherein rendering, using said first mesh model at least some of the received image content, includes using said first map to determine how to wrap an image included in said received image content onto said first mesh model.
11 . The method of claim 10 , further comprising:
receiving updated map information corresponding to said updated mesh model information; and
wherein rendering, using said updated mesh model information at least some of the received additional image content, includes using said updated map information to determine how to wrap an additional image included in said received additional image content onto said updated mesh model.
12 . The method of claim 11 , wherein the updated map information includes map difference information, the method further comprising:
generating an updated map by applying said map difference information to said first map to generate an updated map; and
wherein rendering, using said updated mesh model information, at least some of the received additional image content, includes using said updated map to determine how to wrap an additional image included in said received additional image content onto said updated mesh model.
13 . A computer readable medium including computer executable instructions which, when executed by a computer, control the computer to:
receive information communicating a first mesh model of a 3D environment generated based on measurements of a portion of said environment made using a light field camera at a first time;
receive image content; and
render, using said first mesh model at least some of the received image content.
14 . A playback apparatus, comprising:
a processor configured to control said playback apparatus to:
receive information communicating a first mesh model of a 3D environment generated based on measurements of a portion of said environment made using a light field camera at a first time;
receive image content; and
render, using said first mesh model at least some of the received image content.
15 . The playback apparatus of claim 14 , wherein the processor is further configured to control the playback apparatus to:
receive updated mesh model information, said updated mesh model information including at least some updated mesh model information generated based on measurements of the portion of said environment using said light field camera at a second time.
16 . The playback apparatus of claim 15 , wherein the processor is further configured to control the playback apparatus to:
receive additional image content; and
render, using said updated mesh model information, at least some of the received additional image content.
17 . The playback apparatus of claim 14 , wherein the processor is further configured to control the playback apparatus to:
receive a first map mapping a 2D image space to said first mesh model; and
use said first map to determine how to wrap an image included in said received image content onto said first mesh model as part of being configured to render, using said first mesh model, at least some of the received image content.
18 . The playback apparatus of claim 17 , wherein the processor is further configured to control the playback apparatus to:
receive updated map information corresponding to said updated mesh model information; and
use said updated map information to determine how to wrap an additional image included in said received additional image content onto said updated mesh model as part of being configured to render, using said updated mesh model information, at least some of the received additional image content.
19 . The playback apparatus of claim 18 , wherein the updated map information includes map difference information; and
wherein the processor is further configured to control the playback apparatus to:
generate an updated map by applying said map difference information to said first map to generate an updated map; and
use said updated map to determine how to wrap an additional image included in said received additional image content onto said updated mesh model as part of rendering, using said updated mesh model information, at least some of the received additional image content.
20 . The playback apparatus of claim 16 , wherein said information communicating a first mesh model of the 3D environment includes information defining a complete mesh model.