IP Library Granted Patent US 12,041,361
Granted Patent B2
US 12,041,361 · App. 17/067,154 · Granted Jul 16, 2024

Generating training data using beam splitter with multiple resolution sensors

Inventor: Joseph Wise (Los Angeles, CA)
Assignees: Sony Group Corporation; Sony Pictures Entertainment Inc.
H04N23/951G06N20/00H04N7/0117H04N23/90
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,041,361
App. No.
17/067,154
Granted
Jul 16, 2024
Kind
B2
Abstract

Systems, methods, and apparatus to generate training data include receiving a beam of light reflected from a subject and splitting the beam of light into at least a first beam and a second beam; directing the first beam toward a first sensor and the second beam toward a second sensor; sensing and processing the first beam to generate a first video having a first characteristic; sensing and processing the second beam to generate a second video having a second characteristic; and processing the first video and the second video to generate training data.

Claims (37)

1. A system to generate training data for developing a machine learning video conversion process used in video production, the system comprising:

a beam splitter to receive a beam of light reflected from a subject, and to split the beam of light into two identical beams having an identical spatial perspective, a first beam and a second beam;

a plurality of sensors including a first sensor and a second sensor,

the first sensor configured to sense and process the first beam from the beam splitter to generate a first video having a first resolution, the first sensor to store the first video in an archive, and

the second sensor configured to sense and process the second beam from the beam splitter to generate a second video having a second resolution, the second sensor to store the second video in the archive,

wherein the first and second resolutions of the first and second sensors are selected based on resolutions of input and output data for the machine learning video conversion process,

wherein having the identical spatial perspective obviates a need for calibration of positions of the first and second sensors; and

a converter to receive the first stored video and the second stored video from the archive, and to generate the training data having two sample videos for the machine learning video conversion process.

2. The system of claim 1 , wherein the converter further comprises a formatter to format the first video and the second video into the training data for the machine learning video conversion process.

3. The system of claim 1 , wherein the archive includes a television archive including one or more assets for television programs or other television content.

4. The system of claim 3 , wherein the converter uses the training data to convert a television program from the first resolution to the second resolutions.

5. The system of claim 1 , wherein the first sensor and the second sensor are configured as programmable sensors to sense different colors.

6. The system of claim 1 , wherein the plurality of sensors includes a combination of cameras and sensors.

7. The system of claim 6 , wherein the sensors include at least one of infra-red (IR) sensors, laser sensors, and laser imaging, detection, and ranging (LIDAR) sensors.

8. A method to generate training data for developing a machine learning video conversion process used in video production, the method comprising:

receiving a beam of light reflected from a subject and splitting the beam of light into at least two identical beams having an identical spatial perspective, a first beam and a second beam;

directing the first beam toward a first sensor and the second beam toward a second sensor,

wherein having the identical spatial perspective obviates a need for calibration of positions of the first and second sensors;

sensing and processing the first beam by a first sensor to generate a first video having a first resolution;

storing the first video in an archive;

sensing and processing the second beam by a second sensor to generate a second video having a second resolution;

storing the second video in the archive;

processing the first stored video and the second stored video to generate the training data having two sample videos for the machine learning video conversion process using the two identical beams and each beam being processed by one of the first and second sensors.

9. The method of claim 8 , further comprising

transmitting the training data to a machine learning system.

10. The method of claim 9 , wherein the machine learning system includes the video conversion process.

11. The method of claim 8 , wherein the archive includes a television archive including one or more assets for television programs or other television content.

12. An apparatus to generate training data for developing a machine learning video conversion process used in video production, the apparatus comprising:

means for receiving and splitting a beam of light reflected from a subject into at least two identical beams having an identical spatial perspective, a first beam and a second beam;

means for directing the first beam toward a first sensor and the second beam toward a second sensor,

wherein having the identical spatial perspective obviates a need for calibration of positions of the first and second sensors;

first means for sensing and processing the first beam to generate a first video having a first resolution;

second means for sensing and processing the second beam to generate a second video having a second resolution;

means for storing the first and second video; and

means for processing the first stored video and the second stored video to generate the training data having two sample videos for the machine learning video conversion process using the two identical beams and each beam being processed by one of the first and second means for sensing and processing.

13. The apparatus of claim 12 , further comprising

means for transmitting the training data to a machine learning system.

Assignments (2)
CHANGE OF NAME Recorded May 16, 2023
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 063665/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2020
From: WISE, JOSEPH
To: SONY CORPORATION; SONY PICTURES ENTERTAINMENT INC.
Reel/Frame 054070/0329 →
Continuity (2)
Provisional Application 62947767 · Dec 13, 2019
Related Publication 20210185225A1 · Jun 17, 2021