Granted Patent B2

US 12,387,290 · App. 18/315,485 · Granted Aug 12, 2025

Inventor: Chia-Wei Hsiao (Hsinchu, TW)

Assignee: ASPEED Technology Inc.

G06T3/4038

Abstract

Claims (63)

1. An image processing device of generating a layout comprising a plurality of images, the image processing device comprising:

a transceiver, obtaining a calibration map and a vertex list corresponding to an input image, wherein the vertex list comprises a first vertex on the calibration map, a second vertex on the calibration map, a first data structure corresponding to the first vertex, and a second data structure corresponding to the second vertex;

a storage medium, storing a plurality of modules; and

a processor, coupled to the storage medium and the transceiver, and accessing and executing the plurality of modules, wherein the plurality of modules comprise:

a region of interest composer, receiving layout information through the transceiver to determine a first display region and a second display region in the layout;

a frame tile partition unit, partitioning the layout into a plurality of tiles;

a tile splitter, selecting a first tile located at a border of the first display region and the second display region from the plurality of tiles, and cropping a first sub-block corresponding to the first display region from the first tile;

a coordinate transformer, mapping the first sub-block to the calibration map to obtain a mapping region;

an attribute interpolator, performing an interpolation operation on the first data structure and the second data structure to obtain a third data structure corresponding to a third vertex in response to the third vertex in the mapping region corresponding to the first vertex and the second vertex;

a vertex generator, updating the vertex list according to the third vertex and the third data structure to generate a composed vertex list; and

an image processing module, generating an output image by mapping the input image to the layout according to the composed vertex list, and outputting the output image through the transceiver.

2. The image processing device according to claim 1 , wherein the plurality of modules further comprises:

a primitive assembly unit, partitioning the mapping region into a plurality of primitives according to a size of the mapping region, wherein the plurality of primitives comprise a first primitive with the third vertex.

3. The image processing device according to claim 1 , wherein the plurality of modules further comprise:

a primitive assembly unit, determining whether the mapping region comprises a splicing region of a plurality of original images according to the vertex list, and partitioning the mapping region into a plurality of primitives according to a size of the splicing region in response to the mapping region comprising the splicing region, wherein the plurality of primitives comprise a first primitive with the third vertex.

4. The image processing device according to claim 2 , wherein each of the plurality of primitives is a geometric structure with vertices.

5. The image processing device according to claim 1 , wherein the layout information comprises a first original display region and a second original display region in the layout, wherein

the region of interest composer partitions the first original display region into a plurality of sub-display regions comprising the first display region in response to the first original display region partial overlapped with the second original display region, wherein the plurality of sub-display regions are not overlapped with each other, and each of the plurality of sub-display regions is not overlapped with the second original display region.

6. The image processing device according to claim 5 , wherein the layout information comprises a first priority corresponding to the first original display region and a second priority corresponding to the second original display region, wherein

the region of interest composer partitions the first original display region in response to the first priority being lower than the second priority.

7. The image processing device according to claim 1 , wherein

the region of interest composer determines a projection method according to the layout information, wherein

the coordinate transformer maps the first sub-block to the calibration map to obtain the mapping region according to the projection method.

8. The image processing device according to claim 1 , wherein

the attribute interpolator determines a first weight corresponding to the first vertex and a second weight corresponding to the second vertex according to a distance between the third vertex and the first vertex in a first direction.

9. The image processing device according to claim 8 , wherein

the attribute interpolator calculates a first product of a first attribute value and the first weight in the first data structure and a second product of a second attribute value and the second weight in the second data structure to generate the third data structure.

10. The image processing device according to claim 8 , wherein the first data structure comprises a first scaling factor and a first blending weight corresponding to the first vertex, and the second data structure comprises a second scaling factor and a second blending weight corresponding to the second vertex, wherein

the attribute interpolator calculates a first product of the first blending weight and the first weight to obtain or update a first accumulated value corresponding to the first scaling factor, and calculates a second product of the second blending weight and the second weight to obtain or update a second accumulated value corresponding to the second scaling factor, wherein

the attribute interpolator selects the first scaling factor from the first scaling factor and the second scaling factor to generate the third data structure in response to the first accumulated value being greater than the second accumulated value.

11. The image processing device according to claim 10 , wherein the storage medium further stores a first memory array and a second memory array, wherein

after obtaining the first product, the attribute interpolator determines whether the first scaling factor matches a first address in the first memory array and determines whether data is stored in the first address, wherein

if the data is stored in the first address, in response to determining that the first scaling factor matches the first address, the attribute interpolator accumulates the first product to a second address corresponding to the first address in the second memory array, wherein

if the data is stored in the first address, in response to determining that the first scaling factor does not match the first address, the attribute interpolator stores the first scaling factor in a third address in the first memory array, and accumulates the first product to a fourth address corresponding to the third address in the second memory array, wherein

if the data is not stored in the first address, the attribute interpolator stores the first scaling factor in the first address, and accumulates the first product to the second address.

12. The image processing device according to claim 10 , wherein the storage medium further stores a first memory array and a second memory array, wherein the first scaling factor matches a first address in the first memory array, wherein

the attribute interpolator accumulates the first product to a second address corresponding to the first address in the second memory array according to a lookup table.

13. The image processing device according to claim 1 , wherein

the vertex generator generates a current frame header corresponding to a current frame of the output image according to the composed vertex list, wherein the current frame header indicates a third display region in the layout, wherein

the attribute interpolator copies data in the current frame header to generate a new composed vertex list corresponding to a next frame in response to a fourth display region in the next frame of the output image matching the third display region, wherein the data comprises a fourth vertex corresponding to the third display region and a fourth data structure corresponding to the fourth vertex.

14. The image processing device according to claim 1 , wherein the input image is an equirectangular projection image.

15. The image processing device according to claim 1 , wherein the layout information comprises at least:

coordinates, a width, and a height corresponding to a first region of interest of a source image; and

coordinates, a width, and a height corresponding to a display region of a target image.

16. An image processing method of generating a layout comprising a plurality of images, the image processing method comprising:

obtaining a calibration map and a vertex list corresponding to an input image, wherein the vertex list comprises a first vertex on the calibration map, a second vertex on the calibration map, a first data structure corresponding to the first vertex, and a second data structure corresponding to the second vertex;

receiving layout information to determine a first display region and a second display region in the layout;

partitioning the layout into a plurality of tiles;

selecting a first tile located at a border of the first display region and the second display region from the plurality of tiles, and cropping a first sub-block corresponding to the first display region from the first tile;

mapping the first sub-block to the calibration map to obtain a mapping region;

performing an interpolation operation on the first data structure and the second data structure to obtain a third data structure corresponding to a third vertex in response to the third vertex in the mapping region corresponding to the first vertex and the second vertex;

updating the vertex list according to the third vertex and the third data structure to generate a composed vertex list; and

generating an output image by mapping the input image to the layout according to the composed vertex list, and outputting the output image.

17. An image processing device of generating a layout comprising a plurality of images, the image processing device comprising:

a processor, configured to:

obtain a calibration map and a vertex list corresponding to an input image, wherein the vertex list comprises a first vertex on the calibration map, a second vertex on the calibration map, a first data structure corresponding to the first vertex, and a second data structure corresponding to the second vertex;

receive layout information to determine a first display region and a second display region in the layout;

partition the layout into a plurality of tiles;

select a first tile located at a border of the first display region and the second display region from the plurality of tiles, and crop a first sub-block corresponding to the first display region from the first tile;

map the first sub-block to the calibration map to obtain a mapping region;

perform an interpolation operation on the first data structure and the second data structure to obtain a third data structure corresponding to a third vertex in response to the third vertex in the mapping region corresponding to the first vertex and the second vertex;

update the vertex list according to the third vertex and the third data structure to generate a composed vertex list; and

generate an output image by mapping the input image to the layout according to the composed vertex list, and output the output image.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (14)