Granted Patent B2

US 12,227,134 · App. 17/469,263 · Granted Feb 18, 2025

Inventors: Sunghwan Choi (Seoul, KR); Namyong Park (Seoul, KR); Dukyung Jung (Seoul, KR)

Assignee: LG ELECTRONICS INC.

B60R1/27B60R11/04H04N23/61H04N23/632H04N23/635H04N23/698

Abstract

Claims (78)

1. An image processing device comprising:

a memory configured to store a first image transmitted from a first camera and a second image transmitted from a second camera with a wider view angle than the first camera; and

a processor configured to process the first and second images stored in the memory,

wherein the processor is configured to:

based on a target being specified in a first area in which the first image and the second image overlap each other within an augmented reality screen generated based on at least one of the first image or the second image, extract target information regarding the target based on data for the first image;

generate augmented reality data based on the extracted target information so that a visual object corresponding to the target is displayed on the first area; and

based on the target exiting the first area and entering a second area of the augmented reality screen, track a position of the target based on the extracted target information and generate the augmented reality data so that the visual object is displayed on the second area,

wherein the processor is further configured to configure the augmented reality screen to fit a screen size of a display based on the first image and the second image,

wherein the augmented reality screen is configured to include (i) the first area in which the first image and the second image overlap each other and (ii) the second area in which the first image and the second image do not overlap each other,

wherein the processor is further configured to crop upper and lower portions of the first area, in which the first image and the second image overlap each other, and generate left and right portions of the first area as the second area in an area in which the first image and the second image do not overlap each other, and

wherein the processor is further configured to generate, in the second area, symbol images for objects of the second area.

2. The image processing device of claim 1 , wherein the memory is further configured to receive a third image from a third camera and store the third image, and

wherein the processor is further configured to:

obtain object information regarding objects in the second area from the third image, and

generate, as the augmented reality data, symbol images for the objects to be displayed on the second area based on the object information.

3. The image processing device of claim 1 , wherein the target information includes at least one of a shape, a type, a position, or a distance of the target.

4. The image processing device of claim 3 , wherein the processor is further configured to change a size of the visual object based on a distance to the target or the type of the target.

5. The image processing device of claim 1 , wherein the processor is further configured to, based on the target entering the second area, correct the position of the target in the second area based on the second image corresponding to the second area.

6. The image processing device of claim 1 , wherein the processor is further configured to specify a target included in the first image based on input information obtained from a passenger, and

wherein the input information includes at least one of voice information, gaze information, gesture information, or touch information of the passenger.

7. The image processing device of claim 1 , wherein the memory is further configured to receive a fourth image from a fourth camera and store the fourth image, and

wherein the processor is further configured to generate an image of the first area using the fourth image in addition to the first and second images or instead of the first and second images.

8. The image processing device of claim 1 , wherein the processor is further configured to transmit, to a server, image data of an area specified by a passenger in the first area and receive, from the server, a type of a target included in the specified area and a visual object of the target.

9. An image processing device comprising:

a memory configured to store a first image transmitted from a first camera and a second image transmitted from a second camera with a wider view angle than the first camera; and

a processor configured to process the first and second images stored in the memory,

wherein the processor is configured to:

based on a target being specified in a first area in which the first image and the second image overlap each other within an augmented reality screen generated based on at least one of the first image or the second image, extract target information regarding the target based on data for the first image;

generate augmented reality data based on the extracted target information so that a visual object corresponding to the target is displayed on the first area; and

based on the target exiting the first area and entering a second area of the augmented reality screen, track a position of the target based on the extracted target information and generate the augmented reality data so that the visual object is displayed on the second area,

wherein the processor is further configured to specify a target included in the first image based on input information obtained from a passenger,

wherein the input information includes at least one of voice information, gaze information, gesture information, or touch information of the passenger, and

wherein the processor is further configured to:

obtain the voice information from the passenger,

obtain the gaze information of the passenger based on the voice information, and

specify a target corresponding to the gaze information in the first image.

10. An image processing method comprising:

receiving a first image and a second image from a first camera and a second camera with a wider view angle than the first camera, respectively;

based on a target being specified in a first area in which the first image and the second image overlap each other within an augmented reality screen generated based on at least one of the first image or the second image, extracting target information regarding the target based on data for the first image;

generating augmented reality data based on the extracted target information so that a visual object corresponding to the target is displayed on the first area; and

based on the target exiting the first area and entering a second area of the augmented reality screen, tracking a position of the target based on the extracted target information and generating the augmented reality data so that the visual object is displayed on the second area,

configuring the augmented reality screen to fit a screen size of a display based on the first image and the second image, wherein the augmented reality screen is configured to include (i) the first area in which the first image and the second image overlap each other and (ii) the second area in which the first image and the second image do not overlap each other,

wherein configuring the augmented reality screen comprises:

cropping upper and lower portions of the first area, in which the first image and the second image overlap each other;

generating left and right portions of the first area as the second area in an area in which the first image and the second image do not overlap each other; and

generating, in the second area, symbol images for objects of the second area.

11. The image processing method of claim 10 , further comprising:

receiving a third image from a third camera;

obtaining object information regarding objects in the second area from the third image; and

generating, as the augmented reality data, symbol images for the objects to be displayed on the second area based on the object information.

12. The image processing method of claim 10 , wherein the target information includes one or more of a shape, a type, a position, or a distance of the target, and

wherein a size of the visual object is changed based on a distance to the target or the type of the target.

13. The image processing method of claim 10 , further comprising:

based on the target entering the second area, correcting the position of the target in the second area based on the second image corresponding to the second area.

14. The image processing method of claim 10 , further comprising:

specifying a target included in the first image based on input information obtained from a passenger,

wherein the input information includes at least one of voice information, gaze information, gesture information, or touch information of the passenger.

15. The image processing method of claim 10 , further comprising:

receiving a fourth image from a fourth camera; and

generating an image of the first area using the fourth image in addition to the first and second images or instead of the first and second images.

16. The image processing method of claim 10 , further comprising:

Transmitting, to a server, image data of an area specified by a passenger in the first area; and

receiving, from the server, a type of a target included in the specified area and a visual object of the target.

17. An image processing device comprising:

a first camera configured to output a first image;

a second camera that is configured to output a second image and that has a wider view angle than the first camera;

a memory configured to store the first image and the second image;

a processor configured to generate an augmented reality screen based on at least one of the first image or the second image stored in the memory and generate augmented reality data for a specified target in the first image; and

a display configured to display the augmented reality data by overlapping the augmented reality data with the augmented reality screen,

wherein the processor is further configured to:

based on the target being specified in a first area in which the first image and the second image overlap each other within the augmented reality screen, extract target information regarding the target based on data for the first image;

generate the augmented reality data based on the extracted target information so that a visual object corresponding to the target is displayed on the first area;

based on the target exiting the first area and entering a second area of the augmented reality screen, track a position of the target based on the extracted target information; and

generate the augmented reality data so that the visual object is displayed on the second area,

wherein the processor is further configured to configure the augmented reality screen to fit a screen size of a display based on the first image and the second image,

wherein the augmented reality screen is configured to include (i) the first area in which the first image and the second image overlap each other and (ii) the second area in which the first image and the second image do not overlap each other,

wherein the processor is further configured to crop upper and lower portions of the first area, in which the first image and the second image overlap each other, and generate left and right portions of the first area as the second area in an area in which the first image and the second image do not overlap each other, and

wherein the processor is further configured to generate, in the second area, symbol images for objects of the second area.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (17)