System and method of controlling construction machinery
A control system for construction machinery includes an upper camera installed in a driver cabin in a rear body to photograph the front of the driver cabin, a lower camera installed in a front body rotatably connected to the rear body to photograph the front of the front body, an image processing device configured to synthesize first and second images captured from the upper camera and the lower camera into one image, and configured to detect a shape of a front work apparatus in the first image to determine a steering angle of the front body and determine a position of a transparency processing area in which at least one of the first and second images is transparency-processed in the synthesized image according to the steering angle, and a display device configured to display the synthesized image transparency-processed by the image processing device.
1 . A control system for construction machinery, the control system comprising:
an upper camera installed in a driver cabin in a rear body to photograph the front of the driver cabin;
a lower camera installed in a front body rotatably connected to the rear body through a center pin to photograph the front of the front body;
an image processing device configured to synthesize first and second images captured from the upper camera and the lower camera into one image, and configured to detect a shape of a front work apparatus in the first image to determine a steering angle of the front body and determine a position of a transparency processing area in which at least one of the first and second images is transparency-processed in the synthesized image according to the steering angle; and
a display device configured to display the synthesized image transparency-processed by the image processing device,
wherein the upper camera is installed to coincide with a central axis of the center pin or in the rear of the central axis and the lower camera is installed in the front of the central axis,
wherein the upper camera and the lower camera are installed at a different height from each other,
wherein the image processing device includes:
a shape recognizer configured to recognize the shape of the front work apparatus from the first image to determine the steering angle of the front body; and
a transparency processor configured to determine the position of the transparent processing area in the synthesized image according to the determined steering angle, and
wherein the shape recognizer compares an actual image of the front work apparatus in the first image received from the upper camera with a learning image of the front work apparatus recognized and stored in advance by machine learning to determine the steering angle of the front work apparatus.
2 . The control system for construction machinery of claim 1 , wherein the image processing device further includes a storage portion configured to store a learning image of the front work apparatus by executing a deep learning algorithm using the actual image received from the shape recognizer as input data.
3 . The control system for construction machinery of claim 1 , further comprising:
a work apparatus posture detection portion configured to detect a posture of the front work apparatus,
wherein the image processing device transparency-processes at least one of the first and second images in the transparency processing area according to the posture of the work apparatus detected by the work apparatus posture detection portion.
4 . The control system for construction machinery of claim 3 , wherein the image processing device transparency-processes the first image in the transparency processing area when at least a portion of the front work apparatus invades a predetermined position, and the image processing device transparency-processes the second image in the transparency processing area when the front work apparatus does not invade the predetermined position.
5 . The control system for construction machinery of claim 1 , further comprising:
an input portion configured to set an image processing condition in the image processing device.
6 . The control system for construction machinery of claim 5 , wherein the image processing condition includes a transparency processing switching timing of the first and second images or a size of the transparency processing area of the entire display area of the display device.
7 . A control system for construction machinery, the control system comprising:
an upper camera installed in a driver cabin in a rear body to photograph the front of the driver cabin;
a lower camera installed in a front body rotatably connected to the rear body through a center pin to photograph the front of the front body;
an image processing device configured to synthesize first and second images captured from the upper camera and the lower camera into one image, and configured to detect a shape of a front work apparatus in the first image to determine whether or not the front body turns and determine a position of a transparency processing area in which at least one of the first and second images is transparency-processed in the synthesized image depending on whether or not the front body turns; and
a display device configured to display the synthesized image transparency-processed by the image processing device,
wherein the upper camera is installed to coincide with a central axis of the center pin or in the rear of the central axis and the lower camera is installed in the front of the central axis,
wherein the upper camera and the lower camera are installed at a different height from each other,
wherein the image processing device includes:
a shape recognizer configured to recognize the shape of the front work apparatus from the first image to determine whether or not the front body turns; and
a transparency processor configured to determine the position of the transparent processing area in the synthesized image in response to turning of the front body, and
wherein the shape recognizer compares an actual image of the front work apparatus in the first image received from the upper camera with a learning image of the front work apparatus recognized and stored in advance by machine learning to determine whether or not the front body turns.
8 . The control system for construction machinery of claim 7 , wherein the image processing device recognizes the shape of the front work apparatus from the first image to determine a steering angle of the front body, and determines the position of the transparent processing area in the synthesized image according to the determined steering angle.
9 . A method of controlling construction machinery, the method comprising:
obtaining a first image of the front of a driver cabin from an upper camera installed in the drive cabin in a rear body;
obtaining a second image of the front of a front body from a lower camera installed in the front body rotatably connected to the rear body through a center pin;
detecting, by a shape recognizer, a shape of a front work apparatus from the first image to determine a steering angle of the front body;
synthesizing the first and second images into one image;
determining, by a transparency processor, a position of a transparency processing area in the synthesized image according to the steering angle;
transparency-processing at least one of the first and second images in the transparency processing area; and
displaying the transparency-processed image through a display device,
wherein the upper camera is installed to coincide with a central axis of the center pin or in the rear of the central axis and the lower camera is installed in the front of the central axis,
wherein the upper camera and the lower camera are installed at a different height from each other, and
wherein determining the steering angle of the front body from the first image includes comparing an actual image of the front work apparatus in the first image received from the upper camera with a learning image of the front work apparatus recognized and stored in advance by machine learning to determine the steering angle of the front work apparatus.
10 . The method of claim 9 , further comprising:
obtaining the learning image of the front work apparatus by executing a deep learning algorithm using the actual image as input data.
11 . The method of claim 9 , further comprising:
detecting a posture of the front work apparatus,
wherein transparency-processing the at least one of the first and second images in the transparency processing area includes transparency-processing the at least one of the first and second images according to the detected posture of the front work apparatus.
12 . The method of claim 9 , further comprising:
setting an image processing condition for transparency processing of the first and second images.
13 . The method of claim 12 , wherein the image processing condition includes a transparency processing switching timing of the first and second images or a size of the transparency processing area of the entire display area of the display device.