Control device for industrial vehicle, control system for industrial vehicle, and program for control device for industrial vehicle

1 . A control device for an industrial vehicle, wherein the industrial vehicle is a forklift, the control device comprising:

a display unit;

one or more memories storing instructions; and

a processor coupled to the one or more memories and configured to perform, based on the instructions, a method comprising:

estimating a working state of the forklift,

wherein the estimating comprises:

receiving operation information regarding an operation state with respect to the forklift and photographic information obtained by photographing a package, wherein the photographic information includes at least one image,

acquiring a working state estimation model corresponding to an operator operating the forklift, the working state estimation model including an estimation index set by machine learning based on a past driving performance of the operator operating the forklift, the estimation index defining a transition condition or a transition threshold for transitioning to each of a plurality of working states,

collating the working state estimation model in which the estimation index has been set, with a working state input to estimate a working state, wherein the working state comprises travelling state information of the forklift and package work information of the forklift,

determining, based on the at least one image and the working state estimated by the working state estimation model, whether or not a packing form of the package is in an abnormal state, wherein the abnormal state comprises the package bridging other packages, the package bridging between first and second pallets, or the package protruding from a third pallet, and

outputting the working state on a remote operation device to assist an operator operating the forklift, thereby avoiding a collapse of the package, a fall of the package, or a collision of the package with a fourth pallet,

wherein the display unit comprises a first screen and a second screen, a first video is presented on the first screen, wherein the first video is chosen by the processor according to the current working state, wherein a plurality of images from a plurality of photographic units on the forklift are shown in a reduced size on the second screen.

2 . The control device according to claim 1 , wherein the operation information includes at least one of an accelerator operation amount, a steering angle, a lift operation amount, a reach operation amount, and a tilt operation amount.

3 . The control device according to claim 1 , wherein the estimating further comprises:

acquiring a feature vector of the image from an image obtained by photographing the package, by using machine learning with a second machine learning model,

wherein the second machine learning model is implemented as a convolutional neural network (CNN), and

wherein the working state is estimated on the basis of the feature vector.

4 . A control system for an industrial vehicle comprising the control device for an industrial vehicle according to claim 1 .

5 . A non-transitory computer readable medium storing a program for a control device for an industrial vehicle used in the control device for an industrial vehicle according to claim 1 .

6 . The control device according to claim 1 , wherein the method performed by the processor, based on the instructions, further comprises acquiring, from a database stored in a storage unit, viewpoint information corresponding to the working state by collating the working state with the database, wherein the database stores each of the plurality of working states associated with a respective viewpoint captured by the plurality of photographic units provided at respective portions of the forklift, and

wherein the first video is chosen by the processor according to a viewpoint from the database that corresponds to the current working state.