Apparatus and method for performing a localization of a movable treatment device
An apparatus for performing a localization of a movable treatment device having at least one inertial sensor and configured to treat the target surface. The apparatus has a motion pattern classification device configured to discriminate between two or more motion classes contained in a set of motion classes, and an interface for providing, from the inertial sensor to the motion pattern classification device, inertial sensor data representing a movement of the movable treatment device. The motion pattern classification device has a neural network configured to receive and classify inertial sensor data with respect to the motion classes associated with one or more different zones relating to the target surface so that the classification of inertial sensor data indicates an estimation of the location of the movable treatment device with respect to the one or more zones of the target surface.
1. An apparatus ( 10 ; 100 ) for performing a localization of a movable treatment device ( 11 ) relative to a target surface ( 12 ), the movable treatment device ( 11 ) comprising at least one inertial sensor ( 13 ) and being configured to treat the target surface ( 12 ), the apparatus ( 10 ) comprising:
a motion pattern classification device ( 14 ) configured to discriminate between two or more motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) contained in a set ( 15 ) of motion classes of the movable treatment device ( 11 ), and
an interface ( 16 ) for providing at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) from the inertial sensor ( 13 ) to the motion pattern classification device ( 14 ), the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) representing a movement of the movable treatment device ( 11 ),
wherein the motion pattern classification device ( 14 ) comprises at least one neural network ( 18 ) configured to receive the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) and to classify the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to the motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) contained in the set ( 15 ) of motion classes, wherein said motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) are each associated ( 20 1 , 20 2 , 20 3 , . . . , 20 k ) with one or more different zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) relating to the target surface ( 12 ) so that the classification of the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to the motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) indicates an estimation of the location of the movable treatment device ( 11 ) with respect to the one or more zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) of the target surface ( 12 ),
wherein an output (y t−1 ; y t ; y t+1 ) of the neural network ( 18 ) comprises one or more probability or likelihood values for the estimation of the location of the movable treatment device ( 11 ) with respect to the one or more zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) of the target surface ( 12 ),
wherein performance of the localization does not use input from a camera.
2. The apparatus ( 10 ; 100 ) of claim 1 , wherein the apparatus ( 10 ) is arranged to indicate one zone (Z 3 ) in which the movable treatment device ( 11 ) is located based on the highest probability or likelihood value of the output (y t−1 ; y t ; y t+1 ).
3. The apparatus ( 10 ; 100 ) of claim 1 , wherein at least one motion class ( 15 NB ) contained in the set ( 15 ) of motion classes is associated with a zone ( 21 NB ) outside the target surface ( 12 ), wherein the classification of the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to this motion class ( 15 NB ) indicates that the movable treatment device ( 11 ) is located in said zone ( 21 NB ) outside the target surface ( 12 ).
4. The apparatus ( 10 ; 100 ) of claim 1 , wherein the neural network ( 18 ) is a recurrent neural network, in particular a long short-term memory network or a gated recurrent unit network, further in particular where the recurrent neural network is a bi-directional recurrent neural network and even further in particular wherein the recurrent neural network is at least a two-layer, bi-directional recurrent neural network.
5. The apparatus ( 10 ; 100 ) of claim 1 , wherein the interface ( 16 ) is arranged to provide a sequence of inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) from the inertial sensor ( 13 ) to the motion pattern classification device ( 14 ) as an input (X 1 , X 2 , X 3 , . . . , X n ).
6. The apparatus ( 10 ) of claim 5 , wherein the apparatus ( 10 ) is arranged to indicate one or more of the zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) in which the movable treatment device was located during the time period relating to the sequence of inertial sensor data based on a maximum criterion or a majority criterion.
7. The apparatus ( 10 ) of claim 1 , wherein the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) comprises at least one inertial sensor data portion, in particular at least three inertial sensor portions of the group comprising a linear velocity in x, y and z direction, an angular velocity with respect to the x, y and z axes, a linear acceleration in x, y and z direction, and an angular acceleration with respect to the x, y and z axes.
8. The apparatus ( 10 ) of claim 1 , wherein the movable treatment device ( 11 ) comprises at least one further sensor such as a pressure sensor arranged for measuring the pressure with which the treatment device ( 11 ) is applied against the target surface or a load sensor for sensing a motor load of a motor driving the movable treatment device ( 11 ) and the interface ( 16 ) is arranged for providing at least one further sensor data from the further sensor to the motion pattern classification device ( 14 ) in addition to the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ).
9. The apparatus ( 10 ) of claim 1 , wherein the motion pattern classification device ( 14 ) comprises at least a first and a second neural network ( 18 ) that are each configured to receive the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) and the first neural network is configured to classify the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) contained in a first set ( 15 ) of motion classes and the second neural network is configured to classify the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) contained in a second set ( 15 ) of motion classes, wherein said motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) of the first and second set of motion classes are each associated ( 20 1 , 20 2 , 20 3 , . . . , 20 k ) with one or more different zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) relating to the target surface ( 12 ) so that the classification of the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to the motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) indicates an estimation of the location of the movable treatment device ( 11 ) with respect to the one or more zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) of the target surface ( 12 ) and wherein an output (y t−1 ; y t ; y t +1) of the first neural network and an output (y t−1 ; y t ; y t+1 ) of the second neural network are used to provide an estimation of the location of the movable treatment device ( 11 ) with respect to the one or more zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) of the target surface ( 12 ).
10. The apparatus ( 10 ) of claim 1 , wherein the neural network comprises trained weight matrices and bias vectors.
11. The apparatus ( 10 ) of claim 1 , wherein the movable treatment device ( 11 ) is a personal appliance and the target surface ( 12 ) is a body portion to be treated by the movable treatment device ( 11 ).
12. The apparatus ( 10 ) of claim 1 , wherein the movable treatment device ( 11 ) is an oral care device and the target surface ( 12 ) is an oral cavity, wherein the oral cavity ( 12 ) is separated into a plurality of oral cavity zones ( 1 a - 9 a ; 1 b - 16 b ), wherein the classification of the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to the motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) indicates an estimation of the location of the oral care device ( 11 ) with respect to the one or more oral cavity zones ( 1 a - 9 a ; 1 b - 16 b ).
13. A system comprising an apparatus ( 10 ) of claim 1 and a movable treatment device ( 11 ) configured to treat the target surface ( 12 ) and comprising the at least one inertial sensor ( 13 ).
14. A method for performing a localization of a movable treatment device ( 11 ) relative to a target surface ( 12 ), the movable treatment device ( 11 ) comprising at least one inertial sensor ( 13 ) and being configured to treat the target surface ( 12 ), the method comprising:
providing two or more motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) contained in a set ( 15 ) of motion classes of the movable treatment device ( 11 ),
receiving at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) from the inertial sensor ( 13 ), the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) representing a movement of the movable treatment device ( 11 ),
receiving and processing by means of a neural network ( 18 ) the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) and classifying the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with respect to at least one motion class ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) contained in the set ( 15 ) of motion classes, wherein the motion classes ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) are each associated with one or more different zones ( 21 1 , 21 2 , 21 3 , . . . , 21 n ) of the target surface ( 12 ) so that the classification of the at least one inertial sensor data ( 17 1 , 17 2 , 17 3 , . . . , 17 n ) with the at least one motion class ( 15 1 , 15 2 , 15 3 , . . . , 15 k ) indicates an estimation of the location of the movable treatment device ( 11 ) with respect to the one or more zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) of the target surface ( 12 ),
wherein an output (y t−1 ; y t ; y t+1 ) of the neural network ( 18 ) comprises one or more probability or likelihood values for the estimation of the location of the movable treatment device ( 11 ) with respect to the one or more zones ( 21 1 , 21 2 , 21 3 , . . . , 21 m ) of the target surface ( 12 ),
wherein performance of the method does not use input from a camera.