Electronic device for providing high accuracy bio-signals based on information obtained in a non-contact manner, system, and operation method of the same
According to various embodiments, an electronic device may include a first communication circuit and at least one first processor. At least one first processor is configured to: obtain, via the communication circuit, a plurality of images including a user's face obtained by using a camera of a first external electronic device; obtain first data obtained based on a first sensor of the first external electronic device contacted with a first portion of a body of the user and second data obtained based on a second external electronic device contacting a second portion of the body of the user while obtaining the plurality of images, while simultaneously acquiring the plurality of images; and obtain a first biometric signal of a specific type based on the images, a second biometric signal of the specific type based on the first data, and a third biometric signal based on the second data.
1. An electronic device comprising:
a communication circuit; and
at least one processor, wherein the at least one processor is configured to:
obtain, via the communication circuit, a plurality of images including a user's face captured using a camera of a mobile terminal of the user,
obtain, via the communication circuit, photographing information associated with photographing obtained by the mobile terminal while obtaining the plurality of images,
obtain, via the communication circuit, a first biometric signal indicating photoplethysmography (PPG) based on at least one sensor of the mobile terminal contacting a first finger of a first hand of the user, and a second biometric signal indicating the PPG obtained based on an external electronic device contacting a second finger of a second hand of the user corresponding to the first finger, wherein an accuracy of the second biometric signal is higher than an accuracy of the first biometric signal,
obtain, via the communication circuit, personal information associated with personal characteristics of the user, and by performing learning using the plurality of images, the photographing information, the first biometric signal, the second biometric signal, and the personal information as training data, obtain at least one artificial intelligence model, wherein the plurality of images, the first biometric signal and the second biometric signal are obtained simultaneously, and wherein the second biometric signal is set as an output data of the training data, and the plurality of images, the photographing information, the first biometric signal, and the personal information are set as an input data of the training data,
in response to receiving (i) a plurality of first images received from the camera of the mobile terminal, (ii) first photographing information and first personal information received from the mobile terminal, and (iii) the first biometric signal received from the at least one sensor of the mobile terminal, generate an estimation value of the second biometric signal by the at least one artificial intelligence model, and
determine at least blood pressure of the user by processing the estimation value of the second biometric signal,
wherein the at least one artificial intelligence model is implemented to provide the estimated value of the second biometric signal, in response to receiving the plurality of images obtained using the camera of the mobile terminal, the photographing information, the first biometric signal obtained based on the mobile terminal, and the personal information.
2. The electronic device of claim 1 , wherein the photographing information includes information associated with a state associated with photographing of the mobile terminal and information associated with an external environment of the mobile terminal.
3. The electronic device of claim 1 , wherein the at least one processor is configured to: perform time synchronization of the first biometric signal and the second biometric signal when the learning is performed.
4. The electronic device of claim 3 , wherein the at least one processor is configured to: perform time synchronization of the first biometric signal and the second biometric signal with respect to a biometric signal identified based on the plurality of images when the learning is performed.
5. The electronic device of claim 3 , wherein the at least one processor is configured to: generate the at least one artificial intelligence model before performing the time synchronization.
6. An operation method of an electronic device, the operation method comprising:
obtaining, via a communication circuit of the electronic device, a plurality of images including a user's face captured using a camera of a mobile terminal of the user;
obtaining, via the communication circuit, photographing information associated with photographing obtained by the mobile terminal while captured the plurality of images;
obtaining, via the communication circuit, a first biometric signal indicating photoplethysmography (PPG) based on at least one sensor of the mobile terminal contacting a first finger of a first hand of the user, and a second biometric signal indicating the PPG based on an external electronic device contacting a second finger of a second hand of the user corresponding to the first finger, wherein an accuracy of the second biometric signal is higher than an accuracy of the first biometric signal;
obtaining, via the communication circuit, personal information associated with personal characteristics of the user;
obtaining at least one artificial intelligence model by performing learning using the plurality of images, the photographing information, the first biometric signal, the second biometric signal, and the personal information as training data, wherein the plurality of images, the first biometric signal and the second biometric signal are obtained simultaneously, and wherein the second biometric signal is set as an output data of the training data, and the plurality of images, the photographic information, the first biometric signal, and the personal information are set as an input data of the training data;
in response to receiving (i) a plurality of first images received from the camera of the mobile terminal, (ii) first photographing information and first personal information received from the mobile terminal, and (iii) the first biometric signal received from the at least one sensor of the mobile terminal, generating an estimation value of the second biometric signal by the at least one artificial intelligence model, and
determining at least blood pressure of the user by processing the estimation value of the second biometric signal,
wherein the at least one artificial intelligence model is implemented to provide the estimated value of the second biometric signal, in response to receiving the plurality of images obtained using the camera of the mobile terminal, the photographing information, the first biometric signal obtained based on the mobile terminal, and the personal information.
7. The method of claim 6 , wherein the photographing information includes information associated with a state associated with the photographing of the mobile terminal and information associated with an external environment of the mobile terminal.