IP Library Granted Patent US 12,362,045
Granted Patent B2
US 12,362,045 · App. 17/844,033 · Granted Jul 15, 2025

Information processing apparatus, information processing method, and program

Inventor: Yuya Hamaguchi (Tokyo, JP)
Assignee: FUJIFILM Corporation
G16C20/40G06F16/583G06T7/00G06V30/194G06V40/161
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,362,045
App. No.
17/844,033
Granted
Jul 15, 2025
Kind
B2
Abstract

There are provided an information processing apparatus, an information processing method, and a program with which structural elements in a structural formula can be identified from an image showing the structural formula and the results of identification can be used in a compound search performed later on. An information processing apparatus includes a processor, and the processor is configured to identify, on the basis of feature values of respective regions in a subject image showing a structural formula of each subject compound among subject compounds, structural elements shown by the respective regions among structural elements in the structural formula of the subject compound, by using an identification model, and store element information about the identified structural elements in the structural formula of each subject compound in association with the subject compound The identification model is a model created through machine learning using a learning image showing one structural element in a structural formula of a compound.

Claims (45)

1. An information processing apparatus comprising a processor,

the processor being configured to

identify, on the basis of feature values of respective regions in a subject image showing a structural formula of a subject compound, structural elements shown by the respective regions among structural elements in the structural formula of the subject compound, by using an identification model, and

store element information about the identified structural elements in the structural formula of each subject compound in association with the subject compound,

the identification model being created through machine learning using a plurality of learning images, each of the plurality of learning images showing one structural element in a structural formula of a compound.

2. The information processing apparatus according to claim 1 , wherein when the plurality of learning images, each showing the structural element having the same chemical structure but depicted in different formats from each other, are used in the machine learning, the identification model that derives a common feature value from the plurality of learning images is created through the machine learning.

3. The information processing apparatus according to claim 2 , wherein

the processor is configured to

obtain input information about a search compound, and

search for the subject compound corresponding to the search compound, among the subject compounds for each of which the element information is stored, on the basis of the input information and the element information associated with each subject compound.

4. The information processing apparatus according to claim 3 , wherein

the processor is configured to

calculate a degree of similarity between the search compound and each subject compound on the basis of the input information and the element information stored in association with the subject compound, and

retrieve, as the search compound, the subject compound for which the degree of similarity satisfies a search condition, from among the subject compounds for each of which the element information is stored.

5. The information processing apparatus according to claim 2 , wherein

the processor is configured to

detect the subject image from a document that includes the subject image, and

identify the structural elements shown by the respective regions in the subject image by inputting the detected subject image in the identification model.

6. The information processing apparatus according to claim 5 , wherein the processor is configured to detect the subject image from the document by using an object detection algorithm.

7. The information processing apparatus according to claim 2 , wherein the element information includes information indicating a type of each structural element among the identified structural elements in the structural formula of the subject compound.

8. The information processing apparatus according to claim 1 , wherein

the processor is configured to

obtain input information about a search compound, and

search for a subject compound corresponding to the search compound, among the subject compounds for each of which the element information is stored, on the basis of the input information and the element information associated with each subject compound.

9. The information processing apparatus according to claim 8 , wherein

the processor is configured to

calculate a degree of similarity between the search compound and each subject compound on the basis of the input information and the element information stored in association with the subject compound, and

retrieve, as the search compound, a subject compound for which the degree of similarity satisfies a search condition, from among the subject compounds for each of which the element information is stored.

10. The information processing apparatus according to claim 9 , wherein the processor is configured to obtain the input information that is information about the structural element included in a structural formula of the search compound.

11. The information processing apparatus according to claim 8 , wherein the processor is configured to obtain the input information that is information about a structural element included in a structural formula of the search compound.

12. The information processing apparatus according to claim 1 , wherein

the processor is configured to

detect the subject image from a document that includes the subject image, and

identify the structural elements shown by the respective regions in the subject image by inputting the detected subject image in the identification model.

13. The information processing apparatus according to claim 12 , wherein the processor is configured to detect the subject image from the document by using an object detection algorithm.

14. The information processing apparatus according to claim 1 , wherein the element information includes information indicating a type of each structural element among the identified structural elements in the structural formula of the subject compound.

15. The information processing apparatus according to claim 14 , wherein the information indicating the type of each structural element among the structural elements is information indicating a type of an atom or a bond between atoms corresponding to the structural element.

16. The information processing apparatus according to claim 14 , wherein the information indicating the type of each structural element among the structural elements is information indicating a chemical formula of a functional group corresponding to the structural element.

17. The information processing apparatus according to claim 14 , wherein the information indicating the type of each structural element among the structural elements is information formed of a part of a molecular fingerprint indicating, for each type of structural element, presence or absence of the structural element in the structural formula of the subject compound.

18. The information processing apparatus according to claim 1 , wherein the element information further includes information indicating a location of each structural element among the identified structural elements in the structural formula of the subject compound, in a coordinate space set for the subject image.

19. An information processing method in which a processor is configured to perform

a step of identifying, on the basis of feature values of respective regions in a subject image showing a structural formula of a subject compound among subject compounds, structural elements shown by the respective regions among structural elements included in the structural formula of the subject compound, by using an identification model, and

a step of storing element information about the identified structural elements in the structural formula of each subject compound in association with the subject compound,

the identification model being created through machine learning using a learning image showing one structural element in a structural formula of a compound.

20. A non-transitory computer readable medium storing a program for causing a processor to perform the steps in the information processing method according to claim 19 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 22, 2022
From: HAMAGUCHI, YUYA
To: FUJIFILM CORPORATION
Reel/Frame 060281/0057 →
Priority Claims (1)
JP 2019-236342 · Dec 26, 2019 · national
Continuity (2)
Continuation PCTJP2020040861 · Oct 30, 2020
Related Publication 20220327158A1 · Oct 13, 2022
References Cited (31)
US 5751850A · Rindtorff · 1998 [cited by examiner]
US 10372713B1 · Blake · 2019 [cited by examiner]
US 20120084299A1 · Cai · 2012 [cited by examiner]
US 20190251455A1 · Spangler · 2019 [cited by examiner]
US 20190286669A1 · Li et al. · 2019 [cited by applicant]
US 20200349451A1 · Suzuki · 2020 [cited by examiner]
US 20220309815A1 · Hamaguchi · 2022 [cited by examiner]
CN 102436447 · 2012 [cited by applicant]
CN 108062529 · 2018 [cited by applicant]
CN 108334839 · 2018 [cited by applicant]
CN 110265091 · 2019 [cited by applicant]
JP H04114560 · 1992 [cited by applicant]
JP H0728940 · 1995 [cited by applicant]
JP 2013061886 · 2013 [cited by applicant]
JP 2014091724 · 2014 [cited by applicant]
JP 2014182663 · 2014 [cited by applicant]
WO 2019048965 · 2019 [cited by applicant]
WO 2019175271 · 2019 [cited by applicant]
Staker et al., “Molecular Structure Extraction from Documents Using Deep Learning,” arXiv:1802.04903, https://doi.org/10.48550/ arXiv.1802.04903, Feb. 14, 2018 (Year: 2018). [cited by examiner]
Keyrouz et al., “Chemical Structure Recognition and Prediction: A Machine Learning Technique,” 2018 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) (Year: 2018). [cited by examiner]
Greg Landrum, “RDKit Documentation Release 2012.12.1,” Jan. 22, 2013 (Year: 2013). [cited by examiner]
Tang et al., “A Progressive Structural Analysis Approach for Handwritten Chemical Formula Recognition,” 2013 12th International Conference on Document Analysis and Recognition (Year: 2013). [cited by examiner]
Zheng et al., “Recognition of Handwritten Chemical Organic Ring Structure Symbols Using Convolutional Neural Networks,” 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW) (Year: 2019). [cited by examiner]
Joshua Staker et al., “Molecular Structure Extraction From Documents Using Deep Learning”, Journal of Chemical Information and Modeling, Mar. 2019 , pp. 1-16. [cited by applicant]
“Office Action of China Counterpart Application”, issued on Sep. 3, 2024, with English translation thereof, pp. 1-13. [cited by applicant]
Office Action of Japan Counterpart Application, with English translation thereof, issued on Nov. 14, 2023, pp. 1-6. [cited by applicant]
“Office Action of Japan Counterpart Application” with English translation thereof, issued on Jul. 4, 2023, p. 1-p. 6. [cited by applicant]
Nishiha, “Estimating Molecular Formulae from Structural Formula Images in Rdkit”, Jan. 2019, submit with English description extracted from ISA237, https://qiita.com/nishiha/items/f20f9942alc35elealfd. [cited by applicant]
Hideo Ito, “Image Searching in Patent Information Services”, Japan Patent Office Technology Forum, submit with partial English translation, pp. 66-70. [cited by applicant]
“International Search Report (Form PCT/ISA/210) of PCT/JP2020/040861,” mailed on Feb. 2, 2021, with English translation thereof, pp. 1-5. [cited by applicant]
“Written Opinion of the International Searching Authority (Form PCT/ISA/237) of PCT/JP2020/040861,” mailed on Feb. 2, 2021, with English translation thereof, pp. 1-8. [cited by applicant]