Visually augmenting a graphical rendering of a chemical structure representation or biological sequence representation with multi-dimensional information
In certain embodiments, the invention relates to systems, methods, and apparatus that allow a user to visually augment a graphical rendering of either a chemical structure representation or a biological sequence representation with multi-dimensional information. A user captures a video image using a computing device such as a hand-held smart phone, computerized eye glasses or tablet computer. The video image includes information regarding at least one of a chemical structure and a biological sequence. A processor identifies, within the video image, a graphical representation of at least one of a chemical structure and a biological structure. The processor augments the graphical representation with additional information and provides the video data for presentation upon a display controlled by the computing device. The computing device presents the video data in substantially real time in relation to the capture of the video data by the computing device.
1. A method comprising:
receiving a live video feed comprising two or more video frames captured by a single video camera;
digitally altering, based on a review of the two or more video frames together from the live video feed, at least a portion of the live video feed in order to match a graphical representation of either a chemical structure or biological sequence captured within the two or more video frames to a known chemical compound or biological species, wherein the review of the two or more video frames together from the live video feed comprises:
merging of the two or more video frames from the live video feed; or
appending of one of the two or more video frames from the live video feed to another one of the two or more video frames from the live video feed;
automatically identifying, within the digitally altered live video feed, by a processor of a computing device, an identifier of a chemical compound or a biological species;
querying, by the processor, a database for additional information associated with the chemical compound or biological species;
rendering, by the processor, augmentation data corresponding to the additional information;
generating an output video feed by graphically merging live video feed with the augmentation data, wherein the graphically merging comprises positionally linking the augmentation data with the identifier in the output video feed; and
outputting the output video feed for display in substantially real time in relation to the live video feed.
2. The method of claim 1 , wherein the identifier is a written name or formula of the chemical compound or the biological species.
3. The method of claim 1 , wherein the identifier is written on a curved surface.
4. The method of claim 1 , wherein the additional information comprises one or more of a two-dimensional representation of the identified chemical compound or biological species, a three-dimensional representation of the chemical compound or biological species, or a property of the chemical compound or biological species.
5. The method of claim 1 , wherein the additional information further comprises one or more of:
a storage location of the chemical compound or the biological species; or
information regarding ongoing experiment(s) with use of the chemical compound or the biological species.
6. The method of claim 1 , wherein the augmentation data comprises one or more of graphics corresponding to at least a portion of the additional information, or one or more graphical controls which presents at least a portion of the additional information upon selection.
7. The method of claim 6 , wherein the rendering and the graphically merging comprises rendering a first graphical control of the one or more graphical controls as a semi-transparent image and overlaying the semi-transparent image upon the live video feed.
8. The method of claim 1 , wherein the rendered augmentation data is one or more of:
a two-dimensional representation of the chemical compound or the biological species; or
a three-dimensional representation of the chemical compound or the biological species.
9. The method of claim 8 , wherein graphically merging the received live video feed with the rendered augmentation data comprises superimposing the two-dimensional representation of the chemical compound or the biological species or the three-dimensional representation of the chemical compound or the biological species with at least a portion of the additional information.
10. The method of claim 8 , wherein the two-dimensional representation of the chemical compound or the biological species and the three-dimensional representation of the chemical compound or the biological species are graphical representations of the chemical compound or the biological species.
11. The method of claim 10 , wherein, a graphical representation of the graphical representations of the chemical compound or the biological species is partially obscured, wherein one or more bonds is missing or wherein a portion of an atom identifier is missing.
12. The method of claim 1 , wherein the computing device includes the video camera.
13. The method of claim 1 , wherein the live video feed comprises a sequence of images.
14. A system comprising:
a processor: and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
receive a live video feed comprising two or more video frames captured by a single video camera;
digitally alter, based on a review of the two or more video frames together from the live video feed, at least a portion of the live video feed in order to match a graphical representation of either a chemical structure or biological sequence captured within the two or more video frames to a known chemical compound or biological species, wherein the review of the two or more video frames together from the live video feed comprises:
merging of the two or more video frames from the live video feed; or
appending of one of the two or more video frames from the live video feed to another one of the two or more video frames from the live video feed;
automatically identify, within the digitally altered live video feed, by a processor of a computing device, an identifier of a chemical compound or a biological species;
query a database for additional information associated with the chemical compound or biological species;
render augmentation data corresponding to the additional information;
generate an output video feed by graphically merging the received live video feed with the augmentation data, wherein the graphically merging comprises positionally linking the augmentation data with the identifier in the output video feed; and
output the output video feed for display in substantially real time in relation to the live video feed.
15. The system of claim 14 , wherein the identifier is a written name or formula of the chemical compound or the biological species.
16. The system of claim 14 , wherein the rendered augmentation data comprises one or more of graphics corresponding to at least a portion of the additional information, or one or more graphical controls which presents at least a portion of the additional information upon selection.
17. The system of claim 16 , wherein the rendering and the graphically merging comprises rendering a first graphical control of the one or more graphical controls as a semi-transparent image and overlaying the semi-transparent image upon the live video feed.
18. The system of claim 14 , wherein the rendered augmentation data is one or more of:
a two-dimensional representation of the chemical compound or the biological species; or
a three-dimensional representation of the chemical compound or the biological species.
19. The system of claim 18 , wherein the graphically merging the received live video feed with the rendered augmentation data comprises superimposing the two-dimensional representation of the chemical compound or the biological species or the three-dimensional representation of the chemical compound or the biological species with at least a portion of the additional information.
20. The system of claim 18 , wherein the two-dimensional representation of the chemical compound or the biological species and the three-dimensional representation of the chemical compound or the biological species are graphical representations of the chemical compound or the biological species.
21. The system of claim 14 , wherein the computing device includes the video camera.
22. The system of claim 14 , wherein the live video feed comprises a sequence of images.