IP Library Granted Patent US 12,230,387
Granted Patent B2
US 12,230,387 · App. 18/537,255 · Granted Feb 18, 2025

Pharmaceutical manufacturing process control, support and analysis

Inventor: Matthias Friebe (Glattbrugg, CH)
Assignee: Augmenticon AG
G16H40/20G06F16/9024G06Q10/06316G06Q10/06395G06Q10/103G06Q50/04G06V20/20G16H70/40H04N7/147G06F3/04847
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,230,387
App. No.
18/537,255
Granted
Feb 18, 2025
Kind
B2
Abstract

A computer apparatus and computer-implemented process for generating a batch record (BR) from a master batch record (MBR) during manufacture of a batch of pharmaceutical product. In the control software, a graphical analysis interface is provided for processing graphs of spectroscopic or chromatographic analysis. A group of fields in the MBR is assigned to the analysis. The operator wears an augmented reality headset and uses this to capture an image of a graph from an instrument display. The graph image is then processed to extract the graph and its metadata, and then further processed to find peaks and assign attribute labels to the peaks, and thus populate the analysis fields. Overlay images are then transmitted to AR headset to present to the operator the populated analysis fields. The operator then accepts or rejects the populated analysis fields by issuing user interface commands.

Claims (66)

1. A computer apparatus configured to generate a batch record, BR, during manufacture of a batch of pharmaceutical product by populating a master batch record, MBR, the BR-generating computer apparatus comprising:

a process data structure defining a sequence of manufacturing process steps required to be carried out to manufacture a batch, the batch manufacturing process steps involving respective operator actions, wherein the operator actions include performing at least one of a chromatographic and spectroscopic analysis of the batch using a chromatographic or a spectroscopic instrument;

an MBR comprising a plurality of content items and associated fields, wherein the fields are to be populated as an operator progresses through the operator actions, the MBR including a group of fields relating to the analysis;

a mapping data structure that provides links between ones of the operator actions to ones of the content items and associated fields, including for the analysis; and

a control module configured to:

establish a data communication connection to a handheld computer apparatus of an operator responsible for manufacturing the batch;

receive scene image data from the operator's handheld computer apparatus of a graph image captured by the operator's handheld computer apparatus showing results of the analysis;

process the received scene image data to extract a graph and associated graph metadata from the graph image, wherein the graph includes a plurality of peaks;

interpret the peaks of the graph with reference to the graph metadata to obtain peak data;

populate the analysis fields with reference to the mapping data structure with the graph, metadata associated with the graph and peak data associated with the graph;

transmit image data to the operator's handheld computer apparatus to present to the operator the populated analysis fields; and

receive user interface commands from the operator's handheld computer apparatus to accept or reject the populated analysis fields.

2. The computer apparatus of claim 1 , wherein the control module is further configured to:

transmit image data to the operator's handheld computer apparatus to present to the operator the graph and positioned thereon a baseline which is to be taken as an ordinate zero line;

receive user interface commands from the operator's handheld computer apparatus to reposition the baseline during which the baseline is shown interactively to the operator by amending the image data and to accept the baseline; and

populate an analysis field with the accepted baseline.

3. The computer apparatus of claim 1 , wherein the peak data includes position values for the peaks.

4. The computer apparatus of claim 1 , wherein the peak data includes integral values for the peaks.

5. The computer apparatus of claim 1 , wherein the control module is further configured to assign attribute labels to at least some of the peaks.

6. The computer apparatus of claim 5 , wherein the attribute labels are assigned to peaks by the batch record generating computer apparatus:

processing the graph;

receiving user interface commands from the operator's handheld computer apparatus that specify the attribute label assignment; and/or

processing the graph image, since the attribute label assignment was already displayed in the graph image captured by the operator's handheld computer apparatus, and can be extracted therefrom.

7. The computer apparatus of claim 6 , wherein the control module is further configured to determine relative abundance values between the attributes based on a comparison of the peak integral values.

8. The computer apparatus of claim 1 , wherein the control module is further configured to:

interpret the graph with reference to pre-existing and already interpreted graphs stored in a graph library, wherein the pre-existing graphs are selected based on being from the same kind of analysis using the same kind of instrument on a different batch of the same pharmaceutical product.

9. The computer apparatus of claim 1 , wherein the control module is further configured to:

perform a quality control, QC, check of the batch based on an automated analysis of what has been entered in the analysis fields of the BR, wherein the quality control check compares the analysis field entries with what is permitted in those field entries according to a specification that forms part of the MBR; and

output a QC check outcome selected from the following:

the results indicate that the batch meets specification; and

the results indicate that the batch does not meet specification; and optionally also:

the results indicate that the batch may not meet specification; and/or

the results indicate a systematic error in the completion of the BR.

10. The computer apparatus of a claim 9 , wherein the control module is further configured to:

transmit the BR and QC check outcome to a workstation for review by a qualified person, QP;

receive a batch release decision from the workstation and enter it in a corresponding field of the BR.

11. The computer apparatus of claim 10 , wherein the control module is further configured to:

record at least a subset of scene image data received from the operator's handheld computer apparatus during the manufacture of the batch of pharmaceutical product, said recorded scene image data including video footage; and

transmit at least some of the recorded scene image data to the workstation for review by the QP.

12. The computer apparatus of claim 10 , wherein the control module is further configured to:

establish a live audio communication channel between the QP workstation and the operator's handheld computer apparatus to permit the QP to speak with the operator; and

establish a live video communication channel for transmitting live video feed from the operator's handheld computer apparatus to the QP workstation, thus enabling the QP to view a live video feed from a forward-facing camera of the operator's handheld computer apparatus while speaking with the operator.

13. The computer apparatus of claim 1 , wherein the operator's handheld computer apparatus is selected from the group consisting of: a tablet computer, a mobile phone and a laptop computer.

14. A computer-implemented process for generating a batch record, BR, by populating a master batch record, MBR, stored on a computer apparatus as part of manufacture of a batch of pharmaceutical product by an operator, the operator being provided with a handheld computer apparatus, the process comprising:

providing a process data structure defining a sequence of manufacturing process steps required to be carried out to manufacture a batch, the batch manufacturing process steps involving respective operator actions, wherein the operator actions include performing at least one of a chromatographic and spectroscopic analysis of the batch using a chromatographic or a spectroscopic instrument;

establishing a data communication connection between the operator's handheld computer apparatus and the batch record generating computer apparatus configured to control the generation of the BR during manufacture;

providing the batch record generating computer apparatus with an MBR comprising a plurality of content items and associated fields, wherein the fields are to be populated as an operator progresses through the operator actions, the MBR including a group of fields relating to the analysis;

providing the batch record generating computer apparatus with a mapping data structure that provides links between ones of the operator actions to ones of the MBR content items and associated fields, including for the analysis;

processing received scene image data to extract a graph and associated graph metadata from the graph image, wherein the graph includes a plurality of peaks;

interpreting the peaks of the graph with reference to the graph metadata to obtain peak data;

populating the analysis fields with reference to the mapping data structure with the graph, the graph metadata and graph peak data;

transmitting image data to the operator's handheld computer apparatus to present to the operator the populated analysis fields; and

receiving user interface commands from the operator's handheld computer apparatus to accept or reject the populated analysis fields.

15. The computer-implemented process of claim 14 , wherein the operator's handheld computer apparatus is selected from the group consisting of: a tablet computer, a mobile phone and a laptop computer.

16. A non-transitory computer readable medium storing instructions for performing a computer-implemented process for generating a batch record, BR, by populating a master batch record, MBR, stored on a computer apparatus as part of manufacture of a batch of pharmaceutical product by an operator, the operator being provided with a handheld computer apparatus, the instructions comprising:

one or more instructions which, when executed by one or more processors, cause the one or more processors to:

provide a process data structure defining a sequence of manufacturing process steps required to be carried out to manufacture a batch, the batch manufacturing process steps involving respective operator actions, wherein the operator actions include performing at least one of a chromatographic and spectroscopic analysis of the batch using a chromatographic or a spectroscopic instrument;

establish a data communication connection between the operator's handheld computer apparatus and the batch record generating computer apparatus configured to control the generation of the BR during manufacture;

provide the batch record generating computer apparatus with an MBR comprising a plurality of content items and associated fields, wherein the fields are to be populated as an operator progresses through the operator actions, the MBR including a group of fields relating to the analysis;

provide the batch record generating computer apparatus with a mapping data structure that provides links between ones of the operator actions to ones of the MBR content items and associated fields, including for the analysis;

process the received scene image data to extract a graph and associated graph metadata from the graph image, wherein the graph includes a plurality of peaks;

interpret the peaks of the graph with reference to the graph metadata to obtain peak data;

populate the analysis fields with reference to the mapping data structure with the graph, the graph metadata and graph peak data;

transmit image data to the operator's handheld computer apparatus to present to the operator the populated analysis fields; and

receive user interface commands from the operator's handheld computer apparatus to accept or reject the populated analysis fields.

17. The non-transitory computer readable medium of claim 16 , wherein the operator's handheld computer apparatus is selected from the group consisting of: a tablet computer, a mobile phone and a laptop computer.

Assignments (2)
CHANGE OF NAME Recorded Feb 6, 2024
From: AUGMENTICON GMBH
To: AUGMENTICON AG
Reel/Frame 066396/0359 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: FRIEBE, MATTHIAS
To: AUGMENTICON GMBH
Reel/Frame 065848/0687 →
Priority Claims (3)
GB 1919333 · Dec 26, 2019 · national
GB 1919334 · Dec 26, 2019 · national
GB 1919335 · Dec 26, 2019 · national
Continuity (2)
Continuation 17133167 · Dec 23, 2020
Related Publication 20240120079A1 · Apr 11, 2024
References Cited (34)
US 10366521B1 · Peacock et al. · 2019 [cited by applicant]
US 10467534B1 · Brent · 2019 [cited by applicant]
US 20040193022A1 · Torii · 2004 [cited by applicant]
US 20040219054A1 · Sugawara · 2004 [cited by applicant]
US 20120007852A1 · Morate et al. · 2012 [cited by applicant]
US 20120101837A1 · McCorkle · 2012 [cited by applicant]
US 20130010068A1 · Tiernan et al. · 2013 [cited by applicant]
US 20130038633A1 · Maggiore · 2013 [cited by applicant]
US 20130049976A1 · Maggiore · 2013 [cited by applicant]
US 20130278635A1 · Maggiore · 2013 [cited by applicant]
US 20160055674A1 · Mullins et al. · 2016 [cited by applicant]
US 20160132046A1 · Beoughter et al. · 2016 [cited by applicant]
US 20160328887A1 · Elvezio et al. · 2016 [cited by applicant]
US 20160364913A1 · Montaigne · 2016 [cited by applicant]
US 20170142324A1 · Jost et al. · 2017 [cited by applicant]
US 20180109083A1 · Fenker · 2018 [cited by applicant]
US 20180165978A1 · Wood et al. · 2018 [cited by applicant]
US 20180211447A1 · Spayd · 2018 [cited by applicant]
US 20180356878A1 · Dudekula et al. · 2018 [cited by applicant]
US 20180357823A1 · Duff et al. · 2018 [cited by applicant]
US 20180357922A1 · Dutta et al. · 2018 [cited by applicant]
US 20190316912A1 · Maggiore et al. · 2019 [cited by applicant]
US 20190362556A1 · Ben-Dor et al. · 2019 [cited by applicant]
US 20200125064A1 · Frick et al. · 2020 [cited by applicant]
US 20200188028A1 · Feiner et al. · 2020 [cited by applicant]
WO 2013170204 · 2013 [cited by applicant]
WO 2018106289 · 2018 [cited by applicant]
WO 2018152629 · 2018 [cited by applicant]
WO 2018197349 · 2018 [cited by applicant]
Henderson and Feiner, IEEE International Symposium on Mixed and Augmented Reality 2009, pp. 135-144 (ISBN 978-1-4244-5389-4/09). [cited by applicant]
Henderson and Feiner, IEEE International Symposium on Mixed and Augmented Reality 2011, pp. 191-200 (ISBN 978-1-4577-2185-4/10). [cited by applicant]
Philip Wenig, Label Chromatogram Peaks & Scans, https://spectrometrylab.wordpress.com/2016/08/01/label-chromatogram-peaks-scans/, Aug. 1, 2016. [cited by applicant]
Tom O'Haver, Integration and peak area measurement, https://terpconnect.umd.edu/˜toh/spectrum/Integration.html. [cited by applicant]
Kim et al, Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008-2017), Sep. 1, 2018, IEEE Transactions on Visualization and Computer Graphics pp. (99):1-1 (DOI: 10.1109/TVCG.2018.28… [cited by applicant]