IP Library Granted Patent US 12,380,994
Granted Patent B2
US 12,380,994 · App. 18/537,217 · Granted Aug 5, 2025

Pharmaceutical manufacturing process control, support and analysis

Inventor: Matthias Friebe (Glattbrugg, CH)
Assignee: Augmenticon AG
G16H40/20G06F16/9024G06Q10/06316G06Q10/06395G06Q10/103G06Q50/04G06V20/20G16H70/40H04N7/147G06F3/04847
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,380,994
App. No.
18/537,217
Granted
Aug 5, 2025
Kind
B2
Abstract

A computer apparatus and computer-implemented process for generating a batch record (BR) from a master batch record (MBR) during manufacture of a batch of pharmaceutical product. In the control software, a graphical analysis interface is provided for processing graphs of spectroscopic or chromatographic analysis. A group of fields in the MBR is assigned to the analysis. The operator wears an augmented reality headset and uses this to capture an image of a graph from an instrument display. The graph image is then processed to extract the graph and its metadata, and then further processed to find peaks and assign attribute labels to the peaks, and thus populate the analysis fields. Overlay images are then transmitted to AR headset to present to the operator the populated analysis fields. The operator then accepts or rejects the populated analysis fields by issuing user interface commands.

Claims (74)

1. A computer apparatus configured to generate a batch record, BR, during manufacture of a batch of pharmaceutical product by populating a master batch record, MBR, the batch record generating computer apparatus comprising:

at least one memory; and

at least one processor in communication with the at least one memory to provide a process data structure defining a sequence of manufacturing process steps that are required to be carried out to manufacture a batch, the manufacturing process steps involving respective operator actions;

provide an MBR comprising a plurality of content items and associated fields, wherein the fields are to be populated as an operator progresses through the operator actions;

provide a mapping data structure that links operator actions to content items and associated fields; and

provide a control module configured to:

establish a data communication connection to a portable computer apparatus of an operator responsible for manufacturing the batch;

transmit image data to the operator's portable computer apparatus, the image data presenting ones of the content items and associated fields to the operator in a way that follows the operator's progression through the operator actions as determined with reference to the mapping data structure and that is responsive to the operator populating the MBR fields;

receive user interface commands from the operator's portable computer apparatus; and

populate fields of the MBR, as presented to the operator in the image data, responsive to receipt of the user interface commands.

2. The computer apparatus of claim 1 , wherein the control module is further configured to modify the image data so that the content items and/or associated fields are rendered having regard to a criticality grading of the operator actions.

3. The computer apparatus of claim 2 , wherein the control module is further configured to store a plurality of operator profiles relating to at least one of: operator skill and operator track-record of individual persons, and wherein the criticality grading takes account of an operator profile selected for the operator carrying out the operator actions.

4. The computer apparatus of claim 2 , wherein the content items include text content and the image data is modified by adding visually perceptible markings to distinguish between different portions of the text content having regard to said criticality grading.

5. The computer apparatus of claim 1 , further comprising a library of training units comprising one or more of video clips, stills images, text and audio, each training unit being associated with a specific operator action, group of operator actions or manufacturing process step, wherein the mapping data structure further provides links between ones of the operator actions and ones of the training units.

6. The computer apparatus of claim 1 , wherein the process data structure includes a definition of a group of the operator actions that relate to assembly of a synthesis module by attaching a plurality of vials to specific ones of respective docking stations on the synthesis module.

7. The computer apparatus of claim 6 , wherein the control module is further configured to:

receive scene image data from the operator's portable computer apparatus of at least one image captured by the operator's portable computer apparatus;

process the received scene image data to perform vial identification on any vials found in the scene image data by reading a machine-readable code attached to any such vial; and in response thereto

transmit data to the operator's portable computer apparatus providing feedback information extracted through each code.

8. The computer apparatus of claim 6 , wherein the control module is further configured to:

receive scene image data from the operator's portable computer apparatus of at least one image captured by the operator's portable computer apparatus;

process the received scene image data to identify any vials found in the scene and to identify any docking stations in the scene, at least ones that relate to identified vials, and in response thereto;

transmit data to the operator's portable computer apparatus conveying at least one of:

an indication of docking station in case of an undocked vial; and

an indication of correctness of docking station in case of a docked vial.

9. The computer apparatus of claim 6 , wherein on completion of the group of the operator actions relating to assembly of the synthesis module the control module is further configured to:

receive scene image data from the operator's portable computer apparatus of at least one image captured by the operator's portable computer apparatus;

process the received scene image data to perform a holistic verification of correct synthesis module assembly; and responsive thereto

transmit data to the operator's portable computer apparatus conveying an indication of correctness of the assembly.

10. The computer apparatus of claim 1 , wherein the control module is further configured to:

receive scene image data from the operator's portable computer apparatus of at least one image captured by the operator's portable computer apparatus, wherein the at least one image is of a display of a piece of equipment used during the manufacture of the batch of pharmaceutical product;

process the received scene image data to extract at least one of: graphical data, including a graph and associated graph metadata; and text data;

map the extracted graphical and/or text data to at least one field of the MBR with reference to the mapping data structure;

populate the at least one field according to the mapping; and

transmit image data to the operator's portable computer apparatus to present to the operator the at least one field as populated according to the extracted graphical and/or text data.

11. The computer apparatus of claim 10 , wherein the control module is further configured to:

receive a user interface command to select at least one field of the MBR from the operator's portable computer apparatus;

associate the at least one selected field with the received scene image data from the equipment display; and

apply the data extraction and mapping to the at least one selected field.

12. The computer apparatus of claim 1 , wherein the control module is further configured to:

perform a quality control, QC, check of the batch based on an automated analysis of what has been entered in the fields of the BR, wherein the quality control check compares the field entries with what is permitted in those field entries according to a specification that forms part of the MBR; and

output a QC check outcome selected from the following:

the results indicate that the batch meets specification; and

the results indicate that the batch does not meet specification; and optionally also:

the results indicate that the batch may not meet specification; and/or

the results indicate a systematic error in the completion of the BR.

13. The computer apparatus of a claim 12 , wherein the control module is further configured to:

transmit the BR and QC check outcome to a workstation for review by a qualified person, QP;

receive a batch release decision from the workstation and enter it in a corresponding field of the BR.

14. The computer apparatus of claim 13 , wherein the control module is further configured to:

record at least a subset of scene image data received from the operator's portable computer apparatus during the manufacture of the batch of pharmaceutical product, said recorded scene image data including video footage; and

transmit at least some of the recorded scene image data to the workstation for review by the QP.

15. The computer apparatus of claim 13 , wherein the control module is further configured to:

establish a live audio communication channel between the QP's workstation and the operator's AR headset to permit the QP to speak with the operator; and

establish a live video communication channel for transmitting live video feed from the operator's portable computer apparatus to the QP workstation, thus enabling the QP to view a live video feed from the operator's portable computer apparatus while being able to speak with the operator.

16. The computer apparatus of claim 1 , wherein the operator's portable computer apparatus is an augmented reality headset.

17. The computer apparatus of claim 1 , wherein the operator's portable computer apparatus is selected from the group consisting of: a tablet computer, a mobile phone and a laptop computer.

18. A computer-implemented process for generating a batch record, BR, with a computer apparatus by populating a master batch record, MBR, as part of manufacture of a batch of pharmaceutical product by an operator, the process comprising:

providing a process data structure defining a sequence of manufacturing process steps required to be carried out to manufacture a batch, the batch manufacturing process steps involving respective operator actions;

establishing a data communication connection between an operator's portable computer apparatus and the batch record generating computer apparatus configured to control the generation of the BR during manufacture;

providing the batch record generating computer apparatus with an MBR comprising a plurality of content items and associated fields, wherein the fields are to be populated as an operator progresses through the operator actions;

providing the computer apparatus with a mapping data structure that links operator actions to content items and associated fields;

transmitting image data from the batch record generating computer apparatus to the operator's portable computer apparatus, the image data presenting ones of the content items and associated fields to the operator in a way that follows the operator's progression through the operator actions as determined with reference to the mapping data structure and that is responsive to the operator populating the MBR fields; and

generating the BR by populating the fields of the MBR, as presented to the operator in the image data, responsive to receipt of respective user interface commands from the operator's portable computer apparatus.

19. The computer-implemented process of claim 18 , wherein the operator's portable computer apparatus is selected from the group consisting of: an augmented reality headset, a tablet computer, a mobile phone and a laptop computer.

20. A non-transitory computer readable medium storing instructions for performing a computer-implemented process for generating a batch record, BR, by populating a master batch record, MBR, stored on a computer apparatus as part of manufacture of a batch of pharmaceutical product by an operator, the operator being provided with a portable computer apparatus, the instructions comprising:

one or more instructions which, when executed by one or more processors, cause the one or more processors to:

provide a process data structure defining a sequence of manufacturing process steps required to be carried out to manufacture a batch, the batch manufacturing process steps involving respective operator actions;

establish a data communication connection between an operator's portable computer apparatus and the batch record generating computer apparatus configured to control the generation of the BR during manufacture;

provide the batch record generating computer apparatus with an MBR comprising a plurality of content items and associated fields, wherein the fields are to be populated as an operator progresses through the operator actions;

provide the computer apparatus with a mapping data structure that links operator actions to content items and associated fields;

transmit image data from the batch record generating computer apparatus to the operator's portable computer apparatus, the image data presenting ones of the content items and associated fields to the operator in a way that follows the operator's progression through the operator actions as determined with reference to the mapping data structure and that is responsive to the operator populating the MBR fields; and

generate the BR by populating the fields of the MBR, as presented to the operator in the image data, responsive to receipt of respective user interface commands from the operator's portable computer apparatus.

21. The non-transitory computer readable medium of claim 20 , wherein the operator's portable computer apparatus is selected from the group consisting of: an augmented reality headset, a tablet computer, a mobile phone and a laptop computer.

Assignments (2)
CHANGE OF NAME Recorded Feb 6, 2024
From: AUGMENTICON GMBH
To: AUGMENTICON AG
Reel/Frame 066396/0359 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: FRIEBE, MATTHIAS
To: AUGMENTICON GMBH
Reel/Frame 065848/0687 →
Priority Claims (3)
GB 1919333 · Dec 26, 2019 · national
GB 1919334 · Dec 26, 2019 · national
GB 1919335 · Dec 26, 2019 · national
Continuity (2)
Continuation 17133167 · Dec 23, 2020
Related Publication 20240161917A1 · May 16, 2024
References Cited (37)
US 10366521B1 · Peacock et al. · 2019 [cited by applicant]
US 10467534B1 · Brent · 2019 [cited by applicant]
US 20040193022A1 · Torii · 2004 [cited by applicant]
US 20040219054A1 · Sugawara · 2004 [cited by applicant]
US 20120007852A1 · Morate et al. · 2012 [cited by applicant]
US 20120101837A1 · McCorkle · 2012 [cited by applicant]
US 20130010068A1 · Tiernan et al. · 2013 [cited by applicant]
US 20130038633A1 · Maggiore · 2013 [cited by applicant]
US 20130049976A1 · Maggiore · 2013 [cited by applicant]
US 20130278635A1 · Maggiore · 2013 [cited by applicant]
US 20160055674A1 · Mullins et al. · 2016 [cited by applicant]
US 20160085230A1 · Schlack · 2016 [cited by examiner]
US 20160132046A1 · Beoughter et al. · 2016 [cited by applicant]
US 20160328887A1 · Elvezio et al. · 2016 [cited by applicant]
US 20160364913A1 · Montaigne et al. · 2016 [cited by applicant]
US 20170142324A1 · Jost et al. · 2017 [cited by applicant]
US 20170231490A1 · Toth · 2017 [cited by applicant]
US 20180109083A1 · Fenker · 2018 [cited by applicant]
US 20180114298A1 · Malaika · 2018 [cited by examiner]
US 20180165978A1 · Wood et al. · 2018 [cited by applicant]
US 20180211447A1 · Spayd · 2018 [cited by applicant]
US 20180356878A1 · Dudekula et al. · 2018 [cited by applicant]
US 20180357823A1 · Koniki et al. · 2018 [cited by applicant]
US 20180357922A1 · Dutta et al. · 2018 [cited by applicant]
US 20190316912A1 · Maggiore et al. · 2019 [cited by applicant]
US 20190362556A1 · Ben-Dor et al. · 2019 [cited by applicant]
US 20200125064A1 · Frick et al. · 2020 [cited by applicant]
US 20200188028A1 · Feiner et al. · 2020 [cited by applicant]
WO 2013170204 · 2013 [cited by applicant]
WO 2018106289 · 2018 [cited by applicant]
WO 2018152629 · 2018 [cited by applicant]
WO 2018197349 · 2018 [cited by applicant]
Henderson and Feiner, IEEE International Symposium on Mixed and Augmented Reality 2009, pp. 135-144 (ISBN 978-1-4244-5389-4/09). [cited by applicant]
Henderson and Feiner, IEEE International Symposium on Mixed and Augmented Reality 2011, pp. 191-200 (ISBN 978-1-4577-2185-4/10). [cited by applicant]
Philip Wenig, Label Chromatogram Peaks & Scans, https://spectrometrylab.wordpress.com/2016/08/01/label-chromatogram-peaks-scans/, Aug. 1, 2016. [cited by applicant]
Tom O'Haver, Integration and peak area measurement, https://terpconnect.umd.edu/˜toh/spectrum/Integration.html. [cited by applicant]
Kim et al, Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008-2017), Sep. 1, 2018, IEEE Transactions on Visualization and Computer Graphics pp. (99):1-1 (DOI: 10.1109/TVCG.2018.28… [cited by applicant]