Granted Patent B1

US 12,613,993 · App. 19/209,632 · Granted Apr 28, 2026

Inventors: Idan Habler (Petah Tikva, IL); Tsofit Efroni Zazon (Petah Tikva, IL); Yael Mathov Gome (Be'er Sheva, IL); Itsik Yizhak Mantin (Shoham, IL)

Assignee: Intuit Inc.

G06F21/6245

Abstract

Claims (64)

1 . A method comprising:

receiving a user-provided file as an input to a multi-modal large language model (LLM);

classifying the user-provided file to obtain a file type identifier of the user-provided file;

retrieving a bounding box set of a trusted file corresponding to the file type identifier;

masking the user-provided file with the bounding box set of the trusted file to obtain a secure file, wherein the masking the user-provided file comprises:

overlaying a document image of the user-provided file with a plurality of bounding boxes in the bounding box set of the trusted file to obtain a plurality of content portions of the user-provided file,

wherein the plurality of bounding boxes each encloses a content portion of the plurality of content portions of the user-provided file, and wherein the plurality of bounding boxes exclude a whitespace portion of the user-provided file,

wherein respective content portions of the plurality of content portions correspond to respective bounding boxes of the bounding box set, and

wherein the overlaying the user-provided file with the plurality of bounding boxes masks the user-provided file to retain the plurality of content portions of the user-provided file and blocks the whitespace portion of the user-provided file in order to obtain the secure file,

initializing the secure file comprising a blank document image, and

copying the plurality of content portions of the user-provided file, to the blank document image to obtain the secure file; and

transmitting the secure file to the LLM, wherein the LLM is blocked by the secure file from processing a prompt in the whitespace portion of the user-provided file.

2 . The method of claim 1 , wherein overlaying further comprises:

demarcating at least a first portion of the user-provided file with a first bounding box of the bounding box set of the trusted file as a first content portion to obtain the plurality of content portions, wherein the first bounding box is a set of relative coordinates enclosing a content area of the document image.

3 . The method of claim 1 , wherein copying the plurality of content portions of the user-provided file to the blank document image comprises:

copying pixel values of pixels of a first content portion of the plurality of content portions, corresponding to a first bounding box, to a corresponding second content portion of the secure file, wherein the corresponding second content portion is located in an area of the blank document image corresponding to the first bounding box.

4 . The method of claim 1 , further comprising:

obtaining a raw file; and

extracting, by a layout analysis tool, first bounding box coordinates of a first content portion of the raw file, to obtain a plurality of bounding box coordinates corresponding to a second plurality of content portions of the raw file.

5 . The method of claim 4 , further comprising:

generating a file type identifier corresponding to the raw file based at least on metadata of the raw file and the plurality of bounding box coordinates of the raw file.

6 . The method of claim 4 , further comprising,

storing the raw file as the trusted file, a corresponding file type identifier, and the plurality of bounding box coordinates in a trusted file repository.

7 . The method of claim 1 , further comprising:

receiving the user-provided file comprising the document image from a user application as the input to the LLM.

8 . A system comprising:

at least one computer processor;

a file security manager, executing on the at least one computer processor; and

a multi-modal large language model (LLM), executing on the at least one computer processor,

wherein the file security manager is configured for:

receiving a user-provided file as an input to the LLM,

classifying, by a document classifier, the user-provided file to obtain a file type identifier of the user-provided file,

retrieving, by the file security manager, a bounding box set of a trusted file corresponding to the file type identifier,

masking, using a masking tool of the file security manager, the user-provided file with the bounding box set of the trusted file to obtain a secure file, wherein the masking the user-provided file comprises:

overlaying a document image of the user-provided file with a plurality of bounding boxes in the bounding box set of the trusted file to obtain a plurality of content portions of the user-provided file,

wherein the plurality of bounding boxes each encloses a content portion of the plurality of content portions of the user-provided file, and wherein the plurality of bounding boxes exclude a whitespace portion of the user-provided file,

wherein respective content portions of the plurality of content portions correspond to respective bounding boxes of the bounding box set, and

wherein the overlaying the user-provided file with the plurality of bounding boxes masks the user-provided file to retain the plurality of content portions of the user-provided file and blocks the whitespace portion of the user-provided file in order to obtain the secure file,

initializing the secure file comprising a blank document image, and copying the plurality of content portions of the user-provided file, to the blank document image to obtain the secure file, and

transmitting the secure file to the LLM, wherein the LLM is blocked by the secure file from processing a prompt in the whitespace portion of the user-provided file.

9 . The system of claim 8 , wherein overlaying further comprises:

demarcating at least a first portion of the user-provided file with a first bounding box of the bounding box set of the trusted file as a first content portion to obtain the plurality of content portions, wherein the first bounding box is a set of relative coordinates enclosing a content area of the document image.

10 . The system of claim 8 , further comprising:

obtaining a raw file; and

extracting, by a layout analysis tool, first bounding box coordinates of a first content portion of the raw file, to obtain a plurality of bounding box coordinates corresponding to a second plurality of content portions of the raw file.

11 . The system of claim 10 , further comprising:

generating a file type identifier corresponding to the raw file based at least on metadata of the raw file and the plurality of bounding box coordinates of the raw file.

12 . A method, comprising:

obtaining a plurality of raw files from an external source, wherein a raw file comprises a document image;

generating a plurality of corresponding bounding box sets for each respective raw file of the plurality of raw files, wherein the plurality of corresponding bounding box sets comprises a corresponding bounding box set comprising a bounding box, the bounding box including relative coordinates of a content portion of the document image;

verifying the respective raw file of the plurality of raw files from the external source, based at least on metadata of the respective raw file, and the corresponding bounding box set of the respective raw file;

generating corresponding file type identifiers for the respective raw file of the plurality of raw files, based at least on the metadata of the respective raw file;

storing the plurality of raw files as trusted files, along with the corresponding file type identifiers and the corresponding bounding box sets in a trusted file repository;

receiving, from a user application, a user-provided file, wherein the user-provided file comprises a second document image;

classifying the user-provided file to obtain a file type identifier of the user-provided file;

selecting the corresponding bounding box set of a trusted file corresponding to the file type identifier of the user-provided file from the trusted file repository; and

overlaying the corresponding bounding box set of the trusted file on the second document image to obtain a plurality of content portions of the user-provided file, wherein the overlaying the user-provided file comprises:

overlaying the document image of the user-provided file with a plurality of bounding boxes in the corresponding bounding box set of the trusted file to obtain the plurality of content portions of the user-provided file,

wherein the plurality of bounding boxes each encloses the content portion of the plurality of content portions of the user-provided file, and wherein the plurality of bounding boxes exclude a whitespace portion of the user-provided file,

wherein respective content portions of the plurality of content portions correspond to respective bounding boxes of the corresponding bounding box set, and

wherein the overlaying the user-provided file with the plurality of bounding boxes masks the user-provided file to retain the plurality of content portions of the user-provided file and blocks the whitespace portion of the user-provided file in order to obtain a secure file,

initializing the secure file comprising a blank document image, and

copying the plurality of content portions of the user-provided file, to the blank document image to obtain the secure file,

wherein, based on the overlaying, the corresponding bounding box set of the trusted file blocks a prompt in the whitespace portion of the second document image.

Assignments (1)

References Cited (9)