IP Library Granted Patent US 12,236,500
Granted Patent B2
US 12,236,500 · App. 18/630,072 · Granted Feb 25, 2025

Systems and methods to process electronic images to provide improved visualization and rendering of histopathology slides

Inventors: Alexandre Kirszenberg (Paris, FR); Razik Yousfi (Brooklyn, NY); Thomas Fresneau (Oro Valley, AZ); Peter Schueffler (Munich, DE)
Assignee: Paige.AI, Inc.
G06T1/60G06T7/0012G06V20/69H04N23/58H04N23/63H04N23/69
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,236,500
App. No.
18/630,072
Granted
Feb 25, 2025
Kind
B2
Abstract

A method for processing an electronic image including receiving, by a viewer, the electronic image and a FOV (field of view), wherein the FOV includes at least one coordinate, at least one dimension, and a magnification factor, loading, by the viewer, a plurality of tiles within the FOV, determining, by the viewer, a state of the plurality of tiles in a cache, and in response to determining that the state of the plurality of tiles in the cache is a fully loaded state, rendering, by the viewer, the plurality of tiles to a display.

Claims (64)

1. A computer-implemented method for processing an electronic image, the method comprising:

receiving, by a slide viewer running on a web browser, the electronic image and a field of view (FOV) of the electronic image, wherein the FOV includes at least one coordinate, at least one dimension, and a magnification factor;

loading, by the slide viewer and from a cache of the slide viewer, one or more first tiles of a plurality of tiles within the FOV;

determining, by the slide viewer, that one or more second tiles of the plurality of tiles within the FOV are not in the cache;

requesting one or more electronic images of the one or more second tiles based on the determining, wherein each electronic image of the one or more second tiles is requested sequentially;

loading, by the slide viewer, the one or more electronic images of the one or more second tiles, wherein each electronic image of the one or more second tiles is loaded in parallel;

decoding, by the slide viewer, the one or more electronic images of the one or more second tiles; and

rendering, by the slide viewer, the plurality of tiles to a display after requesting the one or more electronic images.

2. The computer-implemented method of claim 1 , wherein each electronic image of the one or more second tiles is decoded sequentially.

3. The computer-implemented method of claim 1 , wherein each electronic image of the one or more second tiles is decoded in parallel.

4. The computer-implemented method of claim 1 , wherein the loading of the one or more first tiles further comprises:

loading, by the slide viewer, the one or more first tiles from the cache in one or more groups in an order of priority based on the magnification factor of the FOV or based on a magnification level of the one or more first tiles.

5. The computer-implemented method of claim 1 , further comprising:

loading, by the slide viewer, the one or more textures for the one or more second tiles to a graphics processing unit (GPU) of the slide viewer based on determining that the one or more second tiles have the one or more images and do not have the one or more textures in the cache.

6. The computer-implemented method of claim 1 , further comprising:

pruning, by the slide viewer, the cache of one or more unused electronic images or textures for the one or more second tiles prior to rendering the plurality of tiles.

7. The computer-implemented method of claim 1 , wherein the slide viewer comprises at least one of:

the cache,

a core processor,

a renderer, or

a loader.

8. A computer system for processing an electronic image, the computer system comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to perform operations comprising:

receiving, by a slide viewer, the electronic image and a field of view (FOV), wherein the FOV of the electronic image includes at least one coordinate, at least one dimension, and a magnification factor;

loading, by the slide viewer and from a cache of the slide viewer, one or more first tiles of a plurality of tiles within the FOV;

determining, by the slide viewer, that one or more second tiles of the plurality of tiles within the FOV are not in the cache;

requesting one or more electronic images of the one or more second tiles based on the determining, wherein each electronic image of the one or more second tiles is requested sequentially;

loading, by the slide viewer, the one or more electronic images of the one or more second tiles, wherein each electronic image of the one or more second tiles is loaded in parallel;

decoding, by the slide viewer, the one or more electronic images of the one or more second tiles; and

rendering, by the slide viewer, the plurality of tiles to a display after requesting the one or more electronic images.

9. The computer system of claim 8 , wherein each electronic image of the one or more second tiles is decoded sequentially.

10. The computer system of claim 8 , wherein each electronic image of the one or more second tiles is decoded in parallel.

11. The computer system of claim 8 , wherein the loading of the one or more first tiles further comprises:

loading, by the slide viewer, the one or more first tiles from the cache in one or more groups in an order of priority based on the magnification factor of the FOV or based on a magnification level of the one or more first tiles.

12. The computer system of claim 8 , further comprising:

loading, by the slide viewer, the one or more textures for the one or more second tiles to a graphics processing unit (GPU) of the slide viewer based on determining that the one or more second tiles have the one or more images and do not have the one or more textures in the cache.

13. The computer system of claim 8 , further comprising:

pruning, by the slide viewer, the cache of one or more unused electronic images or textures for the one or more second tiles prior to rendering the plurality of tiles.

14. The computer system of claim 8 , wherein the slide viewer comprises at least one of:

the cache,

a core processor,

a renderer, or

a loader.

15. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for processing an electronic image, the operations comprising:

receiving, by a slide viewer, the electronic image and a field of view (FOV) of the electronic image, wherein the FOV includes at least one coordinate, at least one dimension, and a magnification factor;

loading, by the slide viewer and from a cache of the slide viewer, one or more first tiles of a plurality of tiles within the FOV;

determining, by the slide viewer, that one or more second tiles of the plurality of tiles within the FOV are not in the cache;

requesting one or more electronic images of the one or more second tiles based on the determining, wherein each electronic image of the one or more second tiles is requested sequentially;

loading, by the slide viewer, the one or more electronic images of the one or more second tiles, wherein each electronic image of the one or more second tiles is loaded in parallel;

decoding, by the slide viewer, the one or more electronic images of the one or more second tiles; and

rendering, by the slide viewer, the plurality of tiles to a display after requesting the one or more images.

16. The non-transitory computer-readable medium of claim 15 , wherein each electronic image of the one or more second tiles is decoded sequentially or in parallel.

17. The non-transitory computer-readable medium of claim 15 , wherein the loading of the one or more first tiles further comprises:

loading, by the slide viewer, the one or more first tiles from the cache in one or more groups in an order of priority based on the magnification factor of the FOV or based on a magnification level of the one or more first tiles.

18. The non-transitory computer-readable medium of claim 15 , wherein the operation further comprise:

loading, by the slide viewer, the one or more textures for the one or more second tiles to a graphics processing unit (GPU) of the slide viewer based on determining that the one or more second tiles have the one or more images and do not have the one or more textures in the cache.

19. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise:

pruning, by the slide viewer, the cache of one or more unused electronic images or textures for the one or more second tiles prior to rendering the plurality of tiles.

20. The non-transitory computer-readable medium of claim 15 , wherein the slide viewer comprises at least one of:

the cache,

a core processor,

a renderer, or

a loader.

Assignments (3)
RELEASE OF SECURITY INTEREST Recorded May 14, 2026
From: ARES CAPITAL CORPORATION, AS COLLATERAL AGENT
To: PAIGE.AI, INC.
Reel/Frame 075589/0752 →
SECURITY INTEREST Recorded Oct 21, 2025
From: PAIGE.AI, INC.
To: ARES CAPITAL CORPORATION, AS COLLATERAL AGENT
Reel/Frame 073216/0876 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 23, 2024
From: KIRSZENBERG, ALEXANDRE; YOUSFI, RAZIK; FRESNEAU, THOMAS; SCHUEFFLER, PETER
To: PAIGE.AI, INC.
Reel/Frame 067188/0671 →
Continuity (5)
Continuation 17880766 · Aug 4, 2022
Continuation 17535084 · Nov 24, 2021
Continuation 17398388 · Aug 10, 2021
Provisional Application 63064401 · Aug 11, 2020
Related Publication 20240257296A1 · Aug 1, 2024
References Cited (66)
US 20080291201A1 · Lafon · 2008 [cited by examiner]
US 20110131376A1 · Fischer · 2011 [cited by applicant]
US 20190156159A1 · Kopparapu · 2019 [cited by applicant]
US 20190294628A1 · Smart · 2019 [cited by examiner]
WO 2005094462A2 · 2005 [cited by applicant]
WO 2012097438A2 · 2012 [cited by applicant]
WO 2013048640A1 · 2013 [cited by applicant]
WO 2020051113A1 · 2020 [cited by applicant]
WO 2020243556A1 · 2020 [cited by applicant]
510(k) Clearances, “Section 510(k) of the Food, Drug and Cosmetic Act requires device manufacturers who must register, to notify FDA of their intent to market a medical device at least 90 days in advance”, current versi… [cited by applicant]
ACTIX web, “A powerful, pragmatic, and extremely fast web framework for Rust” current version of webpage: https://actix.rs/ ; version of webpage as of Jul. 13, 2020: https://web.archive.org/web/20200713161958/https://ac… [cited by applicant]
Apple, “Metal: Accelerating graphics and much more”, current version of webpage: https://developer.apple.com/metal/ ; version of webpage as of Jul. 14, 2020: https://web.archive.org/web/20200714134955/https://developer.… [cited by applicant]
Catalin Clmpanu, “Microsoft: 70 percent of all security bugs are memory safety issues” percentage of memory safety issues has been hovering at 70 percent for the past 12 years, Feb. 11, 2019. current version of webpage:… [cited by applicant]
Cppreference.com, “C++ reference”, version of webpage as of Jan. 27, 2020: https://en.cppreference.com/mwiki/index.php?title=Main_Page&oldid=115512. [cited by applicant]
Daniel Aleksandersen, “Comparing AVIF vs WebP file sizes at the same DSSIM”, current version of webpage: https://www.ctrl.blog/entry/webp-avif-comparison.html ; version of webpage as of Aug. 10, 2020: https://web.archiv… [cited by applicant]
FLIF, “FLIF—Free Lossless Image Format”, current version of webpage: https://flif.info/ ; version of webpage as of Jul. 27, 2020: https://web.archive.org/web/20200727013520/http://flif.info/. [cited by applicant]
FLIF, “Lossy FLIF: Advantages of lossy FLIF”, current version of webpage: https://flif.info/lossy.html ; version of webpage as of Jul. 5, 2020: https://web.archive.org/web/20200705002145/http://flif.info/lossy.html. [cited by applicant]
GitHub, “WebAssembly/threads”, current version of webpage: https://github.com/WebAssembly/threads ; version of webpage as of Jul. 17, 2020: https://web.archive.org/web/20200717014649/https://github.com/WebAssembly/threa… [cited by applicant]
GitHub, Cyril Concolato et al. “AV1 Image File Format (AFIF)”, current version of webpage: https://aomediacodec.github.io/av1-avif/ ; version of webpage as of Aug. 6, 2020: https://web.archive.org/web/20200806170552/htt… [cited by applicant]
GitHub, Dzmitry Malyshau et al. WebGPU “WebGPU”, current version of webpage: https://gpuweb.github.io/gpuweb/ ; version of webpage as of Aug. 8, 2020: https://web.archive.org/web/20200808003414/https://gpuweb.github.io/… [cited by applicant]
Github, Eugene Kliuchnikov et al. “google/brunsli Practical JPEG Repacker”, current version of webpage: https://github.com/google/brunsli ; version of webpage as of Jul. 28, 2020: https://web.archive.org/web/20200728021… [cited by applicant]
GitHub, Mozilla JPEG Encoder Project “mozilla/mozjpeg: Improved JPEG encoder”, current version of webpage: https://github.com/mozilla/mozjpeg ; version of webpage as of Jul. 10, 2020: https://web.archive.org/web/2020071… [cited by applicant]
Gpu, “wgpu-rs”, current version of webpage: https://wgpu.rs/ ; version of webpage as of Aug. 12, 2020: https://web.archive.org/web/20200812155418/https://wgpu.rs/. [cited by applicant]
Gstreamer, “News—GStreamer 1.20.0 new major stable release”, current version of webpage: https://gstreamer.freedesktop.org/ ; version of webpage as of Aug. 8, 2020: https://web.archive.org/web/20200808114052/https://gst… [cited by applicant]
Jakob Nielson, “Website Response Times” NN/g Nielsen Norman Group, Jun. 20, 2010, current version of webpate: https://www.nngroup.com/articles/website-response-times/ ; version of webpage as of Aug. 9, 2020: https://web… [cited by applicant]
Jon Sneyers, “How JPEG XL Compares to Other Image Codecs”, current version of webpage: https://cloudinary.com/blog/how_jpeg_xl_compares_to_other_image_codecs ; version of webpage as of Jul. 28, 2020: https://web.archive… [cited by applicant]
JPEG, “Overview of JPEG 1” current version of webpage: https://jpeg.org/jpeg/ ; version of webpage as of Aug. 9, 2020: https://web.archive.org/web/20200809233804/https://jpeg.org/jpeg/. [cited by applicant]
JPEG, “Overview of JPEG 200”, current version of webpage: https://jpeg.org/jpeg2000/index.html ; version of webpage as of Aug. 9, 2020: https://web.archive.org/web/20200809232724/https://jpeg.org/jpeg2000/index.html. [cited by applicant]
JPEG, “Overview of JPEG XL”, current version of webpage: https://jpeg.org/jpegxl/ ; version of webpage as of Aug. 6, 2020: https://web.archive.org/web/20200806184032/https://jpeg.org/jpegxl/. [cited by applicant]
Jyrki Alakuijala et al., “Benchmarking JPEG XL image compression”, current version of webpage: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/11353/113530X/Benchmarking-JPEG-XL-image-compression/10.11… [cited by applicant]
MDN Web Docs, “Backround Tasks API”, current version of webpage: https://developer.mozilla.org/en-US/docs/Web/API/Background_Tasks_API ; version of webpage as of Jul. 13, 2020: https://web.archive.org/web/20200713045949… [cited by applicant]
MDN Web Docs, “OffscreenCanvas”, current version of webpage: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas ; version of webpage as of Aug. 8, 2020: https://web.archive.org/web/20200808214151/https://d… [cited by applicant]
MDN Web Docs, “SharedArrayBuffer”, current version of webpage: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer ; version of webpage as of Aug. 8, 2020: https://web.arch… [cited by applicant]
MDN Web Docs, “window.requestldleCallback()”, current version of webpage: https://developer.mozilla.org/en-US/docs/Web/API/Window/requestldleCallback ; version of webpage as of Aug. 4, 2020: https://web.archive.org/web/… [cited by applicant]
Memory Safety, “The Chromium Projects” current version of webpage: https://www.chromium.org/Home/chromium-security/memory-safety ; version of webpage as of Jul. 8, 2020: https://web.archive.org/web/20200807183028/https:… [cited by applicant]
Nokia, “HEIF: Comparison with other formats”, current version of webpage: https://nokiatech.github.io/heif/comparison.html ; version of webpage as of Aug. 7, 2020: https://web.archive.org/web/20200807033756/https://noki… [cited by applicant]
Steven White et al., “What is Direct3D 12?”, current version of webpage: https://docs.microsoft.com/en-us/windows/win32/direct3d12/what-is-directx-12-; version of webpage as of Oct. 22, 2019: https://web.archive.org/web… [cited by applicant]
Techempower, “Web Framework Benchmarks” T-cells that target Lethargic Software Syndrome, current version of webpage: https://www.techempower.com/benchmarks/ ; version of webpage as of Aug. 7, 2020: https://web.archive.o… [cited by applicant]
The WebM Project, “VP9 Video Codec”, current version of webpage: https://www.webmproject.org/vp9/ ; version of the webpage as of Aug. 6, 2020: https://web.archive.org/web/20200806232758/https://www.webmproject.org/vp9/. [cited by applicant]
Vulkan Platform Support, “Vulkan”, current version of webpage: https://www.khronos.org/vulkan/ ; version of webpage as of Aug. 10, 2018: https://web.archive.org/web/20180810065920/https://www.khronos.org/vulkan/. [cited by applicant]
WebP, “An image format for the web”, current version of webpage: https://developers.google.com/speed/webp ; version of webpage as of Aug. 10, 2020: https://web.archive.org/web/20200810164758/https://developers.google.co… [cited by applicant]
WebP, “WebP Compression Study”, current version of webpage: https://developers.google.com/speed/webp/docs/webp_study ; version of webpage as of Jun. 25, 2020: https://web.archive.org/web/20200625100545/https://developer… [cited by applicant]
WebRTC, “Real-time communication for the web”, current version of the webpage: https://webrtc.org/ ; version of the webpage as of Aug. 8, 2020: https://web.archive.org/web/20200808075543/https://webrtc.org/. [cited by applicant]
Wikipedia, “Abstract syntax tree”, version of webpage as of Aug. 10, 2019: https://en.wikipedia.org/w/index.php?title=Abstract_syntax_tree&oldid=910241958. [cited by applicant]
Wikipedia, “Amazon Web Services”, version of webpage as of Aug. 7, 2020: https://en.wikipedia.org/w/index.php?title=Amazon_Web_Services&oldid=971621711. [cited by applicant]
Wikipedia, “An application programming interface (API)”, version of webpage as of Aug. 7, 2020: https://en.wikipedia.org/w/index.php?title=API&oldid=971646630. [cited by applicant]
Wikipedia, “Content delivery network”, version of webpage as of Aug. 10, 2020: https://en.wikipedia.org/w/index.php?title=Content_delivery_network&oldid=972145057. [cited by applicant]
Wikipedia, “Continuous integration”, version of webpage as of Jul. 28, 2020: https://en.wikipedia.org/w/index.php?title=Continuous_integration&oldid=969916004. [cited by applicant]
Wikipedia, “Deep Zoom”, version of webpage as of Feb. 16, 2020: https://en.wikipedia.org/w/index.php?title=Deep_Zoom&oldid=941128067. [cited by applicant]
Wikipedia, “Discrete cosine transform”, version of webpage as of Aug. 10, 2020: https://en.wikipedia.org/w/index.php?title=Discrete_cosine_transform&oldid=972220052. [cited by applicant]
Wikipedia, “Discrete wavelet transform”, version of webpage as of Jul. 9, 2020: https://en.wikipedia.org/w/index.php?title=Discrete_wavelet_transform&oldid=966828588. [cited by applicant]
Wikipedia “File System”, version of webpage as of Aug. 2, 2019: https://en.wikipedia.org/w/index.php?title=File_system&oldid=909082892. [cited by applicant]
Wikipedia “Food and Drug Administration”, version of webpage as of Jun. 7, 2020: https://en.wikipedia.org/w/index.php?title=Food_and_Drug_Administration&oldid=961216493. [cited by applicant]
Wikipedia “H&E stain”, version of webpage as of Jun. 15, 2020: https://en.wikipedia.org/w/index.php?title=H%26E_stain&oldid=962752471. [cited by applicant]
Wikipedia “High Efficiency Image File Format” (HEIF), version of webpage as of Jul. 9, 2020: https://en.wikipedia.org/w/index.php?title=High_Efficiency_Image_File_Format&oldid=966795613. [cited by applicant]
Wikipedia “Just-in-time compilation”, version of webpage as of Aug. 9, 2020: https://en.wikipedia.org/w/index.php?title=Just-in-time_compilation&oldid=971974067. [cited by applicant]
Wikipedia “Lempel-Ziv-Welch”, version of webpage as of May 25, 2020: https://en.wikipedia.org/w/index.php?title=Lempel%E2%80%93Ziv%E2%80%93Welch&oldid=958784425. [cited by applicant]
Wikipedia “LLVM”, version of webpage as of Aug. 4, 2020: https://en.wikipedia.org/w/index.php?title=LLVM&oldid=971184873. [cited by applicant]
Wikipedia “OpenGL Shading Language”, version of webpage as of Jul. 20, 2020: https://en.wikipedia.org/w/index.php?title=OpenGL_Shading_Language&oldid=968635606. [cited by applicant]
Wikipedia “Tagged Image File Format”, version of webpage as of Jul. 26, 2020: https://en.wikipedia.org/w/index.php?title=TIFF&oldid=969688829. [cited by applicant]
Wikipedia, “Scrum (software development)”, version of webpage as of Jul. 26, 2020: https://en.wikipedia.org/w/index.php?title=Scrum_(software_development)&oldid=969619095. [cited by applicant]
Wikipedia, “Single instruction, multiple data”, version of webpage as of Aug. 9, 2020: https://en.wikipedia.org/w/index.php?title=SIMD&oldid=972038390. [cited by applicant]
Wikipedia, “Software development kit”, version of webpage as of Jun. 8, 2020: https://en.wikipedia.org/w/index.php?title=Software_development_kit&oldid=961408787. [cited by applicant]
Wikipedia, “Time to first byte”, version of webpage as of Feb. 17, 2020: https://en.wikipedia.org/w/index.php?title=Time_to_first_byte&oldid=941283786. [cited by applicant]
Wikipedia, “Virtual Network Computing”, version of webpage as of Mar. 7, 2020: https://en.wikipedia.org/w/index.php?title=Virtual_Network_Computing&oldid=944396374. [cited by applicant]
Wikipedia, “WebAssembly”, version of webpage as of Aug. 8, 2020: https://en.wikipedia.org/w/index.php?title=WebAssembly&oldid=971768638. [cited by applicant]