IP Library › Granted Patent US 12,360,768
Granted Patent B2
US 12,360,768 · App. 17/553,780 · Granted Jul 15, 2025

Throttling code fetch for speculative code paths

Inventors: Anant Vithal Nori (Bangalore, IN); Prathmesh Kallurkar (Bangalore, IN); Sreenivas Subramoney (Bangalore, IN); Niranjan Kumar Soundararajan (Bengaluru, IN)
Assignee: Intel Corporation
G06F9/3802
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,360,768
App. No.
17/553,780
Granted
Jul 15, 2025
Kind
B2
Abstract

Methods and apparatus relating to throttling a code fetch for speculative code paths are described. In an embodiment, a first storage structure stores a reference to a code line in response to a request to be received from a cache. A second storage structure to store a reference to the code line in response to an update to an Instruction Dispatch Queue (IDQ). Logic circuitry controls additional code line fetch operations based at least in part on a comparison of a number of ongoing speculative code fetches and a determination that the code line is speculative. Other embodiments are also disclosed and claimed.

Claims (24)

1. An apparatus comprising:

a first storage structure to store a reference to a code line in response to a request to be received from a cache;

a second storage structure to store a reference to the code line in response to an update to an Instruction Dispatch Queue (IDQ); and

logic circuitry to control additional code line fetch operations based at least in part on a comparison of a number of ongoing speculative code fetches and a determination that the code line is speculative, wherein the code line is to be considered as speculative in response to one of: (a) a hit in the first storage structure for the code line and a miss in the second storage structure for the code line; and (b) a hit in the second storage structure for the code line and a miss in the first storage structure for the code line.

2. The apparatus of claim 1 , wherein the logic circuitry is to disallow the additional code line fetch operations based at least in part on a determination that the number of ongoing speculative code fetches has reached a threshold value and the determination that the code line is speculative.

3. The apparatus of claim 1 , wherein the logic circuitry is to allow the additional code line fetch operations based at least in part on a determination that the number of ongoing speculative code fetches has not reached a threshold value and the determination that the code line is speculative.

4. The apparatus of claim 1 , wherein the logic circuitry is to control the additional code line fetch operations for code lines that cause a miss in an instruction cache.

5. The apparatus of claim 1 , wherein the logic circuitry is to control the additional code line fetch operations based at least in part on the comparison of the number of ongoing speculative code fetches in a front end of a processor.

6. The apparatus of claim 1 , wherein the first storage structure comprises a front end Bloom filter.

7. The apparatus of claim 1 , wherein the second storage structure comprises an IDQ Bloom filter.

8. The apparatus of claim 1 , wherein the reference to the code line comprises a virtual address of the code line.

9. The apparatus of claim 1 , wherein the cache is a Level 2 (L2) cache.

10. The apparatus of claim 1 , wherein the second storage structure is to store an indication of at least one of one or more instructions and one or more micro-operations that have been allocated in the IDQ.

11. The apparatus of claim 10 , wherein the IDQ is to store at least one of an instruction and a micro-operation to be allocated in a pre-execution stage of a processor pipeline.

12. The apparatus of claim 1 , wherein the code line is to store at least one of an instruction or a micro-operation.

13. The apparatus of claim 1 , wherein a processor, having one or more processor cores, comprises one or more of: the logic circuitry, the first storage structure, and the second storage structure.

14. One or more non-transitory computer-readable media comprising one or more instructions that when executed on a processor configure the processor to perform one or more operations to cause:

a first storage structure to store a reference to a code line in response to a request to be received from a cache;

a second storage structure to store a reference to the code line in response to an update to an Instruction Dispatch Queue (IDQ); and

logic circuitry to control additional code line fetch operations based at least in part on a comparison of a number of ongoing speculative code fetches and a determination that the code line is speculative, wherein the code line is to be considered as speculative in response to one of: (a) a hit in the first storage structure for the code line and a miss in the second storage structure for the code line; and (b) a hit in the second storage structure for the code line and a miss in the first storage structure for the code line.

15. The one or more computer-readable media of claim 14 , further comprising one or more instructions that when executed on the processor configure the processor to perform one or more operations to cause the logic circuitry to disallow the additional code line fetch operations based at least in part on a determination that the number of ongoing speculative code fetches has reached a threshold value and the determination that the code line is speculative.

16. The one or more computer-readable media of claim 15 , further comprising one or more instructions that when executed on the processor configure the processor to perform one or more operations to cause the logic circuitry to allow the additional code line fetch operations based at least in part on a determination that the number of ongoing speculative code fetches has not reached a threshold value and the determination that the code line is speculative.

17. The one or more computer-readable media of claim 14 , further comprising one or more instructions that when executed on the processor configure the processor to perform one or more operations to cause the logic circuitry to control the additional code line fetch operations for code lines that cause a miss in an instruction cache.

18. The one or more computer-readable media of claim 14 , further comprising one or more instructions that when executed on the processor configure the processor to perform one or more operations to cause the logic circuitry to control the additional code line fetch operations based at least in part on the comparison of the number of ongoing speculative code fetches in a front end of a processor.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 23, 2022
From: NORI, ANANT VITHAL; KALLURKAR, PRATHMESH; SUBRAMONEY, SREENIVAS; SOUNDARARAJAN, NIRANJAN KUMAR
To: INTEL CORPORATION
Reel/Frame 059072/0105 →
Continuity (1)
Related Publication 20230195464A1 · Jun 22, 2023
References Cited (142)
US 5699537A · Sharangpani · 1997 [cited by examiner]
US 5809268A · Chan · 1998 [cited by examiner]
US 6049866A · Earl · 2000 [cited by applicant]
US 6208273B1 · Dye et al. · 2001 [cited by applicant]
US 6233645B1 · Chrysos · 2001 [cited by examiner]
US 6388585B1 · Lacerda · 2002 [cited by applicant]
US 6505293B1 · Jourdan et al. · 2003 [cited by applicant]
US 6625723B1 · Jourday et al. · 2003 [cited by applicant]
US 6862662B1 · Cloud · 2005 [cited by applicant]
US 6879266B1 · Dye et al. · 2005 [cited by applicant]
US 7519796B1 · Golla · 2009 [cited by examiner]
US 8006073B1 · Ali · 2011 [cited by examiner]
US 8447948B1 · Erdogan et al. · 2013 [cited by applicant]
US 8738860B1 · Griffin et al. · 2014 [cited by applicant]
US 9552169B2 · Rappoport et al. · 2017 [cited by applicant]
US 10331558B2 · Sazegari et al. · 2019 [cited by applicant]
US 10671550B1 · Doi · 2020 [cited by applicant]
US 11625349B1 · Randall · 2023 [cited by examiner]
US 12028094B2 · Gaur et al. · 2024 [cited by applicant]
US 20020124142A1 · Har et al. · 2002 [cited by applicant]
US 20020174255A1 · Hayter et al. · 2002 [cited by applicant]
US 20030084274A1 · Gaither · 2003 [cited by examiner]
US 20030088759A1 · Wilkerson · 2003 [cited by applicant]
US 20030217251A1 · Jourdan et al. · 2003 [cited by applicant]
US 20040010679A1 · Moritz · 2004 [cited by examiner]
US 20040039880A1 · Pentkovski et al. · 2004 [cited by applicant]
US 20050160234A1 · Newburn et al. · 2005 [cited by applicant]
US 20050289300A1 · Kim et al. · 2005 [cited by applicant]
US 20060101238A1 · Bose · 2006 [cited by examiner]
US 20060294311A1 · Fu et al. · 2006 [cited by applicant]
US 20070204135A1 · Jiang · 2007 [cited by examiner]
US 20080059765A1 · Svendsen · 2008 [cited by examiner]
US 20080177984A1 · Lataille et al. · 2008 [cited by applicant]
US 20080256345A1 · Bose · 2008 [cited by examiner]
US 20080282034A1 · Jiao et al. · 2008 [cited by applicant]
US 20090150657A1 · Gschwind · 2009 [cited by examiner]
US 20100223237A1 · Mishra et al. · 2010 [cited by applicant]
US 20110072213A1 · Nickolls et al. · 2011 [cited by applicant]
US 20110208918A1 · Raikin et al. · 2011 [cited by applicant]
US 20120089819A1 · Chaudhry · 2012 [cited by examiner]
US 20130111605A1 · Maeda et al. · 2013 [cited by applicant]
US 20130339706A1 · Greiner · 2013 [cited by examiner]
US 20140095814A1 · Marden et al. · 2014 [cited by applicant]
US 20140281240A1 · Willhalm · 2014 [cited by applicant]
US 20140317377A1 · Ould-Ahmed-Vall et al. · 2014 [cited by applicant]
US 20140372736A1 · Greenhalgh · 2014 [cited by examiner]
US 20150106567A1 · Godard et al. · 2015 [cited by applicant]
US 20150178202A1 · Sankaran et al. · 2015 [cited by applicant]
US 20150178214A1 · Alameldeen et al. · 2015 [cited by applicant]
US 20150378731A1 · Lai et al. · 2015 [cited by applicant]
US 20160092373A1 · Doshi et al. · 2016 [cited by applicant]
US 20160179676A1 · Engh-Halstvedt et al. · 2016 [cited by applicant]
US 20160321076A1 · Satpathy et al. · 2016 [cited by applicant]
US 20160321185A1 · Doshi et al. · 2016 [cited by applicant]
US 20160328172A1 · Rappoport et al. · 2016 [cited by applicant]
US 20170046164A1 · Madhavan · 2017 [cited by examiner]
US 20170161076A1 · Alapati · 2017 [cited by examiner]
US 20170199739A1 · Kitchin · 2017 [cited by examiner]
US 20170220475A1 · Bradbury et al. · 2017 [cited by applicant]
US 20170249149A1 · Priyadarshi · 2017 [cited by examiner]
US 20170322811A1 · Abdallah · 2017 [cited by examiner]
US 20170371660A1 · Smith et al. · 2017 [cited by applicant]
US 20180011796A1 · Guilford et al. · 2018 [cited by applicant]
US 20180152201A1 · Gopal et al. · 2018 [cited by applicant]
US 20180165097A1 · Hanley · 2018 [cited by applicant]
US 20190034335A1 · Torre et al. · 2019 [cited by applicant]
US 20190042354A1 · Coquerel et al. · 2019 [cited by applicant]
US 20190044852A1 · Nolan et al. · 2019 [cited by applicant]
US 20190034333A1 · Sazegari et al. · 2019 [cited by applicant]
US 20190391869A1 · Gopal et al. · 2019 [cited by applicant]
US 20200190807A1 · Header · 2020 [cited by applicant]
US 20200249948A1 · Giamei et al. · 2020 [cited by applicant]
US 20200272474A1 · Gabor · 2020 [cited by examiner]
US 20200285580A1 · Subramanian · 2020 [cited by examiner]
US 20210035258A1 · Ray et al. · 2021 [cited by applicant]
US 20210072994A1 · Bainville et al. · 2021 [cited by applicant]
US 20210103550A1 · Appu et al. · 2021 [cited by applicant]
US 20210114495A1 · Battaglia et al. · 2021 [cited by applicant]
US 20210312697A1 · Maiyuran et al. · 2021 [cited by applicant]
US 20210374897A1 · Ray et al. · 2021 [cited by applicant]
US 20220066931A1 · Ray et al. · 2022 [cited by applicant]
US 20220091880A1 · Dutu · 2022 [cited by examiner]
US 20220197643A1 · Gaur et al. · 2022 [cited by applicant]
US 20220197659A1 · Gaur et al. · 2022 [cited by applicant]
US 20220197794A1 · Kallurkar et al. · 2022 [cited by applicant]
US 20220197799A1 · Gaur et al. · 2022 [cited by applicant]
US 20220197813A1 · Gaur et al. · 2022 [cited by applicant]
US 20220272569A1 · Berliner et al. · 2022 [cited by applicant]
US 20220295345A1 · Trim et al. · 2022 [cited by applicant]
US 20230019271A1 · Mukherjee · 2023 [cited by examiner]
CN 103810297A · 2014 [cited by applicant]
CN 114661227A · 2022 [cited by applicant]
CN 114661359A · 2022 [cited by applicant]
CN 114661625A · 2022 [cited by applicant]
CN 115793960A · 2023 [cited by applicant]
EP 4020185A1 · 2022 [cited by applicant]
EP 4020223A1 · 2022 [cited by applicant]
EP 4020230A1 · 2022 [cited by applicant]
EP 4020231A1 · 2022 [cited by applicant]
EP 4149008A1 · 2023 [cited by applicant]
EP 4198749A1 · 2023 [cited by applicant]
JP H0922353A · 1997 [cited by applicant]
WO 2020190799A3 · 2020 [cited by applicant]
WO 2020190807A1 · 2020 [cited by applicant]
Examination report for European Application No. 22206038.6, issued Feb. 21, 2024, 8 pages. [cited by applicant]
Ayers et al., “Asmdb: understanding and mitigating front-end stalls in warehouse-scale computers,” ISCA '19, Jun. 22-26, 2019, 12 pages. [cited by applicant]
Kanev et al., “Profiling a Warehouse-Scale Computer,” ISCA'15, Jun. 13-17, 2015, 12 pages. [cited by applicant]
Reinman et al., “Fetch Directed Instruction Prefetching,” Proceedings of the 32nd Annual International Symposium on Microarchitecture (MICRO-32), Nov. 1999, 12 pages. [cited by applicant]
European Patent Office communication regarding Intention to Grant for application No. 21197700.4, issued Jul. 30, 2024, 78 pages. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 17/133,622, mailed Feb. 29, 2024, 8 pages. [cited by applicant]
Notice of Intent to Grant from the European Patent Office for application No. 21198710.2, issued Aug. 2, 2024, 87 pages. [cited by applicant]
Office Action issued for U.S. Appl. No. 17/133,624, mailed Mar. 4, 2024, 12 pages. [cited by applicant]
Decision to grant European patent for Application No. 21198841.5, Apr. 5, 2024, 2 pages. [cited by applicant]
European Examination Report, application No. 21198874.6, Oct. 23, 2023, 7 pages. [cited by applicant]
European Patent Office, Notice of Grant for Application No. 21198841.5, issued Dec. 14, 2023, 80 pages. [cited by applicant]
Examination report issued by the European Patent Office for Application No. 21198874.6-1203, issued Jan. 19, 2023, 6 pages. [cited by applicant]
Extended European Search Report issued on Mar. 16, 2022 for EP Application No. 21198874.6. [cited by applicant]
European Examination report for application No. 22188197.2, issued Aug. 28, 2024, 6 pages. [cited by applicant]
Intention to Grant Notice issued by the European Patent Office for application No. 22206038.6, issued Jul. 8, 2024, 52 pages. [cited by applicant]
Notice of Intention to Grant for European Patent Application No. 21198710.2, issued Aug. 2, 2024, 89 pages. [cited by applicant]
Abail, et al. “Data Compression Accelerator on IBM POWER9 and z15 Processors,” ISCA 2020, 14 pages. [cited by applicant]
Cao et al. “Characterizing, Modeling, and Benchmarking RocksDB Key-Value Workloads at Facebook,” FAST 2020, retrieved from https://b;log.acolyer.org/2020/03/11/rocks-db-at-facebook/ on Nov. 19, 2020, 12 pages. [cited by applicant]
Colyer, Adrian, “Software-defined far memory in warehouse scale computers,” The Morning Paper, 13 pages, May 22, 2019. [cited by applicant]
Lagar-Cavilla et al. “Software-Defined Far Memory in Warehouse-Scale Computers,” ASPLOS 2019, retrieved from https://blog.acolyer.org/2019/05/22/sw-far-memory/ on Nov. 19, 2020, 11 pages. [cited by applicant]
Lagar-Cavilla, Andres, et al. “Software-Defined Far Memory in Warehouse-Scale Computers,” Session: VM/Memory, ASPLOS '19, Apr. 13-17, 2019, Providence, Rhode Island, pp. 317-330. [cited by applicant]
Zswap, The Linux Kernel documentation, Linux Memory Management Documentation, retrieved from www.kernel.org/doc/html/latest/vm/zswap.html on Aug. 29, 2021. [cited by applicant]
Final Office Action issued in U.S. Appl. No. 17/133,618, issued Jun. 28, 2024, 17 pages. [cited by applicant]
Non-Final Office Action from U.S. Appl. No. 17/133,618, mailed Mar. 15, 2024, 18 pages. [cited by applicant]
Non-final Office Action issued in U.S. Appl. No. 17/133,615 on Feb. 15, 2024, 16 pages. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/470,089, mailed Sep. 25, 2024, 13 pages. [cited by applicant]
Andreas Abel et al., Reverse Engineering of Cache Replacement Policies in Intel Microprocessors and Their Evaluation, 2014 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 3 pages. [cited by applicant]
Andreas Abel et al., Measurement-based Modeling of the Cache Replacement Policy, 2013 IEEE 19th Real-Time and Embedded Technology and Applications Symposium (RTAS), 10 pages. [cited by applicant]
Glenn Reinmany et al., Fetch Directed Instruction Prefetching. International Symposium on Microarchitecture (MICRO-32), Nov. 1999, 12 pages. [cited by applicant]
Pepe Vila et al., CacheQuery: learning replacement policies from hardware caches, 2020 ACM SIGPLAN Conference on Programming Language Design and Implementation, 17 pages. [cited by applicant]
Extended European search report for application No. 22206038.6, issued May 12, 2023, 12 pages. [cited by applicant]
Extended European Search Report for application No. 22188197.2-1224, issued Feb. 3, 2023, 10 pages. [cited by applicant]
Extended European Search Report issued on Apr. 7, 2022 for EP Application No. 21198710.2. [cited by applicant]
Extended European Search Report issued on Mar. 1, 2022 for EP Application No. 2119770.4. [cited by applicant]
Extended European Search Report issued on Mar. 1, 2022 for EP Application No. 21198841.5. [cited by applicant]
Final Office Action issued in U.S. Appl. No. 17/133,624, Aug. 27, 2024, 16 pages. [cited by applicant]
Non-final office action in U.S. Appl. No. 17/551,172, mailed Feb. 13, 2025, 9 pages. [cited by applicant]
Non-Final Office Action issued in U.S. Appl. No. 17/133,618, mailed Jan. 24, 2025. [cited by applicant]