IP Library Granted Patent US 12,293,237
Granted Patent B2
US 12,293,237 · App. 18/545,912 · Granted May 6, 2025

Systems, methods and devices for determining work placement on processor cores

Inventors: Guy M. Therien (Sherwood, OR); Michael D. Powell (Northborough, MA); Venkatesh Ramani (Milpitas, CA); Arijit Biswas (Northborough, MA); Guy G. Sotomayor (San Jose, CA)
Assignee: Intel Corporation
G06F9/5083G06F1/324G06F1/329G06F1/3296G06F9/3009G06F9/5033G06F9/5044G06F9/5094Y02D10/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,293,237
App. No.
18/545,912
Granted
May 6, 2025
Kind
B2
Abstract

Apparatuses, methods and storage medium for computing including determination of work placement on processor cores are disclosed herein. In embodiments, an apparatus may include one or more processors, devices, and/or circuitry to identify a favored core of the processor cores. The one or more processors, devices, and/or circuitry may be configured to determine whether to migrate a thread to or from the favored core. In some embodiments, the determination may be by a process executed by a driver and/or by an algorithm executed by a power control unit of the processor.

Claims (42)

1. A processor, comprising:

a plurality of cores that are architecturally identical;

circuitry to store data indicating physical characteristics of each core of the plurality of cores, the physical characteristics including a maximum frequency of each core of the plurality of cores, at least one core of the plurality of cores having different physical characteristics than another core of the plurality of cores; and

power control circuitry to control frequencies and voltages of the plurality of cores in accordance with corresponding per-core performance values, the power control circuitry including an interface to receive performance requests from system software and to provide information related to the per-core performance values to the system software;

execution circuitry to execute instructions of a driver to determine the per-core performance values,

wherein when a first operational mode is enabled, the execution circuitry is to:

determine a first ranking of the plurality of cores based on the physical characteristics of the plurality of cores, and

provide the first ranking to the system software, and

wherein when the first operational mode is not enabled, the execution circuitry is to:

determine a second ranking of the plurality of cores based on the physical characteristics of the plurality of cores, the second ranking different from the first ranking, and

provide the second ranking to the system software.

2. The processor of claim 1 wherein when the first operational mode is enabled, the power control circuitry is to:

receive a request from the system software indicating a requested performance level for a particular core of the plurality of cores, and

control the frequency and voltage of the particular core based on the requested performance level.

3. The processor of claim 1 , wherein a candidate core of the plurality of cores is to be selected for executing a current workload based on the first ranking when the first operational mode is enabled and based on the second ranking when the first operational mode is not enabled.

4. The processor of claim 3 , wherein the candidate core is to be selected by the power control circuitry.

5. The processor of claim 3 , wherein the candidate core is to be selected by the system software.

6. The processor of claim 1 , wherein the circuitry to store data indicating physical characteristics of each core of the plurality of cores comprises a read-only memory.

7. The processor of claim 1 , wherein the circuitry to store data indicating physical characteristics of each core of the plurality of cores comprises a set of fuses to be programmed to indicate the physical characteristics of each core of the plurality of cores.

8. The processor of claim 1 wherein, when the first operational mode is enabled, the power control circuitry is to receive a request from the system software for a specific level of performance of a first core for executing a current workload.

9. A non-transitory machine-readable medium having program code stored thereon which, when executed by a processor, is to cause the processor to perform:

receiving, over an interface of the processor, performance requests from system software, the performance requests associated with one or more cores of a plurality of cores that are architecturally identical within the processor;

providing information related to per-core performance values to the system software, the per-core performance values based, at least in part, on data indicating physical characteristics of each core of the plurality of cores, the physical characteristics including a maximum frequency of each core of the plurality of cores, at least one core of the plurality of cores having different physical characteristics than another core of the plurality of cores; and

controlling, by power control circuitry of the processor, frequencies and voltages of the plurality of cores in accordance with corresponding per-core performance values;

wherein when a first operational mode is enabled, execution circuitry of the processor performs:

determining a first ranking of the plurality of cores based on the physical characteristics of the plurality of cores, and

providing the first ranking to the system software, and

wherein when the first operational mode is not enabled, the execution circuitry of the processor performs:

determining a second ranking of the plurality of cores based on the physical characteristics of the plurality of cores, the second ranking different from the first ranking, and

providing the second ranking to the system software.

10. The non-transitory machine-readable medium of claim 9 wherein, when the first operational mode is enabled, the performance requests are to indicate a specific level of performance of a first core for executing a current workload.

11. The non-transitory machine-readable medium of claim 9 wherein when the first operational mode is enabled, the program code is to cause the processor to perform:

receiving a request from the system software indicating a requested performance level for a particular core of the plurality of cores, and

controlling the frequency and voltage of the particular core based on the requested performance level.

12. The non-transitory machine-readable medium of claim 9 , wherein a candidate core of the plurality of cores is to be selected for executing a current workload based on the first ranking when the first operational mode is enabled and based on the second ranking when the first operational mode is not enabled.

13. The non-transitory machine-readable medium of claim 12 , wherein the candidate core is to be selected by the system software.

14. The non-transitory machine-readable medium of claim 9 , wherein data indicating physical characteristics of each core of the plurality of cores is stored in a read-only memory.

15. The processor of claim 1 , wherein the power control circuitry further comprises one or more registers to store an indication of a power state of each core of the plurality of cores.

16. The processor of claim 1 , further comprising:

shared circuitry external to the plurality of cores, the shared circuitry including an interconnect and last level cache (LLC) coupled to the plurality of cores, wherein the power control circuitry is to control a shared circuitry frequency and voltage in accordance with a power state of at least a portion of the shared circuitry.

17. The processor of claim 1 , wherein the power control circuitry comprises a programmable state machine or a microcontroller to execute power control code.

18. The processor of claim 1 , wherein the interface includes at least one of: mailbox registers and model-specific registers (MSRs).

Continuity (5)
Continuation 17857394 · Jul 5, 2022
Continuation 17173100 · Feb 10, 2021
Continuation 16107691 · Aug 21, 2018
Continuation 14997032 · Jan 15, 2016
Related Publication 20240118942A1 · Apr 11, 2024
References Cited (207)
US 4228496A · Katzman et al. · 1980 [cited by applicant]
US 4356550A · Katzman et al. · 1982 [cited by applicant]
US 4958273A · Anderson et al. · 1990 [cited by applicant]
US 5367697A · Barlow et al. · 1994 [cited by applicant]
US 5491788A · Cepulis et al. · 1996 [cited by applicant]
US 5761516A · Rostoker et al. · 1998 [cited by applicant]
US 6141762A · Nicol et al. · 2000 [cited by applicant]
US 6407575B1 · Wissell et al. · 2002 [cited by applicant]
US 6425068B1 · Vorbach et al. · 2002 [cited by applicant]
US 6691216B2 · Kelly et al. · 2004 [cited by applicant]
US 6772189B1 · Asselin · 2004 [cited by applicant]
US 6826656B2 · Augsburg et al. · 2004 [cited by applicant]
US 6928566B2 · Nunomura · 2005 [cited by applicant]
US 7111178B2 · Rusu et al. · 2006 [cited by applicant]
US 7146514B2 · Kaushik et al. · 2006 [cited by applicant]
US 7268570B1 · Audet et al. · 2007 [cited by applicant]
US 7398403B2 · Nishioka · 2008 [cited by applicant]
US 7463992B2 · Samaan et al. · 2008 [cited by applicant]
US 7464276B2 · Rusu et al. · 2008 [cited by applicant]
US 7565563B2 · Gappisch et al. · 2009 [cited by applicant]
US 7596708B1 · Halepete et al. · 2009 [cited by applicant]
US 7779287B2 · Lim et al. · 2010 [cited by applicant]
US 7702933B2 · Tsai · 2010 [cited by applicant]
US 7814252B2 · Hoshaku · 2010 [cited by applicant]
US 7882379B2 · Kanakogi · 2011 [cited by applicant]
US 7895453B2 · Kasahara et al. · 2011 [cited by applicant]
US 8112754B2 · Shikano · 2012 [cited by applicant]
US 8281164B2 · Kim · 2012 [cited by applicant]
US 8296773B2 · Bose et al. · 2012 [cited by applicant]
US 8407505B2 · Asaba · 2013 [cited by applicant]
US 8417974B2 · Heller, Jr. · 2013 [cited by applicant]
US 8949633B2 · Belmont et al. · 2015 [cited by applicant]
US 9442559B2 · Biswas et al. · 2016 [cited by applicant]
US 9678792B2 · Doyle et al. · 2017 [cited by applicant]
US 9703613B2 · Saha · 2017 [cited by examiner]
US 10037227B2 · Therien et al. · 2018 [cited by applicant]
US 10372493B2 · Dhanraj et al. · 2019 [cited by applicant]
US 10503542B2 · Therien et al. · 2019 [cited by applicant]
US 20020099976A1 · Sanders et al. · 2002 [cited by applicant]
US 20020120882A1 · Sarangi et al. · 2002 [cited by applicant]
US 20020147932A1 · Brock et al. · 2002 [cited by applicant]
US 20020156611A1 · Lenormand · 2002 [cited by applicant]
US 20030033490A1 · Gappisch et al. · 2003 [cited by applicant]
US 20030046228A1 · Berney · 2003 [cited by applicant]
US 20030076183A1 · Tam et al. · 2003 [cited by applicant]
US 20040015888A1 · Fujii et al. · 2004 [cited by applicant]
US 20040187126A1 · Yoshimura · 2004 [cited by applicant]
US 20050107967A1 · Patel et al. · 2005 [cited by applicant]
US 20050240735A1 · Shen et al. · 2005 [cited by applicant]
US 20060005056A1 · Nishioka · 2006 [cited by applicant]
US 20060107262A1 · Bodas et al. · 2006 [cited by applicant]
US 20070043964A1 · Lim et al. · 2007 [cited by applicant]
US 20070174829A1 · Brockmeyer et al. · 2007 [cited by applicant]
US 20070255929A1 · Kasahara et al. · 2007 [cited by applicant]
US 20070283128A1 · Hoshaku · 2007 [cited by applicant]
US 20080022052A1 · Sakugawa · 2008 [cited by applicant]
US 20080077815A1 · Kanakogi · 2008 [cited by applicant]
US 20080077928A1 · Matsuzaki et al. · 2008 [cited by applicant]
US 20080104425A1 · Gunther et al. · 2008 [cited by applicant]
US 20080196030A1 · Buros et al. · 2008 [cited by applicant]
US 20080235364A1 · Gorbatov et al. · 2008 [cited by applicant]
US 20080301474A1 · Bussa et al. · 2008 [cited by applicant]
US 20090049312A1 · Min · 2009 [cited by applicant]
US 20090055826A1 · Bernstein et al. · 2009 [cited by applicant]
US 20090070772A1 · Shikano · 2009 [cited by applicant]
US 20090164399A1 · Bell, Jr. et al. · 2009 [cited by applicant]
US 20090165007A1 · Aghajanyan · 2009 [cited by applicant]
US 20090195350A1 · Tsern et al. · 2009 [cited by applicant]
US 20090222654A1 · Hum et al. · 2009 [cited by applicant]
US 20090231960A1 · Hutcheson · 2009 [cited by applicant]
US 20090249094A1 · Marshall et al. · 2009 [cited by applicant]
US 20090288092A1 · Yamaoka · 2009 [cited by examiner]
US 20100053005A1 · Mukai et al. · 2010 [cited by applicant]
US 20100058086A1 · Lee · 2010 [cited by applicant]
US 20100077185A1 · Gopalan et al. · 2010 [cited by applicant]
US 20100094572A1 · Chase et al. · 2010 [cited by applicant]
US 20100095137A1 · Bieswanger et al. · 2010 [cited by applicant]
US 20100146310A1 · Kasahara et al. · 2010 [cited by applicant]
US 20100153954A1 · Morrow et al. · 2010 [cited by applicant]
US 20100153956A1 · Capps, Jr. et al. · 2010 [cited by applicant]
US 20100169609A1 · Finkelstein et al. · 2010 [cited by applicant]
US 20100303270A1 · Parkins · 2010 [cited by applicant]
US 20110004774A1 · Hansquine et al. · 2011 [cited by applicant]
US 20110023047A1 · Memik et al. · 2011 [cited by applicant]
US 20110078469A1 · Therien · 2011 [cited by examiner]
US 20110087909A1 · Kanakogi · 2011 [cited by applicant]
US 20110088041A1 · Alameldeen · 2011 [cited by examiner]
US 20110119508A1 · Heller, Jr. · 2011 [cited by applicant]
US 20110161965A1 · Im et al. · 2011 [cited by applicant]
US 20110173477A1 · Asaba · 2011 [cited by applicant]
US 20110252267A1 · Naveh et al. · 2011 [cited by applicant]
US 20110307730A1 · Marshall et al. · 2011 [cited by applicant]
US 20120016793A1 · Peters et al. · 2012 [cited by applicant]
US 20120019379A1 · Ben Ayed · 2012 [cited by applicant]
US 20120042176A1 · Kim · 2012 [cited by applicant]
US 20120079235A1 · Iyer et al. · 2012 [cited by applicant]
US 20120084777A1 · Jayamohan · 2012 [cited by applicant]
US 20120144217A1 · Sistla et al. · 2012 [cited by applicant]
US 20120144218A1 · Brey et al. · 2012 [cited by applicant]
US 20120146708A1 · Naffziger et al. · 2012 [cited by applicant]
US 20120166763A1 · Henry et al. · 2012 [cited by applicant]
US 20120185709A1 · Weissmann et al. · 2012 [cited by applicant]
US 20120284729A1 · Sharda et al. · 2012 [cited by applicant]
US 20120324250A1 · Chakraborty et al. · 2012 [cited by applicant]
US 20130007413A1 · Thomson et al. · 2013 [cited by applicant]
US 20130041977A1 · Wakamiya · 2013 [cited by applicant]
US 20130047166A1 · Penzes et al. · 2013 [cited by applicant]
US 20130080795A1 · Sistla et al. · 2013 [cited by applicant]
US 20130111226A1 · Ananthakrishnan et al. · 2013 [cited by applicant]
US 20130212587A1 · Doyle et al. · 2013 [cited by applicant]
US 20130247046A1 · Asai et al. · 2013 [cited by applicant]
US 20140007097A1 · Chin et al. · 2014 [cited by applicant]
US 20140013330A1 · Wang · 2014 [cited by examiner]
US 20140024145A1 · Bickford et al. · 2014 [cited by applicant]
US 20140026146A1 · Jahagirdar et al. · 2014 [cited by applicant]
US 20140068622A1 · Gmuender et al. · 2014 [cited by applicant]
US 20140089936A1 · Chang et al. · 2014 [cited by applicant]
US 20140130058A1 · Hum et al. · 2014 [cited by applicant]
US 20140181830A1 · Naik et al. · 2014 [cited by applicant]
US 20140189704A1 · Narvaez et al. · 2014 [cited by applicant]
US 20140237274A1 · Murakami · 2014 [cited by applicant]
US 20140281610A1 · Biswas · 2014 [cited by examiner]
US 20140325511A1 · Aasheim · 2014 [cited by applicant]
US 20140344550A1 · Kruglick · 2014 [cited by applicant]
US 20140373025A1 · Kim · 2014 [cited by examiner]
US 20150007187A1 · Shows · 2015 [cited by examiner]
US 20150046685A1 · Park et al. · 2015 [cited by applicant]
US 20150178138A1 · Saha · 2015 [cited by examiner]
US 20150205642A1 · Xu et al. · 2015 [cited by applicant]
US 20150286262A1 · Park · 2015 [cited by examiner]
US 20150355700A1 · Pusukuri et al. · 2015 [cited by applicant]
US 20160139964A1 · Chen et al. · 2016 [cited by applicant]
US 20160246647A1 · Harris · 2016 [cited by examiner]
US 20160357676A1 · Cain, III et al. · 2016 [cited by applicant]
US 20170046198A1 · Solihin · 2017 [cited by examiner]
US 20170090988A1 · Young · 2017 [cited by examiner]
US 20170177415A1 · Dhanraj et al. · 2017 [cited by applicant]
CN 101154169A · 2008 [cited by applicant]
CN 101504618A · 2009 [cited by applicant]
CN 101981529A · 2011 [cited by applicant]
CN 102246117A · 2011 [cited by applicant]
CN 102402458A · 2012 [cited by applicant]
CN 102520912B · 2012 [cited by applicant]
CN 104781803A · 2015 [cited by applicant]
JP 2010039923A · 2010 [cited by applicant]
JP 2014032621A · 2014 [cited by applicant]
WO 2011011155A1 · 2011 [cited by applicant]
WO 2012170746A2 · 2012 [cited by applicant]
WO 2014032621A1 · 2014 [cited by applicant]
Advisory Action dated May 15, 2018 for U.S. Appl. No. 14/978,182. [cited by applicant]
Final Office Action dated Feb. 13, 2020 for U.S. Appl. No. 16/107,691. [cited by applicant]
Final Office Action dated Feb. 16, 2016 for U.S. Appl. No. 13/830,157. [cited by applicant]
Final Office Action dated Mar. 8, 2018 for U.S. Appl. No. 14/978,182. [cited by applicant]
Final Office Action dated Mar. 8, 2018 for U.S. Appl. No. 14/997,032. [cited by applicant]
International Search Report & Written Opinion dated Feb. 17, 2017 for PCT Patent Application No. PCT/ US2016/062327. [cited by applicant]
International Search Report & Written Opinion dated Mar. 21, 2017 for PCT Patent Application No. PCT/ US2016/066351. [cited by applicant]
Non-Final Office Action dated Aug. 9, 2019 for U.S. Appl. No. 16/107,691. [cited by applicant]
Non-Final Office Action dated Aug. 25, 2017 for U.S. Appl. No. 14/997,032. [cited by applicant]
Non-Final Office Action dated Jan. 5, 2017 for U.S. Appl. No. 14/997,032. [cited by applicant]
Non-Final Office Action dated Jun. 11, 2020 for U.S. Appl. No. 16/107,691. [cited by applicant]
Non-Final Office Action dated Mar. 16, 2015 for U.S. Appl. No. 13/830,157. [cited by applicant]
Non-Final Office Action dated Nov. 1, 2017 for U.S. Appl. No. 14/978,182. [cited by applicant]
Non-Final Office Action dated Nov. 1, 2018 for U.S. Appl. No. 14/978,182. [cited by applicant]
Non-Final Office Action dated Oct. 8, 2015 for U.S. Appl. No. 13/830,157. [cited by applicant]
Notice of Allowance dated Jul. 12, 2018 for Japanese Patent Application No. JP2016-235553. [cited by applicant]
Notice of Allowance dated Jun. 14, 2016 for U.S. Appl. No. 13/830,157. [cited by applicant]
Notice of Allowance dated Mar. 21, 2019 for U.S. Appl. No. 14/978,182. [cited by applicant]
Notice of Allowance dated May 22, 2018 for U.S. Appl. No. 14/997,032. [cited by applicant]
Notice of Allowance dated May 25, 2016 for U.S. Appl. No. 13/830,157. [cited by applicant]
Notice of Allowance dated Oct. 27, 2020 for U.S. Appl. No. 16/107,691. [cited by applicant]
Office Action dated Aug. 14, 2019 for European Patent Application No. 16885414.9. [cited by applicant]
Office Action dated Dec. 19, 2017 for Japanese Patent Application No. 2016-235553. [cited by applicant]
Office Action dated Feb. 3, 2020 for Chinese Patent Application No. 201611028185.2. [cited by applicant]
Office Action dated Sep. 6, 2019 for Singapore Patent Application No. 10201610420T. [cited by applicant]
Hozono, T. et al., “A Task Allocation Strategy for Heterogeneous Server Systems with a Machine Learning Approach”, IPSJ SIG Technical Report, vol. 2011, No. 12, Mar. 11, 2011, 9 pages. [cited by applicant]
Non-Final Office Action dated Dec. 9, 2021 for U.S. Appl. No. 17/173,100. [cited by applicant]
Notice of Allowance dated Apr. 4, 2022 for U.S. Appl. No. 17/173,100. [cited by applicant]
Non-Final Office Action dated Nov. 25, 2022 for U.S. Appl. No. 17/857,394. [cited by applicant]
Final Office Action dated Jun. 8, 2023 for U.S. Appl. No. 17/857,394. [cited by applicant]
Notice of Allowance dated Aug. 18, 2023 for U.S. Appl. No. 17/857,394. [cited by applicant]
Advisory Action, U.S. Appl. No. 16/107,691, filed Apr. 28, 2020, 3 pages. [cited by applicant]
European Search Report and Search Opinion, EP App. No. 16885414.9, Aug. 14, 2019, 13 pages. [cited by applicant]
Final Office Action, U.S. Appl. No. 14/997,032, filed Mar. 8, 2018, 11 pages. [cited by applicant]
Final Office Action, U.S. Appl. No. 16/107,691, filed Feb. 13, 2020, 22 pages. [cited by applicant]
Final Office Action, U.S. Appl. No. 17/857,394, filed Jun. 8, 2023, 14 pages. [cited by applicant]
Guy Therien et al., “Systems, Methods and Devices for Work Placement on Processor Cores”, U.S. Appl. No. 14/973,009, filed Dec. 17, 2015, 41 pages. [cited by applicant]
International Preliminary Report on Patentability, PCT App. No. PCT/US16/66351, Jul. 26, 2018, 11 pages. [cited by applicant]
International Search Report and Written Opinion, PCT App. No. PCT/US16/66351, Mar. 21, 2017, 12 pages. [cited by applicant]
Non-Final Office Action dated Mar. 16, 2015 for U.S. Appl. No. 13/830,157, 24 pages. [cited by applicant]
Non-Final Office Action dated Nov. 1, 2018 for U.S. Appl. No. 14/978,182, 24 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 17/857,394, filed Nov. 25, 2022, 26 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 14/997,032, filed Aug. 25, 2017, 12 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 14/997,032, filed Jan. 5, 2017, 14 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 16/107,691, filed Aug. 9, 2019, 20 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 16/107,691, filed Jun. 11, 2020, 22 pages. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 17/173,100, filed Dec. 9, 2021, 15 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 14/997,032, filed May 22, 2018, 7 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 16/107,691, filed Oct. 27, 2020, 11 pages. [cited by applicant]
Notice of Allowance, U.S. App. No. 17/173,100, filed Apr. 4, 2022, 9 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 17/857,394, filed Aug. 18, 2023, 5 pages. [cited by applicant]
Notice of Allowance, U.S. Appl. No. 17/857,394, filed Sep. 7, 2023, 5 pages. [cited by applicant]
Vijay Dhanraj et al., “Thread and/or Virtual Machine Scheduling for Cores With Diverse Capabilities”, U.S. Appl. No. 14/978,182, filed Dec. 22, 2015, 40 pages. [cited by applicant]
Notice of Allowance, CN App. No. 201611163844.3, Apr. 30, 2019, 4 pages (2 pages of English Translation and 2 pages of Original Document). [cited by applicant]
Notice of Allowance, KR App. No. 10-2018-7016836, Feb. 27, 2024, 8 pages (3 pages of English Translation and 5 pages of Original Document). [cited by applicant]
Notice of Allowance, SG App. No. 10201610420T, Apr. 21, 2020, 5 pages. [cited by applicant]
Office Action, CN App. No. 201611163844.3, Aug. 31, 2018, 32 pages (16 pages of English Translation and 16 pages of Original Document). [cited by applicant]
Office Action, KR App. No. 10-2018-7016836, Jul. 12, 2023, 22 pages (12 pages of English Translation and 10 pages of Original Document). [cited by applicant]