Chain of thought reasoning for LLM integrations in spreadsheet environments
Technology is disclosed herein for the integration of spreadsheet environments and LLM services. In an implementation, an application service inputs a first prompt to a LLM service to provide a formula suggestion for a spreadsheet. The application service receives a first output from the LLM service that includes a first formula in a programming language having a syntax that does not support comments. The application service generates a second prompt instructing the LLM service to provide a chain-of-thought breakdown of the first formula and receives a second output. The second output includes a second formula in the same programming language as the first formula and multiple comments corresponding to multiple portions of the second formula.
1. A computing apparatus comprising:
one or more computer-readable storage media;
one or more processors operatively coupled with the one or more computer-readable storage media; and
program instructions stored on the one or more computer-readable storage media that, when executed by the one or more processors, direct the computing apparatus to at least:
receive, in a user interface of an application hosting a spreadsheet, a natural language request from a user for a formula suggestion for the spreadsheet;
input a first prompt to a large language model (LLM) service to provide the formula suggestion for the spreadsheet in response to the natural language request;
receive a first output from the LLM service that includes a first formula in a programming language having a syntax that does not support comments;
generate a second prompt instructing the LLM service to provide a chain-of-thought breakdown of the first formula;
receive a second output comprising the chain-of-thought breakdown of the first formula in the same programming language as the first formula, wherein the chain-of-thought breakdown comprises multiple portions and multiple comments corresponding to the multiple portions;
display, in the user interface, a progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions;
receive, in the user interface, user input comprising an acceptance of a given portion of the multiple portions; and
execute the first formula in the spreadsheet.
2. The computing apparatus of claim 1 , wherein to display the progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions, the program instructions further direct the computing apparatus to advance the progression to a next portion of the multiple portions in response to receiving the user input comprising the acceptance of the given portion.
3. The computing apparatus of claim 2 , wherein the chain-of-thought breakdown comprises a LET function.
4. The computing apparatus of claim 3 , wherein the programming language comprises a non-procedural programming language native to a spreadsheet application hosting the spreadsheet.
5. The computing apparatus of claim 1 , wherein to display the progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions, the program instructions further direct the computing apparatus to receive user input in the user interface of the application to accept or reject each portion of the multiple portions after displaying the respective portion.
6. The computing apparatus of claim 5 , wherein the program instructions further direct the computing apparatus to generate a new prompt to the LLM service when a portion of the multiple portions is rejected.
7. The computing apparatus of claim 1 , wherein the chain-of-thought breakdown comprises a decomposition of the first formula into the multiple portions, wherein each portion of the multiple portions comprises an intermediate step in computing the first formula.
8. The computing apparatus of claim 7 , wherein the first prompt includes at least a portion of the spreadsheet.
9. A method comprising:
receiving, in a user interface of an application hosting a spreadsheet, a natural language request from a user for a formula suggestion for the spreadsheet;
inputting a first prompt to a large language model (LLM) service to provide the formula suggestion for a spreadsheet;
receiving a first output from the LLM service that includes a first formula in a programming language having a syntax that does not support comments;
generating a second prompt instructing the LLM service to provide a chain-of-thought breakdown of the first formula;
receiving a second output comprising the chain-of-thought breakdown of the first formula in the same programming language as the first formula, wherein the chain-of-thought breakdown comprises multiple portions and multiple comments corresponding to the multiple portions;
displaying, in the user interface, a progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions;
receiving, in the user interface, user input comprising an acceptance of a given portion of the multiple portions; and
executing the first formula in the spreadsheet.
10. The method of claim 9 , wherein displaying the progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions comprises advancing the progression to a next portion of the multiple portions in response to receiving the user input comprising the acceptance of the given portion.
11. The method of claim 10 , wherein the chain-of-thought breakdown comprises a LET function.
12. The method of claim 11 , wherein the programming language comprises a non-procedural programming language native to a spreadsheet application hosting the spreadsheet.
13. The method of claim 9 , wherein displaying the progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions comprises receiving user input in the user interface of the application to accept or reject each portion of the multiple portions after displaying the respective portion.
14. The method of claim 13 , further comprising generating a new prompt to the LLM service when a portion of the multiple portions is rejected.
15. The method of claim 9 , wherein the chain-of-thought breakdown comprises a decomposition of the first formula into the multiple portions, wherein each portion of the multiple portions comprises an intermediate step in computing the first formula.
16. One or more computer-readable storage media having program instructions stored thereon that, when executed by one or more processors operatively coupled with the one or more computer-readable storage media, direct a computing device to:
receive, in a user interface of an application hosting a spreadsheet, a natural language request from a user for a formula suggestion for the spreadsheet;
input a first prompt to a large language model (LLM) service to provide a formula suggestion for the spreadsheet in response to the natural language request;
receive a first output from the LLM service that includes a first formula in a programming language having a syntax that does not support comments;
generate a second prompt instructing the LLM service to provide a chain-of-thought breakdown of the first formula;
receive a second output comprising the chain-of-thought breakdown of the first formula in the same programming language as the first formula, wherein the chain-of-thought breakdown comprises multiple portions and multiple comments corresponding to the multiple portions;
display, in the user interface, a progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions;
receive, in the user interface, user input comprising an acceptance of a given portion of the multiple portions; and
execute the first formula in the spreadsheet.
17. The one or more computer-readable storage media of claim 16 , wherein to display the progression of the multiple portions of the chain-of-thought breakdown, the program instructions direct the computing device to advance the progression to a next portion of the multiple portions in response to receiving the user input comprising the acceptance of the given portion.
18. The one or more computer-readable storage media of claim 17 , wherein the chain-of-thought breakdown comprises a LET function.
19. The one or more computer-readable storage media of claim 18 , wherein the programming language comprises a non-procedural programming language native to a spreadsheet application hosting the spreadsheet.
20. The one or more computer-readable storage media of claim 17 , wherein to display the progression of the multiple portions of the chain-of-thought breakdown and the multiple comments corresponding to the multiple portions, the program instructions further direct the computing device to receive user input in the user interface of the application to accept or reject each portion of the multiple portions after displaying the respective portion.