Generating network scenarios to train an LLM-based network troubleshooting agent

1 . A method comprising:

receiving, at a device, feedback from a user regarding an output of a large language model-based troubleshooting agent configured to troubleshoot issues for a network, the output being generated by the large language model-based troubleshooting agent in response to an input request from the user for the large language model-based troubleshooting agent to troubleshoot an issue with the network;

determining, based at least in part on the feedback, that the user was dissatisfied with the output from the large language model-based troubleshooting agent to troubleshoot the issue;

receiving, at the device, network context data that represents a state of the network at a time corresponding to when the output was generated;

determining, by the device, a network scenario associated with the input request and the output, based on network context data received by the device;

causing, by the device and based on the user being dissatisfied, the network scenario to be replicated in a test network that is different than the network;

causing, by the device, the large language model-based troubleshooting agent to assess the network scenario; and

updating, by the device, the large language model-based troubleshooting agent in part by training the large language model-based troubleshooting agent to generate an accurate response to troubleshoot the issue expressed in the network scenario in the test network.

2 . The method as in claim 1 , further comprising:

forming a generic input request based on the input request from the user, wherein the large language model-based troubleshooting agent assesses the network scenario in the test network using the generic input request.

3 . The method as in claim 1 , wherein the device updates the large language model-based troubleshooting agent using reinforcement learning based on a result of an assessment of the network scenario in the test network.

4 . The method as in claim 1 , wherein the device causes the network scenario to be replicated in the test network by:

making a determination that network conditions in the network associated with the input request from the user cannot be reproduced in the test network; and

sending a request, based on the determination, to set up the network scenario.

5 . The method as in claim 1 , wherein the device further receives the network context data associated with the output of the large language model-based troubleshooting agent.

6 . The method as in claim 1 , wherein the output is a final answer by the large language model-based troubleshooting agent.

7 . The method as in claim 1 , wherein the output is an intermediate output of the large language model-based troubleshooting agent.

8 . The method as in claim 1 , wherein the feedback from the user comprises a suggested step for the large language model-based troubleshooting agent to take.

9 . The method as in claim 1 , further comprising:

notifying the user that the large language model-based troubleshooting agent has been updated.

10 . The method as in claim 1 , wherein the input request references a node in the network of a particular type, and wherein the test network includes a node of the particular type.

11 . An apparatus, comprising:

one or more network interfaces;

a processor coupled to the one or more network interfaces and configured to execute one or more processes; and

a memory configured to store a process that is executable by the processor, the process when executed configured to:

receive feedback from a user regarding an output of a large language model-based troubleshooting agent configured to troubleshoot issues for a network, the output being generated by that-the large language model-based troubleshooting agent generates-in response to an input request from the user for the large language model-based troubleshooting agent to troubleshoot an issue with the network;

determine, based at least in part on the feedback, that the user was dissatisfied with the output from the large language model-based troubleshooting agent to troubleshoot the issue;

receive network context data that represents a state of the network at a time corresponding to when the output was generated;

determine a network scenario associated with the input request and the output, based on network context data received by the apparatus;

cause, based on the user being dissatisfied, the network scenario to be replicated in a test network that is different than the network;

cause the large language model-based troubleshooting agent to assess the network scenario; and

update the large language model-based troubleshooting agent in part by training the large language model-based troubleshooting agent to generate an accurate response to troubleshoot the issue expressed in the network scenario in the test network.

12 . The apparatus as in claim 11 , wherein the process when executed is further configured to:

form a generic input request based on the input request from the user, wherein the large language model-based troubleshooting agent assesses the network scenario in the test network using the generic input request.

13 . The apparatus as in claim 11 , wherein the apparatus updates the large language model-based troubleshooting agent using reinforcement learning based on a result of an assessment of the network scenario in the test network.

14 . The apparatus as in claim 11 , wherein the apparatus causes the network scenario to be replicated in the test network by:

making a determination that network conditions in the network associated with the input request from the user cannot be reproduced in the test network; and

sending a request, based on the determination, to set up the network scenario.

15 . The apparatus as in claim 11 , wherein the apparatus further receives network context data associated with the output of the large language model-based troubleshooting agent.

16 . The apparatus as in claim 11 , wherein the output is a final answer by the large language model-based troubleshooting agent.

17 . The apparatus as in claim 11 , wherein the output is an intermediate output of the large language model-based troubleshooting agent.

18 . The apparatus as in claim 11 , wherein the feedback from the user comprises a suggested step for the large language model-based troubleshooting agent to take.

19 . The apparatus as in claim 11 , wherein the process when executed is further configured to:

notify the user that the large language model-based troubleshooting agent has been updated.

20 . A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:

receiving, at a device, feedback from a user regarding an output of a large language model-based troubleshooting agent configured to troubleshoot issues for a network, the output being generated by the large language model-based troubleshooting agent in response to an input request from the user for the large language model-based troubleshooting agent to troubleshoot an issue with the network;

determining, based at least in part on the feedback, that the user was dissatisfied with the output from the large language model-based troubleshooting agent to troubleshoot the issue;

receiving, at the device, network context data that represents a state of the network at a time corresponding to when the output was generated;

determining, by the device, a network scenario associated with the input request and the output, based on network context data received by the device;

causing, by the device and based on the user being dissatisfied, the network scenario to be replicated in a test network that is different than the network;

causing, by the device, the large language model-based troubleshooting agent to assess the network scenario; and

updating, by the device, the large language model-based troubleshooting agent in part by training the large language model-based troubleshooting agent to generate an accurate response to troubleshoot the issue expressed in the network scenario in the test network.