Teaching LLM-based agents to troubleshoot networks using reinforcement learning

1 . A method comprising:

causing a test environment to simulate a scenario related to an impairment with a computer network;

determining, based at least in part on the test environment, ground truth information associated with the impairment;

using, by a device, a large language model-based agent to complete a task related to the scenario;

making, by the device and based at least in part on the ground truth information, an evaluation of an action taken by the large language model-based agent to complete the task;

selecting, by the device and based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task; and

updating, by the device, the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.

2 . The method as in claim 1 , wherein the task comprises troubleshooting an issue in the computer network.

3 . The method as in claim 1 , wherein the large language model-based agent completes the task by interfacing with an application programming interface (API) of a controller for the computer network.

4 . The method as in claim 1 , wherein the device selects the set of allowed actions based in part on user feedback regarding whether an output of the large language model-based agent from the task was satisfactory.

5 . The method as in claim 1 , wherein the large language model-based agent uses a transformer-based policy network to select the action taken by the large language model-based agent to complete the task.

6 . The method as in claim 5 , wherein the device updates the transformer-based policy network with the set of allowed actions.

7 . The method as in claim 1 , wherein the set of allowed actions comprises an action specified by a subject matter expert via a user interface.

8 . The method as in claim 1 , wherein the causing the test environment to simulate the scenario includes using a copy of the large language model-based agent to perform the task.

9 . The method as in claim 1 , wherein the large language model-based agent uses a first large language model to answer a question regarding the task and a second large language model to perform the action.

10 . The method as in claim 9 , wherein the second large language model smaller in size than that of the first large language model.

11 . An apparatus, comprising:

one or more network interfaces;

a processor coupled to the one or more network interfaces; and

a memory configured to store one or more instructions that, when executed by the processor, configure the processor to:

cause a test environment to simulate a scenario related to an impairment with a computer network;

identify, based at least in part on the test environment, ground truth information associated with the impairment;

use a large language model-based agent to complete a task related to the scenario;

make, based at least in part on the ground truth information, an evaluation of an action taken by the large language model-based agent to complete the task;

select, based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task; and

update the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.

12 . The apparatus as in claim 11 , wherein the task comprises troubleshooting an issue in the computer network.

13 . The apparatus as in claim 11 , wherein the large language model-based agent completes the task by interfacing with an application programming interface (API) of a controller for the computer network.

14 . The apparatus as in claim 11 , wherein the apparatus selects the set of allowed actions based in part on user feedback regarding whether an output of the large language model-based agent from the task was satisfactory.

15 . The apparatus as in claim 11 , wherein the large language model-based agent uses a transformer-based policy network to select the action taken by the large language model-based agent to complete the task.

16 . The apparatus as in claim 15 , wherein the apparatus updates the transformer-based policy network with the set of allowed actions.

17 . The apparatus as in claim 11 , wherein the set of allowed actions comprises an action specified by a subject matter expert via a user interface.

18 . The apparatus as in claim 11 , wherein the processor is configured to cause the test environment to simulate the scenario includes by using a copy of the large language model-based agent to perform the task.

19 . The apparatus as in claim 11 , wherein the large language model-based agent uses a first large language model to answer a question regarding the task and a second large language model to perform the action.

20 . A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:

causing a test environment to simulate a scenario related to an impairment with a computer network;

determining, based at least in part on the test environment, ground truth information associated with the impairment;

using, by the device, a large language model-based agent to complete a task related to the scenario;

making, by the device and based at least in part on the ground truth information, an evaluation of an action taken by the large language model-based agent to complete the task;

selecting, by the device and based in part on the evaluation, a set of allowed actions that the large language model-based agent is allowed to perform to complete the task; and

updating, by the device, the large language model-based agent with an indication of the set of allowed actions that it is allowed to perform to complete the task.