Devices, systems, and methods for secure and trustworthy voice assistance

1 . A voice assistance system comprising:

a plurality of individual, secure platforms stored on a computing device of a user, the plurality of platforms comprising:

a secure local edge platform to (i) capture audio spoken by the user, (ii) plan one or more actions to execute one or more commands present in the audio, (iii) perform anomaly detection on the audio, and (iv) manage output to one or more external devices;

an orchestration platform comprising a multi-agent framework, the multi-agent framework comprising a planning agent, an execution agent, and a validation agent;

a transport platform to transmit the audio from the secure local edge platform to the orchestration platform;

a model storage platform storing a plurality of models, the plurality of models comprising a text-to-speech (TTS) model to convert the audio into text data, and a large language model (LLM) stored and hosted entirely on the computing device; and

an external application platform to connect the voice assistance system with one or more external applications, the one or more external applications for performing one or more tasks,

wherein the one or more commands comprise a command to operate the one or more external devices and/or a command to execute the one or more tasks, and

wherein the validation agent prompts the user for explicit verbal confirmation of the one or more commands before the one or more commands are executed by one or more actions, when the one or more commands are flagged for breaking one or more rules.

2 . The system of claim 1 , wherein the planning agent generates plan frames and detects when the one or more commands break the one or more rules violations and flags the one or more commands accordingly, wherein the validation agent performs verbal confirmation loops to trigger the explicit verbal confirmation and generates one or more validated command frames after the user articulates the explicit verbal confirmation, wherein the execution agent transmits the one or more validated command frames to the transport platform.

3 . The system of claim 2 , wherein the transport platform receives the one or more validated command frames from the execution agent and transmits the one or more validated command frames to the one or more external devices and/or the one or more external applications.

4 . The system of claim 3 , wherein the transmission of the one or more validated command frames is performed via a secure channel.

5 . The system of claim 1 , wherein the external application platform utilizes a Model Context Protocol (MCP) to connect the voice assistant system with the one or more external applications.

6 . The system of claim 1 , wherein the one or more external applications comprise a financial transaction application.

7 . The system of claim 6 , further comprising a payment security platform to intercept a command related to the financial transaction application and request, before execution, validation of the command related to the financial transaction application.

8 . The system of claim 1 , wherein the one or more actions are executed only when the planning agent, the validation agent, and the execution agent all approve of the execution.

9 . The system of claim 1 , wherein all communications within the system are represented as typed frames with explicit schemas, thereby enabling deterministic multi-agent processing.

10 . The system of claim 9 , wherein all communications comprise (i) communications between any two or more of the plurality of platforms, and (ii) communications between any one or more of the plurality of platforms and the one or more external devices.

11 . The system of claim 10 , wherein the one or more external devices comprise a home appliance.

12 . A method for providing voice assistance, the method comprising:

capturing, by at least one processor running a secure local edge platform, audio spoken by a user, the audio comprising one or more commands, the one or more commands for operating one or more physical devices and/or executing one or more tasks;

transmitting, by the at least one processor, the audio to an orchestration platform, the orchestration platform comprising a planning agent, a validation agent, and an execution agent;

generating, by the at least one processor running the planning agent, a plurality of plan frames to plan an action to execute the one or more commands;

requesting, by the at least one processor running the validation agent, explicit verbal confirmation from the user of the one or more commands, when the one or more commands are flagged for breaking one or more rules;

receiving, by the at least one processor running the validation agent, the explicit verbal confirmation;

upon receiving the explicit verbal confirmation, validating, by the at least one processor running the validation agent, the one or more commands, to generate one or more validated commands;

transmitting, by the at least one processor, the one or more validated commands from the validation agent to the execution agent; and

executing, by the at least one processor running the execution agent, the one or more validated commands by performing the action.

13 . The method of claim 12 , further comprising:

performing, by the at least one processor, anomaly detection on the audio to determine if the one or more commands are authorized by the user.

14 . The method of claim 13 , wherein the performing the anomaly detection further comprises:

detecting, by the at least one processor running a hybrid machine learning model, one or more anomalous patterns in the audio;

upon detecting the one or more anomalous patterns, generating, by the at least one processor, a security interrupt event;

transmitting, by the at least one processor, the security interrupt event to the orchestration platform via an encrypted transport channel.

15 . The method of claim 14 , wherein the transmitting of the security interrupt event occurs independently of any additional utterances of the user.

16 . The method of claim 12 , further comprising optimizing at least one prompt for the planning agent to improve accuracy of the planning agent in planning the action to execute the one or more commands.

17 . A non-transitory computer-readable storage medium, having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations, the operations comprising:

capturing, by a secure local edge platform, audio spoken by a user, the audio comprising one or more commands, the one or more commands for operating one or more external devices and/or executing one or more tasks;

transmitting, by an encrypted transport platform, the audio to an orchestration platform, the orchestration platform comprising a planning agent, a validation agent, and an execution agent;

generating, by the planning agent, a plurality of plan frames to plan an action to execute the one or more commands;

validating, by the validation agent, the one or more commands by requesting explicit verbal confirmation from the user when the one or more commands are flagged for breaking one or more rules;

upon receiving the explicit verbal confirmation, generating, by the validation agent, one or more validated commands;

transmitting, by the validation agent, the one or more validated commands to the execution agent; and

executing, by the execution agent, the one or more validated commands by performing the action.

18 . The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise:

detecting, by an anomaly detection platform, one or more anomalous patterns in the audio;

upon detecting the one or more anomalous patterns, generating, by the anomaly detection platform, a security interrupt event;

transmitting, by the anomaly detection platform, the security interrupt event to the orchestration platform via an encrypted transport channel.

19 . The non-transitory computer-readable medium of claim 17 , wherein the transmitting of the security interrupt event occurs independently of any additional utterances of the user.

20 . The non-transitory computer-readable medium of claim 17 , wherein the operations further comprise optimizing at least one prompt for the planning agent to improve accuracy of the planning agent in planning the action to execute the one or more commands.