IP Library Granted Patent US 12705577
Granted Patent B2
US 12705577 · App. 17/877,111 · Granted Aug 11, 2026

Collaborative voice-based design and development system

Inventors: Obaid Shaikh (Weatogue, CT); Chandra S. Bathula (Rocky Hill, CT); Rebecca T. Scanlon (Manchester, CT); Annapurna Jagasia (Windsor, CT); Bharghavi Tanamala (East Windsor, CT); Robert O'Connor (Harwinton, CT); Matthew W. Kropp (Manchester, CT); Jhin McGlynn (Waterford, CT)
Assignee: THE TRAVELERS INDEMNITY COMPANY
G10L15/22G06Q10/101G06T11/80G10L15/26G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12705577
App. No.
17/877,111
Granted
Aug 11, 2026
Kind
B2
Abstract

A system includes a multi-user interface and a collaborative drawing tool. The collaborative drawing tool can receive a first set of voice-based drawing commands from a first user, perform a first speech-to-text conversion to transform the first set of voice-based drawing commands into a first set of drawing commands, and render a first drawing update on the multi-user interface based on the first set of drawing commands. The collaborative drawing tool can receive a second set of voice-based drawing commands from a second user, perform a second speech-to-text conversion to transform the second set of voice-based drawing commands into a second set of drawing commands, render a second drawing update on the multi-user interface based on the second set of drawing commands, and save, to a storage system, a drawing output of the multi-user interface resulting from the first drawing update and the second drawing update.

Claims (90)

1 . A system, comprising:

a multi-user interface configured to be shared by a plurality of user connections over a network interface and displayed by a plurality of user systems, each user system of the plurality of user systems comprising a processing system and a display; and

a collaborative drawing tool executable by at least one processing device and comprising a plurality of instructions that when executed by the at least one processing device result in:

receiving a first set of voice-based drawing commands from a first user of the user connections;

performing a first speech-to-text conversion to transform the first set of voice-based drawing commands into a first set of drawing commands;

performing component lookup, positioning, and labelling to form a first message based on keyword processing;

mapping contents of the first message to one or more graph components with connection rules to render a drawing output at each user system of the plurality of user systems by accessing a code library that comprises code to display the one or more graph components corresponding to a device type of each user system of the plurality of user systems;

generating a graph configured to graphically depict the one or more graph components through the multi-user interface using by transmitting the code to each user system of the plurality of user systems, the code corresponding to the device type of each user system of the plurality of user systems;

controlling the multi-user interface to render a first drawing update of the graph on the multi-user interface based on the first set of drawing commands;

receiving a second set of voice-based drawing commands from a second user of the user connections;

performing a second speech-to-text conversion to transform the second set of voice-based drawing commands into a second set of drawing commands;

performing component lookup, positioning, and labelling to form a second message based on keyword processing;

updating the graph based on parsing the second message and accessing the code library that maps contents of the second message to the one or more graph components with connection rules to render the drawing output at each user system of the plurality of user systems;

controlling the multi-user interface to render a second drawing update of the graph on the multi-user interface based on the second set of drawing commands; and

saving, to a storage system, the drawing output of the multi-user interface resulting from the first drawing update and the second drawing update.

2 . The system of claim 1 , wherein the first set of voice-based drawing commands and the second set of voice-based drawing commands are received through separate audio channels.

3 . The system of claim 2 , further comprising instructions that when executed by the at least one processing device result in:

tracking a change history of the first user based on a first sequence comprising one or more sets of drawing commands received on a first audio channel associated with the first user;

tracking a change history of the second user based on a second sequence comprising one or more sets of drawing commands received on a second audio channel associated with the second user; and

providing an undo feature to remove one or more drawing updates based on the change history.

4 . The system of claim 1 , further comprising instructions that when executed by the at least one processing device result in:

mapping the first set of drawing commands and the second set of drawing commands to a visual component palette of shapes, connectors, and attributes; and

selecting one or more of the shapes, connectors, and attributes to add, modify, or delete based on the mapping.

5 . The system of claim 4 , further comprising instructions that when executed by the at least one processing device result in:

parsing the first set of drawing commands and the second set of drawing commands to determine one or more actions to be performed; and

performing one or more of adding, modifying, or deleting based on the one or more actions determined.

6 . The system of claim 1 , further comprising instructions that when executed by the at least one processing device result in:

resizing a field of view of a display on the multi-user interface based on adding, moving, or deleting a visual component upon rendering the first drawing update and/or the second drawing update.

7 . The system of claim 1 , further comprising instructions that when executed by the at least one processing device result in:

broadcasting one or more changes made through the multi-user interface to all the plurality of user systems.

8 . The system of claim 7 , further comprising instructions that when executed by the at least one processing device result in:

assigning a session identifier to a shared session through the multi-user interface for the user connections; and

controlling access to the shared session through a user authentication based on the session identifier.

9 . The system of claim 8 , wherein the first user and the second user are identified based on a sign-in interface that requires user credential entry prior to accepting input from the first user and the second user.

10 . The system of claim 1 , wherein speech-to-text conversion comprises:

performing natural language processing to identify one or more keyword phrases; and

performing keyword processing to extract one or more keywords from the one or more keyword phrases.

11 . The system of claim 10 , wherein the first message and the second message are each a text-based object notation message.

12 . The system of claim 1 , wherein the multi-user interface comprises a shared microphone selector that enables voice detection for one user and disables voice capture from other users of the user connections.

13 . The system of claim 1 , further comprising instructions that when executed by the at least one processing device result in:

identifying a template associated with the first set of commands; and

determining one or more components to include in the rendering based on the template and contents of the first set of commands.

14 . The system of claim 13 , further comprising instructions that when executed by the at least one processing device result in:

customizing at least one of the one or more components based on a change request to make a modification.

15 . The system of claim 1 , further comprising instructions that when executed by the at least one processing device result in:

monitoring for a key phrase to determine which user of a plurality of users is identified as the first user.

16 . The system of claim 1 , further comprising instructions that when executed by the at least one processing device result in:

limiting permission to make one or more updates to a drawing through the multi-user interface based on determining an identity of the first user, the second user, and a third user and one or more access constraints assigned to the first user, the second user, and the third user, wherein limiting permission comprises preventing the third user from making an update to the drawing in a shared session with the first user and the second user based on determining that the third user lacks a drawing update permission.

17 . The system of claim 1 , further comprising instructions that when executed by the at least one processing device result in:

accessing a drawing file defined in a text-based object notation file format;

rendering a drawing on the multi-user interface in a graphical display format based on parsing and interpreting contents of the drawing file in the text-based object notation file format; and

updating the drawing file in the text-based object notation file format based on the first drawing update and the second drawing update, wherein saving the drawing output stores the drawing file in the text-based object notation file format.

18 . The system of claim 1 , wherein the multi-user interface is configured to receive a combination of voice-based input and manual input through the user connections, the manual input comprising one or more of: typed text, a selection action, and a clicking action.

19 . A computer program product comprising a non-transitory storage medium embodied with computer program instructions that when executed by a computer cause the computer to implement:

receiving a first set of voice-based drawing commands from a first user of a plurality of user connections;

performing a first speech-to-text conversion to transform the first set of voice-based drawing commands into a first set of drawing commands;

performing component lookup, positioning, and labelling to form a first message based on keyword processing;

mapping contents of the first message to one or more graph components with connection rules to render a drawing output at each user system of a plurality of user systems by accessing a code library that comprises code to display the one or more graph components corresponding to a device type of each user system of the plurality of user systems, wherein each user system of the plurality of user systems comprises a processing system and a display;

generating a graph configured to graphically depict the one or more graph components through the multi-user interface by transmitting the code to each user system of the plurality of user systems, the code corresponding to the device type of each user system of the plurality of user systems;

controlling a multi-user interface to render a first drawing update of the graph on the multi-user interface based on the first set of drawing commands;

receiving a second set of voice-based drawing commands from a second user of the user connections;

performing a second speech-to-text conversion to transform the second set of voice-based drawing commands into a second set of drawing commands;

performing component lookup, positioning, and labelling to form a second message based on keyword processing;

updating the graph based on parsing the second message and accessing the code library that maps contents of the second message to the one or more graph components with connection rules to render the drawing output at each user system of the plurality of user systems;

controlling the multi-user interface to render a second drawing update of the graph on the multi-user interface based on the second set of drawing commands; and

saving, to a storage system, the drawing output of the multi-user interface resulting from the first drawing update and the second drawing update.

20 . The computer program product of claim 19 , further comprising computer program instructions that when executed by the computer cause the computer to implement:

tracking a change history of the first user based on a first sequence comprising one or more sets of drawing commands received on a first audio channel associated with the first user;

tracking a change history of the second user based on a second sequence comprising one or more sets of drawing commands received on a second audio channel associated with the second user; and

providing an undo feature to remove one or more drawing updates based on the change history.

21 . The computer program product of claim 19 , further comprising computer program instructions that when executed by the computer cause the computer to implement:

parsing the first set of drawing commands and the second set of drawing commands to determine one or more actions to be performed;

mapping the first set of drawing commands and the second set of drawing commands to a visual component palette of shapes, connectors, and attributes;

selecting one or more of the shapes, connectors, and attributes to add, modify, or delete based on the mapping; and

performing one or more of adding, modifying, or deleting based on the one or more actions determined.

22 . The computer program product of claim 19 , further comprising computer program instructions that when executed by the computer cause the computer to implement:

broadcasting one or more changes made through the multi-user interface to the plurality of user systems;

assigning a session identifier to a shared session through the multi-user interface for the user connections; and

controlling access to the shared session through a user authentication based on the session identifier, wherein the first user and the second user are identified based on a sign-in interface that requires user credential entry prior to accepting input from the first user and the second user.

23 . The computer program product of claim 19 , further comprising computer program instructions that when executed by the computer cause the computer to implement:

performing natural language processing to identify one or more keyword phrases; and

performing keyword processing to extract one or more keywords from the one or more keyword phrases.

24 . The computer program product of claim 19 , further comprising computer program instructions that when executed by the computer cause the computer to implement:

identifying a template associated with the first set of commands;

determining one or more components to include in the rendering based on the template and contents of the first set of commands; and

customizing at least one of the one or more components based on a change request to make a modification.

25 . The computer program product of claim 19 , further comprising computer program instructions that when executed by the computer cause the computer to implement:

accessing a drawing file defined in a text-based object notation file format;

rendering a drawing on the multi-user interface in a graphical display format based on parsing and interpreting contents of the drawing file in the text-based object notation file format; and

updating the drawing file in the text-based object notation file format based on the first drawing update and the second drawing update, wherein saving the drawing output stores the drawing file in the text-based object notation file format.