Natural language based computer animation
The present disclosure relates to systems, methods, and non-transitory computer-readable media that facilitate near real-time dynamic generation of graphical content based on audio, text, or gesture input. An example method includes receiving an input from an input device such as a microphone, a keyboard, or a camera. As such, the input may include text, speech, and/or a gesture. The method includes determining, based on the input, one or more command instructions. The method also includes determining, based on the one or more command instructions, a scene layout. Each of the one or more command instructions correspond to at least one element of the scene layout. The method also includes providing a rendered scene based on the determined scene layout.
1. A method comprising:
receiving an input, wherein the input comprises at least one of: text, speech, or a gesture;
determining, based on the input, one or more command instructions, wherein determining the one or more command instructions comprises:
parsing the input into a plurality of line segments;
processing the plurality of line segments with a natural language recognition algorithm, wherein the processing comprises comparing each of the line segments with a natural language command database; and
determining at least one recognized line segment based on the comparison, wherein the one or more command instructions are based on the at least one recognized line segment;
categorizing each of the one or more command instructions into at least one of: an object command instruction, and an environmental command instruction;
storing each categorized command instruction into at least one corresponding database, wherein the corresponding database comprises at least one of: an object database and an environmental database;
determining, based on the one or more command instructions, a scene layout, wherein determining the scene layout comprises:
determining, for each object command instruction, at least one object and at least one corresponding object location;
determining, for each environmental command instruction, at least one environmental element and at least one corresponding environmental element location;
retrieving determined objects and determined environmental elements from a vector object database comprising compressed vector objects, wherein retrieving comprises decompressing a respective vector object based on a corresponding object location or environmental element location; and
providing a rendered scene based on the determined scene layout.
2. The method of claim 1 , wherein each of the one or more command instructions correspond to at least one element of the scene layout.
3. The method of claim 1 , wherein the input comprises text, wherein receiving the input comprises receiving the text via a text entry in a graphical user interface.
4. The method of claim 1 , wherein the input comprises speech, wherein receiving the input comprises receiving the speech via a microphone.
5. The method of claim 1 , wherein the input comprises a gesture, wherein receiving the input comprises receiving the gesture via a camera.
6. The method of claim 1 , wherein processing the plurality of line segments with a natural language recognition algorithm comprises the plurality of line segments being processed sequentially with the natural language recognition algorithm.
7. The method of claim 1 , wherein processing the plurality of line segments with a natural language recognition algorithm comprises at least a portion of the plurality of line segments being processed simultaneously with the natural language recognition algorithm.
8. The method of claim 1 , further comprising generating a pseudo-random seed with a random number generator, wherein determining at least one of: the one or more command instructions or the scene layout is further based on selecting a respective command instruction or respective scene layout associated with the pseudo-random seed.
9. The method of claim 1 , wherein providing a rendered scene comprises rendering, using a dedicated graphics processor, the rendered scene based on the determined scene layout, wherein the dedicated graphics processor comprises a graphics processing unit (GPU).
10. The method of claim 1 , further comprising providing an output in an HTML-compatible format.
11. The method of claim 1 , further comprising providing an output, wherein the output comprises a format that is compatible with at least one of: a virtual reality display or an augmented reality display.
12. The method of claim 1 , wherein the method is performed in near real-time, wherein near real-time comprises at least one of: determining the one or more command instructions at least once every 50 milliseconds or determining the scene layout at least once every 50 milliseconds.
13. The method of claim 1 , wherein one or more steps of the method are carried out by an intelligent agent or artificial intelligence construct that utilizes machine learning.
14. A system comprising:
an input device;
a render processing unit;
a display; and
a controller comprising at least one memory and at least one processor, wherein the controller executes instructions so as to carry out operations, the operations comprising:
receiving, via the input device, input information indicative of at least one of: text, speech, or a gesture;
determining, based on the received input information, one or more command instructions, wherein determining the one or more command instructions comprises:
parsing the received input information into a plurality of line segments;
processing the plurality of line segments with a natural language recognition algorithm, wherein the processing comprises comparing each of the line segments with a natural language command database;
determining at least one recognized line segment based on the comparison, wherein the at least one command instruction is based on the at least one recognized line segment;
categorizing each of the one or more command instructions into at least one of:
an object command instruction, and an environmental command instruction;
storing each categorized command instruction into at least one corresponding database, wherein the corresponding database comprises at least one of: an object database and an environmental database;
determining, based on the one or more command instructions, a scene layout, wherein determining the scene layout comprises:
determining, for each object command instruction, at least one object and at least one corresponding object location;
determining, for each environmental command instruction, at least one environmental element and at least one corresponding environmental element location;
retrieving determined objects and determined environmental elements from a vector object database comprising compressed vector objects, wherein retrieving comprises decompressing a respective vector object based on a corresponding object location or environmental object location; and
rendering, using the render processing unit, a rendered scene based on the determined scene layout; and
displaying, via the display, the rendered scene.
15. The system of claim 14 , wherein the display comprises at least one of: a virtual reality display or an augmented reality display.
16. A non-transitory computer readable medium having stored therein instructions that, when executed by a computing device, cause the computing device to perform operations comprising:
receiving an input, wherein the input comprises at least one of: text, speech, or a gesture;
determining, based on the input, at least one command instruction, wherein determining the at least one command instruction comprises:
parsing the input into a plurality of line segments;
processing the plurality of line segments with a natural language recognition algorithm, wherein the processing comprises comparing each of the line segments with a natural language command database;
determining at least one recognized line segment based on the comparison, wherein the at least one command instruction is based on the at least one recognized line segment;
categorizing each of the one or more command instructions into at least one of: an object command instruction, and an environmental command instruction;
storing each categorized command instruction into at least one corresponding database, wherein the corresponding database comprises at least one of: an object database and an environmental database;
determining, based on the at least one command instruction, a scene layout, wherein determining the scene layout comprises:
determining, for each object command instruction, at least one object and at least one corresponding object location;
determining, for each environmental command instruction, at least one environmental element and at least one corresponding environmental element location;
retrieving determined objects and determined environmental elements from a vector object database comprising compressed vector objects, wherein retrieving comprises decompressing a respective vector object based on a corresponding object location or environmental object location;
providing a rendered scene based on the determined scene layout, wherein each command instruction corresponds to at least one element of the scene layout;
categorizing each command instruction into at least one of: an object command instruction, a language command instruction, or an environmental command instruction;
storing each categorized command instruction into at least one corresponding database, wherein the corresponding database comprises at least one of: an object database, a linguistic database, or an environmental database; and
providing an output in an HTML-compatible format.