Iterative generative AI-assisted video editing system and method
The present disclosure provides a video editing system that receives a recorded video object configured to cause playback of a video recording on a client device. A transcript file object is generated based on the recorded video object. The system applies iterative multi-loop transcript refinement operations to the transcript file object, including inputting the transcript file object and text disfluencies deletion instructions to a large language model (LLM) to generate a first refined transcript file object, and inputting the first refined transcript file object and advanced text deletion instructions to the LLM to generate a second or subsequent refined transcript file object where multiple additional iterative multi-loop transcript refinement operations are deployed to further refine the transcript while mitigating unwanted transcript additions. The system generates video edit instructions based on the second (or subsequent) refined transcript object and generates an updated recorded video object based on the video edit instructions.
1 . A video editing system comprising at least one processor and a memory storing instructions that are operable, when executed by the processor, to cause the video editing system to:
receive a recorded video object that is configured to cause playback, on a client device, of a video recording of at least one speaker;
generate a transcript file object of the video recording based on the recorded video object;
apply iterative multi-loop transcript refinement operations to the transcript file object, wherein the iterative multi-loop transcript refinement operations comprise:
inputting the transcript file object and text disfluencies deletion instructions to a large language model to generate a first refined transcript file object, and
inputting the first refined transcript file object and advanced text deletion instructions to the large language model to generate a second refined transcript file object, wherein the advanced text deletion instructions comprise at least one of: prefatory text deletion instructions or repetitive text deletion instructions;
generate video edit instructions based on the second refined transcript file object; and
generate an updated recorded video object based on the video edit instructions.
2 . The video editing system of claim 1 , wherein the video edit instructions comprise trim ranges defined by the large language model based on the text disfluencies deletion instructions, the prefatory text deletion instructions, or the repetitive text deletion instructions.
3 . The video editing system of claim 1 , wherein the iterative multi-loop transcript refinement operations comprise text confinement instructions.
4 . The video editing system of claim 3 , wherein the text confinement instructions comprise a text confinement threshold.
5 . The video editing system of claim 1 , wherein the video editing system is further configured to:
generate a user interface comprising a refinement adjustment interface component;
receive user input via the refinement adjustment interface component; and
adjust parameters of the iterative multi-loop transcript refinement operations based on the received user input.
6 . The video editing system of claim 1 , wherein the video editing system is further configured to:
chunk the transcript file object into multiple segments;
apply the iterative multi-loop transcript refinement operations to each segment separately; and
combine the refined segments to generate the second refined transcript file object.
7 . The video editing system of claim 1 , wherein generating the updated recorded video object comprises:
identifying video segments corresponding to portions of the second refined transcript file object;
trimming the identified video segments based on the video edit instructions; and
stitching the trimmed video segments together to create the updated recorded video object.
8 . A computer-implemented method comprising:
receiving, by a video editing system, a recorded video object that is configured to cause playback, on a client device, of a video recording of at least one speaker;
generating, by the video editing system, a transcript file object of the video recording based on the recorded video object;
applying, by the video editing system, iterative multi-loop transcript refinement operations to the transcript file object, wherein the iterative multi-loop transcript refinement operations comprise:
inputting the transcript file object and text disfluencies deletion instructions to a large language model to generate a first refined transcript file object, and
inputting the first refined transcript file object and prefatory text deletion instructions to the large language model to generate a second refined transcript file object;
generating, by the video editing system, video edit instructions based on the second refined transcript file object; and
generating, by the video editing system, an updated recorded video object based on the video edit instructions.
9 . The computer-implemented method of claim 8 , wherein the iterative multi-loop transcript refinement operations further comprise:
inputting the second refined transcript file object and repetitive text deletion instructions to the large language model to generate a third refined transcript file object; and
wherein the generating, by the video editing system, video edit instructions is based on the third refined transcript file object.
10 . The computer-implemented method of claim 8 , wherein the video edit instructions comprise trim ranges defined by the large language model based on the text disfluencies deletion instructions or the prefatory text deletion instructions.
11 . The computer-implemented method of claim 8 , wherein the iterative multi-loop transcript refinement operations comprise text confinement instructions.
12 . The computer-implemented method of claim 11 , wherein the text confinement instructions comprise a text confinement threshold.
13 . The computer-implemented method of claim 8 , further comprising:
generating a user interface comprising a refinement adjustment interface component;
receiving user input via the refinement adjustment interface component; and
adjusting parameters of the iterative multi-loop transcript refinement operations based on the received user input.
14 . The computer-implemented method of claim 8 , further comprising:
chunking the transcript file object into multiple segments;
applying the iterative multi-loop transcript refinement operations to each segment separately; and
combining the refined segments to generate the second refined transcript file object.
15 . The computer-implemented method of claim 8 , wherein generating the updated recorded video object comprises:
identifying video segments corresponding to portions of the second refined transcript file object;
trimming the identified video segments based on the video edit instructions; and
stitching the trimmed video segments together to create the updated recorded video object.
16 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a video editing system, cause the video editing system to perform operations comprising:
receiving a recorded video object that is configured to cause playback, on a client device, of a video recording of at least one speaker;
generating a transcript file object of the video recording based on the recorded video object;
applying iterative multi-loop transcript refinement operations to the transcript file object, wherein the iterative multi-loop transcript refinement operations comprise:
inputting the transcript file object and text disfluencies deletion instructions to a large language model to generate a first refined transcript file object, and
inputting the first refined transcript file object and advanced text deletion instructions to the large language model to generate a second refined transcript file object, wherein the advanced text deletion instructions comprise at least one of: prefatory text deletion instructions or repetitive text deletion instructions;
generating video edit instructions based on the second refined transcript file object; and
generating an updated recorded video object based on the video edit instructions.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein the video edit instructions comprise trim ranges defined by the large language model based on the text disfluencies deletion instructions, the prefatory text deletion instructions, or the repetitive text deletion instructions.
18 . The non-transitory computer-readable storage medium of claim 16 , wherein the iterative multi-loop transcript refinement operations comprise text confinement instructions.
19 . The non-transitory computer-readable storage medium of claim 18 , wherein the text confinement instructions comprise a text confinement threshold.
20 . The non-transitory computer-readable storage medium of claim 16 , wherein the operations further comprise:
generating a user interface comprising a refinement adjustment interface component;
receiving user input via the refinement adjustment interface component; and
adjusting parameters of the iterative multi-loop transcript refinement operations based on the received user input.