Tech • AI • Robotics • Game
How to edit your videos with AI for free (complete guide)
5/10TL;DR
A workflow combining Claude Code, Remotion, Whisper, and FFmpeg can automate much of short-form video editing by transcribing speech, cutting pauses, timing subtitles word by word, and generating animations from code.
KEY POINTS
Editing time reduced through automation
Short-form editing had been consuming the largest share of production time, with a 50-second clip requiring 2 to 3 hours for cuts, captions, sound effects, and simple motion graphics. A 20-minute long-form video could take one to two full days. Much of that work consisted of repetitive tasks such as trimming silences, removing failed takes, and aligning subtitles and animations frame by frame.
AI writes the instructions, not the video itself
Large language models do not directly produce polished video files or manually operate editing timelines. Their practical advantage in this workflow is code generation. By describing what should happen at specific moments, such as a counter starting at 3 seconds or a card bouncing in at 4 seconds, the system converts code into rendered video.
Remotion turns code into MP4 files
The core rendering tool is Remotion, an open-source framework that generates video from code and outputs a real MP4. Similar tools exist, including Hyper Frames, but Remotion is used here as the main engine for animation and layout. Because the visuals are defined programmatically, styles and timing can be changed quickly without rebuilding an edit by hand.
A reusable editing system replaces one-off prompts
Rather than requesting a new edit from scratch each time, the setup relies on a custom production environment built around reusable assets. That includes a library of more than 60 animations, multiple visual styles, pacing rules, and automated checks. In practice, the remaining manual step is largely reduced to recording the footage.
Four tools handle the pipeline
The workflow depends on Claude Code, Remotion, Whisper, and FFmpeg. Claude Code works locally on a computer, creates files, launches programs, and coordinates the process. Whisper, developed by OpenAI, transcribes speech and provides word-level timing, while FFmpeg handles technical processing such as cutting silence, reframing, and mixing audio. The tools themselves are presented as free to use, with the paid component being access to Claude Code.
Word-level transcription is critical
Accurate subtitle and animation timing depends on transcription that marks the exact instant each word is spoken. That timing allows captions to appear word by word and lets named visual elements trigger on the relevant spoken word. A stronger transcription model may be needed for reliable French timing, especially when aiming for fully automated output.
Security and installation are treated as a major concern
The installation process emphasizes caution, including verifying downloads, identifying where packages come from, and running security audits. Package management is handled with PNPM, and configuration files are used to limit risk from unsafe or compromised downloads. The approach reflects broader concerns about malware in online developer tools and the need to validate what is installed on a local machine.
The first output is usable but not final
An initial test can already identify voice tracks, process a recorded file, and assemble a basic short with captions and cuts. However, that first result is positioned as only a starting point. The quality gap between a functional automated edit and a polished social clip remains significant without further refinement.
Style can be improved by referencing successful short videos
To push quality higher, the system can be directed to imitate the pacing, transitions, zooms, framing, and subtitle style found in strong-performing clips on platforms such as TikTok and Instagram. The recommendation is not blind copying, but targeted inspiration from formats that already hold attention. Existing design systems and public GitHub repositories can also be imported to expand available motion styles.
Sound design still needs curation
Automated sound effect choices remain a weak point. To improve that layer, royalty-free libraries such as Mixkit can supply stronger audio assets, and the AI can be asked to specify what kind of effect should be downloaded. Over time, repeated corrections and better source assets can significantly improve the final result.
CONCLUSION
Automated editing can remove much of the repetitive labor behind short-form video production, especially subtitles, silence removal, and timed motion graphics. The main competitive edge then shifts from manual editing speed to the quality of the creative system, references, and assets feeding the workflow.
Ask a question