I’ve recently started to think about how to generate videos using code. Here are some links I found.

MoviePy

Python based library to script videos. Works nicely and cleverly wraps the imagemagick and ffmpeg binaries. However, your quickly down to manipulating image bytes manually once you want to do anything beyond what they ship – or you find some other library to generate the image data you want. In the end, this is a helper layer on top to generate frames being sent to ffmpeg.

Scripting Premiere or After Effects

This has a lot going for it. The experience working with ExtendScript is not great, but it is not awful either – you can now even right and debug in VSCode. Obviously AfterEffects lets you do a lot of powerful stuff. But creating a longer sequence can take quite a while since you are just executing UI actions in sequence. The whole thing binds you to an UI-based workflow. nexrender seems to be a pretty cool tool on top of AE to fix this.

Other interesting links are ae-to-json and aepx.js.

Using HTML/JS

This document discussed a number of options.. However, there are essentially two approaches:

captureStream()

When animating on a Canvas, the stream can be captured. This should hopefully give a pretty good framerate. You are essentially going through the normal browser runtime / render loop with this. See:

• Code snippet: Using MediaRecorder to export a canvas animation as video
• Code snippet showing a solution to export an SVG animation in this way

BTW, the guy who posted these snippets has the most amazing personal projects.

Frame-by-Frame

This requires your code to be written in such a way that you can render each frame individually. It is then possible to capture then individually. This approach is used by:

• ccapture.js – hacks into requestAnimationFrame, Date() and so on.
• html5-animation-video-renderer – calls a function for every frame in Chrome headless.
• An in-browser example based on GSAP
• Blog post detailing a PhantomJS-based process (light on details)

Any animation library which lets you go to a particular target frame is suitable for this, including GSAP, Scene.js and anime.

Other things I found which are cool: A timeline widget, another one.

Other Software:

• VideoPuppet – create videos via Markdown Scripts.
• Editly – command-line video editing.
• Slaptrash – create videos via special  HTML instructions.
• Kinetophone
• lang.video – a language for making movies (based on Racket)
• manim – animation engine for math videos
• vapoursynth – video processing framework
• Komposition – Not for scripting, but a UI workflow for screencasters.