Scripting Video

I’ve recently started to think about how to generate videos using code. Here are some links I found.

MoviePy

Python based library to script videos. Works nicely and cleverly wraps the imagemagick and ffmpeg binaries. However, your quickly down to manipulating image bytes manually once you want to do anything beyond what they ship – or you find some other library to generate the image data you want. In the end, this is a helper layer on top to generate frames being sent to ffmpeg.

Scripting Premiere or After Effects

This has a lot going for it. The experience working with ExtendScript is not great, but it is not awful either – you can now even right and debug in VSCode. Obviously AfterEffects lets you do a lot of powerful stuff. But creating a longer sequence can take quite a while since you are just executing UI actions in sequence. The whole thing binds you to an UI-based workflow. nexrender seems to be a pretty cool tool on top of AE to fix this.

Other interesting links are ae-to-json and aepx.js.

Using HTML/JS

This document discussed a number of options.. However, there are essentially two approaches:

captureStream()

When animating on a Canvas, the stream can be captured. This should hopefully give a pretty good framerate. You are essentially going through the normal browser runtime / render loop with this. See:

BTW, the guy who posted these snippets has the most amazing personal projects.

Frame-by-Frame

This requires your code to be written in such a way that you can render each frame individually. It is then possible to capture then individually. This approach is used by:

Any animation library which lets you go to a particular target frame is suitable for this, including GSAP, Scene.js and anime.

Other things I found which are cool: A timeline widget, another one.

Other Software:

  • VideoPuppet – create videos via Markdown Scripts.
  • Editly – command-line video editing.
  • Slaptrash – create videos via special  HTML instructions.
  • Kinetophone
  • lang.video – a language for making movies (based on Racket)
  • manim – animation engine for math videos
  • vapoursynth – video processing framework
  • Komposition – Not for scripting, but a UI workflow for screencasters.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s