ffmpeg commands for image sequences
Scenario: I have a canvas animation, which I'd like to convert to a gif or video file. While Canvas does have a native captureStream()
method which enables you to convert directly to video (more about recording Canvas animations with captureStream()
here) I found that the quality was pretty limited if the animation required a bunch of calculations and things happening in the background - once I started recording the stream, the page slowed down a whole lot, and so did the video. On my slow laptop, at least. So, instead I wrote some code that saved one frame of the drawing at a time as an image, then let me download a whole series of images all at once.
Once I had this series of images, I used ffmpeg to pack them all together into a gif or a video. I have the sense that ffmpeg is quite a powerful tool, but it's a really complex one. Most of my use of it has been limited to trial & error with answers I dig up on stackoverflow. So this was a bit of a challenge but I managed to figure some things out!
images to video
Base ffmpeg command to convert a series of numbered images to video:
ffmpeg -r 60 -f image2 -i image-%d.png -crf 23 -r 30 -pix_fmt yuv420p video.mp4
What the options mean:
-r
: frame rate (fps). Note there are two of these. The first one refers to the input fps and the second to the output.-f
: format... this is probably optional since ffmpeg usually can figure out what your file formats are-i
: input.image-%d.png
means just look for a series of images with ascending digits (image-1.png
,image-2.png
, etc). There are other ways to write this depending on your filenames. For example, if you have a list of images padded with zeroes to 4 digits (img0001.png
,img0002.png
...) useimg%04d.png
.-crf
quality. 0 is lossless, max is 51. I believe the default is 23 and most examples I found online kept the number somewhere around there.-pix_fmt
pixel format. I found that without adding this option andyuv420p
the video was created okay and worked online, but quicktime couldn't open it.- last argument is the output:
video.mp4
images to gif
Base ffmpeg command to convert a series of numbered images to gif:
ffmpeg -f image2 -r 60 -i image-%d.png output.gif
Many of these options are similar to those above.
using filters
The command above on its own creates a file that's completely unoptimized and HUGE. So you can add a bunch of other adjustments through a special filter command.
ffmpeg -f image2 -r 60 -i image-%d.png -filter_complex "FILTER_HERE" output.gif
This was by far the most confusing part for me, so I'm going to break it into pieces.
ffmpeg filter syntax
The filter is separated with commas and semicolons. Commas separate filters, semicolons separate chains of filters. Between semicolons you can also specify the names of inputs & outputs to the chain. If don't specify a name, it's assumed that the output is just coming from the preceding item.
set scale & frame rate
fps=10,scale=320:-1:flags=lanczos,
fps=10
sets output frame rate to 10fpsscale=320:-1
scales images down to 320px wide. -1 keeps it at the same aspect ratioflags=lanczos
scales using lanczos scaling algorithm. There are a bunch of other scaling algortihms you can experiment with if you want.
add a boomerang effect
split[main][back];[back]reverse[r];[main][r]concat=n=2:v=1:a=0,
For this particular project I wanted the animation to play forward and then backward. If you don't want this effect, you can leave this part out.
split
beginning copies the input into 2 segments, one namedmain
and one namedback
reverse
takes theback
segment as input, reverses it, and outputs it as another item labeledr
.concat
concatenatesmain
(the normal, forward sequence) andr
(the reversed one)n=2
= there are 2 input segmentsv=1
= one video stream per segmenta=0
= 0 audio streams per segment
use a custom palette
split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse
GIF is limited to 256 colors. By default ffmpeg will just use a generic palette that attempts to work with a variety of content. In my case, I was working with black & white images, so a palette attempting to cover the entire space would be mostly wasted, and I'd be missing lots of nuance in the shades.
This filter step creates a palette specifically tailored to your content.
split
again copies your input into 2 segments, this time nameds0
ands1
palettegen
creates a palette from thes0
segment- we designate the created palette as
p
paletteuse
tells ffmpeg that when building our output gif froms1
(the sequence with all our previous filters performed on it), use palettep
instead of the default
If you want, you can separate these steps out and generate a palette from your images, save the palette as a png, and then use that saved png as a second input when you create the gif. That looks something like:
filters=fps=10,scale=320:-1:flags=lanczos # or whatever other filters you're using
ffmpeg -f image2 -i image-%d.png -vf "$filters,palettegen" palette.png # first command to create the palette
ffmpeg -i image-%d.png -i palette.png -filter_complex "$filters[x];[x][1:v] paletteuse" output.gif
But the first way, you get it done all at once without having to deal with an extra palette file.
There are a bunch of options and details here if you want to go into them; this blog post discusses palettes in more detail.
full gif command
Using all the above filters.
ffmpeg -f image2 -r 60 -i image-%d.png -filter_complex "fps=10,scale=320:-1:flags=lanczos,split[main][back];[back]reverse[r];[main][r]concat=n=2:v=1:a=0,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif
resources
- pkh.me: high quality gif with ffmpeg
- ffmpeg wiki: filtering guide
- code project: how to use ffmpeg filters to jazz up your audio and video (explains some of the filter syntax details)
- hammad mazhar: using ffmpeg to convert a set of images into video
- a bunch of stackexchange type posts: