ffmpeg commands for image sequences
Scenario: I have a canvas animation, which I'd like to convert to a gif or video file. While Canvas does have a native
captureStream() method which enables you to convert directly to video (more about recording Canvas animations with
captureStream() here) I found that the quality was pretty limited if the animation required a bunch of calculations and things happening in the background - once I started recording the stream, the page slowed down a whole lot, and so did the video. On my slow laptop, at least. So, instead I wrote some code that saved one frame of the drawing at a time as an image, then let me download a whole series of images all at once.
Once I had this series of images, I used ffmpeg to pack them all together into a gif or a video. I have the sense that ffmpeg is quite a powerful tool, but it's a really complex one. Most of my use of it has been limited to trial & error with answers I dig up on stackoverflow. So this was a bit of a challenge but I managed to figure some things out!
images to video
Base ffmpeg command to convert a series of numbered images to video:
ffmpeg -r 60 -f image2 -i image-%d.png -crf 23 -r 30 -pix_fmt yuv420p video.mp4
What the options mean:
-r: frame rate (fps). Note there are two of these. The first one refers to the input fps and the second to the output.
-f: format... this is probably optional since ffmpeg usually can figure out what your file formats are
image-%d.pngmeans just look for a series of images with ascending digits (
image-2.png, etc). There are other ways to write this depending on your filenames. For example, if you have a list of images padded with zeroes to 4 digits (
-crfquality. 0 is lossless, max is 51. I believe the default is 23 and most examples I found online kept the number somewhere around there.
-pix_fmtpixel format. I found that without adding this option and
yuv420pthe video was created okay and worked online, but quicktime couldn't open it.
- last argument is the output:
images to gif
Base ffmpeg command to convert a series of numbered images to gif:
ffmpeg -f image2 -r 60 -i image-%d.png output.gif
Many of these options are similar to those above.
The command above on its own creates a file that's completely unoptimized and HUGE. So you can add a bunch of other adjustments through a special filter command.
ffmpeg -f image2 -r 60 -i image-%d.png -filter_complex "FILTER_HERE" output.gif
This was by far the most confusing part for me, so I'm going to break it into pieces.
ffmpeg filter syntax
The filter is separated with commas and semicolons. Commas separate filters, semicolons separate chains of filters. Between semicolons you can also specify the names of inputs & outputs to the chain. If don't specify a name, it's assumed that the output is just coming from the preceding item.
set scale & frame rate
fps=10sets output frame rate to 10fps
scale=320:-1scales images down to 320px wide. -1 keeps it at the same aspect ratio
flags=lanczosscales using lanczos scaling algorithm. There are a bunch of other scaling algortihms you can experiment with if you want.
add a boomerang effect
For this particular project I wanted the animation to play forward and then backward. If you don't want this effect, you can leave this part out.
splitbeginning copies the input into 2 segments, one named
mainand one named
backsegment as input, reverses it, and outputs it as another item labeled
main(the normal, forward sequence) and
r(the reversed one)
n=2= there are 2 input segments
v=1= one video stream per segment
a=0= 0 audio streams per segment
use a custom palette
GIF is limited to 256 colors. By default ffmpeg will just use a generic palette that attempts to work with a variety of content. In my case, I was working with black & white images, so a palette attempting to cover the entire space would be mostly wasted, and I'd be missing lots of nuance in the shades.
This filter step creates a palette specifically tailored to your content.
splitagain copies your input into 2 segments, this time named
palettegencreates a palette from the
- we designate the created palette as
paletteusetells ffmpeg that when building our output gif from
s1(the sequence with all our previous filters performed on it), use palette
pinstead of the default
If you want, you can separate these steps out and generate a palette from your images, save the palette as a png, and then use that saved png as a second input when you create the gif. That looks something like:
filters=fps=10,scale=320:-1:flags=lanczos # or whatever other filters you're using
ffmpeg -f image2 -i image-%d.png -vf "$filters,palettegen" palette.png # first command to create the palette
ffmpeg -i image-%d.png -i palette.png -filter_complex "$filters[x];[x][1:v] paletteuse" output.gif
But the first way, you get it done all at once without having to deal with an extra palette file.
There are a bunch of options and details here if you want to go into them; this blog post discusses palettes in more detail.
full gif command
Using all the above filters.
ffmpeg -f image2 -r 60 -i image-%d.png -filter_complex "fps=10,scale=320:-1:flags=lanczos,split[main][back];[back]reverse[r];[main][r]concat=n=2:v=1:a=0,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" output.gif
- pkh.me: high quality gif with ffmpeg
- ffmpeg wiki: filtering guide
- code project: how to use ffmpeg filters to jazz up your audio and video (explains some of the filter syntax details)
- hammad mazhar: using ffmpeg to convert a set of images into video
- a bunch of stackexchange type posts: