{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Lab 02: ffmpeg" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Links:\n", "\n", "- [How to use FFmpeg to convert images to video ](https://shotstack.io/learn/use-ffmpeg-to-convert-images-to-video/)\n", "- [Using ffmpeg to convert a set of images into a video](https://hamelot.io/visualization/using-ffmpeg-to-convert-a-set-of-images-into-a-video/)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Make Plots" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We'll store our plots in folder `build`. If it does not exist, create it." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# if folder build does not exist, create it\n", "import os\n", "if not os.path.exists('build'):\n", " os.makedirs('build')" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "We can make animations by stacking images together. First let's make some plots.\n", "\n", "As an example, we'll make a plot of a 1D wave\n", "\n", "$$y = \\sin(x + t)$$\n", "\n", "over a range of $-\\pi < x < \\pi$ and $0 < t < 2\\pi$." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# make plots of y = sin(x + t) for -pi < x < pi and 0 < t < 2pi, save as png\n", "x = np.linspace(-np.pi, np.pi, 100)\n", "for i, t in enumerate(np.linspace(0, 2*np.pi, 100)):\n", " y = np.sin(x + t)\n", " # draw the plot on canvas\n", " plt.plot(x, y)\n", " # save the plot in the format demo_XX.png, e.g. demo_00.png, demo_01.png, ...\n", " plt.savefig('build/demo_{:02d}.png'.format(i))\n", " # clear the canvas\n", " plt.clf()\n", " " ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Create a Video Using ffmpeg" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "A minimal example of using ffmpeg to create a video is\n", "\n", "```bash\n", "ffmpeg -i build/demo_%02d.png -r 25 build/demo.mp4\n", "```\n", "\n", "where `-i` specifies the input file names, `build/%02d.png` represents all files in the format `demo_00.png`, `demo_01.png`, `...` in the `build` folder, `-r 25` sets the frame rate to 25fps, and `build/demo.mp4` sets the output file name." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "ffmpeg version 5.0.1 Copyright (c) 2000-2022 the FFmpeg developers\n", " built with Apple clang version 13.1.6 (clang-1316.0.21.2)\n", " configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/5.0.1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-neon\n", " libavutil 57. 17.100 / 57. 17.100\n", " libavcodec 59. 18.100 / 59. 18.100\n", " libavformat 59. 16.100 / 59. 16.100\n", " libavdevice 59. 4.100 / 59. 4.100\n", " libavfilter 8. 24.100 / 8. 24.100\n", " libswscale 6. 4.100 / 6. 4.100\n", " libswresample 4. 3.100 / 4. 3.100\n", " libpostproc 56. 3.100 / 56. 3.100\n", "Input #0, image2, from 'build/demo_%02d.png':\n", " Duration: 00:00:04.00, start: 0.000000, bitrate: N/A\n", " Stream #0:0: Video: png, rgba(pc), 432x288 [SAR 2835:2835 DAR 3:2], 25 fps, 25 tbr, 25 tbn\n", "Stream mapping:\n", " Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))\n", "Press [q] to stop, [?] for help\n", "[libx264 @ 0x155639bc0] using SAR=1/1\n", "[libx264 @ 0x155639bc0] using cpu capabilities: ARMv8 NEON\n", "[libx264 @ 0x155639bc0] profile High 4:4:4 Predictive, level 2.1, 4:4:4, 8-bit\n", "[libx264 @ 0x155639bc0] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00\n", "Output #0, mp4, to 'build/demo.mp4':\n", " Metadata:\n", " encoder : Lavf59.16.100\n", " Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv444p(tv, progressive), 432x288 [SAR 1:1 DAR 3:2], q=2-31, 25 fps, 12800 tbn\n", " Metadata:\n", " encoder : Lavc59.18.100 libx264\n", " Side data:\n", " cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A\n", "frame= 100 fps=0.0 q=-1.0 Lsize= 25kB time=00:00:03.88 bitrate= 53.5kbits/s speed=41.2x \n", "video:23kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 8.260633%\n", "[libx264 @ 0x155639bc0] frame I:1 Avg QP:18.66 size: 4544\n", "[libx264 @ 0x155639bc0] frame P:39 Avg QP:16.97 size: 291\n", "[libx264 @ 0x155639bc0] frame B:60 Avg QP:15.81 size: 123\n", "[libx264 @ 0x155639bc0] consecutive B-frames: 6.0% 30.0% 36.0% 28.0%\n", "[libx264 @ 0x155639bc0] mb I I16..4: 13.0% 65.6% 21.4%\n", "[libx264 @ 0x155639bc0] mb P I16..4: 1.4% 1.3% 0.1% P16..4: 8.1% 2.7% 0.5% 0.0% 0.0% skip:85.9%\n", "[libx264 @ 0x155639bc0] mb B I16..4: 0.2% 0.1% 0.0% B16..8: 11.0% 1.9% 0.0% direct: 0.0% skip:86.8% L0:53.9% L1:40.6% BI: 5.5%\n", "[libx264 @ 0x155639bc0] 8x8 transform intra:53.4% inter:19.0%\n", "[libx264 @ 0x155639bc0] coded y,u,v intra: 5.6% 2.1% 2.2% inter: 0.2% 0.0% 0.0%\n", "[libx264 @ 0x155639bc0] i16 v,h,dc,p: 39% 52% 9% 0%\n", "[libx264 @ 0x155639bc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 10% 61% 0% 0% 0% 0% 0% 0%\n", "[libx264 @ 0x155639bc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 27% 30% 3% 3% 4% 1% 3% 1%\n", "[libx264 @ 0x155639bc0] Weighted P-Frames: Y:0.0% UV:0.0%\n", "[libx264 @ 0x155639bc0] ref P L0: 61.4% 1.0% 24.5% 13.1%\n", "[libx264 @ 0x155639bc0] ref B L0: 78.4% 19.3% 2.3%\n", "[libx264 @ 0x155639bc0] ref B L1: 96.3% 3.7%\n", "[libx264 @ 0x155639bc0] kb/s:46.54\n" ] }, { "data": { "text/plain": [ "0" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# create a video from the png files\n", "os.system('ffmpeg -r 25 -i build/demo_%02d.png build/demo.mp4 -y')" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You can also run commands in a notebook using `!`:\n", "\n", "```python\n", "!ffmpeg -i build/demo_%02d.png -r 25 build/demo.mp4\n", "```\n", "\n", "Either way works. You can also directly run the command in the terminal." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ffmpeg version 5.0.1 Copyright (c) 2000-2022 the FFmpeg developers\n", " built with Apple clang version 13.1.6 (clang-1316.0.21.2)\n", " configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/5.0.1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-neon\n", " libavutil 57. 17.100 / 57. 17.100\n", " libavcodec 59. 18.100 / 59. 18.100\n", " libavformat 59. 16.100 / 59. 16.100\n", " libavdevice 59. 4.100 / 59. 4.100\n", " libavfilter 8. 24.100 / 8. 24.100\n", " libswscale 6. 4.100 / 6. 4.100\n", " libswresample 4. 3.100 / 4. 3.100\n", " libpostproc 56. 3.100 / 56. 3.100\n", "Input #0, image2, from 'build/demo_%02d.png':\n", " Duration: 00:00:04.00, start: 0.000000, bitrate: N/A\n", " Stream #0:0: Video: png, rgba(pc), 432x288 [SAR 2835:2835 DAR 3:2], 25 fps, 25 tbr, 25 tbn\n", "Stream mapping:\n", " Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))\n", "Press [q] to stop, [?] for help\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0musing SAR=1/1\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0musing cpu capabilities: ARMv8 NEON\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mprofile High 4:4:4 Predictive, level 2.1, 4:4:4, 8-bit\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0m264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00\n", "Output #0, mp4, to 'build/demo.mp4':\n", " Metadata:\n", " encoder : Lavf59.16.100\n", " Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv444p(tv, progressive), 432x288 [SAR 1:1 DAR 3:2], q=2-31, 25 fps, 12800 tbn\n", " Metadata:\n", " encoder : Lavc59.18.100 libx264\n", " Side data:\n", " cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A\n", "frame= 100 fps=0.0 q=-1.0 Lsize= 25kB time=00:00:03.88 bitrate= 53.5kbits/s speed=38.7x \n", "video:23kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 8.260633%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mframe I:1 Avg QP:18.66 size: 4544\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mframe P:39 Avg QP:16.97 size: 291\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mframe B:60 Avg QP:15.81 size: 123\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mconsecutive B-frames: 6.0% 30.0% 36.0% 28.0%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mmb I I16..4: 13.0% 65.6% 21.4%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mmb P I16..4: 1.4% 1.3% 0.1% P16..4: 8.1% 2.7% 0.5% 0.0% 0.0% skip:85.9%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mmb B I16..4: 0.2% 0.1% 0.0% B16..8: 11.0% 1.9% 0.0% direct: 0.0% skip:86.8% L0:53.9% L1:40.6% BI: 5.5%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0m8x8 transform intra:53.4% inter:19.0%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mcoded y,u,v intra: 5.6% 2.1% 2.2% inter: 0.2% 0.0% 0.0%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mi16 v,h,dc,p: 39% 52% 9% 0%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mi8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 10% 61% 0% 0% 0% 0% 0% 0%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mi4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 27% 30% 3% 3% 4% 1% 3% 1%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mWeighted P-Frames: Y:0.0% UV:0.0%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mref P L0: 61.4% 1.0% 24.5% 13.1%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mref B L0: 78.4% 19.3% 2.3%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mref B L1: 96.3% 3.7%\n", "\u001b[1;36m[libx264 @ 0x126a0bf20] \u001b[0mkb/s:46.54\n" ] } ], "source": [ "!ffmpeg -i build/demo_%02d.png -r 25 build/demo.mp4 -y" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Watch the video" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# watch the video\n", "import IPython.display as ipd\n", "ipd.Video('build/demo.mp4')\n" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "Congratulations! You've made your first video using ffmpeg." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "playground", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.4" }, "orig_nbformat": 4 }, "nbformat": 4, "nbformat_minor": 2 }