Videos: Upgrade FFmpeg from 6.1.1 to 7.x for faster transcoding

[lastzero]

As a user with videos that need to be transcoded or videos in the new EVC and VVC formats, I would like to see FFmpeg shipped with and used by PhotoPrism as it supports multi-threaded transcoding and the latest codecs.

There are static builds provided directly by the FFmpeg project, but it's unclear if we can use them since (a) they only support a subset of codecs which are freely available (so our file format support could shrink rather than grow) and (b) hardware video transcoding must continue to work for (at least) all currently supported GPUs and CPUs, which might require dynamic linking:

• https://johnvansickle.com/ffmpeg/
• https://johnvansickle.com/ffmpeg/git-readme.txt

Video Formats currently supported by PhotoPrism:

• 3G2 Mobile Multimedia Container (CDMA2000) Video .3g2
• 3GP Mobile Multimedia Container (3G) Video .3gp
• ASF Advanced Systems Format Video .asf
• AV1 AOMedia Video 1 Video .av1
• AVC Advanced Video Coding (H.264, MPEG-4 Part 10) Video .avc
• AVI Microsoft Audio Video Interleave Video .avi
• DV DV Video Video .dv
• FLV Adobe Flash Video .f4v, .flv
• HEVC High Efficiency Video Coding (H.265) Video .hevc
• M2TS Blu-ray MPEG-2 Transport Stream Video .m2ts
• M4V Apple iTunes Multimedia Container Video .m4v
• MJPG Motion JPEG Video .mjpeg, .mjpg
• MKV Matroska Multimedia Container Video .mkv
• MOV Apple QuickTime Video .mov, .qt
• MP2 MPEG 2 (H.262) Video .mp2, .mpv
• MP4 Multimedia Container (MPEG-4 Part 14) Video .mp, .mp4
• MPG Moving Picture Experts Group (MPEG) Video .mpeg, .mpg
• MTS Advanced Video Coding High Definition (AVCHD) Video .mts
• OGV Ogg Media (OGG) Video .ogg, .ogv, .ogx
• VVC Versatile Video Coding (H.266) Video .vvc
• WEBM Google WebM Video .webm
• WMV Windows Media Video .wmv

Sample Files for Testing:

• https://dl.photoprism.app/samples/Formats/Video/

Related Documentation:

• https://docs.photoprism.app/getting-started/advanced/transcoding/
• https://docs.photoprism.app/user-guide/organize/video/#transcoding

Acceptance Criteria:

• Our official Docker image MUST include FFmpeg 7.x for all currently supported architectures: AMD64, ARM64 and ARMv7
• If there is an official Linux distribution backport (e.g. from Ubuntu), it SHOULD be used; otherwise, we can ideally install pre-compiled FFmpeg 7.x binaries from a trusted, external source or, if there is no other option, compile our own binaries from source code
• For all other architectures, there SHOULD be a way to either compile binaries or download and install pre-compiled FFmpeg 7.x binaries
• All currently supported video formats (free and non-free) MUST continue to work, in addition to the newly supported EVC and VVC video file formats
• Hardware video transcoding MUST continue to work for (at least) all currently supported GPUs and CPUs

Any help figuring out the best way to upgrade to FFmpeg 7.x and then testing it would be much appreciated!

[lastzero]

These changes install FFmpeg 7 as described at https://ubuntuhandbook.org/index.php/2024/04/ffmpeg-7-0-ppa-ubuntu/ for testing purposes. Unfortunately, this appears to be a privately maintained PPA (so no guaranteed updates) and there is no ARMv7 build.

[lastzero]

It also seems like the h264_nvenc codec, required to support NVIDIA hardware transcoding, is missing in this package...

[lastzero]

AVC Decoder / Encoder Support

Running the ffmpeg -codecs command shows the supported H.264 (MPEG-4 AVC) codecs for the FFmpeg version currently installed:

FFmpeg 6.1.1 (standard Ubuntu 24.04 LTS distribution package)

H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
(decoders: h264 h264_v4l2m2m h264_qsv h264_cuvid)
(encoders: libx264 libx264rgb h264_nvenc h264_qsv h264_v4l2m2m h264_vaapi)

FFmpeg 7.0 (installed from ppa:ubuntuhandbook1/ffmpeg7)

H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
(decoders: h264 h264_v4l2m2m h264_qsv libopenh264)
(encoders: libx264 libx264rgb libopenh264 h264_amf h264_omx h264_qsv h264_v4l2m2m h264_vaapi)

Conclusion

While the new version (originally compiled by the Debian maintainers) additionally supports h264_amf for AMD CPUs and h264_omx for Rapberry Pi's, it does not support NVIDIA GPUs (which are popular among our users). This is probably because NVIDIA support needs to be added from an external repository when building FFmpeg and so it is not available by default.

Upgrade Strategy

We will therefore wait for an official Ubuntu distribution package for FFmpeg 7, which should include NVIDIA GPU support. Everyone else can use the new install-ffmpeg.sh script included with our Docker image (or PHOTOPRISM_INIT: "ffmpeg" if you are using Docker Compose).

[lastzero]

The install-ffmpeg.sh script has been updated to install the static build available at johnvansickle.com/ffmpeg/.

If you use Docker Compose, you can add PHOTOPRISM_INIT: "ffmpeg" to the environment section of your compose.yaml or docker-compose.yml file to run this script when the PhotoPrism container starts for the first time:

services:
  photoprism:
    environment:
      PHOTOPRISM_INIT: "ffmpeg"

Please note, though, that this version does not support hardware transcoding and is therefore only suitable if you use standard software transcoding. The static build may also not support all video formats.