FFmpeg開發系列1 - FFmpeg、OpenCV、CUDA、NV_Codec GPU加速環境搭建

OS:Ubuntu: 16.04 / 18.04

OpencCV: 4.2

Cuda: 8.0 / 9.0

Nivida Driver: 418.56 (GTX 1050ti) / 435 (GTX 1060)

CPU: i3-8300 3.70Ghz

ffmpeg: 4.2 / 4.2.4

nv_codec_headers: 8.2 / 9.0

Install Nvidia Driver

list available Nvidia Driver
1

ubuntu-drivers devices

add repoitory

1
2


sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update

instll Nvidia Driver
1

sudo apt install nvidia-xxx

Install CUDA

Check CUDA and Nvidia driver compatibility https://docs.nvidia.com/deploy/cuda-compatibility/index.html
Download Cuda runfile which determines by your os https://developer.nvidia.com/cuda-downloads

please choose not to install nvidia graphics driver, or the driver will be updated**
install cuda (ex: cuda 8.0 for ubuntnu 16.04 )
1

sudo sh cuda_8.0.61_375.26_linux.run

Add environment variables to ~/.bashrc

1
2
3
4


sudo nano ~/.bashrc
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH 
source ~/.bashrc

Install NV_codec (optional for gpu )

download nv_codec_headers 8.2: https://github.com/FFmpeg/nv-codec-headers/tree/sdk/8.2
install nv_codec_headers 8.2
1

sudo make install
nv_codec_headers 9.0需要cuda 9.0以上

Install FFmpeg 4.2.4

install necessary tool

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


sudo apt-get update -qq && sudo apt-get -y install \
  autoconf \
  automake \
  build-essential \
  cmake \
  git-core \
  libass-dev \
  libfreetype6-dev \
  libgnutls28-dev \
  libsdl2-dev \
  libtool \
  libva-dev \
  libvdpau-dev \
  libvorbis-dev \
  libxcb1-dev \
  libxcb-shm0-dev \
  libxcb-xfixes0-dev \
  pkg-config \
  texinfo \
  wget \
  yasm \
  zlib1g-dev

install libx264:

1
2
3
4


git clone https://code.videolan.org/videolan/x264.git
cd x264
sudo ./configure --enable-shared --disable-asm
sudo make && sudo make install

installl ffplay
1

sudo apt-get install libsdl2-dev

install libfdk_aac (optional)

https://sourceforge.net/projects/opencore-amr/files/fdk-aac/

1
2


./configure
make && sudo make install

1
2
3


vi /etc/ld/so.conf.d/aac.conf
/usr/local/lib
sudo ldconfig

install mp3lame (optional)

1

sudo apt-get install yasm libmp3lame-dev

download ffmpeg:

1

git clone https://git.ffmpeg.org/ffmpeg.git

configure command (if you need cuvid)

1

sudo ./configure --enable-libmp3lame --enable-libfdk_aac --enable-libx264 --enable-gpl --enable-cuda --enable-cuvid --enable-nvenc --enable-nonfree --enable-pic --enable-rpath --enable-shared --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --enable-libnpp --enable-ffplay

configure command (if you don’t need cuvid and sound but want cuda )

1

sudo ./configure --enable-libx264 --enable-gpl --enable-cuda --enable-nvenc --enable-nonfree --enable-rpath --enable-pic --enable-shared --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --enable-ffplay

compile ffmpeg
1

sudo make && sudo make install

add to envirnment

1
2
3
4
5
6
7


sudo vi ~/.bashrc 
export PATH=/usr/local/bin:$PATH
source ~/.bashrc
sudo nano /etc/ld.so.conf
# dertermine on your ffmpeg install location
/usr/local/ffmpeg/lib  
sudo ldconfig

for more build options https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

Test ffmpeg

Hardware acceleration methods:

1

ffmpeg -hwaccels

cpu:

1

time ffmpeg -rtsp_transport tcp -i rtsp://admin:ai123456@192.168.0.111 -r 30 -t 100 -c:v h264 -b:v 2048k -vf scale=1280:-1 -y -c:v libx264 tcp_cpu_output.mp4

gpu:

1

time ffmpeg -rtsp_transport tcp -hwaccel cuvid -c:v h264_cuvid -i rtsp://admin:ai123456@192.168.0.111 -r 30 -t 100 -b:v 2048k -vf scale_npp=1280:-1 -y -c:v h264_nvenc tcp_gpu_output.mp4

默認下ffmpeg抓rtsp使用UDP,這會lose大量packet，使用TCP避免。

Video.mp4	CPU	GPU
Real Time	16s	0.5s
Bit rate	1497kbps	1479kbps

rtsp://admin:ai123456@192.168.0.111 (30幀)	CPU	GPU
Real Time	1m49s	1m41s
Bit rate	1911kbps	2067kbps

Install Opencv

install required and optional package

1

sudo add-apt-repository -y "deb http://security.ubuntu.com/ubuntu xenial-security main"

1

sudo apt-get install -y libjpeg8-dev libjasper-dev libpng12-dev

1

sudo apt install build-essential cmake git pkg-config libgtk-3-dev libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev

1

sudo apt install libjpeg-dev libpng-dev libtiff-dev gfortran openexr libatlas-base-dev python3-dev python3-numpy libtbb2 libtbb-dev libdc1394-22-dev

download opencv 4.2 and extra library

1
2
3
4
5
6
7


mkdir opencv_base
cd opencv_base
git clone https://github.com/opencv/opencv.git --branch=4.2.0
git clone https://github.com/opencv/opencv_contrib.git --branch=4.2.0	 
cd opencv_base/opencv/
mkdir build
cd build

with cuda and opencv_world

check your cuda_arch_bin:https://developer.nvidia.com/cuda-gpus

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


sudo cmake -D CMAKE_BUILD_TYPE=Release \
	 -D OPENCV_GENERATE_PKGCONFIG=YES \
	 -D CMAKE_INSTALL_PREFIX=/usr/local \
	 -D BUILD_JAVA=YES \
	 -D WITH_CUDA=ON \
	 -D BUILD_opencv_world=ON \
	 -D WITH_FFMPEG=ON \
	 -D OPENCV_GENERATE_PKGCONFIG=ON \
	 -D OPENCV_EXTRA_MODULES_PATH=/usr/local/opencv_base/opencv_contrib/modules \
	 -D BUILD_opencv_python3=yes \
	 -D CUDA_ARCH_BIN=6.1 ..

without cuda (Simple)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


sudo cmake -D CMAKE_BUILD_TYPE=RELEASE \
	-D OPENCV_GENERATE_PKGCONFIG=YES \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D BUILD_JAVA=YES \
	-D BUILD_opencv_world=ON \
	-D WITH_FFMPEG=ON \
    -D INSTALL_C_EXAMPLES=ON \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D OPENCV_GENERATE_PKGCONFIG=ON \
    -D OPENCV_EXTRA_MODULES_PATH=/usr/local/opencv_base/opencv_contrib/modules \
    -D BUILD_EXAMPLES=ON ..

Compile cuda && ffmpeg with Opencv 開發指令

include及lib都為相對路徑 (if you want to pack code )

Two .so files need to be at system location

libx264.so

1

sudo cp lib/libx264.so.160 /lib/x86_64-linux-gnu/

libswresample.so

1

sudo cp lib/libswresample.so.3 /usr/local/lib/

CUDA compiles to shared file (.so): (NV12toBGR)

1

nvcc -shared cuda_convert.cu -o ../lib/libcuda_convert.so --compiler-options '-fPIC'

FFmpeg gpu server compiles to shared file (.so)

1

g++ -shared -fPIC ffmpeg_gpu_server.cpp -o ../lib/libffmpeg_gpu_server.so -I../include/ -L ../lib -Wl,--rpath='../lib' -lavformat -lavcodec -lavutil -lswscale -lswresample -lcuda_convert -lx264

FFmpeg gpu client compiles

1

g++ ffmpeg_gpu_client.cpp -o ../bin/ffmpeg_gpu_client -I../include/ -Wl,--rpath='../lib' -L -lffmpeg_gpu_server -lopencv_world -ldl -lm -lz -lX11 -std=gnu++11

Install Nvidia Driver#

Install CUDA#

Install NV_codec (optional for gpu )#

Install FFmpeg 4.2.4#

Test ffmpeg#

Install Opencv#

with cuda and opencv_world#

without cuda (Simple)#

Compile cuda && ffmpeg with Opencv 開發指令#

Two .so files need to be at system location#

CUDA compiles to shared file (.so): (NV12toBGR)#

FFmpeg gpu server compiles to shared file (.so)#

FFmpeg gpu client compiles#