如何使用VideoWriter从OpenCV打开GStreamer管道

Question

我使用OpenCV VideoCapture捕获视频帧。捕捉工作正常，因为我能够使用这样的帧：

cv::VideoCapture cap("v4l2src device=/dev/video1 ! videoscale ! videorate ! video/x-raw, width=640, height=360, framerate=30/1 ! videoconvert ! appsink"); 
cv::imshow("feed", frame); 

我也想通过网络发送流，这里是我卡住的地方。不知何故，我在appsrc管道部分失败。我想将流编码为jpeg并通过udp发送。这就是我得到的：

cv::VideoWriter writer 
writer.open("appsrc ! videoconvert ! jpegenc ! jpegparse ! rtpjpegpay pt=96 ! udpsink host=192.168.1.25 port=5000", 0, (double)30, cv::Size(640, 360), true); 

看起来上面的线没有做任何事。 没有做任何事情。另外这款GStreamer的命令不显示任何内容：

gst-launch-1.0 udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG, payload=(int)96" ! rtpjpegdepay ! jpegdec ! decodebin ! videoconvert ! autovideosink 

我不知道我在哪里在writer.open部分失败。如果我像这样运行波纹管GStreamer的命令，他们的工作：

gst-launch-1.0 v4l2src device=/dev/video1 ! videoscale ! videorate ! video/x-raw, width=640, height=360, framerate=30/1 ! jpegenc ! jpegparse ! rtpjpegpay pt=96 ! udpsink host=192.168.1.25 port=5000 
gst-launch-1.0 udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG, payload=(int)96" ! rtpjpegdepay ! jpegdec ! decodebin ! videoconvert ! autovideosink 

Answer 1

使用的OpenCV的API的Gstreamer之前，我们需要使用GStreamer的命令行工具工作的管道。

发件人： OP使用JPEG编码，所以这个管道将使用相同的编码。

gst-launch-1.0 -v v4l2src \ 
! video/x-raw,format=YUY2,width=640,height=480 \ 
! jpegenc \ 
! rtpjpegpay \ 
! udpsink host=127.0.0.1 port=5000 

接收机：宿caps为rtpjpegdepay需要匹配SRC发件人管道的rtpjpegpay的caps。

gst-launch-1.0 -v udpsrc port=5000 \ 
! application/x-rtp, media=video, clock-rate=90000, encoding-name=JPEG, payload=26 \ 
! rtpjpegdepay \ 
! jpegdec \ 
! xvimagesink sync=0 

既然我们有发送方和接收方的工作管道，我们可以将它们移植到OpenCV。

发件人：

void sender() 
{ 
    // VideoCapture: Getting frames using 'v4l2src' plugin, format is 'BGR' because 
    // the VideoWriter class expects a 3 channel image since we are sending colored images. 
    // Both 'YUY2' and 'I420' are single channel images. 
    VideoCapture cap("v4l2src ! video/x-raw,format=BGR,width=640,height=480,framerate=30/1 ! appsink",CAP_GSTREAMER); 

    // VideoWriter: 'videoconvert' converts the 'BGR' images into 'YUY2' raw frames to be fed to 
    // 'jpegenc' encoder since 'jpegenc' does not accept 'BGR' images. The 'videoconvert' is not 
    // in the original pipeline, because in there we are reading frames in 'YUY2' format from 'v4l2src' 
    VideoWriter out("appsrc ! videoconvert ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! jpegenc ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000",CAP_GSTREAMER,0,30,Size(640,480),true); 

    if(!cap.isOpened() || !out.isOpened()) 
    { 
     cout<<"VideoCapture or VideoWriter not opened"<<endl; 
     exit(-1); 
    } 

    Mat frame; 

    while(true) { 

     cap.read(frame); 

     if(frame.empty()) 
      break; 

     out.write(frame); 

     imshow("Sender", frame); 
     if(waitKey(1) == 's') 
      break; 
    } 
    destroyWindow("Sender"); 
} 

接收机：

void receiver() 
{  
    // The sink caps for the 'rtpjpegdepay' need to match the src caps of the 'rtpjpegpay' of the sender pipeline 
    // Added 'videoconvert' at the end to convert the images into proper format for appsink, without 
    // 'videoconvert' the receiver will not read the frames, even though 'videoconvert' is not present 
    // in the original working pipeline 
    VideoCapture cap("udpsrc port=5000 ! application/x-rtp,media=video,payload=26,clock-rate=90000,encoding-name=JPEG,framerate=30/1 ! rtpjpegdepay ! jpegdec ! videoconvert ! appsink",CAP_GSTREAMER); 

    if(!cap.isOpened()) 
    { 
     cout<<"VideoCapture not opened"<<endl; 
     exit(-1); 
    } 

    Mat frame; 

    while(true) { 

     cap.read(frame); 

     if(frame.empty()) 
      break; 

     imshow("Receiver", frame); 
     if(waitKey(1) == 'r') 
      break; 
    } 
    destroyWindow("Receiver"); 
}