2012-12-13 123 views
7

我一直在努力编写一个从Live555中的FramedSource派生出来的类,它将允许我将实时数据从我的D3D9应用程序传输到MP4或类似。如何编写一个Live555 FramedSource让我可以流式传输H.264生活

我所做的每一帧都是将缓冲区作为纹理抓取到系统内存中,然后将其转换为RGB→YUV420P,然后使用x264对其进行编码,然后理想地将NAL数据包传递给Live555。我创建了一个名为H264FramedSource的类,它基本上通过复制DeviceSource文件从FramedSource派生而来。输入是一个输入文件,而不是输入文件,我已经使它成为一个NAL数据包,我更新了每个帧。

我对编解码器和流媒体相当陌生,所以我可能会做一切完全错误的。在每个doGetNextFrame()我应该抓住NAL包,并做一些像

memcpy(fTo, nal->p_payload, nal->i_payload) 

我认为有效载荷是我的帧字节的数据?如果任何人有一个他们从FramedSource派生出来的类的例子,至少可以接近我想要做的事情,我很乐意看到它,这对我来说都是新的,对于弄清楚发生了什么有点棘手。 Live555的文档几乎是代码本身,这并不能让我很容易弄清楚。

+1

我不知道这是否会解决您的问题或没有,但要头脑中,NAL数据包不是1-1帧,您可能会从单帧编码中获得0多个NAL数据包。 – jeremy

+0

感谢您的回复,我很确定是这样。一个人的FramedSource如何解释这个问题? – Garviel

+1

你有没有考虑使用H264VideoStreamDiscreteFramer而不是H264VideoStreamFramer?我会做更多的研究,看看是否有任何东西出现。 (见http://comments.gmane.org/gmane.comp.multimedia.live555.devel/8450) – jeremy

回答

12

好吧,我终于有一些时间花在这个,并得到它的工作!我确信还有其他人会乞求知道如何做到这一点。

您将需要您自己的FramedSource来获取每个帧,进行编码并为流处理做好准备,我将尽快提供一些源代码。

本质上把你的FramedSource放到H264VideoStreamDiscreteFramer中,然后把它扔到H264RTPSink中。这样

scheduler = BasicTaskScheduler::createNew(); 
env = BasicUsageEnvironment::createNew(*scheduler); 

framedSource = H264FramedSource::createNew(*env, 0,0); 

h264VideoStreamDiscreteFramer 
= H264VideoStreamDiscreteFramer::createNew(*env, framedSource); 

// initialise the RTP Sink stuff here, look at 
// testH264VideoStreamer.cpp to find out how 

videoSink->startPlaying(*h264VideoStreamDiscreteFramer, NULL, videoSink); 

env->taskScheduler().doEventLoop(); 

现在在你的主循环渲染的东西,扔在你的后备缓冲您已保存到系统内存的FramedSource因此它可以编码等有关的详细信息如何设置编码的东西检查出这个答案How does one encode a series of images into H264 using the x264 C API?

我的实现是非常多的哈克状态,还没有被优化,我的d3d应用程序运行在15fps左右,由于编码,哎,所以我将不得不看看这个。但对于所有的意图和目的,这个StackOverflow的问题得到解答,因为我主要是在如何对它进行流式传输之后。我希望这可以帮助其他人。

至于我FramedSource它看起来有点像这样

concurrent_queue<x264_nal_t> m_queue; 
SwsContext* convertCtx; 
x264_param_t param; 
x264_t* encoder; 
x264_picture_t pic_in, pic_out; 


EventTriggerId H264FramedSource::eventTriggerId = 0; 
unsigned H264FramedSource::FrameSize = 0; 
unsigned H264FramedSource::referenceCount = 0; 

int W = 720; 
int H = 960; 

H264FramedSource* H264FramedSource::createNew(UsageEnvironment& env, 
               unsigned preferredFrameSize, 
               unsigned playTimePerFrame) 
{ 
     return new H264FramedSource(env, preferredFrameSize, playTimePerFrame); 
} 

H264FramedSource::H264FramedSource(UsageEnvironment& env, 
            unsigned preferredFrameSize, 
            unsigned playTimePerFrame) 
    : FramedSource(env), 
    fPreferredFrameSize(fMaxSize), 
    fPlayTimePerFrame(playTimePerFrame), 
    fLastPlayTime(0), 
    fCurIndex(0) 
{ 
     if (referenceCount == 0) 
     { 

     } 
     ++referenceCount; 

     x264_param_default_preset(&param, "veryfast", "zerolatency"); 
     param.i_threads = 1; 
     param.i_width = 720; 
     param.i_height = 960; 
     param.i_fps_num = 60; 
     param.i_fps_den = 1; 
     // Intra refres: 
     param.i_keyint_max = 60; 
     param.b_intra_refresh = 1; 
     //Rate control: 
     param.rc.i_rc_method = X264_RC_CRF; 
     param.rc.f_rf_constant = 25; 
     param.rc.f_rf_constant_max = 35; 
     param.i_sps_id = 7; 
     //For streaming: 
     param.b_repeat_headers = 1; 
     param.b_annexb = 1; 
     x264_param_apply_profile(&param, "baseline"); 


     encoder = x264_encoder_open(&param); 
     pic_in.i_type   = X264_TYPE_AUTO; 
     pic_in.i_qpplus1   = 0; 
     pic_in.img.i_csp   = X264_CSP_I420; 
     pic_in.img.i_plane  = 3; 


     x264_picture_alloc(&pic_in, X264_CSP_I420, 720, 920); 

     convertCtx = sws_getContext(720, 960, PIX_FMT_RGB24, 720, 760, PIX_FMT_YUV420P, SWS_FAST_BILINEAR, NULL, NULL, NULL); 


     if (eventTriggerId == 0) 
     { 
      eventTriggerId = envir().taskScheduler().createEventTrigger(deliverFrame0); 
     } 
} 

H264FramedSource::~H264FramedSource() 
{ 
    --referenceCount; 
    if (referenceCount == 0) 
    { 
     // Reclaim our 'event trigger' 
     envir().taskScheduler().deleteEventTrigger(eventTriggerId); 
     eventTriggerId = 0; 
    } 
} 

void H264FramedSource::AddToBuffer(uint8_t* buf, int surfaceSizeInBytes) 
{ 
    uint8_t* surfaceData = (new uint8_t[surfaceSizeInBytes]); 

    memcpy(surfaceData, buf, surfaceSizeInBytes); 

    int srcstride = W*3; 
    sws_scale(convertCtx, &surfaceData, &srcstride,0, H, pic_in.img.plane, pic_in.img.i_stride); 
    x264_nal_t* nals = NULL; 
    int i_nals = 0; 
    int frame_size = -1; 


    frame_size = x264_encoder_encode(encoder, &nals, &i_nals, &pic_in, &pic_out); 

    static bool finished = false; 

    if (frame_size >= 0) 
    { 
     static bool alreadydone = false; 
     if(!alreadydone) 
     { 

      x264_encoder_headers(encoder, &nals, &i_nals); 
      alreadydone = true; 
     } 
     for(int i = 0; i < i_nals; ++i) 
     { 
      m_queue.push(nals[i]); 
     } 
    } 
    delete [] surfaceData; 
    surfaceData = NULL; 

    envir().taskScheduler().triggerEvent(eventTriggerId, this); 
} 

void H264FramedSource::doGetNextFrame() 
{ 
    deliverFrame(); 
} 

void H264FramedSource::deliverFrame0(void* clientData) 
{ 
    ((H264FramedSource*)clientData)->deliverFrame(); 
} 

void H264FramedSource::deliverFrame() 
{ 
    x264_nal_t nalToDeliver; 

    if (fPlayTimePerFrame > 0 && fPreferredFrameSize > 0) { 
     if (fPresentationTime.tv_sec == 0 && fPresentationTime.tv_usec == 0) { 
      // This is the first frame, so use the current time: 
      gettimeofday(&fPresentationTime, NULL); 
     } else { 
      // Increment by the play time of the previous data: 
      unsigned uSeconds = fPresentationTime.tv_usec + fLastPlayTime; 
      fPresentationTime.tv_sec += uSeconds/1000000; 
      fPresentationTime.tv_usec = uSeconds%1000000; 
     } 

     // Remember the play time of this data: 
     fLastPlayTime = (fPlayTimePerFrame*fFrameSize)/fPreferredFrameSize; 
     fDurationInMicroseconds = fLastPlayTime; 
    } else { 
     // We don't know a specific play time duration for this data, 
     // so just record the current time as being the 'presentation time': 
     gettimeofday(&fPresentationTime, NULL); 
    } 

    if(!m_queue.empty()) 
    { 
     m_queue.wait_and_pop(nalToDeliver); 

     uint8_t* newFrameDataStart = (uint8_t*)0xD15EA5E; 

     newFrameDataStart = (uint8_t*)(nalToDeliver.p_payload); 
     unsigned newFrameSize = nalToDeliver.i_payload; 

     // Deliver the data here: 
     if (newFrameSize > fMaxSize) { 
      fFrameSize = fMaxSize; 
      fNumTruncatedBytes = newFrameSize - fMaxSize; 
     } 
     else { 
      fFrameSize = newFrameSize; 
     } 

     memcpy(fTo, nalToDeliver.p_payload, nalToDeliver.i_payload); 

     FramedSource::afterGetting(this); 
    } 
} 

哦,谁想要知道我的并发队列是什么,在这里它是,它出色的作品http://www.justsoftwaresolutions.co.uk/threading/implementing-a-thread-safe-queue-using-condition-variables.html

享受还有祝你好运!

+0

感谢分享 – user718146

+0

嗨Garviel,在您的H264FramedSource中,如果有效负载大于目标缓冲区,它只会设置fNumTruncatedBytes值,playload的剩余部分是否会丢失?如果没有,你可以告诉我引擎盖下的机制 – user718146

+0

嗨,数据简单地丢失。如果我没有记错,则不必执行截断。这一切都取决于您的应用程序/设置中可以接受的内容。在我的情况下,我是通过网络进行流式传输的,如果我没有限制它,我会收到很多视觉文物和延迟不一致的情况。我不得不看这个,因为我没有超越原型。 – Garviel

2

deliverFrame方法缺乏在其启动以下检查:

if (!isCurrentlyAwaitingData()) return;  

看到DeviceSource.cpp在LIVE

+0

您好koraxkorakos,我实际上有一个类的外部方法,调用isLive555Waiting的东西。我记得它的存在,但不是执行抱歉。 – Garviel