MediaMuxer有时会产生黑色视频

我正在使用AndroidEncoder产生带有AAC音频的H264视频，我通过RTMP网络同时发送这些音频，并使用MediaMuxer将其本地存储在MP4文件中。MediaMuxer有时会产生黑色视频

广播在RTMP上运行得很好，但是在本地保存结果视频有时可以，有时它只是带有声音的黑色帧（或者只是带有一些信息块的第一部分帧），有时它首先播放音频，然后是具有随机FPS速度的视频。

这是我的复用器包装类和MPEG4Writer SDK类所示的输出：

02-28 11:57:38.521 6420-6475/com.myapp.broadcast W/AndroidMuxer: addTrack: Requested adding of track to class AndroidMuxer of type AUDIO 
02-28 11:57:38.522 6420-6475/com.myapp.broadcast W/AndroidMuxer: addTrack: Adding track for audio/mp4a-latm 
02-28 11:57:38.522 6420-6475/com.myapp.broadcast W/AndroidMuxer: addTrack: Track index to class AndroidMuxer of type AUDIO, track saved for future use! 
02-28 11:57:38.621 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: Requested adding of track to class AndroidMuxer of type VIDEO 
02-28 11:57:38.621 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: Adding track for video/avc 
02-28 11:57:38.621 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: All tracks added, marking Local muxer as ready! 
02-28 11:57:38.622 6420-6470/com.myapp.broadcast W/AndroidMuxer: addTrack: Track index to class AndroidMuxer of type VIDEO, track saved for future use! 
02-28 11:57:52.050 6420-6420/com.myapp.broadcast I/AndroidMuxer: init: Added audio track widh id 0 
02-28 11:57:52.050 6420-6420/com.myapp.broadcast I/AndroidMuxer: init: Added video track widh id 1 
02-28 11:57:52.050 6420-6420/com.myapp.broadcast W/AndroidMuxer: init: Muxer was successfully created! Starting to record 
02-28 11:57:52.056 6420-6420/com.myapp.broadcast I/MPEG4Writer: limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3195 bytes 
02-28 11:57:52.057 6420-6420/com.myapp.broadcast W/AndroidMuxer: start: Muxer started! 
02-28 11:57:52.070 6420-6965/com.myapp.broadcast I/MPEG4Writer: setStartTimestampUs: 0 
02-28 11:57:52.070 6420-6965/com.myapp.broadcast I/MPEG4Writer: Earliest track starting time: 0 
02-28 11:57:52.083 6420-6966/com.myapp.broadcast I/MPEG4Writer: setStartTimestampUs: 0 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast W/AndroidMuxer: writeSampleData: All tracks finished! Calling Stop 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast W/AndroidMuxer: stop: Calling stop to the muxer 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track stopping 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track source stopping 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track source stopped 
02-28 11:58:02.617 6420-6965/com.myapp.broadcast I/MPEG4Writer: Received total/0-length (452/0) buffers and encoded 452 frames. - Audio 
02-28 11:58:02.617 6420-6965/com.myapp.broadcast I/MPEG4Writer: Audio track drift time: 0 us 
02-28 11:58:02.617 6420-6475/com.myapp.broadcast D/MPEG4Writer: Audio track stopped 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track stopping 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track source stopping 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track source stopped 
02-28 11:58:02.618 6420-6966/com.myapp.broadcast I/MPEG4Writer: Received total/0-length (315/0) buffers and encoded 315 frames. - Video 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Video track stopped 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Duration from tracks range is [10143021, 10495420] us 
02-28 11:58:02.618 6420-6475/com.myapp.broadcast D/MPEG4Writer: Stopping writer thread 
02-28 11:58:02.620 6420-6964/com.myapp.broadcast D/MPEG4Writer: 0 chunks are written in the last batch 
02-28 11:58:02.620 6420-6475/com.myapp.broadcast D/MPEG4Writer: Writer thread stopped 
02-28 11:58:02.621 6420-6475/com.myapp.broadcast I/MPEG4Writer: The mp4 file will not be streamable. 
02-28 11:58:02.621 6420-6475/com.myapp.broadcast W/AndroidMuxer: stop: muxer stopped!

不，虽然本地保存的视频是不行的，同样的画面的直播流（来自同一导致编码过程）很好，所以问题必须在复用器本身（或者更确切地说，就是我处理它的方式）。

这是我复用器包装（由bigflake的mediacodec samples严重的启发）：

import android.media.MediaCodec; 
import android.media.MediaFormat; 
import android.media.MediaMuxer; 
import android.util.Log; 

import java.io.IOException; 
import java.nio.ByteBuffer; 

/** 
* @hide 
*/ 
public class AndroidMuxer extends Muxer { 
    private static final String TAG = "AndroidMuxer"; 
    private static final boolean VERBOSE = false; 

    private MediaMuxer mMuxer; 
    private boolean mStarted = false; 
    private boolean mStopped = false; 
    private boolean mReadyToStart; 
    private MediaFormat mVideoTrackFormat; 
    private MediaFormat mAudioTrackFormat; 

    private long mStartTimeUs = 0; 
    @Override 
    public void reset() { 
     Log.w(TAG, "reset: Reset is called on LOCAL MUXER!"); 

      mStarted = false; 
      mReadyToStart = false; 
      mNumTracks = 0; 
      mNumTracksFinished = 0; 
      Log.w(TAG, "reset: Reset worked on the Muxer"); 

    } 

    /** 
    * This will only initialize the fields, we only want the real muxer to be initialized 
    * when we go live. 
    * @param outputFile outputPath of the file 
    * @param format Format of the muxer (will always be MPEG4 TBH) 
    */ 
    public AndroidMuxer(String outputFile, FORMAT format) { 
     super(outputFile, format); 
     mStarted = false; 
    } 

    @Override 
    public void init() { 
     if(!mReadyToStart) { 
      return; 
     } 

     if(mStarted){ 
      Log.i(TAG, "init: Muxer already created! Skipping!"); 
      return; 
     } 


     try { 
      switch (mFormat) { 
       case MPEG4: 
        mMuxer = new MediaMuxer(mOutputPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4); 
        int audioTrack = mMuxer.addTrack(mAudioTrackFormat); 
        Log.i(TAG, "init: Added audio track widh id " + audioTrack); 
        mAudioTrackIndex = audioTrack; 

        int videoTrack = mMuxer.addTrack(mVideoTrackFormat); 
        Log.i(TAG, "init: Added video track widh id " + videoTrack); 
        mVideoTrackIndex = videoTrack; 
        Log.w(TAG, "init: Muxer was successfully created! Starting to record"); 
        start(); 
        break; 
       default: 
        throw new IllegalArgumentException("Unrecognized format!"); 
      } 
     } catch (IOException e) { 
      throw new RuntimeException("MediaMuxer creation failed", e); 
     } 
    } 

    public static AndroidMuxer create(String outputFile, FORMAT format) { 
     return new AndroidMuxer(outputFile, format); 
    } 

    @Override 
    public int addTrack(MediaFormat trackFormat, AndroidEncoder.TrackType trackType) { 
     Log.w(TAG, "addTrack: Requested adding of track to class " + getClass().getSimpleName() + " of type " + trackType); 
     //Super method just keeps track of amount of tracks added 
     super.addTrack(trackFormat, trackType); 
     Log.w(TAG, "addTrack: Adding track for " + trackFormat.getString(MediaFormat.KEY_MIME)); 
     if (mStarted) 
      throw new RuntimeException("format changed twice"); 

     //What we do is store the media info for later use once the muxer is created. 
     //This is done to prevent file creation until the event goes live. 

     if(trackType == AndroidEncoder.TrackType.VIDEO){ 
      mVideoTrackFormat = trackFormat; 
     } 
     else if(trackType == AndroidEncoder.TrackType.AUDIO){ 
      mAudioTrackFormat = trackFormat; 
     } 



     if (allTracksAdded()) { 
      Log.w(TAG, "addTrack: All tracks added, marking Local muxer as ready!"); 
      markReadyToStart(); 
     } 
     Log.w(TAG, "addTrack: Track index to class " + getClass().getSimpleName() + " of type " + trackType + ", track saved for future use!"); 
     //Return dummy 
     return 1; 
    } 


    /** 
    * 
    * Marking the Muxer as ready to be started but not yet! 
    * 
    */ 
    private void markReadyToStart() { 
     mReadyToStart = true; 
    } 



    private void start() { 
     if(mStarted){ 
      Log.i(TAG, "start: Skipped start due to muxer already started"); 
     } 
     mStartTimeUs = 0; 
     mMuxer.start(); 
     Log.w(TAG, "start: Muxer started!"); 
     mStarted = true; 
    } 

    protected void stop() { 
     if(mStarted) { 
      Log.w(TAG, "stop: Calling stop to the muxer"); 
      mMuxer.stop(); 
      Log.w(TAG, "stop: muxer stopped!"); 
      mStarted = false; 
      mStopped = true; 
      mReadyToStart = false; 
     } 
    } 

    @Override 
    public void release() { 
     super.release(); 
     if(mStopped) { 
      mMuxer.release(); 
      mStopped = false; 
      mStarted = false; 
     } 
    } 

    @Override 
    public boolean isStarted() { 
     return mStarted; 
    } 

    @Override 
    public void writeSampleData(MediaCodec encoder, int trackIndex, int bufferIndex, ByteBuffer encodedData, MediaCodec.BufferInfo bufferInfo) { 
     if(!mStarted) { 

      return; 
     } 

     super.writeSampleData(encoder, trackIndex, bufferIndex, encodedData, bufferInfo); 
     if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) { 
      // MediaMuxer gets the codec config info via the addTrack command 
      if (VERBOSE) Log.d(TAG, "ignoring BUFFER_FLAG_CODEC_CONFIG"); 
      return; 
     } 

     if (bufferInfo.size == 0) { 
      if (VERBOSE) Log.d(TAG, "ignoring zero size buffer"); 
      return; 
     } 

     if (!mStarted) { 
      Log.w(TAG, "writeSampleData called before muxer started. Ignoring packet. Track index: " + trackIndex + " tracks added: " + mNumTracks); 
//   encoder.releaseOutputBuffer(bufferIndex, false); 
      return; 
     } 

//  bufferInfo.presentationTimeUs = getNextRelativePts(bufferInfo.presentationTimeUs, trackIndex); 


     if(mStartTimeUs == 0) { 
       mStartTimeUs = bufferInfo.presentationTimeUs; 
      } 
     bufferInfo.presentationTimeUs -= mStartTimeUs; 
     if(bufferInfo.presentationTimeUs < 0) { 
      bufferInfo.presentationTimeUs = 0; 
     } 
     bufferInfo.presentationTimeUs = getSafePts(bufferInfo.presentationTimeUs, trackIndex); 


     mMuxer.writeSampleData(trackIndex, encodedData, bufferInfo); 

//  encoder.releaseOutputBuffer(bufferIndex, false); 

     if (allTracksFinished()) { 
      Log.w(TAG, "writeSampleData: All tracks finished! Calling Stop"); 
      stop(); 
     } 
    } 

    @Override 
    public void forceStop() { 
     Log.w(TAG, "forceStop: ForceStop called!"); 
     stop(); 
    } 
}

注意，RTMP流合并和地方复用器可能无法启动在同一时间工作，这就是为什么我存储介质的信息，然后使用它只有当用户请求复用器启动时。

来源

2017-02-28 Gabriel Sanmartin

如果本地复用器还没有准备好，并且您会跳过写入数据包，那么您可以跳过写入关键帧。后来不是关键帧的视频帧在到达下一个关键帧之前无法解码（并且如果文件以非关键帧开始，谁知道视频播放器是否会感到困惑？）。如果关键帧之间的间隔时间很长，则在合理的时间内可能完全没有关键帧。（虽然，如果你是通过RTMP进行流式传输的话，你可能会经常使用关键帧。）

我的建议是确保存储本地复用器准备好之前生成的所有数据包，然后写入它们一旦准备就绪，就可以向本地复用器发送数据包，或者继续跳过数据包直到达到下一个关键帧。

来源

2017-02-28 12:19:26 mstorsjo

不能这样做，我的应用程序的设计使您可以开始直播，然后选择任何时刻在本地开始录制。但是，我可以做的只是当我收到一个关键帧时才开始提供本地编码器。不过我不确定这是否是问题，因为如果这是问题，那么a）我几乎不会获得关键帧作为第一帧，因此我几乎永远不会有适当的录音和b）我认为如果这是问题，然后我可能会得到2黑色秒，然后视频将恢复正常，一旦我得到一个关键帧（我有GOP设置为每2秒）。 –

但是，我会尝试这种方法。我想，这也不能解释为什么我有时会得到第一个音频，然后是一些本地录制文件中的所有视频。 –

MediaMuxer有时会产生黑色视频

回答

相关问题