ReplayKit 制作的视频在 Xamarin 中不断跳帧

Question

我有一个非常基本的视频录制项目，它在 Swift 中运行良好，但是移植到 Xamarin 空白项目中的相同代码生成的视频每隔几秒就会不断跳帧。代码从 ViewDidLoad 开始，通过 UIButton 停止这里是下面的记录代码：

RPScreenRecorder rp = RPScreenRecorder.SharedRecorder;
AVAssetWriter assetWriter;
AVAssetWriterInput videoInput;

public override void ViewDidLoad()
{
    base.ViewDidLoad();
    StartScreenRecording();
}

public void StartScreenRecording()
{
    VideoSettings videoSettings = new VideoSettings();
    NSError wError;
    assetWriter = new AVAssetWriter(videoSettings.OutputUrl, AVFileType.AppleM4A, out wError);
    videoInput = new AVAssetWriterInput(AVMediaType.Video, videoSettings.OutputSettings);

    videoInput.ExpectsMediaDataInRealTime = true;

    assetWriter.AddInput(videoInput);

    if (rp.Available)
    {
        rp.StartCaptureAsync((buffer, sampleType, error) =>
        {
            if (buffer.DataIsReady)
            {

                if (assetWriter.Status == AVAssetWriterStatus.Unknown)
                {

                    assetWriter.StartWriting();

                    assetWriter.StartSessionAtSourceTime(buffer.PresentationTimeStamp);

                }

                if (assetWriter.Status == AVAssetWriterStatus.Failed)
                {
                    return;
                }

                if (sampleType == RPSampleBufferType.Video)
                {
                    if (videoInput.ReadyForMoreMediaData)
                    {
                        videoInput.AppendSampleBuffer(buffer);
                    }
                }

            }

        });
    }

}

public void StopRecording()
{
    rp.StopCapture((error) => {
        if (error == null)
        {
            assetWriter.FinishWriting(() => { });
        }
    });
}

下面是 VideoSettings 文件的样子：

public class VideoSettings
{
    public string VideoFilename => "render";
    public string VideoFilenameExt = "mp4";
    public nfloat Width { get; set; }
    public nfloat Height { get; set; }
    public AVVideoCodec AvCodecKey => AVVideoCodec.H264;

    public NSUrl OutputUrl
    {
        get
        {
            return GetFilename(VideoFilename,VideoFilenameExt);
        }
    }

    private NSUrl GetFilename(string filename, string extension)
    {
        NSError error;
        var docs = new NSFileManager().GetUrl(NSSearchPathDirectory.DocumentDirectory, NSSearchPathDomain.User, null, true, out error).ToString() + filename + 1 + "." + extension;
        if (error == null)
        {
            return new NSUrl(docs);
        }
        return null;
    }


    public AVVideoSettingsCompressed OutputSettings
    {
        get
        {
            return new AVVideoSettingsCompressed
            {
                Codec = AvCodecKey,
                Width = Convert.ToInt32(UIScreen.MainScreen.Bounds.Size.Width),
                Height = Convert.ToInt32(UIScreen.MainScreen.Bounds.Size.Height)
            };
        }
    }
}

Answer 1

TL;DR：您需要一个 try {} finally {} 块来帮助 GC 及时释放您的样本缓冲区：

try {
    // Do stuff with the buffer.
} finally {
    buffer.Dispose ();
}

长话短说：

这是因为 GC 的速度不够快，无法意识到需要释放 CMSampleBuffer 而您运行样本缓冲区不足，这就是您看到延迟的原因，因为直到一个新的缓冲区可用，它拍摄了该实际帧的快照。

也不必担心调用 Dispose () Xamarin 会做正确的事情，因此不需要额外的检查

public void Dispose ()
{
    this.Dispose (true);
    GC.SuppressFinalize (this);
}

protected virtual void Dispose (bool disposing)
{
    if (this.invalidate.IsAllocated) {
        this.invalidate.Free ();
    }
    if (this.handle != IntPtr.Zero) {
        CFObject.CFRelease (this.handle);
        this.handle = IntPtr.Zero;
    }
}

因此您的代码应如下所示：

if (rp.Available)
{
    // TODO: Also note that we are not using the Async version here
    rp.StartCapture((buffer, sampleType, error) =>
    {
        try {
            if (buffer.DataIsReady) {

                if (assetWriter.Status == AVAssetWriterStatus.Unknown) {

                    assetWriter.StartWriting ();

                    assetWriter.StartSessionAtSourceTime (buffer.PresentationTimeStamp);

                }

                if (assetWriter.Status == AVAssetWriterStatus.Failed) {
                    return;
                }

                if (sampleType == RPSampleBufferType.Video) {
                    if (videoInput.ReadyForMoreMediaData) {
                        videoInput.AppendSampleBuffer (buffer);
                    }
                }

            }
        } finally {
            buffer.Dispose ();
        }


    }, null);
}

ReplayKit 制作的视频在 Xamarin 中不断跳帧

Video produced by ReplayKit is constantly skipping frames in Xamarin

c#

ios

xamarin

replaykit