Xcode 调试 iOS 时 GStreamer 不需要的启动延迟和单元测试崩溃

GStreamer unwanted startup delay and unit test crashes when Xcode debugging for iOS

我有一个 iOS 应用程序,它使用 Gstreamer 通过 UDP 从源流式传输实时视频。当我 运行 这个应用程序附加了 Xcode 调试器时,在视频出现之前会有 5-10 秒 的延迟。然后它似乎快速显示所有以前的帧,直到它 "catches up" 到当前时间戳。起初我以为这是由我的管道转换为动态代码引起的,但后来我将问题隔离到仅在调试时:

这是我的控制台输出,其中包含一些调试消息、GStreamer 回调以及我在代码注释语法中的注释:

WARNING: no real random source present!
 GST_STATE_READY
playOrStopVideo
2019-09-26 15:45:19.442411-0400  pipeline is null
2019-09-26 15:45:19.502487-0400  pad_added_handler: tee
2019-09-26 15:45:19.534644-0400  Metal GPU Frame Capture Enabled
2019-09-26 15:45:19.535959-0400  Metal API Validation Enabled
[GStreamer check_initialization_complete]:[00m Initialization complete, notifying application.
ViewController gstreamerInitialized()
2019-09-26 15:45:19.646378-0400   message NEED_CONTEXT
2019-09-26 15:45:19.646436-0400   message HAVE_CONTEXT
2019-09-26 15:45:19.646465-0400   message NEED_CONTEXT
2019-09-26 15:45:19.646492-0400   message ELEMENT
2019-09-26 15:45:19.646547-0400   message NEED_CONTEXT
2019-09-26 15:45:19.646636-0400   message NEED_CONTEXT
2019-09-26 15:45:19.646685-0400   message NEED_CONTEXT
 GST_STATE_READY
2019-09-26 15:45:19.649381-0400   message STREAM_STATUS CREATE for src
2019-09-26 15:45:19.652090-0400   message STREAM_STATUS CREATE for src
2019-09-26 15:45:19.652148-0400   message STREAM_STATUS CREATE for src
 GST_STATE_PAUSED
2019-09-26 15:45:19.652808-0400   message STREAM_STATUS ENTER for src
2019-09-26 15:45:19.652926-0400   message NEW_CLOCK
2019-09-26 15:45:19.652980-0400   message STREAM_STATUS ENTER for src
2019-09-26 15:45:19.653166-0400   message STREAM_STATUS ENTER for src
2019-09-26 15:45:19.987250-0400  pad_added_handler: decodebin0
2019-09-26 15:45:19.999452-0400   message STREAM_START
2019-09-26 15:45:20.032557-0400   message TAG
=================================================================
Main Thread Checker: UI API called on a background thread: -[UIView frame]
PID: 3788, TID: 1475176, Thread name: (none), Queue name: org.freedesktop.gstreamer.glwindow, QoS: 0
Backtrace:
4   MyApp                               0x0000000105ef14f8 gst_gl_context_eagl_update_layer + 652
5   MyApp                               0x0000000105edd4b8 gst_gl_window_eagl_set_window_handle + 92
6   MyApp                               0x0000000105ef5f6c _set_window_handle_cb + 104
7   MyApp                               0x0000000105edd790 __gst_gl_window_eagl_send_message_async_block_invoke + 36
8   libdispatch.dylib                   0x00000001097e51dc _dispatch_call_block_and_release + 24
9   libdispatch.dylib                   0x00000001097e519c _dispatch_client_callout + 16
10  libdispatch.dylib                   0x00000001097f3dfc _dispatch_queue_serial_drain + 768
11  libdispatch.dylib                   0x00000001097e86ac _dispatch_queue_invoke + 328
12  libdispatch.dylib                   0x00000001097f4d54 _dispatch_root_queue_drain_deferred_wlh + 352
13  libdispatch.dylib                   0x00000001097fbe38 _dispatch_workloop_worker_thread + 676
14  libsystem_pthread.dylib             0x0000000184c83e70 _pthread_wqthread + 860
15  libsystem_pthread.dylib             0x0000000184c83b08 start_wqthread + 4
2019-09-26 15:45:20.049748-0400 MyApp[3788:1475176] [reports] Main Thread Checker: UI API called on a background thread: -[UIView frame]
PID: 3788, TID: 1475176, Thread name: (none), Queue name: org.freedesktop.gstreamer.glwindow, QoS: 0
Backtrace:
4   MyApp                               0x0000000105ef14f8 gst_gl_context_eagl_update_layer + 652
5   MyApp                               0x0000000105edd4b8 gst_gl_window_eagl_set_window_handle + 92
6   MyApp                               0x0000000105ef5f6c _set_window_handle_cb + 104
7   MyApp                               0x0000000105edd790 __gst_gl_window_eagl_send_message_async_block_invoke + 36
8   libdispatch.dylib                   0x00000001097e51dc _dispatch_call_block_and_release + 24
9   libdispatch.dylib                   0x00000001097e519c _dispatch_client_callout + 16
10  libdispatch.dylib                   0x00000001097f3dfc _dispatch_queue_serial_drain + 768
11  libdispatch.dylib                   0x00000001097e86ac _dispatch_queue_invoke + 328
12  libdispatch.dylib                   0x00000001097f4d54 _dispatch_root_queue_drain_deferred_wlh + 352
13  libdispatch.dylib                   0x00000001097fbe38 _dispatch_workloop_worker_thread + 676
14  libsystem_pthread.dylib             0x0000000184c83e70 _pthread_wqthread + 860
15  libsystem_pthread.dylib             0x0000000184c83b08 start_wqthread + 4
=================================================================
Main Thread Checker: UI API called on a background thread: -[UIView layer]
PID: 3788, TID: 1475176, Thread name: (none), Queue name: org.freedesktop.gstreamer.glwindow, QoS: 0
Backtrace:

4   MyApp                               0x0000000105ef1300 gst_gl_context_eagl_update_layer + 148
5   MyApp                               0x0000000105edd4b8 gst_gl_window_eagl_set_window_handle + 92
6   MyApp                               0x0000000105ef5f6c _set_window_handle_cb + 104
7   MyApp                               0x0000000105edd790 __gst_gl_window_eagl_send_message_async_block_invoke + 36
8   libdispatch.dylib                   0x00000001097e51dc _dispatch_call_block_and_release + 24
9   libdispatch.dylib                   0x00000001097e519c _dispatch_client_callout + 16
10  libdispatch.dylib                   0x00000001097f3dfc _dispatch_queue_serial_drain + 768
11  libdispatch.dylib                   0x00000001097e86ac _dispatch_queue_invoke + 328
12  libdispatch.dylib                   0x00000001097f4d54 _dispatch_root_queue_drain_deferred_wlh + 352
13  libdispatch.dylib                   0x00000001097fbe38 _dispatch_workloop_worker_thread + 676
14  libsystem_pthread.dylib             0x0000000184c83e70 _pthread_wqthread + 860
15  libsystem_pthread.dylib             0x0000000184c83b08 start_wqthread + 4
2019-09-26 15:45:27.554390-0400 MyApp[3788:1475176] [reports] Main Thread Checker: UI API called on a background thread: -[UIView layer]
PID: 3788, TID: 1475176, Thread name: (none), Queue name: org.freedesktop.gstreamer.glwindow, QoS: 0
Backtrace:
4   MyApp                               0x0000000105ef1300 gst_gl_context_eagl_update_layer + 148
5   MyApp                               0x0000000105edd4b8 gst_gl_window_eagl_set_window_handle + 92
6   MyApp                               0x0000000105ef5f6c _set_window_handle_cb + 104
7   MyApp                               0x0000000105edd790 __gst_gl_window_eagl_send_message_async_block_invoke + 36
8   libdispatch.dylib                   0x00000001097e51dc _dispatch_call_block_and_release + 24
9   libdispatch.dylib                   0x00000001097e519c _dispatch_client_callout + 16
10  libdispatch.dylib                   0x00000001097f3dfc _dispatch_queue_serial_drain + 768
11  libdispatch.dylib                   0x00000001097e86ac _dispatch_queue_invoke + 328
12  libdispatch.dylib                   0x00000001097f4d54 _dispatch_root_queue_drain_deferred_wlh + 352
13  libdispatch.dylib                   0x00000001097fbe38 _dispatch_workloop_worker_thread + 676
14  libsystem_pthread.dylib             0x0000000184c83e70 _pthread_wqthread + 860
15  libsystem_pthread.dylib             0x0000000184c83b08 start_wqthread + 4

//
// THIS IS WHERE IT HANGS (check the timestamps)
//

2019-09-26 15:45:27.692918-0400   message ASYNC_DONE
 GST_STATE_PLAYING
2019-09-26 15:45:28.023985-0400   message QOS
2019-09-26 15:45:28.190131-0400   message QOS
2019-09-26 15:45:28.418494-0400   message QOS
2019-09-26 15:45:28.657004-0400   message QOS
2019-09-26 15:45:28.665464-0400   message QOS
2019-09-26 15:45:28.668784-0400   message QOS
2019-09-26 15:45:28.672105-0400   message QOS
2019-09-26 15:45:28.675492-0400   message QOS

//
// Sometimes I see output with QoS about "a lot of buffers being dropped"
//

//
// STOPPING VIDEO VIA UI AND RESTARTING (plays immediately)
//
playOrStopVideo
2019-09-26 15:45:32.959316-0400   SENT EOS OK
2019-09-26 15:45:32.962257-0400   GST_MESSAGE_EOS
2019-09-26 15:45:32.969720-0400   Pipeline set to NULL
 GST_STATE_NULL
[GStreamer setupVideoWithRecording:]:[00m Exited main loop
2019-09-26 15:45:32.970650-0400   Exited main loop
2019-09-26 15:45:32.970712-0400   destroy()
playOrStopVideo
2019-09-26 15:45:35.330103-0400  pipeline is null
2019-09-26 15:45:35.369882-0400  pad_added_handler: tee
[GStreamer check_initialization_complete]:[00m Initialization complete, notifying application.
ViewController gstreamerInitialized()
2019-09-26 15:45:35.385008-0400   message NEED_CONTEXT
2019-09-26 15:45:35.385064-0400   message HAVE_CONTEXT
2019-09-26 15:45:35.385099-0400   message NEED_CONTEXT
2019-09-26 15:45:35.385134-0400   message ELEMENT
2019-09-26 15:45:35.385241-0400   message NEED_CONTEXT
2019-09-26 15:45:35.385305-0400   message NEED_CONTEXT
2019-09-26 15:45:35.385368-0400   message NEED_CONTEXT
 GST_STATE_READY
2019-09-26 15:45:35.389410-0400   message STREAM_STATUS CREATE for src
2019-09-26 15:45:35.389992-0400   message STREAM_STATUS CREATE for src
2019-09-26 15:45:35.390051-0400   message STREAM_STATUS CREATE for src
 GST_STATE_PAUSED
2019-09-26 15:45:35.390270-0400   message STREAM_STATUS ENTER for src
2019-09-26 15:45:35.390301-0400   message NEW_CLOCK
2019-09-26 15:45:35.390437-0400   message STREAM_STATUS ENTER for src
2019-09-26 15:45:35.390519-0400   message STREAM_STATUS ENTER for src
2019-09-26 15:45:35.659082-0400  pad_added_handler: decodebin1
2019-09-26 15:45:35.676625-0400   message STREAM_START
2019-09-26 15:45:35.682690-0400   message TAG
2019-09-26 15:45:35.693208-0400   message ASYNC_DONE
 GST_STATE_PLAYING

这没什么大不了的,因为它只是在调试时,但是这给我的开发增加了很多不必要的时间。这似乎与 iOS 线程投诉有关。自从我使用它以来,它们就一直在 GStreamer 中(甚至可能已经提交了错误),而且我不确定是否有解决方法,因为它在库本身中。想法?

编辑

此处报告:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/568 https://www.bountysource.com/issues/48924739-glwindow-eagl-bug-with-ios11-and-xcode-9-0-beta-4 https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/372 https://gitlab.freedesktop.org/gstreamer/gstreamer/issues/394

不确定正确的修复方法是什么。我正在尝试各种调度队列组合。

编辑 2:

使用 XCTWaiter 创建一些单元测试,但 运行ning 应用程序在同一时间停止。假设这也与线程有关,因为在测试(主)线程中等待 GStreamer 在后台修改应该在主线程(UIView 属性视图和框架)上修改的内容。

我能够通过重新编译 GStreamer 并添加一些调度调用来解决这个问题,其中 [UIView layer][UIView frame] 用于 gst_gl_context_eagl_update_layer 和 window 的 draw_cb.

使用此提交作为参考,但您也可以查看堆栈跟踪,因为我需要的调用更少。
gstglcontext_eagl.m gstglwindow_eagl.m

调度依据: