如何使用 camera2 的 reprocessCaptureRequest API
How to use a reprocessCaptureRequest with camera2 API
我正在尝试将相机项目更新为 Android N，因此我要移动旧的 CameraCaptureSession to a ReprocessableCaptureSession. I did it and it is working fine, but with this new feature I can use the CameraDevice.TEMPLATE_ZERO_SHUTTER_LAG template in my device and I can reprocess frames with the reprocessCaptureRequest。
这是我的问题出现的地方。因为我没有找到任何示例，而且我不太了解有关如何使用 reprocessCaptureRequest:
的小文档
Each reprocess CaptureRequest processes one buffer from CameraCaptureSession's input Surface to all output Surfaces included in the reprocess capture request. The reprocess input images must be generated from one or multiple output images captured from the same camera device. The application can provide input images to camera device via queueInputImage(Image). The application must use the capture result of one of those output images to create a reprocess capture request so that the camera device can use the information to achieve optimal reprocess image quality. For camera devices that support only 1 output Surface, submitting a reprocess CaptureRequest with multiple output targets will result in a CaptureFailure.
我试图查看 google.sources 中关于相机的 CTS 测试，但他们做的和我一样。使用多个imageReader，将TotalCaptureResult张图片保存在一张LinkedBlockingQueue<TotalCaptureResult>中。稍后只需调用：
TotalCaptureResult totalCaptureResult = state.captureCallback.getTotalCaptureResult();
CaptureRequest.Builder reprocessCaptureRequest = cameraStore.state().cameraDevice.createReprocessCaptureRequest(totalCaptureResult);
reprocessCaptureRequest.addTarget(state.yuvImageReader.getSurface());
sessionStore.state().session.capture(reprocessCaptureRequest.build(), null, this.handlers.bg());
但它总是抛出一个 RuntimeException：
java.lang.RuntimeException: Capture failed: Reason 0 in frame 170,
我只想知道使用 ReprocessableCaptureSession 的正确方法是什么，因为我已经尝试了所有方法，但我不知道我做错了什么。
我终于找到了使 reprocessableCaptureSession 正常工作的解决方案。
我使用的是 Flux 架构，所以当您看到 Dispatcher.dispatch(action) 时不要混淆，只需将其视为回调即可。所以，这是我的代码：
首先会话是如何创建的：
//Configure preview surface
Size previewSize = previewState.previewSize;
previewState.previewTexture.setDefaultBufferSize(previewSize.getWidth(), previewSize.getHeight());
ArrayList<Surface> targets = new ArrayList<>();
for (SessionOutputTarget outputTarget : state.outputTargets) {
Surface surface = outputTarget.getSurface();
if (surface != null) targets.add(surface);
}
targets.add(previewState.previewSurface);
CameraCharacteristics cameraCharacteristics = cameraStore.state().availableCameras.get(cameraStore.state().selectedCamera);
Size size = CameraCharacteristicsUtil.getYuvOutputSizes(cameraCharacteristics).get(0);
InputConfiguration inputConfiguration = new InputConfiguration(size.getWidth(),
size.getHeight(), ImageFormat.YUV_420_888);
CameraCaptureSession.StateCallback sessionStateCallback = new CameraCaptureSession.StateCallback() {
@Override
public void onConfigured(@NonNull CameraCaptureSession session) {
if (sessionId != currentSessionId) {
Timber.e("Session opened for an old open request, skipping. Current %d, Request %d", currentSessionId, sessionId);
//performClose(session);
return;
}
try {
session.getInputSurface();
//This call is irrelevant,
//however session might have closed and this will throw an IllegalStateException.
//This happens if another camera app (or this one in another PID) takes control
//of the camera while its opening
} catch (IllegalStateException e) {
Timber.e("Another process took control of the camera while creating the session, aborting!");
}
Dispatcher.dispatchOnUi(new SessionOpenedAction(session));
}
@Override
public void onConfigureFailed(@NonNull CameraCaptureSession session) {
if (sessionId != currentSessionId) {
Timber.e("Configure failed for an old open request, skipping. Current %d, request %d", currentSessionId, sessionId);
return;
}
Timber.e("Failed to configure the session");
Dispatcher.dispatchOnUi(new SessionFailedAction(session, new IllegalStateException("onConfigureFailed")));
}
};
if (state.outputMode == OutputMode.PHOTO) {
cameraState.cameraDevice.createReprocessableCaptureSession(inputConfiguration, targets, sessionStateCallback, handlers.bg());
} else if (state.outputMode == OutputMode.VIDEO) {
cameraState.cameraDevice.createCaptureSession(targets, sessionStateCallback, handlers.bg());
}
} catch (IllegalStateException | IllegalArgumentException e) {
Timber.e(e, "Something went wrong trying to start the session");
} catch (CameraAccessException e) {
//Camera will throw CameraAccessException if another we try to open / close the
//session very fast.
Timber.e("Failed to access camera, it was closed");
}
使用 4 个表面（预览、YUV（输入）、JPEG 和 RAW）创建的照片会话。之后，我配置我的 imageWriter:
Dispatcher.subscribe(Dispatcher.VERY_HIGH_PRIORITY, SessionOpenedAction.class)
.filter(a -> isInPhotoMode())
.subscribe(action -> {
PhotoState newState = new PhotoState(state());
newState.zslImageWriter = ImageWriter.newInstance(action.session.getInputSurface(), MAX_REPROCESS_IMAGES);
setState(newState);
});
好的，现在我们已经创建了 ImageWriter 和会话。不，我们使用重复请求开始流式传输：
CaptureRequest.Builder captureRequestBuilder =
cameraStore.state().cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_ZERO_SHUTTER_LAG);
captureRequestBuilder.addTarget(previewStore.state().previewSurface);
captureRequestBuilder.addTarget(photoStore.state().yuvImageReader.getSurface());
state.session.setRepeatingRequest(captureRequestBuilder.build(), state.zslCaptureCallback, handlers.bg());
为了不添加很多代码，只说 zslCaptureCallback 是一个自定义回调，它保存在 LinkedBlockingQueue<TotalCaptureRequest> X 最后一个 TotalCaptureRequests 中。此外，我对 yuvImageReader（输入一个）执行相同操作，将最后 X 个图像保存在队列中。
最后是我的 "take photo" 方法：
try {
//Retrieve the last image stored by the zslImageReader
Image image = zslImageReaderListener.getImage();
//Retrieve the last totalCaptureResult from the zslCaptureCallback and create a reprocessableCaptureRequest with it
TotalCaptureResult captureResult = sessionStore.state().zslCaptureCallback.getCaptureResult(image.getTimestamp());
CaptureRequest.Builder captureRequest = cameraStore.state().cameraDevice.createReprocessCaptureRequest(captureResult);
//Add the desired target and values to the captureRequest
captureRequest.addTarget(state().jpegImageReader.getSurface());
//Queued back to ImageWriter for future consumption.
state.zslImageWriter.queueInputImage(image);
//Drain all the unused and queued CapturedResult from the CaptureCallback
sessionStore.state().zslCaptureCallback.drain();
//Capture the desired frame
CaptureRequest futureCaptureResult = captureRequest.build();
sessionStore.state().session.capture(futureCaptureResult, new CameraCaptureSession.CaptureCallback() {
@Override
public void onCaptureCompleted(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull TotalCaptureResult result) {
Dispatcher.dispatchOnUi(new PhotoStatusChangedAction(PhotoState.Status.SUCCESS));
}
@Override
public void onCaptureFailed(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull CaptureFailure failure) {
super.onCaptureFailed(session, request, failure);
Exception captureFailedException = new RuntimeException(
String.format("Capture failed: Reason %s in frame %d, was image captured? -> %s",
failure.getReason(),
failure.getFrameNumber(),
failure.wasImageCaptured()));
Timber.e(captureFailedException, "Cannot take mediaType, capture failed!");
Dispatcher.dispatchOnUi(new PhotoStatusChangedAction(PhotoState.Status.ERROR, captureFailedException));
}
}, this.handlers.bg());
//Capture did not blow up, we are taking the photo now.
newState.status = PhotoState.Status.TAKING;
} catch (CameraAccessException | InterruptedException| IllegalStateException | IllegalArgumentException | SecurityException e) {
Timber.e(e, "Cannot take picture, capture error!");
newState.status = PhotoState.Status.ERROR;
}
我正在尝试将相机项目更新为 Android N，因此我要移动旧的 CameraCaptureSession to a ReprocessableCaptureSession. I did it and it is working fine, but with this new feature I can use the CameraDevice.TEMPLATE_ZERO_SHUTTER_LAG template in my device and I can reprocess frames with the reprocessCaptureRequest。
这是我的问题出现的地方。因为我没有找到任何示例，而且我不太了解有关如何使用 reprocessCaptureRequest:
Each reprocess CaptureRequest processes one buffer from CameraCaptureSession's input Surface to all output Surfaces included in the reprocess capture request. The reprocess input images must be generated from one or multiple output images captured from the same camera device. The application can provide input images to camera device via queueInputImage(Image). The application must use the capture result of one of those output images to create a reprocess capture request so that the camera device can use the information to achieve optimal reprocess image quality. For camera devices that support only 1 output Surface, submitting a reprocess CaptureRequest with multiple output targets will result in a CaptureFailure.
我试图查看 google.sources 中关于相机的 CTS 测试，但他们做的和我一样。使用多个imageReader，将TotalCaptureResult张图片保存在一张LinkedBlockingQueue<TotalCaptureResult>中。稍后只需调用：
TotalCaptureResult totalCaptureResult = state.captureCallback.getTotalCaptureResult();
CaptureRequest.Builder reprocessCaptureRequest = cameraStore.state().cameraDevice.createReprocessCaptureRequest(totalCaptureResult);
reprocessCaptureRequest.addTarget(state.yuvImageReader.getSurface());
sessionStore.state().session.capture(reprocessCaptureRequest.build(), null, this.handlers.bg());
但它总是抛出一个 RuntimeException：
java.lang.RuntimeException: Capture failed: Reason 0 in frame 170,
我只想知道使用 ReprocessableCaptureSession 的正确方法是什么，因为我已经尝试了所有方法，但我不知道我做错了什么。
我终于找到了使 reprocessableCaptureSession 正常工作的解决方案。
我使用的是 Flux 架构，所以当您看到 Dispatcher.dispatch(action) 时不要混淆，只需将其视为回调即可。所以，这是我的代码：
首先会话是如何创建的：
//Configure preview surface
Size previewSize = previewState.previewSize;
previewState.previewTexture.setDefaultBufferSize(previewSize.getWidth(), previewSize.getHeight());
ArrayList<Surface> targets = new ArrayList<>();
for (SessionOutputTarget outputTarget : state.outputTargets) {
Surface surface = outputTarget.getSurface();
if (surface != null) targets.add(surface);
}
targets.add(previewState.previewSurface);
CameraCharacteristics cameraCharacteristics = cameraStore.state().availableCameras.get(cameraStore.state().selectedCamera);
Size size = CameraCharacteristicsUtil.getYuvOutputSizes(cameraCharacteristics).get(0);
InputConfiguration inputConfiguration = new InputConfiguration(size.getWidth(),
size.getHeight(), ImageFormat.YUV_420_888);
CameraCaptureSession.StateCallback sessionStateCallback = new CameraCaptureSession.StateCallback() {
@Override
public void onConfigured(@NonNull CameraCaptureSession session) {
if (sessionId != currentSessionId) {
Timber.e("Session opened for an old open request, skipping. Current %d, Request %d", currentSessionId, sessionId);
//performClose(session);
return;
}
try {
session.getInputSurface();
//This call is irrelevant,
//however session might have closed and this will throw an IllegalStateException.
//This happens if another camera app (or this one in another PID) takes control
//of the camera while its opening
} catch (IllegalStateException e) {
Timber.e("Another process took control of the camera while creating the session, aborting!");
}
Dispatcher.dispatchOnUi(new SessionOpenedAction(session));
}
@Override
public void onConfigureFailed(@NonNull CameraCaptureSession session) {
if (sessionId != currentSessionId) {
Timber.e("Configure failed for an old open request, skipping. Current %d, request %d", currentSessionId, sessionId);
return;
}
Timber.e("Failed to configure the session");
Dispatcher.dispatchOnUi(new SessionFailedAction(session, new IllegalStateException("onConfigureFailed")));
}
};
if (state.outputMode == OutputMode.PHOTO) {
cameraState.cameraDevice.createReprocessableCaptureSession(inputConfiguration, targets, sessionStateCallback, handlers.bg());
} else if (state.outputMode == OutputMode.VIDEO) {
cameraState.cameraDevice.createCaptureSession(targets, sessionStateCallback, handlers.bg());
}
} catch (IllegalStateException | IllegalArgumentException e) {
Timber.e(e, "Something went wrong trying to start the session");
} catch (CameraAccessException e) {
//Camera will throw CameraAccessException if another we try to open / close the
//session very fast.
Timber.e("Failed to access camera, it was closed");
}
使用 4 个表面（预览、YUV（输入）、JPEG 和 RAW）创建的照片会话。之后，我配置我的 imageWriter:
Dispatcher.subscribe(Dispatcher.VERY_HIGH_PRIORITY, SessionOpenedAction.class)
.filter(a -> isInPhotoMode())
.subscribe(action -> {
PhotoState newState = new PhotoState(state());
newState.zslImageWriter = ImageWriter.newInstance(action.session.getInputSurface(), MAX_REPROCESS_IMAGES);
setState(newState);
});
好的，现在我们已经创建了 ImageWriter 和会话。不，我们使用重复请求开始流式传输：
CaptureRequest.Builder captureRequestBuilder =
cameraStore.state().cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_ZERO_SHUTTER_LAG);
captureRequestBuilder.addTarget(previewStore.state().previewSurface);
captureRequestBuilder.addTarget(photoStore.state().yuvImageReader.getSurface());
state.session.setRepeatingRequest(captureRequestBuilder.build(), state.zslCaptureCallback, handlers.bg());
为了不添加很多代码，只说 zslCaptureCallback 是一个自定义回调，它保存在 LinkedBlockingQueue<TotalCaptureRequest> X 最后一个 TotalCaptureRequests 中。此外，我对 yuvImageReader（输入一个）执行相同操作，将最后 X 个图像保存在队列中。
最后是我的 "take photo" 方法：
try {
//Retrieve the last image stored by the zslImageReader
Image image = zslImageReaderListener.getImage();
//Retrieve the last totalCaptureResult from the zslCaptureCallback and create a reprocessableCaptureRequest with it
TotalCaptureResult captureResult = sessionStore.state().zslCaptureCallback.getCaptureResult(image.getTimestamp());
CaptureRequest.Builder captureRequest = cameraStore.state().cameraDevice.createReprocessCaptureRequest(captureResult);
//Add the desired target and values to the captureRequest
captureRequest.addTarget(state().jpegImageReader.getSurface());
//Queued back to ImageWriter for future consumption.
state.zslImageWriter.queueInputImage(image);
//Drain all the unused and queued CapturedResult from the CaptureCallback
sessionStore.state().zslCaptureCallback.drain();
//Capture the desired frame
CaptureRequest futureCaptureResult = captureRequest.build();
sessionStore.state().session.capture(futureCaptureResult, new CameraCaptureSession.CaptureCallback() {
@Override
public void onCaptureCompleted(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull TotalCaptureResult result) {
Dispatcher.dispatchOnUi(new PhotoStatusChangedAction(PhotoState.Status.SUCCESS));
}
@Override
public void onCaptureFailed(@NonNull CameraCaptureSession session,
@NonNull CaptureRequest request,
@NonNull CaptureFailure failure) {
super.onCaptureFailed(session, request, failure);
Exception captureFailedException = new RuntimeException(
String.format("Capture failed: Reason %s in frame %d, was image captured? -> %s",
failure.getReason(),
failure.getFrameNumber(),
failure.wasImageCaptured()));
Timber.e(captureFailedException, "Cannot take mediaType, capture failed!");
Dispatcher.dispatchOnUi(new PhotoStatusChangedAction(PhotoState.Status.ERROR, captureFailedException));
}
}, this.handlers.bg());
//Capture did not blow up, we are taking the photo now.
newState.status = PhotoState.Status.TAKING;
} catch (CameraAccessException | InterruptedException| IllegalStateException | IllegalArgumentException | SecurityException e) {
Timber.e(e, "Cannot take picture, capture error!");
newState.status = PhotoState.Status.ERROR;
}