ARCamera.projectPoint swift 实施

Question

我正在尝试使用相机的 projectionMatrix 和 viewMatrix 实现 ARCamera.projectPoint 功能。

打算创建这样的东西：

let position = simd_float3(x: 1, y: 2, z: 3)
let position4 = simd_float4(x: position.x, y: position.y, z: position.z, w: 1)

let projectionMatrix = frame.camera.projectionMatrix(for: .landscapeRight, viewportSize: frame.camera.imageResolution, zNear: 0.001, zFar: 1000.0)
let viewMatrix = frame.camera.viewMatrix(for: .landscapeRight)

let projection = position4 * projectionMatrix * viewMatrix
let arkitProjection = frame.camera.projectPoint(position, orientation: .landscapeRight, viewportSize: frame.camera.imageResolution)

assert(projection.x == Float(arkitProjection.x))
assert(projection.y == Float(arkitProjection.y))

但我不知道如何正确实现它。希望对你有所帮助。

Answer 1

你们非常亲密！你在 projection 中拥有的实际上是一个“剪辑 space” 位置（另请注意，我固定了矩阵乘法的顺序）。然后，您需要使用“透视划分”将其转换为“规范化设备坐标”（在 xyz 维度中具有从 -1 到 +1 的坐标）。然后最后你需要根据图像大小将它线性映射到图像中的位置。

let position = simd_float3(x: 1, y: 2, z: 3)
let position4 = simd_float4(x: position.x, y: position.y, z: position.z, w: 1)

let projectionMatrix = frame.camera.projectionMatrix(for: .landscapeRight, viewportSize: frame.camera.imageResolution, zNear: 0.001, zFar: 1000.0)
let viewMatrix = frame.camera.viewMatrix(for: .landscapeRight)

let clipSpacePosition = projectionMatrix * viewMatrix * position4
let normalizedDeviceCoordinate = clipSpacePosition / clipSpacePosition.w
let projection = CGPoint(
    x: (CGFloat(normalizedDeviceCoordinate.x) + 1) * (frame.camera.imageResolution.width / CGFloat(2)),
    y: ((-CGFloat(normalizedDeviceCoordinate.y) + 1)) * (frame.camera.imageResolution.height / CGFloat(2))
)

let arkitProjection = frame.camera.projectPoint(position, orientation: .landscapeRight, viewportSize: frame.camera.imageResolution)

assert(abs(projection.x - arkitProjection.x) < 0.01)
assert(abs(projection.y - arkitProjection.y) < 0.01)

进一步阅读：

http://www.opengl-tutorial.org/beginners-tutorials/tutorial-3-matrices/#the-projection-matrix（关于剪辑 space 和投影的一般有用信息）
https://developer.apple.com/documentation/arkit/tracking_and_visualizing_faces（搜索“透视鸿沟”）
https://learnopengl.com/Getting-started/Coordinate-Systems

ARCamera.projectPoint swift 实施

ARCamera.projectPoint swift implementation

render

linear-algebra

ios

swift

arkit