在 warpAffine 变换后如何将点重新映射或恢复到其以前的坐标系？
How to remap or revert a point into its former coordinate system after warpAffine has transformed it?
我正在使用 模板匹配 (TM) 来查找图片中所有 M 的位置（第一张图片向左），但我无法将匹配点的位置（指的是旋转 ROI 内的位置）重新映射回原始图像：
问题是我需要在这一点上反转（撤消）warpAffine 转换，而我的计算并不完美，如上图最右侧所示用橙色框。
我已经查看了 SO 中与该主题相关的所有帖子，但 none 确实有帮助，因为我要反转的操作稍微复杂一些：
- Center of rotated cv::Rect
简单来说，这个应用程序是做什么的？
- 它首先加载图像： original image and the template;
- 它创建了 8 个具有所需旋转角度的 ROI。稍后使用旋转角度来校正 M 的方向，使其保持水平，并且“看起来很漂亮”对于 TM；
- 循环迭代列表中的每个 ROI：选择一个 ROI，使用
rotate_bound() 旋转它，然后对其执行 TM。;
- 当 TM 操作成功并找到字母时，它会尝试重新映射定义匹配位置的点 从 旋转的 ROI 到 原始 ROI 中的坐标，然后可用于指定原始图像内匹配项的正确位置。
主要问题似乎是撤消 rotate_bound() 创建的旋转矩阵中定义的所有操作。顺便说一句，如果你从来没有听说过这个功能，这里有一个 good reference.
如何修复重映射计算？
这是一个Short, Self Contained, Correct (Compilable), Example:
import cv2
import numpy as np
# rotate_bound: helper function that rotates the image adds some padding to avoid cutting off parts of it
# reference: https://www.pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/
def rotate_bound(image, angle):
# grab the dimensions of the image and then determine the center
(h, w) = image.shape[:2]
(cX, cY) = (w // 2, h // 2)
# grab the rotation matrix (applying the negative of the angle to rotate clockwise), then grab the sine and cosine
# (i.e., the rotation components of the matrix)
M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
cos = np.abs(M[0, 0])
sin = np.abs(M[0, 1])
# compute the new bounding dimensions of the image
nW = int(np.multiply(h, sin) + np.multiply(w, cos))
nH = int(np.multiply(h, cos) + np.multiply(w, sin))
# adjust the rotation matrix to take into account translation
M[0, 2] += (nW / 2) - cX
M[1, 2] += (nH / 2) - cY
# perform rotation and return the image (white background) along with the Rotation Matrix
return cv2.warpAffine(image, M, (nW, nH), borderValue=(255,255,255)), M
# Step 1 - Load images
input_img = cv2.imread("target.png", cv2.IMREAD_GRAYSCALE)
template_img = cv2.imread("template.png", cv2.IMREAD_GRAYSCALE)
matches_dbg_img = cv2.cvtColor(input_img, cv2.COLOR_GRAY2BGR) # for debugging purposes
# Step 2 - Generate some ROIs
# each ROI contains the x,y,w,h and angle (degree) to rotate the box and make its M appear horizontal
roi_w = 26
roi_h = 26
roi_list = []
roi_list.append((112, 7, roi_w, roi_h, 0))
roi_list.append((192, 36, roi_w, roi_h, -45))
roi_list.append((227, 104, roi_w, roi_h, -90))
roi_list.append((195, 183, roi_w, roi_h, -135))
roi_list.append((118, 216, roi_w, roi_h, -180))
roi_list.append((49, 196, roi_w, roi_h, -225))
roi_list.append((10, 114, roi_w, roi_h, -270))
roi_list.append((36, 41, roi_w, roi_h, -315))
# debug: draw green ROIs
rois_dbg_img = cv2.cvtColor(input_img, cv2.COLOR_GRAY2BGR)
for roi in roi_list:
x, y, w, h, angle = roi
x2 = x + w
y2 = y + h
cv2.rectangle(rois_dbg_img, (x, y), (x2, y2), (0,255,0), 2)
cv2.imwrite('target_rois.png', rois_dbg_img)
cv2.imshow('ROIs', rois_dbg_img)
cv2.waitKey(0)
cv2.destroyWindow('ROIs')
# Step 3 - Select a ROI, crop and rotate it, then perform Template Matching
for i, roi in enumerate(roi_list):
x, y, w, h, angle = roi
roi_cropped = input_img[y:y+h, x:x+w]
roi_rotated, M = rotate_bound(roi_cropped, angle)
# debug: display each rotated ROI
#cv2.imshow('ROIs-cropped-rotated', roi_rotated)
#cv2.waitKey(0)
# debug: dump roi to the disk (before/after rotation)
filename = 'target_roi' + str(i)
cv2.imwrite(filename + '.png', roi_cropped)
cv2.imwrite(filename + '_rotated.png', roi_rotated)
# perform template matching
res = cv2.matchTemplate(roi_rotated, template_img, cv2.TM_CCOEFF_NORMED)
(_, score, _, (pos_x, pos_y)) = cv2.minMaxLoc(res)
print('TM score=', score)
# Step 4 - When a TM is found, revert the rotation of matched point so that it represents a location in the original image
# Note: pos_x and pos_y define the location of the matched template in a rotated ROI
threshold = 0.75
if (score >= threshold):
# debug in cropped image
print('find_k_symbol: FOUND pos_x=', pos_x, 'pos_y=', pos_y, 'w=', template_img.shape[1], 'h=', template_img.shape[0])
rot_output_roi = cv2.cvtColor(roi_rotated, cv2.COLOR_GRAY2BGR)
cv2.rectangle(rot_output_roi, (pos_x, pos_y), (pos_x + template_img.shape[1], pos_y + template_img.shape[0]), (0, 165, 255), 2) # orange
cv2.imshow('rot-matched-template', rot_output_roi)
cv2.waitKey(0)
cv2.destroyWindow('rot-matched-template')
###
# How to convert the location of the matched template (pos_x, pos_y) to points in roi_cropped?
# (which is the ROI before rotation)
###
# extract variables from the rotation matrix
M_x = M[0][2]
M_y = M[1][2]
#print('M_x=', M_x, '\tM_y=', M_y)
M_cosx = M[0][0]
M_msinx = M[0][1]
#print('M_cosx=', M_cosx, '\tM_msinx=', M_msinx)
M_siny = M[1][0]
M_cosy = M[1][1]
#print('M_siny=', M_siny, '\tM_cosy=', M_cosy)
# undo translation:
dst1_x = pos_x - M_x
dst1_y = pos_y - M_y
# undo rotation:
# after this operation, (new_pos_x, new_pos_y) should already be a valid point in the original ROI
new_pos_x = M_cosx * dst1_x - M_msinx * dst1_y
new_pos_y = -M_siny * dst1_x + M_cosy * dst1_y
# debug: create the bounding rect of the detected symbol in the original input image
detected_x = x + int(new_pos_x)
detected_y = y + int(new_pos_y)
detected_w = template_img.shape[1]
detected_h = template_img.shape[0]
detected_rect = (detected_x, detected_y, detected_w, detected_h)
print('find_k_symbol: detected_x=', detected_x, 'detected_y=', detected_y, 'detected_w=', detected_w, 'detected_h=', detected_h)
print()
cv2.rectangle(matches_dbg_img, (detected_x, detected_y), (detected_x + detected_w, detected_y + detected_h), (0, 165, 255), 2) # orange
cv2.imwrite('target_matches.png', matches_dbg_img)
cv2.imshow('matches', matches_dbg_img)
cv2.waitKey(0)
这里是 运行 应用程序所需的图像：original image and template image。
您几乎完成了 - 所缺少的只是将边界框矩形围绕其左上角旋转已知角度，然后绘制这个旋转后的矩形。
自 cv2.rectangle only draws up-right rectangles, we need some alternative. One option is to represent the rectangle as a list of its corner points (for consistency, let's say, in clockwise order, starting from top-left). We can then draw it as a closed polyline going through those 4 points, using cv2.polylines.
要旋转矩形，我们需要对其所有角点应用几何变换。为此，我们首先使用 cv2.getRotationMatrix2D.
获得一个变换矩阵
我们将角点转换为齐次坐标，并计算变换矩阵与坐标转置数组的点积。
为了方便（让每个点都在一行上）我们转置结果。
# Rotate rectangle defined by (x,y,w,h) around its top left corner (x,y) by given angle
def rotate_rectangle(x, y, w, h, angle):
# Generate homogenous coordinates of the corners
# Start top left, go clockwise
corners = np.array([
(x, y, 1)
, (x + w, y, 1)
, (x + w, y + h, 1)
, (x, y + h, 1)
], np.int32)
# Create rotation matrix to transform the coordinates
m_rot = cv2.getRotationMatrix2D((x, y), angle, 1.0)
# Apply transformation
rotated_points = np.dot(m_rot, corners.T).T
return rotated_points
现在，我们首先确定旋转边界框的角，而不是调用 cv2.rectangle：
rot_points = rotate_rectangle(detected_x, detected_y, detected_w, detected_h, angle)
由于cv2.polylines需要整数坐标，我们round the values and convert the datatype数组：
rot_points = np.round(rot_points).astype(np.int32)
最后通过4个角点绘制一条闭合折线：
cv2.polylines(matches_dbg_img, [rot_points], True, (0, 165, 255), 2)
我正在使用 模板匹配 (TM) 来查找图片中所有 M 的位置（第一张图片向左），但我无法将匹配点的位置（指的是旋转 ROI 内的位置）重新映射回原始图像：
问题是我需要在这一点上反转（撤消）warpAffine 转换，而我的计算并不完美，如上图最右侧所示用橙色框。
我已经查看了 SO 中与该主题相关的所有帖子，但 none 确实有帮助，因为我要反转的操作稍微复杂一些：
- Center of rotated cv::Rect
简单来说，这个应用程序是做什么的？
- 它首先加载图像： original image and the template;
- 它创建了 8 个具有所需旋转角度的 ROI。稍后使用旋转角度来校正 M 的方向，使其保持水平，并且“看起来很漂亮”对于 TM；
- 循环迭代列表中的每个 ROI：选择一个 ROI，使用
rotate_bound()旋转它，然后对其执行 TM。; - 当 TM 操作成功并找到字母时，它会尝试重新映射定义匹配位置的点 从 旋转的 ROI 到 原始 ROI 中的坐标，然后可用于指定原始图像内匹配项的正确位置。
主要问题似乎是撤消 rotate_bound() 创建的旋转矩阵中定义的所有操作。顺便说一句，如果你从来没有听说过这个功能，这里有一个 good reference.
如何修复重映射计算？
这是一个Short, Self Contained, Correct (Compilable), Example:
import cv2
import numpy as np
# rotate_bound: helper function that rotates the image adds some padding to avoid cutting off parts of it
# reference: https://www.pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/
def rotate_bound(image, angle):
# grab the dimensions of the image and then determine the center
(h, w) = image.shape[:2]
(cX, cY) = (w // 2, h // 2)
# grab the rotation matrix (applying the negative of the angle to rotate clockwise), then grab the sine and cosine
# (i.e., the rotation components of the matrix)
M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
cos = np.abs(M[0, 0])
sin = np.abs(M[0, 1])
# compute the new bounding dimensions of the image
nW = int(np.multiply(h, sin) + np.multiply(w, cos))
nH = int(np.multiply(h, cos) + np.multiply(w, sin))
# adjust the rotation matrix to take into account translation
M[0, 2] += (nW / 2) - cX
M[1, 2] += (nH / 2) - cY
# perform rotation and return the image (white background) along with the Rotation Matrix
return cv2.warpAffine(image, M, (nW, nH), borderValue=(255,255,255)), M
# Step 1 - Load images
input_img = cv2.imread("target.png", cv2.IMREAD_GRAYSCALE)
template_img = cv2.imread("template.png", cv2.IMREAD_GRAYSCALE)
matches_dbg_img = cv2.cvtColor(input_img, cv2.COLOR_GRAY2BGR) # for debugging purposes
# Step 2 - Generate some ROIs
# each ROI contains the x,y,w,h and angle (degree) to rotate the box and make its M appear horizontal
roi_w = 26
roi_h = 26
roi_list = []
roi_list.append((112, 7, roi_w, roi_h, 0))
roi_list.append((192, 36, roi_w, roi_h, -45))
roi_list.append((227, 104, roi_w, roi_h, -90))
roi_list.append((195, 183, roi_w, roi_h, -135))
roi_list.append((118, 216, roi_w, roi_h, -180))
roi_list.append((49, 196, roi_w, roi_h, -225))
roi_list.append((10, 114, roi_w, roi_h, -270))
roi_list.append((36, 41, roi_w, roi_h, -315))
# debug: draw green ROIs
rois_dbg_img = cv2.cvtColor(input_img, cv2.COLOR_GRAY2BGR)
for roi in roi_list:
x, y, w, h, angle = roi
x2 = x + w
y2 = y + h
cv2.rectangle(rois_dbg_img, (x, y), (x2, y2), (0,255,0), 2)
cv2.imwrite('target_rois.png', rois_dbg_img)
cv2.imshow('ROIs', rois_dbg_img)
cv2.waitKey(0)
cv2.destroyWindow('ROIs')
# Step 3 - Select a ROI, crop and rotate it, then perform Template Matching
for i, roi in enumerate(roi_list):
x, y, w, h, angle = roi
roi_cropped = input_img[y:y+h, x:x+w]
roi_rotated, M = rotate_bound(roi_cropped, angle)
# debug: display each rotated ROI
#cv2.imshow('ROIs-cropped-rotated', roi_rotated)
#cv2.waitKey(0)
# debug: dump roi to the disk (before/after rotation)
filename = 'target_roi' + str(i)
cv2.imwrite(filename + '.png', roi_cropped)
cv2.imwrite(filename + '_rotated.png', roi_rotated)
# perform template matching
res = cv2.matchTemplate(roi_rotated, template_img, cv2.TM_CCOEFF_NORMED)
(_, score, _, (pos_x, pos_y)) = cv2.minMaxLoc(res)
print('TM score=', score)
# Step 4 - When a TM is found, revert the rotation of matched point so that it represents a location in the original image
# Note: pos_x and pos_y define the location of the matched template in a rotated ROI
threshold = 0.75
if (score >= threshold):
# debug in cropped image
print('find_k_symbol: FOUND pos_x=', pos_x, 'pos_y=', pos_y, 'w=', template_img.shape[1], 'h=', template_img.shape[0])
rot_output_roi = cv2.cvtColor(roi_rotated, cv2.COLOR_GRAY2BGR)
cv2.rectangle(rot_output_roi, (pos_x, pos_y), (pos_x + template_img.shape[1], pos_y + template_img.shape[0]), (0, 165, 255), 2) # orange
cv2.imshow('rot-matched-template', rot_output_roi)
cv2.waitKey(0)
cv2.destroyWindow('rot-matched-template')
###
# How to convert the location of the matched template (pos_x, pos_y) to points in roi_cropped?
# (which is the ROI before rotation)
###
# extract variables from the rotation matrix
M_x = M[0][2]
M_y = M[1][2]
#print('M_x=', M_x, '\tM_y=', M_y)
M_cosx = M[0][0]
M_msinx = M[0][1]
#print('M_cosx=', M_cosx, '\tM_msinx=', M_msinx)
M_siny = M[1][0]
M_cosy = M[1][1]
#print('M_siny=', M_siny, '\tM_cosy=', M_cosy)
# undo translation:
dst1_x = pos_x - M_x
dst1_y = pos_y - M_y
# undo rotation:
# after this operation, (new_pos_x, new_pos_y) should already be a valid point in the original ROI
new_pos_x = M_cosx * dst1_x - M_msinx * dst1_y
new_pos_y = -M_siny * dst1_x + M_cosy * dst1_y
# debug: create the bounding rect of the detected symbol in the original input image
detected_x = x + int(new_pos_x)
detected_y = y + int(new_pos_y)
detected_w = template_img.shape[1]
detected_h = template_img.shape[0]
detected_rect = (detected_x, detected_y, detected_w, detected_h)
print('find_k_symbol: detected_x=', detected_x, 'detected_y=', detected_y, 'detected_w=', detected_w, 'detected_h=', detected_h)
print()
cv2.rectangle(matches_dbg_img, (detected_x, detected_y), (detected_x + detected_w, detected_y + detected_h), (0, 165, 255), 2) # orange
cv2.imwrite('target_matches.png', matches_dbg_img)
cv2.imshow('matches', matches_dbg_img)
cv2.waitKey(0)
这里是 运行 应用程序所需的图像：original image and template image。
您几乎完成了 - 所缺少的只是将边界框矩形围绕其左上角旋转已知角度，然后绘制这个旋转后的矩形。
自 cv2.rectangle only draws up-right rectangles, we need some alternative. One option is to represent the rectangle as a list of its corner points (for consistency, let's say, in clockwise order, starting from top-left). We can then draw it as a closed polyline going through those 4 points, using cv2.polylines.
要旋转矩形，我们需要对其所有角点应用几何变换。为此，我们首先使用 cv2.getRotationMatrix2D.
我们将角点转换为齐次坐标，并计算变换矩阵与坐标转置数组的点积。
为了方便（让每个点都在一行上）我们转置结果。
# Rotate rectangle defined by (x,y,w,h) around its top left corner (x,y) by given angle
def rotate_rectangle(x, y, w, h, angle):
# Generate homogenous coordinates of the corners
# Start top left, go clockwise
corners = np.array([
(x, y, 1)
, (x + w, y, 1)
, (x + w, y + h, 1)
, (x, y + h, 1)
], np.int32)
# Create rotation matrix to transform the coordinates
m_rot = cv2.getRotationMatrix2D((x, y), angle, 1.0)
# Apply transformation
rotated_points = np.dot(m_rot, corners.T).T
return rotated_points
现在，我们首先确定旋转边界框的角，而不是调用 cv2.rectangle：
rot_points = rotate_rectangle(detected_x, detected_y, detected_w, detected_h, angle)
由于cv2.polylines需要整数坐标，我们round the values and convert the datatype数组：
rot_points = np.round(rot_points).astype(np.int32)
最后通过4个角点绘制一条闭合折线：
cv2.polylines(matches_dbg_img, [rot_points], True, (0, 165, 255), 2)