OpenCV warpPerspective的问题

Question

我正在开发一个Android背景减法项目，使用移动摄像头。我尝试使用特征匹配，findHomography和warpPerspective来查找两个帧之间的重叠像素。但是，我得到的输出稍微不正确。我对图像处理相当陌生，所以我不熟悉所有的术语。我有两个主要问题：

1）warpPerspective的结果过度失真 - 例如，图像歪斜，图像中的物体被翻转，挤压等。我该如何解决这个问题？

2）我有时会得到'OpenCV Error：As thesis failed'错误，这会导致我的应用程序崩溃。此错误映射到warpPerspective。注意：image1（前一帧）和image2（当前帧）中的尺寸相同。我在检测功能之前将图像转换为灰色（当前来自RGB）。我有时会在findHomography上得到类似的'OpenCV assertion failed'错误，但我知道它至少需要4分 - 所以添加if语句解决了它，但不知道如何用warpPerspective解决错误。

的错误，我得到：

02-24 15:30:49.554: E/cv::error()(4589): OpenCV Error: Assertion failed (type == src2.type() && src1.cols == src2.cols && (type == CV_32F || type == CV_8U)) 
    in void cv::batchDistance(cv::InputArray, cv::InputArray, cv::OutputArray, int, cv::OutputArray, int, int, cv::InputArray, int, bool), 
    file /home/reports/ci/slave_desktop/50-SDK/opencv/modules/core/src/stat.cpp, line 2473 

我的代码：

void stitchFrames(){ 

    //convert frames to grayscale 
    image1 = prevFrame.clone(); 
    image2 = currFrame.clone(); 

    if(colourSpace==1){ //convert from RGB to gray 
     cv::cvtColor(image1, image1Gray,CV_RGB2GRAY); 
     cv::cvtColor(image2, image2Gray,CV_RGB2GRAY); 
    } 
    else if(colourSpace==2){ //convert from HSV to gray 
     cv::cvtColor(image1, image1Gray,CV_HSV2RGB); 
     cv::cvtColor(image1Gray,image1Gray,CV_RGB2GRAY); 
     cv::cvtColor(image2, image1Gray,CV_HSV2RGB); 
     cv::cvtColor(image2Gray,image1Gray,CV_RGB2GRAY); 
    } 

    else if(colourSpace==3){ //no need for conversion 
     image1Gray = image1; 
     image2Gray = image2; 
    } 

    //----FEATURE DETECTION---- 

    //key points 
    std::vector<KeyPoint> keypoints1, keypoints2; 

    int minHessian; 

    cv::FastFeatureDetector detector; 

    detector.detect(image1Gray,keypoints1); //prevFrame 
    detector.detect(image2Gray,keypoints2); //currFrame 

    KeyPoint kp = keypoints2[4]; 
    Point2f p = kp.pt; 
    float i = p.y; 

    //---FEATURE EXTRACTION---- 

    //extracted descriptors 
    cv::Mat descriptors1,descriptors2; 

    OrbDescriptorExtractor extractor; 
    extractor.compute(image1,keypoints1,descriptors1); //prevFrame 
    extractor.compute(image2,keypoints2,descriptors2); //currFrame 

    //----FEATURE MATCHING---- 

    //BruteForceMacher 

    BFMatcher matcher; 

    std::vector<cv::DMatch> matches; //result of matching descriptors 
    std::vector<cv::DMatch> goodMatches; //result of sifting matches to get only 'good' matches 

    matcher.match(descriptors1,descriptors2,matches); 

    //----HOMOGRAPY - WARP-PERSPECTIVE - PERSPECTIVE-TRANSFORM---- 

    double maxDist = 0.0; //keep track of max distance from the matches 
    double minDist = 80.0; //keep track of min distance from the matches 

    //calculate max & min distances between keypoints 
    for(int i=0; i<descriptors1.rows;i++){ 
     DMatch match = matches[i]; 

     float dist = match.distance; 
     if (dist<minDist) minDist = dist; 
     if(dist>maxDist) maxDist=dist; 
    } 

    //get only the good matches 
    for(int i = 0; i < descriptors1.rows; i++){ 
     DMatch match = matches[i]; 
     if(match.distance< 500){ 
      goodMatches.push_back(match); 
     } 
    } 

    std::vector<Point2f> obj; 
    std::vector<Point2f> scene; 

    //get the keypoints from the good matches 
    for(int i = 0; i < goodMatches.size(); i++){ 

     //--keypoints from image1 
     DMatch match1 = goodMatches[i]; 
     int qI1 = match1.trainIdx; 
     KeyPoint kp1 = keypoints2[qI1]; 
     Point2f point1 = kp1.pt; 
     obj.push_back(point1); 

     //--keypoints from image2 
     DMatch match2 = goodMatches[i]; 
     int qI2 = match2.queryIdx; 
     KeyPoint kp2 = keypoints1[qI2]; 
     Point2f point2 = kp2.pt; 
     scene.push_back(point2); 

    } 

    //calculate the homography matrix 
    if(goodMatches.size() >=4){ 
     Mat H = findHomography(obj,scene, CV_RANSAC); 

     warpPerspective(image2,warpResult,H,Size(image1.cols,image1.rows)); 
    } 
} 

Answer 1

关于你的第一个问题，我觉得你提的失真是由于这样的事实：

• 您估计的单恋H映射图像1坐标图像2.坐标当你Mat H = findHomography(obj,scene, CV_RANSAC);，obj在图像点坐标1和scene是像点坐标2.

• 您然后在功能warpPerspective使用H就好像贴图坐标在图像2到图像1中的坐标，因为您期望它将image2转换为warpResult，我猜测它应该被拼接到image1。

因此，你应该估计单应H如下：Mat H = findHomography(scene, obj, CV_RANSAC);。

关于你的第二个问题，我认为它是由该指令提出：

matcher.match(descriptors1,descriptors2,matches); 

错误说，表达

(type == src2.type() && src1.cols == src2.cols && (type == CV_32F || type == CV_8U)) 

被发现是假的，而应该是真正的功能工作。类似的问题也解决了here：调用match功能之前，你需要手动检查是否符合下列条件：

(descriptors1.type()==descriptors2.type() && descriptors1.cols==descriptors2.cols) 

Answer 2

关于（1），我的猜测是，你估计的单应是基于糟糕的比赛。

首先，我将首先使用ORB检测器代替FAST，然后更改findHomographyransacReprojThreshold参数。默认值是3，细节：

ransacReprojThreshold：

Maximum allowed reprojection error to treat a point pair as an inlier (used in the RANSAC method only). That is, if:

| dstPoints_i - convertPointsHomogeneous（H * srcPoints_i）| > ransacReprojThreshold

then the point i is considered an outlier. If srcPoints and dstPoints are measured in pixels, it usually makes sense to set this parameter somewhere in the range of 1 to 10.

换句话说，假设默认3个像素，如果将所述单应性的srcPoint后，其对dstPoint距离超过3个像素，即一对被认为是内点（即：好） 。

这仅仅是个开始，它也将帮助你找到好的比赛，好的单应更好的过滤器，你会发现关于这些几个答案：

OpenCV Orb not finding matches..

How can you tell if a homography matrix is acceptable or not?