YAIS

Yet Another Image Stitcher

Fundamentals and Trends in Vision and Image Processing - Final Project

Eric Biagioli


Description

YAIS is a portable OpenCV-based image stitcher. It can be installed and used on computers and (in brief) on mobile devices (android and iOS). The goal of YAIS is to be a useful tool to create panoramas using mobile devices.

Downloads

Some results

Building in Singapore

Somewhere in Singapore

Botafogo Praia Shopping

How do I see the world

Brief walk through the algorithm

Suppose we have the following images, and we want to create a 2x3 panorama using the selected images.

Step 1. Features detection

Here is used a SIFT-based algorithm to detect features. This work uses OpenCV's implementation of the SURF algorithm.

2592 x 1944 894 x 671
3416 features
2592 x 1944 894 x 671
3128 features
2592 x 1944 894 x 671
3694 features
2592 x 1944 894 x 671
3855 features
2592 x 1944 894 x 671
3666 features
2592 x 1944 894 x 671
3593 features

Step 2: (a) Match feaures. (b) Find geometrically consistent matches using RANSAC. (c) Find the homography and the confidence.

BestOf2NearestMatcher (A FLANN-based algorithm) (FLANN means Fast Library for Approximate Nearest Neighbors) is OpenCV's default implementation for features matching. The default threshold for the confidence, in order to consider two images connected, is 1.

Originals Features Matched Features Aligned using homography
717 matched features
512 ransac inliers
Confidence: 2.29494
361 matched features
302 ransac inliers
Confidence: 2.59673
234 matched features
157 ransac inliers
Confidence: 2.00767
532 matched features
411 ransac inliers
Confidence: 2.45227
819 matched features
665 ransac inliers
Confidence: 2.62121
560 matched features
408 ransac inliers
Confidence: 2.31818
145 matched features
92 ransac inliers
Confidence: 1.78641
400 matched features
291 ransac inliers
Confidence: 2.27344
744 matched features
494 ransac inliers
Confidence: 2.13668
Confidence too small. Cannot match images!
41 matched features
8 ransac inliers
Confidence: 0.394089
137 matched features
68 ransac inliers
Confidence: 1.38493
373 matched features
275 ransac inliers
Confidence: 2.29358
667 matched features
478 ransac inliers
Confidence: 2.29697
232 matched features
157 ransac inliers
Confidence: 2.0232
663 matched features
498 ransac inliers
Confidence: 2.40696

We can align every image to the same dest image.


7765 x 3624

Step 3: Find Connected Components of image matches

Consider all images as nodes, and consider that there exists an arc between two images if and only if the confidence computed in the previous step is equal to or higher that 1.

Step 4: Bundle Adjustment & Wave Correction

In order to compute all cammera rotations and focal lengths, the implementation uses Bundle Adjustment (Triggs et al., 1999).

One problem that we still need to adress is thw following wavy effect:

In order to correct it, we can observe that if the cammera is not twisted with respect to the horizon, then the $X$’s vectors (i.e.: directions of the horizontal axis) of all images lies in a plane.

Then, consider the vector $u$ such that $$\left(\sum_{i=1}^{n}X_iX_i^t\right)u = 0$$ $u$ is normal to the plane that contains the cammera centre and the horizon.

Rotating in a way such that the vector $u$ is vertical removes the shown wavy effect.

Step 5: Warp Images

The warping is selected among the many OpenCV built-in warpers

Step 6: Exposure Compensation

In this step, we use OpenCV's implementation of the Gain Compansation algorithm.

Step 7: Blending

OpenCV implements feather blending and multiband blending. We use in this work the multiband blending.