Digital Photography with Flash and
No-Flash Image
Pairs
A report by Gabriela Martínez December 2005.
This project is based on the
paper "Digital Photography with
Flash and No-Flash Image Pairs" . Petschnigg et al.ACM. 2004
RED EYES
DETECTION MASK
Red eye is a common problem in flash photography and is due to light
reflection by a well vascularized retina. The classical approach to
solve this problem is assume a single image as input. Once the pupil
mask has been detected the techniques darken the pixels within the mask
to make the image appear more natural.
ALGORITHM
They have developed a red eye
removal algorithm that considers the change in pupil color between the
ambient image (where it is usually very dark) and the flash image
(where it may be red). They convert the image pair into YCbCr space to
decorrelate luminance from chrominance and compute a relative redness
measure as follows:
- Calculate: R=FCr-ACr
,
where F is the flash image and A is the ambient image.
- Segmentation starts with: R > teye .
They set teye
to 0.05 so that the resulting segmentation defines regions where flash
image is redder than ambient image.
- Look
for seed pixels: R > max[0.6 mR
+ 3sigmaR]
and Al
< tdark
IMPLEMENTATION
Using Matlab:
- Convert the space color image using rgb2ycbcr command<>
- Create detection mask (step 1): mask = flash_ycbcr(:,:,3) -
noflash_ycbcr(:,:,3); mask = mask - min(mask(:)); mask = mask /
max(max(mask));
- Find seed
pixels and mark them (step 2,3): TLower = max([ 0.4 mean(mask(:)) +
var(mask(:)) ^ 0.5]); low = mask > TLower; TUpper = min([0.8
mean(mask(:)) + 6 * var(mask(:)) ^ 0.5]); high = mask > TUpper; [r,
c] = find(high); low = bwlabel(low, 8); final_c = bwselect(low,c,r,8);
- Red eyes mask result:
RESULTS
DENOISING AND DETAIL
TRANSFER
Reducing noise in photographic images has been a long standing problem
in image processing and computer vision. One common solution is to
apply an edge-preserving smoothing filter to the image such as
bilateral filter (Tomasi and Manduchi 1998).
The image denoising technique presented by the authors consist in
builds on the bilateral filter. Let's summarize Tomasi and Manduchi's
basic bilateral filter and then show the generalization done.
BILATERAL FILTER. The basic
idea underlying bilateral filter is to do in the range of an image what
traditional filters do in its domain. Two pixels can be close to one another, that is,
occupy nearby spatial location, or they can be similar to one another, that is,
have nearby values, possibly in perceptually meaningful fashion.
It combines a classic low pass filter with an edge stopping function
that attenuates the filter kernel weight, when the intensity difference
between pixels is large.
The bilateral filter computes the values of pixels p for ambient image A as:
Ap=(1/k(p))sum(gd(p'-p)gr(Ap-Ap')Ap')
where k(p) is normalization term: k(p)=sum(gd(p'-p)gr(Ap-Ap'))
The functions gd,
gr
are Gaussian with standard deviation sigmad
(geometric closeness) and sigmar
(photometric range) respectively.
JOINT BILATERAL FILTER. Flash
image contains a much better estimate of the true high frequency
information than the ambient image. Based on this observation the
authors modify bilateral filter to compute the edge stopping function gr
using the flash image F
instead of A.
Apn=(1/k(p))sum(gd(p'-p)gr(Fp-Fp')Ap')
where the constant k(p) is
modified similarly.
ALGORITHM
- Bilateral filter, compute: A_base,
F_base.
- Joint Bilateral filter, compute: A_nr.
- Compute F_detail=(F+0.02)/(F_base+0.02).
- Transfer detail to A_nr: A_nr*F_detail.
- Create shadows mask M : diff(F,A) < tshadows
- Result: (1-M)A_nrF_detail+MA_base
RESULTS
- Other results click here
- Power Point Presentation here
REFERENCES
- PETSCHNIGG., G., AGRAWALA., M., HOPPE., H., SZELISKI., R.,
COHEN., M., TOYOMA., K., "Digital
Photography with Flash and No-Flash Image Pairs". ACM. 2004.
- GONZALEZ, R., WOODS, R., AND EDDINS, S., "Digital Image Processing: Using MATLAB"
Editorial Prentice Hall.
- PETSCHNIGG., G., Vision Project.
http://graphics.standford.edu/~georgp/vision.html
- TOMASI., C., AND MANDUCHI., R., "Bilateral
Filtering for Gray and Color Images". Proceedings of the 1998
IEEE. International Conference on Computer Vision, Bombay-India.