Skip to main content

Projects

Near Light Correction for Image Relighting and 3D Shape Recovery

About this Project

nearlightestimatedpos.jpg3D Light Position Estimation: A comparison of 3D light position estimation using triangulation from multiple mirror balls (left) and the method proposed in this paper (right). In the left figure, the light positions of 81 dome lights obtained by least square error triangulation from five mirror spheres placed in the center, top left, top right, bottom left, bottom right of the scene. In the right figure, the light positions of the same setup are estimated from just a piece of white matte printing paper. The light positions from the mirror balls are subject to large triangulation errors for lights near the top of the dome. Our technique, however, produces highly accurate estimates of 3D light position. Dataset and Matlab Source Code.

The Method

We propose a near-light illumination model for image relighting and 3D shape recovery. We correct the near-light distance effect to provide much more uniformly lit images that yield a more appealing image for relighting applications. Furthermore, we use our near-light model for more accurate photometric stereo calculations of surface normals, eliminating the “potato-chip” shaped surface reconstruction error that results from violating the far-light assumption. We verify our model with both free-form capture using hand-held flash as the illumination source and capture using LED lights mounted on a dome-shaped surface.

Hardware

We used two types of capture. Left shows free-form capture using hand-held flash as the illumination source. The right image shows capture using LED lights mounted on a dome shaped surface.

Comparision of Image Relighting without and with Near Light Correction

Relighting comparisons for two works by Paul Gauguin housed at the Art Institute of Chicago, a woodblock (top), and a transfer print (bottom). The concession number for the woodblock
is 1940-91, and 2002-237 for the print. A comparison is shown between the raw captured images (left) and after the near-light correction technique introduced in this paper(right). We use the calibrated light position to compute the light attenuation due to the distance squared fall-off. The inverse of this attenuation mask (middle) is used to produce relit images with even illumination (right). The corrected images look uniformly lit and more visually pleasing.

"POTATO-CHIP" ERROR OF RECONSTRUCTED SHAPE DUE TO WRONG LIGHT MODEL

This figure shows the typical "Potato-chip" shaped error resulted from modeling near lights as far lights, which typically occur in the Reflectance Transfer Image (RTI). In this example, the object is 50cm wide, smooth and uniformly flat, and the light is 150cm above the object. The center of the object has an incident illumination angle of 90
degrees, but the left and right border of the central scan line have incident illumination angles of 80.5 degrees and 99.5 degrees respectively, with an error range of 19=2atan(25/150)
degrees from left to right. The erroneous lighting angles cause mis-estimated surface normals in RTI, which causes the ``potato-chip'' shaped surface.

Comparision of Surface Reconstruction without and with Near Light Correction I

nearlightshapepaper.jpg

Experimental comparison between far-light and near-light models for surface normal and 3d shape reconstruction. We use a flat matte paper as a ground truth test object. The first column shows the captured and corrected images. The position of the red dot in the circle approximates lighting direction. The second column shows the surface normal. The ground truth should be uniform, but the normal from the distant light model has a large error especially at the border. The third column shows the reconstructed 3d surface. The last column shows a horizontal scan line of the recovered depth map. The near-light model results are close to the ground truth, while the far-light model results in a large potato chipped distortion.

Comparision of Surface Reconstruction without and with Near Light Correction II:

nearlightshapewoodblock.jpg

Comparison of far-light and near-light models for 3d shape reconstruction. The work of art shown is a woodblock produced by artist Paul Gauguin, housed at the Art Institute of Chicago. The concession number of the artwork is 2238-56. (a) One of the captured images showing the near-light illumination effect. (b) A 3D reconstruction without correcting for the near-light effect. The woodblock appears to be bent like a ``potato chip'' so surface details are difficult to resolve. (c) A 3D reconstruction using our method to correct for the near-light effect. The woodblock is now flat and details of the carving can be discerned from the reconstruction.

Learn more about the project

Read the Project Publication

View All Projects

Back to top