Hardware
(a). two polarizers with opposite polarization are attached to the camera and the flash light of the mobile device, respectively; (b). photo of our prototype Mobile SfS with an iPhone 6 and a custom
3D printed widget.
About this Project
Our shape-from-shifting technique uses a mobile device camera to capture images around the object with the built-in flash used as a source of illumination. SIFT-based image registration renders the images to the same viewpoint but each with a unique illumination direction. The synthesized images are further processed by uncalibrated photometric stereo to acquire dense surface normal vector maps.
The Method
Surface shape scanning techniques, such as laser scanning and photometric stereo, are widespread analytical tools used in the field of cultural heritage. Compared to regular 2D RGB photos, 3D surface scans provide higher fidelity of an object’s surface shape which assist conservators, art historians, and archaeologists in understanding how these artworks and artifacts are made and to digitally document them for purposes of conservation. However, current state-of-the-art 3D surface scanning tools used in art conservation are often expensive and bulky- such as light dome structures that are often over 1 m in diameter. In this paper, we introduce mobile shape-from-shifting (SfS): a simple, low-cost and streamlined photometric stereo framework for scanning planar surfaces with a consumer mobile device coupled to a low-cost add-on component. Our free-form mobile SfS framework relaxes the rigorous hardware and other complex requirements inherent to conventional 3D scanning tools. This is achieved by taking a sequence of photos with the on-board camera and flash of a mobile device. The sequence of captures are used to reconstruct high quality normal maps using near- light photometric stereo algorithms, which are of comparable quality to conventional photometric stereo. We demonstrate 3D surface reconstructions with SfS on different materials and scales. Moreover, the mobile SfS technique can be used ”in the wild” so that 3D scans may be performed in their natural environment, eliminating the need for transport to a laboratory setting. With the elegant design and low cost, we believe our Mobile SfS can greatly benefit the conservation community by providing a user-friendly and cost-effective solution for 3D surface scanning.
Samples for evaluation with Mobile SfS
A replica (a) of an Aztec calendar stone (b); a portion of the wall of the Bah´a’´ı Temple, Evanston, Il (c); and a page of an illuminated manuscript from the Isabella Stewart Gardner Museum (d).
Acquisition procedure and image pre-processing
Images are taken at several positions over the object with slight lateral and vertical shifts of the phone. Typically no fewer than nine images are taken for the surface normal reconstruction. Images are registered with the SIFT function and cropped to display the same region.
Surface normals measured from a plaster replica of an Aztec calendar stone
(a). the photo of the imaged sample; (b). the surface normal generated with near-light PS using light dome; (c). the surface normal generated using mobile SfS but without polarizers; (d). the surface normal reconstructed with the proposed mobile SfS. Close-up images show the ”nose” on the plate. Scale bar: 5 cm.
Surface normals measured from architecture
(a). photo of the region of Bah´a’´ı Temple which is been scanned; (b). the surface normal generated with mobile SfS; (c). The rendering with surface normal generated with mobile SfS. Close-up shows the detailed feature on the stone. Scale bar: 10 cm.