Menu:

 

IMAGE-BASED MODELLING FROM UNCALIBRATED IMAGES

Researches:   Miguel Sainz, Antonio Susin

Description:

Image Based Modeling and Rendering (IBMR) is a fairly new field of research and experimentation with the goal of reducing the complexity of use and the execution time of the modeling of photorealistic 3D graphics.

A

Fig 1. Complete pipeline: from data to the final model.

These new techniques can be used to develop methods and tools for acquiring and modeling objects, using pictures and/or video sequences as source data. In this work we present a novel pipeline to reconstruct full 3D photorealistic model reconstruction of real world objects from a set of images.

A
Fig 2. Different input images of a monster model
 
The reconstruction process begins with the acquisition of an unrestricted video walkthrough inside or around the scene to be modeled. Robust digital image processing  and computer vision techniques allow the use of low cost acquisition systems such as photo or video cameras.

After acquiring and digitizing the frames, a tracking algorithm extracts significant 2D information of the video sequence. Then, a calibration process automatically calculates the 3D position of the tracked features, the 3D position and orientation of cameras and their internal parameters, such as the focal length. We have successfully developed novel algorithms that perform these calculations robust and computationally efficiently.

A
Fig 3. Camera parameters recovered from the images.

We perform a spatial carving operation on a piece of virtual material containing the scene using a voxel carving methodology highly optimized to use the computation power of the actual videocards.

 

A

A

Fig. 4. Carved voxelized model (left) and the smoothing version (left)

Once this carving is finished, a modeling process obtains a representation suitable to be rendered. We propose three different representations which are a trade-off of different factors with the goal of achieving fast rendering: the final application might require from the 3D model representation several characteristics such as support for level of details (LODs), collision detection, interactive illumination and photorealistic quality. The proposed pipeline targets applications in areas such as Virtual Reality, Computer Gaming and Special Effects. All these environments require high quality 3D models for increased realism. Our approach allows obtaining such data directly from real world objects using images in a fast and automatic way.


A

A

A

Fig. 5. Voxelized model after the carving process.

A

Fig. 6. Facial Model recovered from he images


Publications:

Sainz M.
Image-Based Modelling from Uncalibrated Images PhD Dissertation Electrical and Computer Engineering Dep. University of California, Irvine. 16/06/2003

 


Example image - aligned to the right

Sainz M., Pajarola R., Susin A. and Mercade A., SPOC: Simple Point-Based Object Capturing.
IEEE Journal of Computer Graphics and Applications (CG&A), vol. 24 (4), pp.24--33,2004.


Example image - aligned to the right

Sainz M., Pajarola R. and Susin A., Photorealistic Image Based Objects from Uncalibrated Images.
In Posters of IEEE Visualization Conference (VIS'03), (PDF 294Kb)


Example image - aligned to the right

Sainz M., Susin A. and Bagherzadeh N., MTMesh: Image Based Mesh Reconstruction and Rendering
IAESTED Conference in Visualization, Imaging and Image Processing (VIIP'03), ed. M.H. Hamza, pp 785-790 (2003). (PDF 4.3Mb)


Example image - aligned to the right

Sainz M., Susin A., Cervantes, A. and Bagherzadeh N., Persepolis: Recovering history with a handheld camera. In Posters of Eurographics 2003 (EG'03).(PDF 6.6Mb)


Example image - aligned to the right

Sainz M., Susin A. and Bagherzadeh N., Camera Calibration of Long Image Sequences with the Presence of Occlusions. In Proceedings of the International Conference in Image Processing 2003 (ICIP'03), vol I, pp 317-320 (2003). (PDF 273Kb)


Example image - aligned to the right

Sainz M., Bagherzadeh N. and Susin A.,  Carving 3D Models from Uncalibrated Views. 
5th IASTED International Conference Computer Graphics and Imaging (CGIM 2002)August 12-14, 2002 Kauai, Hawaii, USA pp 144-149. (PDF 1.54Mb)


Example image - aligned to the right

Sainz M., Bagherzadeh N. and Susin A.,  Hardware Accelerated Voxel Carving. 
Proc. 1st Iberoamerican Symposium in Computer Graphics (SIACG 2002), M.P. dos Santos, L. Velho, X. Pueyo edit.pp. 289-297 (2002) Guimarães, Portugal  (PDF 3.0Mb)


Example image - aligned to the right

Sainz M., Bagherzadeh N. and Susin A.,  Recovering 3D Metric Structure and Motion from Multiple Uncalibrated Cameras.  IEEE Proc. International Conference on Information Technology: Coding and Computingpp 268-273, Las Vegas, 2002. (PDF 419Kb


References:

[1] Berthilsson R., Heyden A. and Sparr G., Recursive structure and motion from image sequences using shape and depthspaces. Proc. IEEE Computer Vision and Pattern Recognition, 444–449, 1999.
[2] Broadhurst A. and Cipolla R., A statistical consistency check for the space carving algorithm. Proc. British Machine Vision Conference I, 282–291, 2000.
[3] Broadhurst A., Drummond T.W. and Cipolla R., A probabilistic framework for space carving. International Conference on Computer Vision I, 282–291, 2001.
[4] Chen Q., Multi-view Image-Based Rendering and Modeling. PhD. Dissertation, Computer Science USC, 1–141, 2000.
[5] Chen Q. and Medioni G., Efficient, iterative solution to Multiview projective reconstruction problem. Proc. IEEE Computer Vision and Pattern Recognition 1, 55–61, 1999.
[6] CulbertsonW.B., Malzbender T. and Slabaugh G., Generalized Voxel Coloring. Proc. of Vision Algorithms Theory and Practice Workshop, 100–114, Corfu, Greece, 1999.
[7] Golub G.H., Van Loan C.F. Matrix Computations. Third Edition, Ed. Johns Hopkins, 1996.
[8] Fusiello A., Uncalibrated Euclidean reconstruction: a review. Image and Vision Computing, 18, 555–563, 2000.
[9] Heyden A., Berthilsson R. and Sparr G., An iterative factorization method for projective structure and motion from image sequences. Image Vision and Computing, 17, 981–991, 1999.
[10] Han M. and Kanade T., Creating 3D Models with Uncalibrated Cameras IEEE Computer Society Workshop on the Application of Computer Vision (WACV2000), 9(2), 137–154, 2000.
[11] Jacobs D., Linear fitting with missing data for structure-from-motion Computing Visison and Image Understanding , 82, 57–81, 2001.
[12] Kutulakos K. and Seitz S., A theory of shape by space carving. International Journal of Computer Vision 38(3), 198–218, 2000.
[13] Mahamud S. and Hebert M., Iterative projective reconstruction from multiple views. motion IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’00),2, 430–437, 2000. 2000.
[14] Poelman C.J. and Kanade T., A paraperspective factorization method for shape and motion recovery Technical Report CMU-CS 93-219, School of Computer Science, Carnegie Mellon University , December 1993.
[15] Pollefeys M., 3D Modeling From Images SIGGRAPH 2000 Course Notes, July 2000.
[16] Seitz S. and Dyer C., Photorealistic scene reconstruction by voxel coloring. International Journal of Computer Vision 35(2), 1067–1073, 1999.
[17] Sturm P., Critical motion sequences for monocular selfcalibration and uncalibrated euclidean reconstruction. Proc. IEEE Computer Vision and Pattern Recognition, 1100–
1105, 1997.
[18] Tomasi C. and Kanade T., Shape and Motion from Image Streams Under Orthography: a factorization method International Journal of Computer Vision, 9(2), 137–154, 1992.
[19] Triggs B., Factorization methods for projective structure and motion IEEE CVPR96, 845–851, 1996.
[20] Triggs B., Autocalibration and the Absolute Quadric IEEE CVPR97, 609–614, 1997.


Toni Susin