2016
Journal article  Open Access

Relative Scale Estimation and 3D Registration of Multi-Modal Geometry Using Growing Least Squares

Mellado N., Dellepiane M., Scopigno R.

[INFO.INFO-CG]Computer Science [cs]/Computational Geometry [cs.CG]  Computer Vision and Pattern Recognition  Multi-modal data  Software  [INFO.INFO-CV]Computer Science [cs]/Computer Vision and Pattern Recognition [cs.CV]  Computer Graphics and Computer-Aided Design  3D registration  Multi-scale descriptors  Signal Processing 

The advent of low cost scanning devices and the improvement of multi-view stereo techniques have made the acquisition of 3D geometry ubiquitous. Data gathered from different devices, however, result in large variations in detail, scale, and coverage. Registration of such data is essential before visualizing, comparing and archiving them. However, state-of-the-art methods for geometry registration cannot be directly applied due to intrinsic differences between the models, e.g., sampling, scale, noise. In this paper we present a method for the automatic registration of multi-modal geometric data, i.e., acquired by devices with different properties (e.g., resolution, noise, data scaling). The method uses a descriptor based on Growing Least Squares, and is robust to noise, variation in sampling density, details, and enables scale-invariant matching. It allows not only the measurement of the similarity between the geometry surrounding two points, but also the estimation of their relative scale. As it is computed locally, it can be used to analyze large point clouds composed of millions of points. We implemented our approach in two registration procedures (assisted and automatic) and applied them successfully on a number of synthetic and real cases. We show that using our method, multi-modal models can be automatically registered, regardless of their differences in noise, detail, scale, and unknown relative coverage.

Source: IEEE transactions on visualization and computer graphics 22 (2016): 2160–2173. doi:10.1109/TVCG.2015.2505287

Publisher: IEEE Computer Society,, New York, NY , Stati Uniti d'America


[1] S. Du, N. Zheng, L. Xiong, S. Ying, and J. Xue, “Scaling iterative closest point algorithm for registration of m-d point sets,” J. Vis. Comun. Image Repr., vol. 21, no. 5-6, pp. 442-452, Jul. 2010.
[2] K. Parmar and R. Kher, “A comparative analysis of multimodality medical image fusion methods,” in Modelling Symp. (AMS), May 2012, pp. 93-97.
[3] M. Bhattacharya and A. Das, “Multimodality medical image registration and fusion techniques using mutual information and genetic algorithm-based approaches,” in Software Tools and Algorithms for Biological Systems, 2011, vol. 696, pp. 441-449.
[4] I. Reducindo, E. Arce-Santana, D. Campos-Delgado, and A. Alba, “Evaluation of multimodal medical image registration based on particle filter,” in Electrical Engineering Computing Science and Automatic Control, Sept 2010, pp. 406-411.
[5] N. Mellado, G. Guennebaud, P. Barla, P. Reuter, and C. Schlick, “Growing least squares for the analysis of manifolds in scalespace,” Comp. Graph. Forum, vol. 31, no. 5, Aug. 2012.
[6] J. Salvi, C. Matabosch, D. Fofi, and J. Forest, “A review of recent range image registration methods with accuracy evaluation,” Image Vision Comput., vol. 25, no. 5, pp. 578-596, May 2007.
[7] O. van Kaick, H. Zhang, G. Hamarneh, and D. Cohen-Or, “A survey on shape correspondence,” Computer Graphics Forum, vol. 30, no. 6, pp. 1681-1707, 2011.
[8] G. Tam, Z.-Q. Cheng, Y.-K. Lai, F. Langbein, Y. Liu, D. Marshall, R. Martin, X.-F. Sun, and P. Rosin, “Registration of 3d point clouds and meshes: A survey from rigid to nonrigid,” Visualization and Computer Graphics, IEEE Trans. on, vol. 19, no. 7, 2013.
[9] H. Pottmann, S. Leopoldseder, and M. Hofer, “Registration without icp,” Comput. Vis. Image Underst., vol. 95, no. 1, 2004.
[10] S. Rusinkiewicz and M. Levoy, “Efficient variants of the ICP algorithm,” in Third International Conference on 3D Digital Imaging and Modeling (3DIM), Jun. 2001.
[11] N. J. Mitra, N. Gelfand, H. Pottmann, and L. Guibas, “Registration of point cloud data from a geometric optimization perspective,” in Proc. of the Symp. on Geometry Processing, 2004.
[12] S. Bouaziz, A. Tagliasacchi, and M. Pauly, “Sparse iterative closest point,” Proc. of the Symp. on Geometry Processing, vol. 32, no. 5, pp. 1-11, 2013.
[13] P. Heider, A. Pierre-Pierre, R. Li, and C. Grimm, “Local shape descriptors, a survey and evaluation,” in Proc. of EG 3DOR 2011, Aire-la-Ville, Switzerland, Switzerland, 2011, pp. 49-56.
[14] A. Johnson, “Spin-images: A representation for 3-d surface matching,” Ph.D. dissertation, Robotics Institute, Carnegie Mellon University, August 1997.
[15] E. Kalogerakis, D. Nowrouzezahrai, P. Simari, and K. Singh, “Extracting lines of curvature from noisy point clouds,” Comput. Aided Des., vol. 41, no. 4, pp. 282-292, Apr. 2009.
[16] R. B. Rusu, N. Blodow, and M. Beetz, “Fast point feature histograms (fpfh) for 3d registration,” in Robotics and Automation, 2009. ICRA'09. IEEE, 2009, pp. 3212-3217.
[17] X. Li and I. Guskov, “Multi-scale features for approximate alignment of point-based surfaces,” in Proc. of the Symp. on Geometry Processing. Eurographics Association, 2005.
[18] L. Skelly and S. Sclaroff, “Improved feature descriptors for 3-D surface matching,” in Proc. SPIE Conf. on Two- and 3-Dimensional Methods for Inspection and Metrology, 2007.
[19] N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, “Robust global registration,” in Proc. of the Symp. on Geometry Processing, 2005.
[20] A. Makadia, A. I. Patterson, and K. Daniilidis, “Fully automatic registration of 3d point clouds,” in Proc. of the CVPR 2006. Washington, DC, USA: IEEE Computer Society, 2006, pp. 1297-1304.
[21] H. Pottmann, Q.-X. Huang, Y.-L. Yang, and S.-M. Hu, “Geometry and convergence analysis of algorithms for registration of 3d shapes,” Int. J. Comput. Vision, vol. 67, no. 3, pp. 277-296, 2006.
[22] S. Krishnan, P. Y. Lee, J. B. Moore, and S. Venkatasubramanian, “Global registration of multiple 3d point sets via optimizationon-a-manifold.” in Symposium on Geometry Processing, 2005, pp. 187-196.
[23] F. Bonarrigo and A. Signoroni, “An enhanced 'optimization-on-amanifold' framework for global registration of 3d range data,” in Proc. of 3DIM/PVT 2011. IEEE Computer Society, 2011.
[24] D. Aiger, N. J. Mitra, and D. Cohen-Or, “4-points congruent sets for robust pairwise surface registration,” ACM Trans. Graph., vol. 27, no. 3, pp. 85:1-85:10, Aug. 2008.
[25] N. Mellado, D. Aiger, and N. J. Mitra, “Super 4pcs fast global pointcloud registration via smart indexing,” Computer Graphics Forum, vol. 33, no. 5, pp. 205-215, 2014.
[26] E. Rodola', A. Albarelli, F. Bergamasco, and A. Torsello, “A scale independent selection process for 3d object recognition in cluttered scenes,” Int. Journal of Computer Vision, vol. 102, no. 1-3, 2013.
[27] M. Corsini, M. Dellepiane, F. Ganovelli, R. Gherardi, A. Fusiello, and R. Scopigno, “Fully automatic registration of image sets on approximate geometry,” International Journal of Computer Vision, vol. 102, no. 1-3, pp. 91-111, 2013.
[28] B. Lin, T. Tamaki, F. Zhao, B. Raytchev, K. Kaneda, and K. Ichii, “Scale alignment of 3d point clouds with different scales,” Machine Vision and Applications, vol. 25, no. 8, pp. 1989-2002, 2014.
[29] R. Pintus, E. Gobbetti, and R. Combet, “Fast and robust semiautomatic registration of photographs to 3d geometry,” in Proc. of VAST 2011, Aire-la-Ville, Switzerland, Switzerland, 2011, pp. 9-16.
[30] C. Wu, B. Clipp, X. Li, J.-M. Frahm, and M. Pollefeys, “3d model matching with viewpoint-invariant patches (vip),” in Computer Vision and Pattern Recognition, CVPR 2008, 2008.
[31] H. Kim and A. Hilton, “Evaluation of 3d feature descriptors for multi-modal data registration,” in Proc. of 3DV, International Conference on 3D Vision, June 2013.
[32] S. Lee, M. Park, and K. Lee, “Full 3d surface reconstruction of partial scan data with noise and different levels of scale,” Journal of Mechanical Science and Technology, vol. 28, no. 8, 2014.
[33] L. Quan and K. Tang, “Polynomial local shape descriptor on interest points for 3d part-in-whole matching,” Computer-Aided Design, vol. 59, no. 0, 2015.
[34] D. Cohen-Steiner, P. Alliez, and M. Desbrun, “Variational shape approximation,” ACM Tr. on Graph., vol. 23, pp. 905-914, Aug. 2004.
[35] R. Raguram and J.-M. Frahm, “Recon: Scale-adaptive robust estimation via residual consensus,” Computer Vision, IEEE International Conference on, vol. 0, pp. 1299-1306, 2011.
[36] C. Wang, M. M. Bronstein, A. M. Bronstein, and N. Paragios, “Discrete minimum distortion correspondence problems for nonrigid shape matching,” in Proc. of SSVM, Berlin, 2012.
[37] D. Raviv, A. M. Bronstein, M. M. Bronstein, R. Kimmel, and N. Sochen, “Affine-invariant geodesic geometry of deformable 3d shapes,” Computers & Graphics, vol. 35, no. 3, 2011.
[38] R. M. Rustamov, “Laplace-beltrami eigenfunctions for deformation invariant shape representation,” in Proc. of the Symp. on Geometry Processing, 2007, pp. 225-233.
[39] A. Dubrovina and R. Kimmel, “Matching shapes by eigendecomposition of the laplace-beltrami operator,” in Proc. 3DPVT, vol. 2, no. 3, 2010.
[40] J. Sun, M. Ovsjanikov, and L. Guibas, “A concise and provably informative multi-scale signature based on heat diffusion,” in Proc. of the Symp. on Geometry Processing, 2009.
[41] M. M. Bronstein and I. Kokkinos, “Scale-invariant heat kernel signatures for non-rigid shape recognition.” in CVPR, 2010, pp. 1704-1711.
[42] K. Crane, C. Weischedel, and M. Wardetzky, “Geodesics in heat: A new approach to computing distance based on heat flow,” ACM Trans. Graph., vol. 32, no. 5, pp. 152:1-152:11, Oct. 2013.
[43] G. Patane´ and M. Spagnuolo, “Heat diffusion kernel and distance on surface meshes and point sets,” Computers & Graphics, vol. 37, no. 6, 2013.
[44] H. Fadaifard, G. Wolberg, and R. Haralick, “Multiscale 3d feature extraction and matching with an application to 3d face recognition,” Graphical Models, vol. 75, no. 4, pp. 157 - 176, 2013.
[45] B. Romeny, Front-End Vision and Multi-Scale Image Analysis: Multiscale Computer Vision Theory and Applications, written in Mathematica, 1st ed. Springer Publishing, 2009.
[46] D. G. Lowe, “Object recognition from local scale-invariant features,” in Proc. of the International Conference on Computer Vision. Washington, DC, USA: IEEE Computer Society, 1999.
[47] P. Scovanner, S. Ali, and M. Shah, “A 3-dimensional sift descriptor and its application to action recognition,” in Proc. of the 15th International Conference on Multimedia, 2007, pp. 357-360.
[48] G. Flitton, T. Breckon, and N. Megherbi Bouallagu, “Object recognition using 3d sift in complex ct volumes,” in Proc. of BMVS 2010, 2010, pp. 11.1-11.12.
[49] A. Zaharescu, E. Boyer, K. Varanasi, and R. Horaud, “Surface feature detection and description with applications to mesh matching,” in CVPR 2009, 2009.
[50] C. Maes, T. Fabry, J. Keustermans, D. Smeets, P. Suetens, and D. Vandermeulen, “Feature detection on 3d face surfaces for pose normalisation and recognition,” in Biometrics: Theory Applications and Systems, 2010., 2010.
[51] T. Darom and Y. Keller, “Scale-invariant features for 3-d mesh models,” IEEE Trans. on Image Processing, vol. 21, 2012.
[52] G. Guennebaud and M. Gross, “Algebraic point set surfaces,” ACM Trans. Graph., vol. 26, no. 3, Jul. 2007.
[53] M. Berger, J. A. Levine, L. G. Nonato, G. Taubin, and C. T. Silva, “A benchmark for surface reconstruction,” ACM Trans. Graph., vol. 32, no. 2, pp. 20:1-20:17, Apr. 2013.
[54] M. Corsini, P. Cignoni, and R. Scopigno, “Efficient and flexible sampling with blue noise properties of triangular meshes,” Trans. on Visualization and Computer Graphics, vol. 18, no. 6, 2012.
[55] N. Mellado, G. Ciaudo, G. Guennebaud, and P. Barla, “Patate lib,” http://patate.gforge.inria.fr/, 2013.
[56] P. Cignoni, M. Callieri, M. Corsini, M. Dellepiane, F. Ganovelli, and G. Ranzuglia, “Meshlab: an open-source mesh processing tool,” in Sixth Eurographics Italian Chapter Conference, 2008, p. 129.
[57] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381-395, 1981.
[58] A. Albarelli, E. Rodola, and A. Torsello, “Loosely distinctive features for robust surface alignment,” in Computer Vision-ECCV 2010. Springer, 2010, pp. 519-532.
[59] A. Johnson and M. Hebert, “Using spin images for efficient object recognition in cluttered 3d scenes,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 21, no. 5, May 1999.
[60] T. Tamaki, S. Tanigawa, Y. Ueno, B. Raytchev, and K. Kaneda, “Scale matching of 3d point clouds by finding keyscales with spin images,” in International Conference on Pattern Recognition, 2010.
[61] P. Besl and N. D. McKay, “A method for registration of 3-d shapes,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 14, no. 2, pp. 239-256, Feb 1992.

Metrics



Back to previous page
BibTeX entry
@article{oai:it.cnr:prodotti:358987,
	title = {Relative Scale Estimation and 3D Registration of Multi-Modal Geometry Using Growing Least Squares},
	author = {Mellado N. and Dellepiane M. and Scopigno R.},
	publisher = {IEEE Computer Society,, New York, NY , Stati Uniti d'America},
	doi = {10.1109/tvcg.2015.2505287},
	journal = {IEEE transactions on visualization and computer graphics},
	volume = {22},
	pages = {2160–2173},
	year = {2016}
}

HARVEST4D
Harvesting Dynamic 3D Worlds from Commodity Sensor Clouds

SMARTGEOMETRY
Structure-Aware Geometry Processing


OpenAIRE