Reconstructing head models from photographs for individualized 3D-audio processing
Dellepiane M, Pietroni N, Tsingos N, Asselot M, Scopigno R
[ INFO.INFO-GR ] Computer Science [cs]/Graphics [cs.GR] [INFO.INFO-GR]Computer Science [cs]/Graphics [cs.GR] HRTF calculation Sound scattering Computer Graphics and Computer-Aided Design
Visual fidelity and interactivity are the main goals in Computer Graphics research, but recently also audio is assuming an important role. Binaural rendering can provide extremely pleasing and realistic three-dimensional sound, but to achieve best results it's necessary either to measure or to estimate individual Head Related Transfer Function (HRTF). This function is strictly related to the peculiar features of ears and face of the listener. Recent sound scattering simulation techniques can calculate HRTF starting from an accurate 3D model of a human head. Hence, the use of binaural rendering on large scale (i.e. video games, entertainment) could depend on the possibility to produce a sufficiently accurate 3D model of a human head, starting from the smallest possible input. In this paper we present a completely automatic system, which produces a 3D model of a head starting from simple input data (five photos and some key-points indicated by user). The geometry is generated by extracting information from images and accordingly deforming a 3D dummy to reproduce user head features. The system proves to be fast, automatic, robust and reliable: geometric validation and preliminary assessments show that it can be accurate enough for HRTF calculation.
Source: COMPUTER GRAPHICS FORUM (PRINT), vol. 27 (issue 7), pp. 1719-1727
[Beg94] BEGAULT D. R.: 3D Sound for Virtual Reality and Multimedia. Academic Press Professional, 1994.
[Bla97] BLAUERT J.: Spatial Hearing : The Psychophysics of Human Sound Localization. M.I.T. Press, Cambridge, MA, 1997.
[Bla06] BLANZ V.: Face recognition based on a 3d morphable model. In FGR '06: Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (Washington, DC, USA, 2006), IEEE Computer Society, pp. 617-624.
[BSS07] BLANZ V., SCHERBAUM K., SEIDEL H.-P.: Fitting a morphable model to 3d scans of faces. In IEEE ICCV 2007 (2007), pp. 1-8.
[BV99] BLANZ V., VETTER T.: A morphable model for the synthesis of 3D faces. In Siggraph 1999, Computer Graphics Proceedings (Los Angeles, 1999), Rockwood A., (Ed.), Addison Wesley Longman, pp. 187-194.
[CB05] CHEN H., BHANU B.: Contour matching for 3d ear recognition. In WACV-MOTION '05: Volume 1 (Washington, DC, USA, 2005), pp. 123-128.
[CB07] CHEN H., BHANU B.: Human ear recognition in 3d. Pattern Analysis and Machine Intelligence, IEEE Transactions on 29, 4 (April 2007), 718-737.
[CCCS08] CALLIERI M., CIGNONI P., CORSINI M., SCOPIGNO R.: Masked photo blending: mapping dense photographic dataset on high-resolution sampled 3d models. Computers and Graphics (2008), (under publication).
[CRS98] CIGNONI P., ROCCHINI C., SCOPIGNO R.: Metro: measuring error on simplified surfaces. Computer Graphics Forum 17, 2 (June 1998), 167-174.
[D'A01] D'APUZZO N.: Human face modeling frommulti images. In Proc. of 3rd Int. Image Sensing Seminar on New Dev. in Digital Photogrammetry, Gifu, Japan (2001), pp. 28-29.
[DAA99] DUDA R. O., AVENDANO C., ALGAZI V. R.: An adaptable ellipsoidal head model for the interaural time difference. In Proc. IEEE (ICASSP) (1999), pp. II:965-968.
[FOT04] FUJIMURA K., OUE Y., TERAUCHI T.: Improved 3d head reconstruction system based on combining shape-from-silhouette with two-stage stereo algorithm. In ICPR '04: Volume 3 (Washington, DC, USA, 2004), pp. 127-130.
[Gar05] GARDNER W.: Spatial audio reproduction: Towards individualized binaural sound. National Academy of Engineering (2005).
[HA04] HASSANPOUR R., ATALAY V.: Delaunay triangulation based 3d human face modeling from uncalibrated images. Computer Vision and Pattern Rec. Workshop (2004), 75-75.
[Ian89] IANNARELLI A.: Ear identification. Paramount Publishing Company, Freemont, California (1989).
[JHY∗05] JIANG D., HU Y., YAN S., ZHANG L., ZHANG H., GAO W.: Efficient 3d reconstruction for face recognition. J. of Pattern Recogn. 38, 6 (June 2005), 787-798.
[JM06] JEGES E., MATE L.: Model-based human ear identification. World Automation Congress, 2006. WAC '06 (24-26 July 2006), 1-6.
[Kat01] KATZ B.: Boundary element method calculation of individual head-related transfer function. part I: Rigid model calculation. Journal Acoustical Soc. Am. 110, 5 (2001), 2440-2448.
[KB07] KATZ B., BEGAULT D.: Round robin comparison of HRTF measurement systems: preliminary results. In Proc. 19th Intl. Congress on Acoustics (ICA2007), Madrid, Spain (2007).
[KN06] KAHANA Y., NELSON P.: Numerical modelling of the spatial acoustic response of the human pinna. Journal of Sound and Vibration 292, 1-2 (2006), 148-178.
[Lar01] LARCHER V.: Techniques de spatialisation des sons pour la réalité virtuelle. Thèse de doctorat, Université Paris 6 (Pierre et Marie Curie), Paris, 2001.
[LLY04] LEE T.-Y., L IN P.-H., Y ANG T.-H.: Photorealistic 3d head modeling using multi-view images. In ICCSA (2) (2004), pp. 713-720.
[LMT98] LEE W.-S., M AGNENAT-T HALMANN N.: Head modeling from pictures and morphing in 3d with image metamorphosis based on triangulation. In CAPTECH (1998), pp. 254-267.
[MMO00] MIDDLEBROOKS J., MACPHERSON E., ONSAN Z.: Psychophysical customization of directional transfer functions for virtual sound localization. Journal Acoustical Soc. Am. 108, 6 (2000), 3088-3091.
Metrics
Back to previous page
@article{oai:it.cnr:prodotti:44190,
title = {Reconstructing head models from photographs for individualized 3D-audio processing},
author = {Dellepiane M and Pietroni N and Tsingos N and Asselot M and Scopigno R},
doi = {10.1111/j.1467-8659.2008.01316.x},
year = {2008}
}
0000-0003-1815-4756
