2016
Journal article  Open Access

Wize Mirror - a smart, multisensory cardio-metabolic risk monitoring system

Andreu Y., Chiarugi F., Colantonio S., Giannakakis G., Giorgi D., Henriquez P., Kazantzaki E., Manousos D., Kostas M., Matuszewski B. J., Pascali M. A., Pediaditis M., Raccichini G., Tsiknakis M.

I440  Computer Vision and Pattern Recognition  Unobtrusive health monitoring  Software  3D face detection  3D morphometric analysis  Multimodal data integration  Psycho-somatic status recognition  Tracking and reconstruction  Signal Processing 

In the recent years personal health monitoring systems have been gaining popularity, both as a result of the pull from the general population, keen to improve well-being and early detection of possibly serious health conditions and the push from the industry eager to translate the current significant progress in computer vision and machine learning into commercial products. One of such systems is the Wize Mirror, built as a result of the FP7 funded SEMEOTICONS (SEMEiotic Oriented Technology for Individuals CardiOmetabolic risk self-assessmeNt and Self-monitoring) project. The project aims to translate the semeiotic code of the human face into computational descriptors and measures, automatically extracted from videos, multispectral images, and 3D scans of the face. The multisensory platform, being developed as the result of that project, in the form of a smart mirror, looks for signs related to cardio-metabolic risks. The goal is to enable users to self-monitor their well-being status over time and improve their life-style via tailored user guidance. This paper is focused on the description of the part of that system, utilising computer vision and machine learning techniques to perform 3D morphological analysis of the face and recognition of psycho-somatic status both linked with cardio-metabolic risks. The paper describes the concepts, methods and the developed implementations as well as reports on the results obtained on both real and synthetic datasets.

Source: Computer vision and image understanding (Print) 148 (2016): 3–22. doi:10.1016/j.cviu.2016.03.018

Publisher: Academic Press,, San Diego , Stati Uniti d'America


SEMEOTICONS FP7-ICT-2013-10 European project. 2013. URL http://www. semeoticons.eu/
Anxiety. 2015a. URL http://www.apa.org/topics/anxiety/
Anxiety disorders and effective treatment. 2015b. URL http://www.apa.org/ helpcenter/anxiety-treatment.aspx
Common signs and symptoms of stress - The American institute of stress. 2015c. URL http://www.stress.org/stress-effects/
Alarcón, G., Valentn, A. (Eds.), 2012, Introduction to Epilepsy. Cambridge University Press, Cambridge, United Kingdom.
Balakrishnan, G., Durand, F., Guttag, J., 2013. Detecting pulse from head motions in video. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR '13), pp. 3430-3437.
Besl, P.J., McKay, N.D., 1992. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14 (2), 239-256.
Biasotti, S., Falcidieno, B., Giorgi, D., Spagnuolo, M., 2014. Mathematical tools for shape analysis and description. Synth. Lect. Comput. Graph. Anim. 6 (2), 1-138.
Bleiweiss, A., Werman, M., 2010. Robust head pose estimation by fusion time-of-flight, depth and color. In: IEEE Automatic Face and Gesture Recognition, pp. 116-121.
Cai, Q., Gallup, D., Zhang, C., Zhang, Z., 2010. 3d deformable face tracking with a commodity depth camera. In: European Conference on Computer Vision, pp. 229-242.
Chiarugi, F., Iatraki, G., Christinaki, E., Manousos, D., Giannakakis, G., Pediaditis, M., Pampouchidou, A., Marias, K., Tsiknakis, M.N., 2014. Facial signs and psychophysical status estimation for well-being assessment. In: 7th IEEE International Conference on Health Informatics (BIOSTEC 2014), Angers, France, pp. 555-562.
Choi, J., Tran, A., Dumortier, Y., Medioni, G., 2014. Real-time 3-d face tracking and modeling framework for mid-res cam. In: IEEE Winter Conference on Applications of Computer Vision, pp. 660-667.
Christinaki, E., Giannakakis, G., Chiarugi, F., Pediaditis, M., Iatraki, G., Manousos, D., Marias, K., Tsiknakis, M., 2014. Comparison of blind source separation algorithms for optical heart rate monitoring. In: Wireless Mobile Communication and Healthcare (Mobihealth), 2014 EAI 4th International Conference on 3-5 Nov. 2014, pp. 339-342.
Cootes, T.F., Edwards, G.J., Taylor, C.J., 2001. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23 (6), 681-685.
Djordjevic, J., Lawlor, D.A., Zhurov, A.L., et al., 2013. A population-based cross-sectional study of the association between facial morphology and cardiometabolic risk factors in adolescence. In: BMJ Open, pp. 1-10.
Ekman, P., Friesen, W.V., 1971. Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17 (2), 124-129.
Fanelli, G., Weise, T., Gall, J., Van Gool, L., 2011. Real time head pose estimation from consumer depth cameras. In: Annual Symposium of the German Association for Pattern Recognition, 6835, pp. 101-110.
Farkas, L.G., 1994. Anthropometry of the Head and Face, 2nd ed. Raven Press, New York.
Farneback, G., 2003. Two-frame motion estimation based on polynomial expansion. In: The 13th Scandinavian conference on Image analysis (SCIA'03), Gteborg, Sweden, pp. 363-370.
Ferrario, V., Dellavia, C., Tartaglia, G., Turci, M., Sforza, C., 2004. Soft-tissue facial morphology in obese adolescents: a three-dimensional non invasive assessment. Angle Orthod. 74 (1).
Giachetti, A., Lovato, C., Piscitelli, F., Milanese, C., Zancanaro, C., 2015. Robust automatic measurement of 3d scanned models for human body fat estimation. IEEE J. Biomed. Health Inform. 19 (2), 660-667.
Gunes, H., Piccardi, M., 2007. Bi-modal emotion recognition from expressive face and body gestures. J. Netw. Comput. Appl. 30 (4), 1334-1345.
Haldiki, M., Batistakis, Y., Vazirgiannis, M., 2001. On clustering validation techniques. J. Intell. Inf. Syst. 17 (2-3), 107-145.
Hamilton, M., 1959. The assessment of anxiety-states by rating. Br. J. Med. Psychol. 32 (1), 50-55.
Hammond, P., 2007. The use of 3d face shape modelling in dismorphology. Arch. Dis. Child. 92, 1120-1126.
Harrigan, J.A., O'Conell, D., 1996. How do you look when feeling anxious? facial displays of anxiety. Pers. individ. Differences 21 (2), 205-212.
Henriquez, P., Higuera, O., Matuszewski, B.J., 2014. Head pose tracking for immersive applications. In: IEEE International Conference on Image Processing, pp. 1957-1961.
Hernandez, M., Choi, J., Medioni, G., 2015. Near laser-scan quality 3-d face reconstruction from a low-quality depth stream. Image Vis. Comput. 36, 61- 69.
Hojo, H., Hamada, N., 2009. Mouth motion analysis with space-time interest points. In: IEEE Region 10 Conference (TENCON 2009), Singapore, Singapore, pp. 1- 6.
Huang, X., Chen, X., Tang, T., Huang, Z., 2013. Marching cubes algorithm for fast 3d modeling human face by incremental data fusion. Math. probl. Eng. 2013, 1- 7.
Irani, R., Nasrollahi, K., Moeslund, T.B., 2014. Improved pulse detection from head motions using dct. In: 9th International Conference on Computer Vision Theory and Applications, pp. 118-124.
Kojovic, M., Cordivari, C., Bhatia, K., 2011. Myoclonic disorders: a practical approach for diagnosis and treatment. Ther. adv. neurol. disord. 4 (1), 47-62.
Koolhaas, J., Bartolomucci, A., Buwalda, B., de Boer, S.F., Flgge, G., Korte, S.M., Meerlo, P., Murison, R., Olivier, B., Palanza, P., Richter-Levin, G., Sgoifo, A., Steimer, T., Stiedl, O., van Dijk, G., Whr, M., Fuchs, E., 2010. Stress revisited: A critical evaluation of the stress concept. Neurosci. Biobehav. Rev. 35 (5), 1291-1301.
Lee, B.J., Do, J.H., Kim, J.K., 2012. A classification method of normal and overweight females based on facial features for automated medical applications. J Biomed. Biotechnol.
Lee, B.J., Kim, J.K., 2014. Predicting visceral obesity based on facial characteristics.. BMC Complement. Altern. Med. 14 (248).
Li, C., Ford, E.S., McGuire, L.C., Mokdad, A.H., 2007. Increasing trends in waist circumference and abdominal obesity among u.s. adults. Obesity 15, 216- 223.
Lin, J.D., Chiou, W.K., Weng, H.F., Fang, J.T., Liu, T.H., 2004. Application of three-dimensional body scanner: Observation of prevalence of metabolic syndrome. Clin. Nutr. 23 (6), 1313-1323.
Lucas, B.D., Kanade, T., 1981. An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th international joint conference on Artificial intelligence (IJCAI'81), pp. 674-679.
Macedo, M., Apolinario, A., Souza., A., 2013. Kinectfusion for faces: real-time 3d tracking and modeling using a kinect camera for a markerless ar system. SBC J. 3D Inter. Syst. 4 (2), 2-7.
Malassiotis, S., Strintzis, M., 2005. Robust real-time 3d head pose estimation from range data. Pattern Recognit. 38 (8), 1153-1165.
Manousos, D., Iatraki, G., Christinaki, E., Pediaditis, M., Chiarugi, F., Tsiknakis, M., Marias, K., 2014. Contactless detection of facial signs related to stress: A preliminary study. In: EAI 4th International Conference on Wireless Mobile Communication and Healthcare (Mobihealth 2014), Athens, Greece, pp. 335- 338.
Mase, K., Pentland, A., 1991. Automatic lipreading by optical-flow analysis. Syst. Comput. Jpn. 22 (6), 796-803.
Matthew, I., Baker, S., 2004. Active appearance models revisited. Int. J. Comput. Vis. 60 (2), 135-164.
Mou, X., Wang, A., 2012. A fast and robust head pose estimation system based on depth data. In: International Conference on Robotics and Biomimetics, pp. 470-475.
Murphy-Chutorian, E., Trivedi, M.M., 2009. Head pose estimation in computer vision: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 31 (4), 607-626.
Newcombe, R., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A., 2011. Kinectfusion: Real-time dense surface mapping and tracking. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 127-136.
Niles, A.N., Dour, H.J., Stanton, A.L., Roy-Byrne, P.P., Stein, M.B., Sullivan, G., Sherbourne, C.D., Rose, R.D., Craske, M.G., 2015. Anxiety and depressive symptoms and medical illness among adults with anxiety disorders. J. Psychosom. Res. 78 (2), 109-115.
Oliver, N., Pentland, A., Brard, F., 2000. Lafter: A real-time face and lips tracker with facial expression recognition. Pattern Recognit. 33 (8), 1369-1382.
Padeleris, P., Zabulis, X., Argyros, A., 2012. Head pose estimation on depth data based on particle swarm optimization. In: Computer Vision and Pattern Recognition Workshop (CVPRW), pp. 42-49.
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T., 2009. A 3d face model for pose and illumination invariant face recognition. In: IEEE Proc. of the 6th IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS) for Security, Safety and Monitoring in Smart Environments, Genova (Italy) - September 2-4, 2009, pp. 296-301.
Pediaditis, M., Giannakakis, G., Chiarugi, F., Manousos, D., Pampouchidou, A., Christinaki, E., Iatraki, G., Kazantzaki, E., Simos, P.G., Marias, K., Tsiknakis, M., 2015. Extraction of facial features as indicators of stress and anxiety. In: 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), Milano, Italy, pp. 3711-3714.
Quan, W., Matuszewski, B., Shark, L.-K., 2010. Improved 3-d facial representation through statistical shape model. In: IEEE International Conference on Image Processing, pp. 2433-2436.
Raytchev, B., Yoda, I., Katsuhiko, R., 2004. Head pose estimation by nonlinear manifold learning. In: IEEE International Conference on Pattern Recognition, pp. 462-466.
Reyment, R.A., 1996. An Idiosyncratic History of Early Morphometrics. In: Marcus, L.F., Corti, M., Loy, A., Naylor, G.J.P., Slice, D.E. (Eds.), Advances in Morphometrics. Springer, US, pp. 15-22.
Romero, L.M., 2004. Physiological stress in ecology: Lessons from biomedical research. Trend. Ecol. Evol. 19 (5), 249-255.
Sardinha, A., Nardi, A.E., 2012. The role of anxiety in metabolic syndrome. Expert Rev. Endocrinol. Metab. 7 (1), 63-71.
Seeman, E., Nickel, K., Stiefelhagen, R., 2004. Head pose estimation using stereo vision for human-robot interaction. In: IEEE Automatic Face and Gesture Recognition, pp. 626-631.
Selye, H., 1950. The Physiology and Pathology of Exposures to Stress. Montreal, Canada: Acta Endocrinologica.
Sharma, N., Gedeon, T., 2012. Objective measures, sensors and computational techniques for stress recognition and classification: A survey. Comput. Methods Programs Biomed. 108 (3), 1287-1301.
Shin, L.M., Liberzon, I., 1996. The neurocircuitry of fear, stress, and anxiety disorders. Neuropsychopharmacology 35 (1), 169-191.
Sierra-Johnson, J., Johnson, B.D., 2004. Facial fat and its relationship to abdominal fat: a marker for insulin resistance? Med. Hypotheses 63, 783-786.
Smeets, D., Keustermans, J., Vandermeulen, D., Suetens, P., 2013. meshsift: Local surface features for 3d face recognition under expression variations and partial data. Comput. Vis. Image Understanding 117 (2), 158-169.
Thejaswi, N.S., Sengupta, S., 2008. Lip localization and viseme recognition from video sequences. In: National Communications Conference (NCC), Mumbai, India.
Thompson, D.W., 1942. On Growth and Form. Cambridge University Press, Cambridge.
Velardo, C., Dugelay, J.-L., 2010. Weight estimation from visual body appearance. In: BTAS 2010, 4th IEEE International Conference on Biometrics: Theory, Applications and Systems, September 27-29, 2010, Washington DC, USA, pp. 1-6.
Velardo, C., Dugelay, J.-L., Paleari, M., Ariano, P., 2012. Building the space scale or how to weight a person with no gravity. In: ESPA 2012, IEEE 1st International Conference on Emerging Signal Processing Applications, January 12-14, 2012, Las Vegas, USA, pp. 67-70. http://dx.doi.org/10.1109/ESPA.2012.6152447.
Wang, J., Gallagher, D., Thornton, J.C., Yu, W., Horlick, M., Pi-Sunyer, F.X., 2006. Validation of a 3-dimensional photonic scanner for the measurement of body volumes, dimensions and percentage body fat.. Am. J. Clin. Nutr. 809-816.
Wells, J.C., Cole, T.J., Bruner, D., Treleaven, P., 2008. Body shape in american and british adults: between-country and inter-ethnic comparisons.. Int. J. Obes. 32 (1), 152-159.
Zollhofer, M., Martinek, M., Greiner, G., Stamminger, M., J., S., 2011. Automatic reconstruction of personalized avatars from 3d face scans. Comput. Anim. Virtual Worlds 22, 195-202.

Metrics



Back to previous page
BibTeX entry
@article{oai:it.cnr:prodotti:354429,
	title = {Wize Mirror - a smart, multisensory cardio-metabolic risk monitoring system},
	author = {Andreu Y. and Chiarugi F. and Colantonio S. and Giannakakis G. and Giorgi D. and Henriquez P. and Kazantzaki E. and Manousos D. and Kostas M. and Matuszewski B.  J. and Pascali M.  A. and Pediaditis M. and Raccichini G. and Tsiknakis M.},
	publisher = {Academic Press,, San Diego , Stati Uniti d'America},
	doi = {10.1016/j.cviu.2016.03.018},
	journal = {Computer vision and image understanding (Print)},
	volume = {148},
	pages = {3–22},
	year = {2016}
}

SEMEOTICONS
SEMEiotic Oriented Technology for Individual’s CardiOmetabolic risk self-assessmeNt and Self-monitoring


OpenAIRE