2019
Journal article  Open Access

Towards a topological-geometrical theory of group equivariant non-expansive operators for data analysis and machine learning

Bergomi M. G., Frosini P., Giorgi D., Quercioli N.

55N35 (Primary)  Machine Learning (stat.ML)  Mathematics - Algebraic Topology  topological data analysis  FOS: Mathematics  Computer Vision and Pattern Recognition  Computer Networks and Communications  68U05  Machine Learning  65D18 (Secondary)  Computer Vision and Pattern Recognition (cs.CV)  FOS: Computer and information sciences  Equivariant operators  57S10  Topological Data Analysis  persistent homology  47H09  natural pseudo-distance  Machine Learning (cs.LG)  Mathematics - Operator Algebras  Computer Science - Machine Learning  Statistics - Machine Learning  persistent topology  Artificial Intelligence  Shape classification  Operator Algebras (math.OA)  Algebraic Topology (math.AT)  Human-Computer Interaction  54H15  Group equivariant non-expansive operator  Software  Metric learning  Computer Science - Computer Vision and Pattern Recognition 

We provide a general mathematical framework for group and set equivariance in machine learning. We define group equivariant non-expansive operators (GENEOs) as maps between function spaces associated with groups of transformations. We study the topological and metric properties of the space of GENEOs to evaluate their approximating power and set the basis for general strategies to initialize and compose operators. We define suitable pseudo-metrics for the function spaces, the equivariance groups and the set of non-expansive operators. We prove that, under suitable assumptions, the space of GENEOs is compact and convex. These results provide fundamental guarantees in a machine learning perspective. By considering isometry-equivariant non-expansive operators, we describe a simple strategy to select and sample operators. Thereafter, we show how selected and sampled operators can be used both to perform classical metric learning and to inject knowledge in artificial neural networks.

Source: Nature Machine Intelligence 1 (2019): 423–433. doi:10.1038/s42256-019-0087-3

Publisher: Springer Nature


[1] Y. LeCun, Y. Bengio, et al., Convolutional networks for images, speech, and time series, The handbook of brain theory and neural networks 3361 (10) (1995) 1995.
[2] F. Anselmi, L. Rosasco, T. Poggio, On invariance and selectivity in representation learning, Information and Inference: A Journal of the IMA 5 (2) (2016) 134{158. arXiv:/oup/backfile/content_public/ journal/imaiai/5/2/10.1093_imaiai_iaw009/2/iaw009.pdf, doi:10.1093/imaiai/iaw009. URL http://dx.doi.org/10.1093/imaiai/iaw009
[3] T. Cohen, M. Welling, Group equivariant convolutional networks, in: International conference on machine learning, 2016, pp. 2990{2999.
[4] D. E. Worrall, S. J. Garbin, D. Turmukhambetov, G. J. Brostow, Harmonic networks: Deep translation and rotation equivariance, in: Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Vol. 2, 2017.
[5] P. Frosini, Towards an Observer-oriented Theory of Shape Comparison, in: A. Ferreira, A. Giachetti, D. Giorgi (Eds.), Eurographics Workshop on 3D Object Retrieval, The Eurographics Association, 2016. doi:10.2312/3dor.20161080.
[6] S. Biasotti, L. De Floriani, B. Falcidieno, P. Frosini, D. Giorgi, C. Landi, L. Papaleo, M. Spagnuolo, Describing shapes by geometrical-topological properties of real functions, ACM Comput. Surv. 40 (4) (2008) 12:1{12:87. doi:10.1145/1391729.1391731. URL http://doi.acm.org/10.1145/1391729.1391731
[7] G. Carlsson, A. Zomorodian, The theory of multidimensional persistence, Discrete Comput. Geom. 42 (1) (2009) 71{93. doi:10.1007/s00454-009-9176-0. URL http://dx.doi.org/10.1007/s00454-009-9176-0
[8] H. Edelsbrunner, J. Harer, Persistent homology|a survey, in: Surveys on discrete and computational geometry, Vol. 453 of Contemp. Math., Amer. Math. Soc., Providence, RI, 2008, pp. 257{282. doi: 10.1090/conm/453/08802. URL http://dx.doi.org/10.1090/conm/453/08802
[9] D. Cohen-Steiner, H. Edelsbrunner, J. Harer, Stability of persistence diagrams, Discrete Comput. Geom. 37 (1) (2007) 103{120. doi:10.1007/s00454-006-1276-5. URL http://dx.doi.org/10.1007/s00454-006-1276-5
[10] A. Cerri, B. Di Fabio, M. Ferri, P. Frosini, C. Landi, Betti numbers in multidimensional persistent homology are stable functions, Math. Methods Appl. Sci. 36 (12) (2013) 1543{1557. doi:10.1002/mma. 2704. URL http://dx.doi.org/10.1002/mma.2704
[11] S. A. Gaal, Point set topology, Pure and Applied Mathematics, Vol. XVI, Academic Press, New York-London, 1964.
[12] P. Frosini, G. Jablonski, Combining persistent homology and invariance groups for shape comparison, Discrete Comput. Geom. 55 (2) (2016) 373{409. doi:10.1007/s00454-016-9761-y. URL http://dx.doi.org/10.1007/s00454-016-9761-y
[13] S. Y. Oudot, Persistence theory: from quiver representations to data analysis, Vol. 209 of Mathematical Surveys and Monographs, American Mathematical Society, Providence, RI, 2015. doi:10.1090/surv/ 209. URL https://doi.org/10.1090/surv/209
[14] X. Glorot, Y. Bengio, Understanding the di culty of training deep feedforward neural networks, in: Proceedings of the thirteenth international conference on arti cial intelligence and statistics, 2010, pp. 249{256.
[15] G. E. Hinton, A. Krizhevsky, S. D. Wang, Transforming auto-encoders, in: International Conference on Arti cial Neural Networks, Springer, 2011, pp. 44{51.
[16] S. Sabour, N. Frosst, G. E. Hinton, Dynamic routing between capsules, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems 30, Curran Associates, Inc., 2017, pp. 3856{3866. URL http://papers.nips.cc/paper/6975-dynamic-routing-between-capsules.pdf

Metrics



Back to previous page
BibTeX entry
@article{oai:it.cnr:prodotti:408494,
	title = {Towards a topological-geometrical theory of group equivariant non-expansive operators for data analysis and machine learning},
	author = {Bergomi M. G. and Frosini P. and Giorgi D. and Quercioli N.},
	publisher = {Springer Nature},
	doi = {10.1038/s42256-019-0087-3 and 10.48550/arxiv.1812.11832},
	journal = {Nature Machine Intelligence},
	volume = {1},
	pages = {423–433},
	year = {2019}
}

5HTCircuits
Modulation of cortical circuits and predictive neural coding by serotonin


OpenAIRE