2021
Other
Open Access
NAUSICAA - D1.2: Prototipi Analisi Visuale
Vadicamo L, Gennaro C, Cafarelli D, Falchi F
In questo documento vengono descritte le principali attività svolte nell'ambito dell'Obiettivo Operativo n. 1 (OO1) "Progettazione dei sistemi di Intelligenza Artificiale e di Visione Artificiale per la sicurezza dell'imbarcazione" e in particolare dell'Attività A1.2 "Realizzazione prima versione prototipi Analisi Visuale".
Vadicamo L, Gennaro C, Cafarelli D, Falchi F
In questo documento vengono descritte le principali attività svolte nell'ambito dell'Obiettivo Operativo n. 1 (OO1) "Progettazione dei sistemi di Intelligenza Artificiale e di Visione Artificiale per la sicurezza dell'imbarcazione" e in particolare dell'Attività A1.2 "Realizzazione prima versione prototipi Analisi Visuale".
See at:
CNR IRIS 
| ISTI Repository | CNR IRIS 
2021
Software
Restricted
Visione III
Amato G, Bolettieri P, Carrara F, Falchi F, Gennaro C, Messina N, Vadicamo L, Vairo C
VISIONE III is a content-based retrieval system that supports various search functionalities (text search, object/color-based search, semantic and visual similarity search, temporal search). It uses a full-text search engine as a search backend. In this third version of our system, we modified the user interface, and we made some changes to the techniques used to analyze and search for videos.
Amato G, Bolettieri P, Carrara F, Falchi F, Gennaro C, Messina N, Vadicamo L, Vairo C
VISIONE III is a content-based retrieval system that supports various search functionalities (text search, object/color-based search, semantic and visual similarity search, temporal search). It uses a full-text search engine as a search backend. In this third version of our system, we modified the user interface, and we made some changes to the techniques used to analyze and search for videos.
See at:
bilioso.isti.cnr.it | CNR IRIS | CNR IRIS
2021
Conference article
Open Access
VISIONE at Video Browser Showdown 2021
Amato G, Bolettieri P, Falchi F, Gennaro C, Messina N, Vadicamo L, Vairo C
This paper presents the second release of VISIONE, a tool for effective video search on large-scale collections. It allows users to search for videos using textual descriptions, keywords, occurrence of objects and their spatial relationships, occurrence of colors and their spatial relationships, and image similarity. One of the main features of our system is that it employs specially designed textual encodings for indexing and searching video content using the mature and scalable Apache Lucene full-text search engine.DOI: 10.1007/978-3-030-67835-7_47
Project(s): AI4EU
, AI4Media
Metrics:
Amato G, Bolettieri P, Falchi F, Gennaro C, Messina N, Vadicamo L, Vairo C
This paper presents the second release of VISIONE, a tool for effective video search on large-scale collections. It allows users to search for videos using textual descriptions, keywords, occurrence of objects and their spatial relationships, occurrence of colors and their spatial relationships, and image similarity. One of the main features of our system is that it employs specially designed textual encodings for indexing and searching video content using the mature and scalable Apache Lucene full-text search engine.DOI: 10.1007/978-3-030-67835-7_47
Project(s): AI4EU
, AI4Media Metrics:
See at:
CNR IRIS | link.springer.com | ISTI Repository | ZENODO | zenodo.org | Lecture Notes in Computer Science | CNR IRIS | CNR IRIS
2021
Journal article
Open Access
The VISIONE video search system: exploiting off-the-shelf text search engines for large-scale video retrieval
Amato G, Bolettieri P, Carrara F, Debole F, Falchi F, Gennaro C, Vadicamo L, Vairo C
This paper describes in detail VISIONE, a video search system that allows users to search for videos using textual keywords, the occurrence of objects and their spatial relationships, the occurrence of colors and their spatial relationships, and image similarity. These modalities can be combined together to express complex queries and meet users' needs. The peculiarity of our approach is that we encode all information extracted from the keyframes, such as visual deep features, tags, color and object locations, using a convenient textual encoding that is indexed in a single text retrieval engine. This offers great flexibility when results corresponding to various parts of the query (visual, text and locations) need to be merged. In addition, we report an extensive analysis of the retrieval performance of the system, using the query logs generated during the Video Browser Showdown (VBS) 2019 competition. This allowed us to fine-tune the system by choosing the optimal parameters and strategies from those we tested.Source: JOURNAL OF IMAGING, vol. 7 (issue 5)
DOI: 10.3390/jimaging7050076
DOI: 10.48550/arxiv.2008.02749
Project(s): AI4Media
Metrics:
Amato G, Bolettieri P, Carrara F, Debole F, Falchi F, Gennaro C, Vadicamo L, Vairo C
This paper describes in detail VISIONE, a video search system that allows users to search for videos using textual keywords, the occurrence of objects and their spatial relationships, the occurrence of colors and their spatial relationships, and image similarity. These modalities can be combined together to express complex queries and meet users' needs. The peculiarity of our approach is that we encode all information extracted from the keyframes, such as visual deep features, tags, color and object locations, using a convenient textual encoding that is indexed in a single text retrieval engine. This offers great flexibility when results corresponding to various parts of the query (visual, text and locations) need to be merged. In addition, we report an extensive analysis of the retrieval performance of the system, using the query logs generated during the Video Browser Showdown (VBS) 2019 competition. This allowed us to fine-tune the system by choosing the optimal parameters and strategies from those we tested.Source: JOURNAL OF IMAGING, vol. 7 (issue 5)
DOI: 10.3390/jimaging7050076
DOI: 10.48550/arxiv.2008.02749
Project(s): AI4Media
Metrics:
See at:
arXiv.org e-Print Archive | Journal of Imaging | Journal of Imaging | CNR IRIS | ISTI Repository | ISTI Repository | DOAJ-Articles | www.mdpi.com | Journal of Imaging | ZENODO | doi.org | CNR IRIS | CNR IRIS
2021
Conference article
Open Access
On generalizing permutation-based representations for approximate search
Vadicamo L, Gennaro C, Amato G
In the domain of approximate metric search, the Permutation-based Indexing (PBI) approaches have been proved to be particularly suitable for dealing with large data collections. These methods employ a permutation-based representation of the data, which can be efficiently indexed using data structures such as inverted files. In the literature, the definition of the permutation of a metric object was derived by reordering the distances of the object to a set of pivots. In this paper, we aim at generalizing this definition in order to enlarge the class of permutations that can be used by PBI approaches. As a practical outcome, we defined a new type of permutation that is calculated using distances from pairs of pivots. The proposed technique permits us to produce longer permutations than traditional ones for the same number of object-pivot distance calculations. The advantage is that the use of inverted files built on permutation prefixes leads to greater efficiency in the search phase when longer permutations are used.DOI: 10.1007/978-3-030-89657-7_6
Project(s): "Research Initiation Award: Fabrication and Characterization of Composite Contacts on Wide Band Gap Semiconductor for High Temperature Application in Air.", AI4Media
Metrics:
Vadicamo L, Gennaro C, Amato G
In the domain of approximate metric search, the Permutation-based Indexing (PBI) approaches have been proved to be particularly suitable for dealing with large data collections. These methods employ a permutation-based representation of the data, which can be efficiently indexed using data structures such as inverted files. In the literature, the definition of the permutation of a metric object was derived by reordering the distances of the object to a set of pivots. In this paper, we aim at generalizing this definition in order to enlarge the class of permutations that can be used by PBI approaches. As a practical outcome, we defined a new type of permutation that is calculated using distances from pairs of pivots. The proposed technique permits us to produce longer permutations than traditional ones for the same number of object-pivot distance calculations. The advantage is that the use of inverted files built on permutation prefixes leads to greater efficiency in the search phase when longer permutations are used.DOI: 10.1007/978-3-030-89657-7_6
Project(s): "Research Initiation Award: Fabrication and Characterization of Composite Contacts on Wide Band Gap Semiconductor for High Temperature Application in Air.", AI4Media
Metrics:
See at:
CNR IRIS | link.springer.com | ISTI Repository | CNR IRIS | CNR IRIS | link.springer.com
2021
Other
Open Access
AIMH research activities 2021
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Coccomini D., Ciampi L., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2021 activities of the research group.DOI: 10.32079/isti-ar-2021/003
Metrics:
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Coccomini D., Ciampi L., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2021 activities of the research group.DOI: 10.32079/isti-ar-2021/003
Metrics:
See at:
CNR IRIS | ISTI Repository | CNR IRIS
2021
Journal article
Open Access
Re-ranking via local embeddings: A use case with permutation-based indexing and the nSimplex projection
Vadicamo L, Gennaro C, Falchi F, Chavez E, Connor R, Amato G
Approximate Nearest Neighbor (ANN) search is a prevalent paradigm for searching intrinsically high dimensional objects in large-scale data sets. Recently, the permutation-based approach for ANN has attracted a lot of interest due to its versatility in being used in the more general class of metric spaces. In this approach, the entire database is ranked by a permutation distance to the query. Typically, permutations allow the efficient selection of a candidate set of results, but typically to achieve high recall or precision this set has to be reviewed using the original metric and data. This can lead to a sizeable percentage of the database being recalled, along with many expensive distance calculations. To reduce the number of metric computations and the number of database elements accessed, we propose here a re-ranking based on a local embedding using the nSimplex projection. The nSimplex projection produces Euclidean vectors from objects in metric spaces which possess the n-point property. The mapping is obtained from the distances to a set of reference objects, and the original metric can be lower bounded and upper bounded by the Euclidean distance of objects sharing the same set of references. Our approach is particularly advantageous for extensive databases or expensive metric function. We reuse the distances computed in the permutations in the first stage, and hence the memory footprint of the index is not increased. An extensive experimental evaluation of our approach is presented, demonstrating excellent results even on a set of hundreds of millions of objects.Source: INFORMATION SYSTEMS, vol. 95
DOI: 10.1016/j.is.2020.101506
Project(s): AI4EU
Metrics:
Vadicamo L, Gennaro C, Falchi F, Chavez E, Connor R, Amato G
Approximate Nearest Neighbor (ANN) search is a prevalent paradigm for searching intrinsically high dimensional objects in large-scale data sets. Recently, the permutation-based approach for ANN has attracted a lot of interest due to its versatility in being used in the more general class of metric spaces. In this approach, the entire database is ranked by a permutation distance to the query. Typically, permutations allow the efficient selection of a candidate set of results, but typically to achieve high recall or precision this set has to be reviewed using the original metric and data. This can lead to a sizeable percentage of the database being recalled, along with many expensive distance calculations. To reduce the number of metric computations and the number of database elements accessed, we propose here a re-ranking based on a local embedding using the nSimplex projection. The nSimplex projection produces Euclidean vectors from objects in metric spaces which possess the n-point property. The mapping is obtained from the distances to a set of reference objects, and the original metric can be lower bounded and upper bounded by the Euclidean distance of objects sharing the same set of references. Our approach is particularly advantageous for extensive databases or expensive metric function. We reuse the distances computed in the permutations in the first stage, and hence the memory footprint of the index is not increased. An extensive experimental evaluation of our approach is presented, demonstrating excellent results even on a set of hundreds of millions of objects.Source: INFORMATION SYSTEMS, vol. 95
DOI: 10.1016/j.is.2020.101506
Project(s): AI4EU
Metrics:
See at:
CNR IRIS | ISTI Repository | www.sciencedirect.com | ZENODO | Information Systems | Information Systems | CNR IRIS | CNR IRIS | CNR IRIS
2021
Journal article
Open Access
Query filtering using two-dimensional local embeddings
Vadicamo L, Connor R, Chávez E
In high dimensional data sets, exact indexes are ineffective for proximity queries, and a sequential scan over the entire data set is unavoidable. Accepting this, here we present a new approach employing two-dimensional embeddings. Each database element is mapped to the XY plane using the four-point property. The caveat is that the mapping is local: in other words, each object is mapped using a different mapping. The idea is that each element of the data is associated with a pair of reference objects that is well-suited to filter that particular object, in cases where it is not relevant to a query. This maximises the probability of excluding that object from a search. At query time, a query is compared with a pool of reference objects which allow its mapping to all the planes used by data objects. Then, for each query/object pair, a lower bound of the actual distance is obtained. The technique can be applied to any metric space that possesses the four-point property, therefore including Euclidean, Cosine, Triangular, Jensen-Shannon, and Quadratic Form distances. Our experiments show that for all the data sets tested, of varying dimensionality, our approach can filter more objects than a standard metric indexing approach. For low dimensional data this does not make a good search mechanism in its own right, as it does not scale with the size of the data: that is, its cost is linear with respect to the data size. However, we also show that it can be added as a post-filter to other mechanisms, increasing efficiency with little extra cost in space or time. For high-dimensional data, we show related approximate techniques which, we believe, give the best known compromise for speeding up the essential sequential scan. The potential uses of our filtering technique include pure GPU searching, taking advantage of the tiny memory footprint of the mapping.Source: INFORMATION SYSTEMS, vol. 101
DOI: 10.1016/j.is.2021.101808
Metrics:
Vadicamo L, Connor R, Chávez E
In high dimensional data sets, exact indexes are ineffective for proximity queries, and a sequential scan over the entire data set is unavoidable. Accepting this, here we present a new approach employing two-dimensional embeddings. Each database element is mapped to the XY plane using the four-point property. The caveat is that the mapping is local: in other words, each object is mapped using a different mapping. The idea is that each element of the data is associated with a pair of reference objects that is well-suited to filter that particular object, in cases where it is not relevant to a query. This maximises the probability of excluding that object from a search. At query time, a query is compared with a pool of reference objects which allow its mapping to all the planes used by data objects. Then, for each query/object pair, a lower bound of the actual distance is obtained. The technique can be applied to any metric space that possesses the four-point property, therefore including Euclidean, Cosine, Triangular, Jensen-Shannon, and Quadratic Form distances. Our experiments show that for all the data sets tested, of varying dimensionality, our approach can filter more objects than a standard metric indexing approach. For low dimensional data this does not make a good search mechanism in its own right, as it does not scale with the size of the data: that is, its cost is linear with respect to the data size. However, we also show that it can be added as a post-filter to other mechanisms, increasing efficiency with little extra cost in space or time. For high-dimensional data, we show related approximate techniques which, we believe, give the best known compromise for speeding up the essential sequential scan. The potential uses of our filtering technique include pure GPU searching, taking advantage of the tiny memory footprint of the mapping.Source: INFORMATION SYSTEMS, vol. 101
DOI: 10.1016/j.is.2021.101808
Metrics:
See at:
CNR IRIS | ISTI Repository | ISTI Repository | www.sciencedirect.com | CNR IRIS | CNR IRIS | CNR IRIS