Typology
operator:
and / or
Language
operator:
and / or
Date
operator:
and / or
Rights
operator:
and / or
2022
Journal article
Open Access
Interactive video retrieval evaluation at a distance: comparing sixteen interactive video search systems in a remote setting at the 10th Video Browser Showdown
Heller S., Gsteiger V., Bailer W., Gurrin C., Jonsson B. T., Lokoc J., Leibetseder A., Mejzlik F., Peska L., Rossetto L., Schall K., Schoeffmann K., Schuldt H., Spiess F., Tran L. D., Vadicamo L., Vesely P., Vrochidis S., Wu J.
The Video Browser Showdown addresses difficult video search challenges through an annual interactive evaluation campaign attracting research teams focusing on interactive video retrieval. The campaign aims to provide insights into the performance of participating interactive video retrieval systems, tested by selected search tasks on large video collections. For the first time in its ten year history, the Video Browser Showdown 2021 was organized in a fully remote setting and hosted a record number of sixteen scoring systems. In this paper, we describe the competition setting, tasks and results and give an overview of state-of-the-art methods used by the competing systems. By looking at query result logs provided by ten systems, we analyze differences in retrieval model performances and browsing times before a correct submission. Through advances in data gathering methodology and tools, we provide a comprehensive analysis of ad-hoc video search tasks, discuss results, task design and methodological challenges. We highlight that almost all top performing systems utilize some sort of joint embedding for text-image retrieval and enable specification of temporal context in queries for known-item search. Whereas a combination of these techniques drive the currently top performing systems, we identify several future challenges for interactive video search engines and the Video Browser Showdown competition itself.Source: International journal of multimedia information retrieval Print 11 (2022). doi:10.1007/s13735-021-00225-2
DOI: 10.1007/s13735-021-00225-2
Project(s): AI4Media
Metrics:
Heller S., Gsteiger V., Bailer W., Gurrin C., Jonsson B. T., Lokoc J., Leibetseder A., Mejzlik F., Peska L., Rossetto L., Schall K., Schoeffmann K., Schuldt H., Spiess F., Tran L. D., Vadicamo L., Vesely P., Vrochidis S., Wu J.
The Video Browser Showdown addresses difficult video search challenges through an annual interactive evaluation campaign attracting research teams focusing on interactive video retrieval. The campaign aims to provide insights into the performance of participating interactive video retrieval systems, tested by selected search tasks on large video collections. For the first time in its ten year history, the Video Browser Showdown 2021 was organized in a fully remote setting and hosted a record number of sixteen scoring systems. In this paper, we describe the competition setting, tasks and results and give an overview of state-of-the-art methods used by the competing systems. By looking at query result logs provided by ten systems, we analyze differences in retrieval model performances and browsing times before a correct submission. Through advances in data gathering methodology and tools, we provide a comprehensive analysis of ad-hoc video search tasks, discuss results, task design and methodological challenges. We highlight that almost all top performing systems utilize some sort of joint embedding for text-image retrieval and enable specification of temporal context in queries for known-item search. Whereas a combination of these techniques drive the currently top performing systems, we identify several future challenges for interactive video search engines and the Video Browser Showdown competition itself.Source: International journal of multimedia information retrieval Print 11 (2022). doi:10.1007/s13735-021-00225-2
DOI: 10.1007/s13735-021-00225-2
Project(s): AI4Media
Metrics:
See at:
ISTI Repository 
| link.springer.com 
| CNR ExploRA
2022
Conference article
Open Access
MOBDrone: a drone video dataset for Man OverBoard Rescue
Cafarelli D., Ciampi L., Vadicamo L., Gennaro C., Berton A., Paterni M., Benvenuti C., Passera M., Falchi F.
Modern Unmanned Aerial Vehicles (UAV) equipped with cameras can play an essential role in speeding up the identification and rescue of people who have fallen overboard, i.e., man overboard (MOB). To this end, Artificial Intelligence techniques can be leveraged for the automatic understanding of visual data acquired from drones. However, detecting people at sea in aerial imagery is challenging primarily due to the lack of specialized annotated datasets for training and testing detectors for this task. To fill this gap, we introduce and publicly release the MOBDrone benchmark, a collection of more than 125K drone-view images in a marine environment under several conditions, such as different altitudes, camera shooting angles, and illumination. We manually annotated more than 180K objects, of which about 113K man overboard, precisely localizing them with bounding boxes. Moreover, we conduct a thorough performance analysis of several state-of-the-art object detectors on the MOBDrone data, serving as baselines for further research.Source: ICIAP 2022 - 21st International Conference on Image Analysis and Processing, pp. 633–644, Lecce, Italia, 23-27/05/2022
DOI: 10.1007/978-3-031-06430-2_53
Metrics:
Cafarelli D., Ciampi L., Vadicamo L., Gennaro C., Berton A., Paterni M., Benvenuti C., Passera M., Falchi F.
Modern Unmanned Aerial Vehicles (UAV) equipped with cameras can play an essential role in speeding up the identification and rescue of people who have fallen overboard, i.e., man overboard (MOB). To this end, Artificial Intelligence techniques can be leveraged for the automatic understanding of visual data acquired from drones. However, detecting people at sea in aerial imagery is challenging primarily due to the lack of specialized annotated datasets for training and testing detectors for this task. To fill this gap, we introduce and publicly release the MOBDrone benchmark, a collection of more than 125K drone-view images in a marine environment under several conditions, such as different altitudes, camera shooting angles, and illumination. We manually annotated more than 180K objects, of which about 113K man overboard, precisely localizing them with bounding boxes. Moreover, we conduct a thorough performance analysis of several state-of-the-art object detectors on the MOBDrone data, serving as baselines for further research.Source: ICIAP 2022 - 21st International Conference on Image Analysis and Processing, pp. 633–644, Lecce, Italia, 23-27/05/2022
DOI: 10.1007/978-3-031-06430-2_53
Metrics:
See at:
ISTI Repository | link.springer.com | CNR ExploRA
2022
Dataset
Open Access
MOBDrone: a large-scale drone-view dataset for man overboard detection
Cafarelli D., Ciampi L., Vadicamo L., Gennaro C., Berton A., Paterni M., Benvenuti C., Passera M., Falchi F.
The Man OverBoard Drone (MOBDrone) dataset is a large-scale collection of aerial footage images. It contains 126,170 frames extracted from 66 video clips gathered from one UAV flying at an altitude of 10 to 60 meters above the mean sea level. Images are manually annotated with more than 180K bounding boxes localizing objects belonging to 5 categories --- person, boat, lifebuoy, surfboard, wood. More than 113K of these bounding boxes belong to the person category and localize people in the water simulating the need to be rescued.
Cafarelli D., Ciampi L., Vadicamo L., Gennaro C., Berton A., Paterni M., Benvenuti C., Passera M., Falchi F.
The Man OverBoard Drone (MOBDrone) dataset is a large-scale collection of aerial footage images. It contains 126,170 frames extracted from 66 video clips gathered from one UAV flying at an altitude of 10 to 60 meters above the mean sea level. Images are manually annotated with more than 180K bounding boxes localizing objects belonging to 5 categories --- person, boat, lifebuoy, surfboard, wood. More than 113K of these bounding boxes belong to the person category and localize people in the water simulating the need to be rescued.
See at:
ISTI Repository | CNR ExploRA | zenodo.org
2022
Conference article
Open Access
A task category space for user-centric comparative multimedia search evaluations
Lokoc J., Bailer W., Barthel K. U., Gurrin C., Heller S., Jónsson B. Þ., Peska L., Rossetto L., Schoeffmann K., Vadicamo L., Vrochidis S., Wu J.
In the last decade, user-centric video search competitions have facilitated the evolution of interactive video search systems. So far, these competitions focused on a small number of search task categories, with few attempts to change task category configurations. Based on our extensive experience with interactive video search contests, we have analyzed the spectrum of possible task categories and propose a list of individual axes that define a large space of possible task categories. Using this concept of category space, new user-centric video search competitions can be designed to benchmark video search systems from different perspectives. We further analyse the three task categories considered so far at the Video Browser Showdown and discuss possible (but sometimes challenging) shifts within the task category space.Source: MMM 2022 - 28th International Conference on Multi Media Modeling, pp. 193–204, Phu Quoc, Vietnam, 06-10/06/2022
DOI: 10.1007/978-3-030-98358-1_16
Project(s): AI4Media
Metrics:
Lokoc J., Bailer W., Barthel K. U., Gurrin C., Heller S., Jónsson B. Þ., Peska L., Rossetto L., Schoeffmann K., Vadicamo L., Vrochidis S., Wu J.
In the last decade, user-centric video search competitions have facilitated the evolution of interactive video search systems. So far, these competitions focused on a small number of search task categories, with few attempts to change task category configurations. Based on our extensive experience with interactive video search contests, we have analyzed the spectrum of possible task categories and propose a list of individual axes that define a large space of possible task categories. Using this concept of category space, new user-centric video search competitions can be designed to benchmark video search systems from different perspectives. We further analyse the three task categories considered so far at the Video Browser Showdown and discuss possible (but sometimes challenging) shifts within the task category space.Source: MMM 2022 - 28th International Conference on Multi Media Modeling, pp. 193–204, Phu Quoc, Vietnam, 06-10/06/2022
DOI: 10.1007/978-3-030-98358-1_16
Project(s): AI4Media
Metrics:
See at:
ISTI Repository | doi.org | link.springer.com | CNR ExploRA
2022
Conference article
Open Access
VISIONE at Video Browser Showdown 2022
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
VISIONE is a content-based retrieval system that supports various search functionalities (text search, object/color-based search, semantic and visual similarity search, temporal search). It uses a full-text search engine as a search backend. In the latest version of our system, we modified the user interface, and we made some changes to the techniques used to analyze and search for videos.Source: MMM 2022 - 28th International Conference on Multimedia Modeling, pp. 543–548, Phu Quoc, Vietnam, 06-10/06/2022
DOI: 10.1007/978-3-030-98355-0_52
Project(s): AI4EU, AI4Media
Metrics:
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
VISIONE is a content-based retrieval system that supports various search functionalities (text search, object/color-based search, semantic and visual similarity search, temporal search). It uses a full-text search engine as a search backend. In the latest version of our system, we modified the user interface, and we made some changes to the techniques used to analyze and search for videos.Source: MMM 2022 - 28th International Conference on Multimedia Modeling, pp. 543–548, Phu Quoc, Vietnam, 06-10/06/2022
DOI: 10.1007/978-3-030-98355-0_52
Project(s): AI4EU
, AI4Media Metrics:
See at:
ISTI Repository | doi.org | link.springer.com | CNR ExploRA
2022
Conference article
Open Access
Approximate nearest neighbor search on standard search engines
Carrara F., Vadicamo L., Gennaro C., Amato G.
Approximate search for high-dimensional vectors is commonly addressed using dedicated techniques often combined with hardware acceleration provided by GPUs, FPGAs, and other custom in-memory silicon. Despite their effectiveness, harmonizing those optimized solutions with other types of searches often poses technological difficulties. For example, to implement a combined text+image multimodal search, we are forced first to query the index of high-dimensional image descriptors and then filter the results based on the textual query or vice versa. This paper proposes a text surrogate technique to translate real-valued vectors into text and index them with a standard textual search engine such as Elasticsearch or Apache Lucene. This technique allows us to perform approximate kNN searches of high-dimensional vectors alongside classical full-text searches natively on a single textual search engine, enabling multimedia queries without sacrificing scalability. Our proposal exploits a combination of vector quantization and scalar quantization. We compared our approach to the existing literature in this field of research, demonstrating a significant improvement in performance through preliminary experimentation.Source: SISAP 2022 - 15th International Conference on Similarity Search and Applications, pp. 214–221, Bologna, Italy, 7-9/10/2022
DOI: 10.1007/978-3-031-17849-8_17
Project(s): AI4Media
Metrics:
Carrara F., Vadicamo L., Gennaro C., Amato G.
Approximate search for high-dimensional vectors is commonly addressed using dedicated techniques often combined with hardware acceleration provided by GPUs, FPGAs, and other custom in-memory silicon. Despite their effectiveness, harmonizing those optimized solutions with other types of searches often poses technological difficulties. For example, to implement a combined text+image multimodal search, we are forced first to query the index of high-dimensional image descriptors and then filter the results based on the textual query or vice versa. This paper proposes a text surrogate technique to translate real-valued vectors into text and index them with a standard textual search engine such as Elasticsearch or Apache Lucene. This technique allows us to perform approximate kNN searches of high-dimensional vectors alongside classical full-text searches natively on a single textual search engine, enabling multimedia queries without sacrificing scalability. Our proposal exploits a combination of vector quantization and scalar quantization. We compared our approach to the existing literature in this field of research, demonstrating a significant improvement in performance through preliminary experimentation.Source: SISAP 2022 - 15th International Conference on Similarity Search and Applications, pp. 214–221, Bologna, Italy, 7-9/10/2022
DOI: 10.1007/978-3-031-17849-8_17
Project(s): AI4Media
Metrics:
See at:
ISTI Repository | link.springer.com | CNR ExploRA
2022
Other
Open Access
COCO, LVIS, Open Images V4 classes mapping
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
This repository contains a mapping between the classes of COCO, LVIS, and Open Images V4 datasets into a unique set of 1460 classes. COCO [Lin et al 2014] contains 80 classes, LVIS [gupta2019lvis] contains 1460 classes, Open Images V4 [Kuznetsova et al. 2020] contains 601 classes. We built a mapping of these classes using a semi-automatic procedure in order to have a unique final list of 1460 classes. We also generated a hierarchy for each class, using wordnet.Project(s): AI4Media
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
This repository contains a mapping between the classes of COCO, LVIS, and Open Images V4 datasets into a unique set of 1460 classes. COCO [Lin et al 2014] contains 80 classes, LVIS [gupta2019lvis] contains 1460 classes, Open Images V4 [Kuznetsova et al. 2020] contains 601 classes. We built a mapping of these classes using a semi-automatic procedure in order to have a unique final list of 1460 classes. We also generated a hierarchy for each class, using wordnet.Project(s): AI4Media
See at:
CNR ExploRA | zenodo.org
2022
Conference article
Open Access
Investigating binary partition power in metric query
Connor R., Dearle A., Vadicamo L.
It is generally understood that, as dimensionality increases, the minimum cost of metric query tends from O(log n) to O (n) in both space and time, where n is the size of the data set. With low dimensionality, the former is easy to achieve; with very high dimensionality, the latter is inevitable. We previously described BitPart as a novel mechanism suitable for performing exact metric search in "high(er)" dimensions. The essential tradeoff of BitPart is that its space cost is linear with respect to the size of the data, but the actual space required for each object may be small as log2 n bits, which allows even very large data sets to be queried using only main memory. Potentially the time cost still scales with O (log n). Together these attributes give exact search which outperforms indexing structures if dimensionality is within a certain range. In this article, we reiterate the design of BitPart in this context. The novel contribution is an indepth examination of what the notion of "high(er)" means in practical terms. To do this we introduce the notion of exclusion power, and show its application to some generated data sets across different dimensions.Source: SEBD 2022 - 30th Italian Symposium on Advanced Database Systems, pp. 415–426, Tirrenia (PI), Italia, 19-22/06/2022
Connor R., Dearle A., Vadicamo L.
It is generally understood that, as dimensionality increases, the minimum cost of metric query tends from O(log n) to O (n) in both space and time, where n is the size of the data set. With low dimensionality, the former is easy to achieve; with very high dimensionality, the latter is inevitable. We previously described BitPart as a novel mechanism suitable for performing exact metric search in "high(er)" dimensions. The essential tradeoff of BitPart is that its space cost is linear with respect to the size of the data, but the actual space required for each object may be small as log2 n bits, which allows even very large data sets to be queried using only main memory. Potentially the time cost still scales with O (log n). Together these attributes give exact search which outperforms indexing structures if dimensionality is within a certain range. In this article, we reiterate the design of BitPart in this context. The novel contribution is an indepth examination of what the notion of "high(er)" means in practical terms. To do this we introduce the notion of exclusion power, and show its application to some generated data sets across different dimensions.Source: SEBD 2022 - 30th Italian Symposium on Advanced Database Systems, pp. 415–426, Tirrenia (PI), Italia, 19-22/06/2022
See at:
ceur-ws.org | ISTI Repository | ISTI Repository | CNR ExploRA
2022
Conference article
Open Access
On the expected exclusion power of binary partitions for metric search
Vadicamo L., Dearle A., Connor R.
The entire history and, we dare say, future of similarity search is governed by the underlying notion of partition. A partition is an equivalence relation defined over the space, therefore each element of the space is contained within precisely one of the equivalence classes of the partition. All attempts to search a finite space efficiently, whether exactly or approximately, rely on some set of principles which imply that if the query is within one equivalence class, then one or more other classes either cannot, or probably do not, contain any of its solutions. In most early research, partitions relied only on the metric postulates, and logarithmic search time could be obtained on low dimensional spaces. In these cases, it was straightforward to identify multiple partitions, each of which gave a relatively high probability of identifying subsets of the space which could not contain solutions. Over time the datasets being searched have become more complex, leading to higher dimensional spaces. It is now understood that even an approximate search in a very high-dimensional space is destined to require O(n) time and space. Almost entirely missing from the research literature however is any analysis of exactly when this effect takes over. In this paper, we make a start on tackling this important issue. Using a quantitative approach, we aim to shed some light on the notion of the exclusion power of partitions, in an attempt to better understand their nature with respect to increasing dimensionality.Source: SISAP 2022 - 15th International Conference on Similarity Search and Applications, pp. 104–117, Bologna, Italy, 7-9/10/2022
DOI: 10.1007/978-3-031-17849-8_9
Project(s): AI4Media
Metrics:
Vadicamo L., Dearle A., Connor R.
The entire history and, we dare say, future of similarity search is governed by the underlying notion of partition. A partition is an equivalence relation defined over the space, therefore each element of the space is contained within precisely one of the equivalence classes of the partition. All attempts to search a finite space efficiently, whether exactly or approximately, rely on some set of principles which imply that if the query is within one equivalence class, then one or more other classes either cannot, or probably do not, contain any of its solutions. In most early research, partitions relied only on the metric postulates, and logarithmic search time could be obtained on low dimensional spaces. In these cases, it was straightforward to identify multiple partitions, each of which gave a relatively high probability of identifying subsets of the space which could not contain solutions. Over time the datasets being searched have become more complex, leading to higher dimensional spaces. It is now understood that even an approximate search in a very high-dimensional space is destined to require O(n) time and space. Almost entirely missing from the research literature however is any analysis of exactly when this effect takes over. In this paper, we make a start on tackling this important issue. Using a quantitative approach, we aim to shed some light on the notion of the exclusion power of partitions, in an attempt to better understand their nature with respect to increasing dimensionality.Source: SISAP 2022 - 15th International Conference on Similarity Search and Applications, pp. 104–117, Bologna, Italy, 7-9/10/2022
DOI: 10.1007/978-3-031-17849-8_9
Project(s): AI4Media
Metrics:
See at:
ISTI Repository | link.springer.com | CNR ExploRA
2022
Journal article
Open Access
A leap among quantum computing and quantum neural networks: a survey
Massoli F. V., Vadicamo L., Amato G., Falchi F.
In recent years, Quantum Computing witnessed massive improvements in terms of available resources and algorithms development. The ability to harness quantum phenomena to solve computational problems is a long-standing dream that has drawn the scientific community's interest since the late 80s. In such a context, we propose our contribution. First, we introduce basic concepts related to quantum computations, and then we explain the core functionalities of technologies that implement the Gate Model and Adiabatic Quantum Computing paradigms. Finally, we gather, compare and analyze the current state-of-the-art concerning Quantum Perceptrons and Quantum Neural Networks implementations.Source: ACM computing surveys (2022). doi:10.1145/3529756
DOI: 10.1145/3529756
DOI: 10.48550/arxiv.2107.03313
Project(s): AI4EU, AI4Media
Metrics:
Massoli F. V., Vadicamo L., Amato G., Falchi F.
In recent years, Quantum Computing witnessed massive improvements in terms of available resources and algorithms development. The ability to harness quantum phenomena to solve computational problems is a long-standing dream that has drawn the scientific community's interest since the late 80s. In such a context, we propose our contribution. First, we introduce basic concepts related to quantum computations, and then we explain the core functionalities of technologies that implement the Gate Model and Adiabatic Quantum Computing paradigms. Finally, we gather, compare and analyze the current state-of-the-art concerning Quantum Perceptrons and Quantum Neural Networks implementations.Source: ACM computing surveys (2022). doi:10.1145/3529756
DOI: 10.1145/3529756
DOI: 10.48550/arxiv.2107.03313
Project(s): AI4EU
, AI4Media Metrics:
See at:
arXiv.org e-Print Archive | ISTI Repository | ACM Computing Surveys | doi.org | CNR ExploRA
2022
Report
Open Access
AIMH research activities 2022
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2022 activities of the research group.Source: ISTI Annual reports, 2022
DOI: 10.32079/isti-ar-2022/002
Metrics:
Aloia N., Amato G., Bartalesi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Metilli D., Molinari A., Moreo A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2022 activities of the research group.Source: ISTI Annual reports, 2022
DOI: 10.32079/isti-ar-2022/002
Metrics:
See at:
ISTI Repository | CNR ExploRA
2022
Software
Unknown
Visione IV
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
VISIONE IV is the fourth release of a tool for fast and effective video search on a large-scale dataset. It includes several search functionalities like text search, object and color-based search, semantic and visual similarity search, and temporal search.
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
VISIONE IV is the fourth release of a tool for fast and effective video search on a large-scale dataset. It includes several search functionalities like text search, object and color-based search, semantic and visual similarity search, and temporal search.
See at: CNR ExploRA | visione.isti.cnr.it
2022
Software
Unknown
MOBDrone App
Cafarelli D., Vairo C., Gennaro C., Vadicamo L., Falchi F.
MOBdrone is an android app developed as part of the NAUSICAA project for automatically searching for people who have fallen overboard. It uses DJI's sdk for automatic drone flight and integrates with a DLL for interaction with the ship's dashboard and a python app for visual analysis of captured video.
Cafarelli D., Vairo C., Gennaro C., Vadicamo L., Falchi F.
MOBdrone is an android app developed as part of the NAUSICAA project for automatically searching for people who have fallen overboard. It uses DJI's sdk for automatic drone flight and integrates with a DLL for interaction with the ship's dashboard and a python app for visual analysis of captured video.
See at: gitea-s2i2s.isti.cnr.it | CNR ExploRA
2022
Software
Unknown
NADLibrary
Cafarelli D., Vairo C., Gennaro C., Vadicamo L., Falchi F.
NADLibrary is a DLL library developed as part of the NAUSICAA project, which acts as a communication interface between the Windows application running on the ship's dashboard for remote control of the drone and the android application that controls the drone.
Cafarelli D., Vairo C., Gennaro C., Vadicamo L., Falchi F.
NADLibrary is a DLL library developed as part of the NAUSICAA project, which acts as a communication interface between the Windows application running on the ship's dashboard for remote control of the drone and the android application that controls the drone.
See at: gitea-s2i2s.isti.cnr.it | CNR ExploRA
2022
Software
Unknown
Dummy drone dashboard
Cafarelli D., Vairo C., Gennaro C., Vadicamo L., Falchi F.
Dummy drone dashboard is a C# interface, developed as part of the NAUSICAA project, that emulates the ship's dashboard to test the communication DLL developed for communication between the ship's dashboard and the android application that controls the drone.
Cafarelli D., Vairo C., Gennaro C., Vadicamo L., Falchi F.
Dummy drone dashboard is a C# interface, developed as part of the NAUSICAA project, that emulates the ship's dashboard to test the communication DLL developed for communication between the ship's dashboard and the android application that controls the drone.
See at: gitea-s2i2s.isti.cnr.it | CNR ExploRA
2022
Software
Unknown
VisioneRAI
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
A release of the VISIONE tool on RAI's real media content. This first prototipe was developed as part of AI4media European project , and provides a set of integrated components for browsing and searching videos by similar frames, by objects occurring in videos, by spatial relationships among objects in videos, and by cross-model search functionality (text-to-video search).Project(s): AI4Media
Amato G., Bolettieri P., Carrara F., Falchi F., Gennaro C., Messina N., Vadicamo L., Vairo C.
A release of the VISIONE tool on RAI's real media content. This first prototipe was developed as part of AI4media European project , and provides a set of integrated components for browsing and searching videos by similar frames, by objects occurring in videos, by spatial relationships among objects in videos, and by cross-model search functionality (text-to-video search).Project(s): AI4Media
See at: CNR ExploRA | visionerai.isti.cnr.it
2022
Report
Restricted
AI4media D8.2 - Initial use case demonstrators and applications
Tzoannos S., Tsabouraki D., Konios D., Varsou E., Gray B., Dimitrov A., Dimitrov E., Kostadinov G., Gravina D., Melhart D., Henriksen L., Holmgård C., Kompatsiaris I., Papadopoulos S., Cuccovillo L., Van Kemenade P., Bocyte R., Amato G., Vadicamo L., Bolettieri P., Negro F., Montagnuolo M., Messina A., Bruccoleri A., Mignot R., Bauwens R., Overmeire L., Matton M., Garcia A.
This report - accompanying the first release of the AI4Media's WP8 demonstrators - is the first one in a series of three releases. The aim of the report is to provide the reader with information about the seven AI4Media use cases, focusing on the operational environment of each use case, the features and epics covered in the first release, and the technical and non-technical activities that took place leading to the first release of the demonstrators.Source: ISTI Project Report, AI4Media, D8.2, 2022
Project(s): AI4Media
Tzoannos S., Tsabouraki D., Konios D., Varsou E., Gray B., Dimitrov A., Dimitrov E., Kostadinov G., Gravina D., Melhart D., Henriksen L., Holmgård C., Kompatsiaris I., Papadopoulos S., Cuccovillo L., Van Kemenade P., Bocyte R., Amato G., Vadicamo L., Bolettieri P., Negro F., Montagnuolo M., Messina A., Bruccoleri A., Mignot R., Bauwens R., Overmeire L., Matton M., Garcia A.
This report - accompanying the first release of the AI4Media's WP8 demonstrators - is the first one in a series of three releases. The aim of the report is to provide the reader with information about the seven AI4Media use cases, focusing on the operational environment of each use case, the features and epics covered in the first release, and the technical and non-technical activities that took place leading to the first release of the demonstrators.Source: ISTI Project Report, AI4Media, D8.2, 2022
Project(s): AI4Media
See at:
CNR ExploRA
2021
Journal article
Open Access
Re-ranking via local embeddings: A use case with permutation-based indexing and the nSimplex projection
Vadicamo L., Gennaro C., Falchi F., Chavez E., Connor R., Amato G.
Approximate Nearest Neighbor (ANN) search is a prevalent paradigm for searching intrinsically high dimensional objects in large-scale data sets. Recently, the permutation-based approach for ANN has attracted a lot of interest due to its versatility in being used in the more general class of metric spaces. In this approach, the entire database is ranked by a permutation distance to the query. Typically, permutations allow the efficient selection of a candidate set of results, but typically to achieve high recall or precision this set has to be reviewed using the original metric and data. This can lead to a sizeable percentage of the database being recalled, along with many expensive distance calculations. To reduce the number of metric computations and the number of database elements accessed, we propose here a re-ranking based on a local embedding using the nSimplex projection. The nSimplex projection produces Euclidean vectors from objects in metric spaces which possess the n-point property. The mapping is obtained from the distances to a set of reference objects, and the original metric can be lower bounded and upper bounded by the Euclidean distance of objects sharing the same set of references. Our approach is particularly advantageous for extensive databases or expensive metric function. We reuse the distances computed in the permutations in the first stage, and hence the memory footprint of the index is not increased. An extensive experimental evaluation of our approach is presented, demonstrating excellent results even on a set of hundreds of millions of objects.Source: Information systems (Oxf.) 95 (2021). doi:10.1016/j.is.2020.101506
DOI: 10.1016/j.is.2020.101506
Project(s): AI4EU
Metrics:
Vadicamo L., Gennaro C., Falchi F., Chavez E., Connor R., Amato G.
Approximate Nearest Neighbor (ANN) search is a prevalent paradigm for searching intrinsically high dimensional objects in large-scale data sets. Recently, the permutation-based approach for ANN has attracted a lot of interest due to its versatility in being used in the more general class of metric spaces. In this approach, the entire database is ranked by a permutation distance to the query. Typically, permutations allow the efficient selection of a candidate set of results, but typically to achieve high recall or precision this set has to be reviewed using the original metric and data. This can lead to a sizeable percentage of the database being recalled, along with many expensive distance calculations. To reduce the number of metric computations and the number of database elements accessed, we propose here a re-ranking based on a local embedding using the nSimplex projection. The nSimplex projection produces Euclidean vectors from objects in metric spaces which possess the n-point property. The mapping is obtained from the distances to a set of reference objects, and the original metric can be lower bounded and upper bounded by the Euclidean distance of objects sharing the same set of references. Our approach is particularly advantageous for extensive databases or expensive metric function. We reuse the distances computed in the permutations in the first stage, and hence the memory footprint of the index is not increased. An extensive experimental evaluation of our approach is presented, demonstrating excellent results even on a set of hundreds of millions of objects.Source: Information systems (Oxf.) 95 (2021). doi:10.1016/j.is.2020.101506
DOI: 10.1016/j.is.2020.101506
Project(s): AI4EU
Metrics:
See at:
ZENODO | Information Systems | Information Systems | CNR ExploRA | www.sciencedirect.com
2021
Journal article
Open Access
The VISIONE video search system: exploiting off-the-shelf text search engines for large-scale video retrieval
Amato G., Bolettieri P., Carrara F., Debole F., Falchi F., Gennaro C., Vadicamo L., Vairo C.
This paper describes in detail VISIONE, a video search system that allows users to search for videos using textual keywords, the occurrence of objects and their spatial relationships, the occurrence of colors and their spatial relationships, and image similarity. These modalities can be combined together to express complex queries and meet users' needs. The peculiarity of our approach is that we encode all information extracted from the keyframes, such as visual deep features, tags, color and object locations, using a convenient textual encoding that is indexed in a single text retrieval engine. This offers great flexibility when results corresponding to various parts of the query (visual, text and locations) need to be merged. In addition, we report an extensive analysis of the retrieval performance of the system, using the query logs generated during the Video Browser Showdown (VBS) 2019 competition. This allowed us to fine-tune the system by choosing the optimal parameters and strategies from those we tested.Source: JOURNAL OF IMAGING 7 (2021). doi:10.3390/jimaging7050076
DOI: 10.3390/jimaging7050076
DOI: 10.48550/arxiv.2008.02749
Project(s): AI4Media
Metrics:
Amato G., Bolettieri P., Carrara F., Debole F., Falchi F., Gennaro C., Vadicamo L., Vairo C.
This paper describes in detail VISIONE, a video search system that allows users to search for videos using textual keywords, the occurrence of objects and their spatial relationships, the occurrence of colors and their spatial relationships, and image similarity. These modalities can be combined together to express complex queries and meet users' needs. The peculiarity of our approach is that we encode all information extracted from the keyframes, such as visual deep features, tags, color and object locations, using a convenient textual encoding that is indexed in a single text retrieval engine. This offers great flexibility when results corresponding to various parts of the query (visual, text and locations) need to be merged. In addition, we report an extensive analysis of the retrieval performance of the system, using the query logs generated during the Video Browser Showdown (VBS) 2019 competition. This allowed us to fine-tune the system by choosing the optimal parameters and strategies from those we tested.Source: JOURNAL OF IMAGING 7 (2021). doi:10.3390/jimaging7050076
DOI: 10.3390/jimaging7050076
DOI: 10.48550/arxiv.2008.02749
Project(s): AI4Media
Metrics:
See at:
arXiv.org e-Print Archive | Journal of Imaging | Journal of Imaging | ISTI Repository | ISTI Repository | CNR ExploRA | DOAJ-Articles | www.mdpi.com | Journal of Imaging | ZENODO | doi.org
2021
Journal article
Embargo
Query filtering using two-dimensional local embeddings
Vadicamo L., Connor R., Chávez E.
In high dimensional data sets, exact indexes are ineffective for proximity queries, and a sequential scan over the entire data set is unavoidable. Accepting this, here we present a new approach employing two-dimensional embeddings. Each database element is mapped to the XY plane using the four-point property. The caveat is that the mapping is local: in other words, each object is mapped using a different mapping. The idea is that each element of the data is associated with a pair of reference objects that is well-suited to filter that particular object, in cases where it is not relevant to a query. This maximises the probability of excluding that object from a search. At query time, a query is compared with a pool of reference objects which allow its mapping to all the planes used by data objects. Then, for each query/object pair, a lower bound of the actual distance is obtained. The technique can be applied to any metric space that possesses the four-point property, therefore including Euclidean, Cosine, Triangular, Jensen-Shannon, and Quadratic Form distances. Our experiments show that for all the data sets tested, of varying dimensionality, our approach can filter more objects than a standard metric indexing approach. For low dimensional data this does not make a good search mechanism in its own right, as it does not scale with the size of the data: that is, its cost is linear with respect to the data size. However, we also show that it can be added as a post-filter to other mechanisms, increasing efficiency with little extra cost in space or time. For high-dimensional data, we show related approximate techniques which, we believe, give the best known compromise for speeding up the essential sequential scan. The potential uses of our filtering technique include pure GPU searching, taking advantage of the tiny memory footprint of the mapping.Source: Information systems (Oxf.) 101 (2021). doi:10.1016/j.is.2021.101808
DOI: 10.1016/j.is.2021.101808
Metrics:
Vadicamo L., Connor R., Chávez E.
In high dimensional data sets, exact indexes are ineffective for proximity queries, and a sequential scan over the entire data set is unavoidable. Accepting this, here we present a new approach employing two-dimensional embeddings. Each database element is mapped to the XY plane using the four-point property. The caveat is that the mapping is local: in other words, each object is mapped using a different mapping. The idea is that each element of the data is associated with a pair of reference objects that is well-suited to filter that particular object, in cases where it is not relevant to a query. This maximises the probability of excluding that object from a search. At query time, a query is compared with a pool of reference objects which allow its mapping to all the planes used by data objects. Then, for each query/object pair, a lower bound of the actual distance is obtained. The technique can be applied to any metric space that possesses the four-point property, therefore including Euclidean, Cosine, Triangular, Jensen-Shannon, and Quadratic Form distances. Our experiments show that for all the data sets tested, of varying dimensionality, our approach can filter more objects than a standard metric indexing approach. For low dimensional data this does not make a good search mechanism in its own right, as it does not scale with the size of the data: that is, its cost is linear with respect to the data size. However, we also show that it can be added as a post-filter to other mechanisms, increasing efficiency with little extra cost in space or time. For high-dimensional data, we show related approximate techniques which, we believe, give the best known compromise for speeding up the essential sequential scan. The potential uses of our filtering technique include pure GPU searching, taking advantage of the tiny memory footprint of the mapping.Source: Information systems (Oxf.) 101 (2021). doi:10.1016/j.is.2021.101808
DOI: 10.1016/j.is.2021.101808
Metrics:
See at:
CNR ExploRA | www.sciencedirect.com