2025
Journal article
Open Access
HDRT: a large-scale dataset for infrared-guided HDR imaging
Peng Ji., Bashford-Rogers T., Banterle F., Zhao H., Debattista K.
Capturing images with enough details to solve imaging tasks is a long-standing challenge in imaging, particularly due to the limitations of standard dynamic range (SDR) images which often lose details in underexposed or overexposed regions. Traditional high dynamic range (HDR) methods, like multi-exposure fusion or inverse tone mapping, struggle with ghosting and incomplete data reconstruction. Infrared (IR) imaging offers a unique advantage by being less affected by lighting conditions, providing consistent detail capture regardless of visible light intensity. In this paper, we introduce the HDRT dataset, the first comprehensive dataset that consists of HDR and thermal IR images. The HDRT dataset comprises 50,000 images captured across three seasons over six months in eight cities, providing a diverse range of lighting conditions and environmental contexts. Leveraging this dataset, we propose HDRTNet, a novel deep neural method that fuses IR and SDR content to generate HDR images. Extensive experiments validate HDRTNet against the state-of-the-art, showing substantial quantitative and qualitative quality improvements. The HDRT dataset not only advances IR-guided HDR imaging but also offers significant potential for broader research in HDR imaging, multi-modal fusion, domain transfer, and beyond. The dataset is available at https://huggingface.co/datasets/jingchao-peng/HDRTDataset.Source: INFORMATION FUSION, vol. 120
DOI: 10.1016/j.inffus.2025.103109
DOI: 10.48550/arxiv.2406.05475
Metrics:
Peng Ji., Bashford-Rogers T., Banterle F., Zhao H., Debattista K.
Capturing images with enough details to solve imaging tasks is a long-standing challenge in imaging, particularly due to the limitations of standard dynamic range (SDR) images which often lose details in underexposed or overexposed regions. Traditional high dynamic range (HDR) methods, like multi-exposure fusion or inverse tone mapping, struggle with ghosting and incomplete data reconstruction. Infrared (IR) imaging offers a unique advantage by being less affected by lighting conditions, providing consistent detail capture regardless of visible light intensity. In this paper, we introduce the HDRT dataset, the first comprehensive dataset that consists of HDR and thermal IR images. The HDRT dataset comprises 50,000 images captured across three seasons over six months in eight cities, providing a diverse range of lighting conditions and environmental contexts. Leveraging this dataset, we propose HDRTNet, a novel deep neural method that fuses IR and SDR content to generate HDR images. Extensive experiments validate HDRTNet against the state-of-the-art, showing substantial quantitative and qualitative quality improvements. The HDRT dataset not only advances IR-guided HDR imaging but also offers significant potential for broader research in HDR imaging, multi-modal fusion, domain transfer, and beyond. The dataset is available at https://huggingface.co/datasets/jingchao-peng/HDRTDataset.Source: INFORMATION FUSION, vol. 120
DOI: 10.1016/j.inffus.2025.103109
DOI: 10.48550/arxiv.2406.05475
Metrics:
See at:
arXiv.org e-Print Archive 
| Information Fusion 
| doi.org | CNR IRIS | CNR IRIS | www.sciencedirect.com
2024
Conference article
Open Access
A study on the use of high dynamic range imaging for gaussian splatting methods: are 8 bits enough?
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
The recent rise of Neural Radiance Fields (NeRFs)-like methods has revolutionized high-fidelity scene reconstruction, with 3D Gaussian Splatting (3DGS) standing out for its ability to generate photorealistic images while maintaining fast, efficient rendering. 3DGS delivers high-fidelity representations of complex scenes at any scale (from very small objects to entire cities), accurately capturing geometry, materials, and lighting, while meeting the need for fast and efficient rendering-crucial for applications requiring real-time performance. Although High Dynamic Range (HDR) technology, which enables the capture of comprehensive real-world lighting information, has been used in novel view synthesis, several questions remain unanswered. For example, does HDR improve the overall quality of reconstruction? Are 8 bits enough? Can tone mapped images be a balanced compromise regarding quality and details? To answer such questions, in this work, we study the application of HDR technology on the 3DGS method for acquiring real-world scenes.DOI: 10.2312/stag.20241341
Project(s): Future-Oriented REsearch LABoratory
Metrics:
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
The recent rise of Neural Radiance Fields (NeRFs)-like methods has revolutionized high-fidelity scene reconstruction, with 3D Gaussian Splatting (3DGS) standing out for its ability to generate photorealistic images while maintaining fast, efficient rendering. 3DGS delivers high-fidelity representations of complex scenes at any scale (from very small objects to entire cities), accurately capturing geometry, materials, and lighting, while meeting the need for fast and efficient rendering-crucial for applications requiring real-time performance. Although High Dynamic Range (HDR) technology, which enables the capture of comprehensive real-world lighting information, has been used in novel view synthesis, several questions remain unanswered. For example, does HDR improve the overall quality of reconstruction? Are 8 bits enough? Can tone mapped images be a balanced compromise regarding quality and details? To answer such questions, in this work, we study the application of HDR technology on the 3DGS method for acquiring real-world scenes.DOI: 10.2312/stag.20241341
Project(s): Future-Oriented REsearch LABoratory
Metrics:
See at:
diglib.eg.org | CNR IRIS | CNR IRIS
2024
Journal article
Open Access
Perceptual quality assessment of NeRF and neural view synthesis methods for front-facing views
Liang H., Wu T., Hanji P., Banterle F., Gao H., Mantiuk R., Oztireli C.
Neural view synthesis (NVS) is one of the most successful techniques for synthesizing free viewpoint videos, capable of achieving high fidelity from only a sparse set of captured images. This success has led to many variants of the techniques, each evaluated on a set of test views typically using image quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research on how NVS methods perform with respect to perceived video quality. We present the first study on perceptual evaluation of NVS and NeRF variants. For this study, we collected two datasets of scenes captured in a controlled lab environment as well as in-the-wild. In contrast to existing datasets, these scenes come with reference video sequences, allowing us to test for temporal artifacts and subtle distortions that are easily overlooked when viewing only static images. We measured the quality of videos synthesized by several NVS methods in a well-controlled perceptual quality assessment experiment as well as with many existing state-of-the-art image/video quality metrics. We present a detailed analysis of the results and recommendations for dataset and metric selection for NVS evaluation.Source: COMPUTER GRAPHICS FORUM, vol. 43 (issue 2)
DOI: 10.1111/cgf.15036
DOI: 10.17863/cam.106658
DOI: 10.48550/arxiv.2303.15206
Project(s): RealVision
, Machine learning methods for rendering on a high-dynamic-range multi-focal-plane display
Metrics:
Liang H., Wu T., Hanji P., Banterle F., Gao H., Mantiuk R., Oztireli C.
Neural view synthesis (NVS) is one of the most successful techniques for synthesizing free viewpoint videos, capable of achieving high fidelity from only a sparse set of captured images. This success has led to many variants of the techniques, each evaluated on a set of test views typically using image quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research on how NVS methods perform with respect to perceived video quality. We present the first study on perceptual evaluation of NVS and NeRF variants. For this study, we collected two datasets of scenes captured in a controlled lab environment as well as in-the-wild. In contrast to existing datasets, these scenes come with reference video sequences, allowing us to test for temporal artifacts and subtle distortions that are easily overlooked when viewing only static images. We measured the quality of videos synthesized by several NVS methods in a well-controlled perceptual quality assessment experiment as well as with many existing state-of-the-art image/video quality metrics. We present a detailed analysis of the results and recommendations for dataset and metric selection for NVS evaluation.Source: COMPUTER GRAPHICS FORUM, vol. 43 (issue 2)
DOI: 10.1111/cgf.15036
DOI: 10.17863/cam.106658
DOI: 10.48550/arxiv.2303.15206
Project(s): RealVision
, Machine learning methods for rendering on a high-dynamic-range multi-focal-plane displayMetrics:
See at:
arXiv.org e-Print Archive | Apollo | Apollo | Apollo | IRIS Cnr | IRIS Cnr | Apollo | Apollo | Apollo | doi.org | CNR IRIS
2024
Journal article
Open Access
Evaluating image-based interactive 3D modeling tools
Siddique A., Cignoni P., Corsini M., Banterle F.
Structure from Motion (SfM) is a computer vision technique used to reconstruct three-dimensional (3D) structures from a series of two-dimensional (2D) images or video frames. However, SfM tools struggle with transparent objects, reflective surfaces, and low-resolution frames. In such situations, image-based interactive 3D modeling software packages are employed to model 3D objects and measure dimensions. Our contributions to this work are twofold. First, we have introduced new tools to improve 3D modeling software packages; such tools are aimed at easing the workload for users. Second, we have conducted a comprehensive user study to evaluate the efficacy of popular 3d modeling software packages. The task is to measure certain dimensions for which ground truth measurements are already known. A relative error is calculated for every measurement. The evaluation of each software tool is done through survey form, event logs, and measurement relative error. The results of this user study clearly show that our approach to 3D modeling using multiple images has a lower relative error and produces higher quality 3D models than other software packages. In addition, it shows our new tools reduce the required time for completing a task.Source: IEEE ACCESS, vol. 12, pp. 104138-104152
DOI: 10.1109/access.2024.3434584
Project(s): "Photogrammetric Method for Determining BWR Internals Dimensions, EVOCATION
Metrics:
Siddique A., Cignoni P., Corsini M., Banterle F.
Structure from Motion (SfM) is a computer vision technique used to reconstruct three-dimensional (3D) structures from a series of two-dimensional (2D) images or video frames. However, SfM tools struggle with transparent objects, reflective surfaces, and low-resolution frames. In such situations, image-based interactive 3D modeling software packages are employed to model 3D objects and measure dimensions. Our contributions to this work are twofold. First, we have introduced new tools to improve 3D modeling software packages; such tools are aimed at easing the workload for users. Second, we have conducted a comprehensive user study to evaluate the efficacy of popular 3d modeling software packages. The task is to measure certain dimensions for which ground truth measurements are already known. A relative error is calculated for every measurement. The evaluation of each software tool is done through survey form, event logs, and measurement relative error. The results of this user study clearly show that our approach to 3D modeling using multiple images has a lower relative error and produces higher quality 3D models than other software packages. In addition, it shows our new tools reduce the required time for completing a task.Source: IEEE ACCESS, vol. 12, pp. 104138-104152
DOI: 10.1109/access.2024.3434584
Project(s): "Photogrammetric Method for Determining BWR Internals Dimensions, EVOCATION
Metrics:
See at:
IEEE Access | IEEE Access | IRIS Cnr | IRIS Cnr | CNR IRIS
2024
Journal article
Open Access
Self-supervised high dynamic range imaging: what can be learned from a single 8-bit video?
Banterle F., Marnerides D., Bashford-Rogers T., Debattista K.
Recently, Deep Learning-based methods for inverse tone mapping standard dynamic range (SDR) images to obtain high dynamic range (HDR) images have become very popular. These methods manage to fill over-exposed areas convincingly both in terms of details and dynamic range. To be effective, deep learning-based methods need to learn from large datasets and transfer this knowledge to the network weights. In this work, we tackle this problem from a completely different perspective. What can we learn from a single SDR 8-bit video? With the presented self-supervised approach, we show that, in many cases, a single SDR video is sufficient to generate an HDR video of the same quality or better than other state-of-the-art methods.Source: ACM TRANSACTIONS ON GRAPHICS, vol. 43 (issue 2), pp. 1-16
DOI: 10.1145/3648570
Metrics:
Banterle F., Marnerides D., Bashford-Rogers T., Debattista K.
Recently, Deep Learning-based methods for inverse tone mapping standard dynamic range (SDR) images to obtain high dynamic range (HDR) images have become very popular. These methods manage to fill over-exposed areas convincingly both in terms of details and dynamic range. To be effective, deep learning-based methods need to learn from large datasets and transfer this knowledge to the network weights. In this work, we tackle this problem from a completely different perspective. What can we learn from a single SDR 8-bit video? With the presented self-supervised approach, we show that, in many cases, a single SDR video is sufficient to generate an HDR video of the same quality or better than other state-of-the-art methods.Source: ACM TRANSACTIONS ON GRAPHICS, vol. 43 (issue 2), pp. 1-16
DOI: 10.1145/3648570
Metrics:
See at:
dl.acm.org | CNR IRIS | IRIS Cnr | ACM Transactions on Graphics | IRIS Cnr | CNR IRIS | CNR IRIS
2024
Conference article
Open Access
Re:Draw - context aware translation as a controllable method for artistic production
Cardoso J. L., Banterle F., Cignoni P., Wimmer M.
We introduce context-aware translation, a novel method that combines the benefits of inpainting and image-to-image translation, respecting simultane- ously the original input and contextual relevance – where existing methods fall short. By doing so, our method opens new avenues for the controllable use of AI within artistic creation, from animation to digital art. As an use case, we apply our method to redraw any hand-drawn animated character eyes based on any design specifications – eyes serve as a focal point that captures viewer attention and conveys a range of emotions; however, the labor-intensive na- ture of traditional animation often leads to compro- mises in the complexity and consistency of eye de- sign. Furthermore, we remove the need for produc- tion data for training and introduce a new charac- ter recognition method that surpasses existing work by not requiring fine-tuning to specific productions. This proposed use case could help maintain consis- tency throughout production and unlock bolder and more detailed design choices without the produc- tion cost drawbacks. A user study shows context- aware translation is preferred over existing work 95.16% of the time.DOI: 10.24963/ijcai.2024/842
Metrics:
Cardoso J. L., Banterle F., Cignoni P., Wimmer M.
We introduce context-aware translation, a novel method that combines the benefits of inpainting and image-to-image translation, respecting simultane- ously the original input and contextual relevance – where existing methods fall short. By doing so, our method opens new avenues for the controllable use of AI within artistic creation, from animation to digital art. As an use case, we apply our method to redraw any hand-drawn animated character eyes based on any design specifications – eyes serve as a focal point that captures viewer attention and conveys a range of emotions; however, the labor-intensive na- ture of traditional animation often leads to compro- mises in the complexity and consistency of eye de- sign. Furthermore, we remove the need for produc- tion data for training and introduce a new charac- ter recognition method that surpasses existing work by not requiring fine-tuning to specific productions. This proposed use case could help maintain consis- tency throughout production and unlock bolder and more detailed design choices without the produc- tion cost drawbacks. A user study shows context- aware translation is preferred over existing work 95.16% of the time.DOI: 10.24963/ijcai.2024/842
Metrics:
See at:
arXiv.org e-Print Archive | CNR IRIS | www.ijcai.org | doi.org | CNR IRIS | CNR IRIS
2023
Journal article
Open Access
NoR-VDPNet++: real-time no-reference image quality metrics
Banterle F, Artusi A, Moreo A, Carrara F, Cignoni P
Efficiency and efficacy are desirable properties for any evaluation metric having to do with Standard Dynamic Range (SDR) imaging or with High Dynamic Range (HDR) imaging. However, it is a daunting task to satisfy both properties simultaneously. On the one side, existing evaluation metrics like HDR-VDP 2.2 can accurately mimic the Human Visual System (HVS), but this typically comes at a very high computational cost. On the other side, computationally cheaper alternatives (e.g., PSNR, MSE, etc.) fail to capture many crucial aspects of the HVS. In this work, we present NoR-VDPNet++, a deep learning architecture for converting full-reference accurate metrics into no-reference metrics thus reducing the computational burden. We show NoR-VDPNet++ can be successfully employed in different application scenarios.Source: IEEE ACCESS, vol. 11, pp. 34544-34553
DOI: 10.1109/access.2023.3263496
Project(s): ENCORE
Metrics:
Banterle F, Artusi A, Moreo A, Carrara F, Cignoni P
Efficiency and efficacy are desirable properties for any evaluation metric having to do with Standard Dynamic Range (SDR) imaging or with High Dynamic Range (HDR) imaging. However, it is a daunting task to satisfy both properties simultaneously. On the one side, existing evaluation metrics like HDR-VDP 2.2 can accurately mimic the Human Visual System (HVS), but this typically comes at a very high computational cost. On the other side, computationally cheaper alternatives (e.g., PSNR, MSE, etc.) fail to capture many crucial aspects of the HVS. In this work, we present NoR-VDPNet++, a deep learning architecture for converting full-reference accurate metrics into no-reference metrics thus reducing the computational burden. We show NoR-VDPNet++ can be successfully employed in different application scenarios.Source: IEEE ACCESS, vol. 11, pp. 34544-34553
DOI: 10.1109/access.2023.3263496
Project(s): ENCORE
Metrics:
See at:
IEEE Access | CNR IRIS | ieeexplore.ieee.org | ISTI Repository | ISTI Repository | CNR IRIS | CNR IRIS
2023
Journal article
Open Access
MoReLab: a software for user-assisted 3D reconstruction
Siddique A., Banterle F., Corsini M., Cignoni P., Sommerville D., Joffe C.
We present MoReLab, a tool for user-assisted 3D reconstruction. This reconstruction requires an understanding of the shapes of the desired objects. Our experiments demonstrate that existing Structure from Motion (SfM) software packages fail to estimate accurate 3D models in low-quality videos due to several issues such as low resolution, featureless surfaces, low lighting, etc. In such scenarios, which are common for industrial utility companies, user assistance becomes necessary to create reliable 3D models. In our system, the user first needs to add features and correspondences manually on multiple video frames. Then, classic camera calibration and bundle adjustment are applied. At this point, MoReLab provides several primitive shape tools such as rectangles, cylinders, curved cylinders, etc., to model different parts of the scene and export 3D meshes. These shapes are essential for modeling industrial equipment whose videos are typically captured by utility companies with old video cameras (low resolution, compression artifacts, etc.) and in disadvantageous lighting conditions (low lighting, torchlight attached to the video camera, etc.). We evaluate our tool on real industrial case scenarios and compare it against existing approaches. Visual comparisons and quantitative results show that MoReLab achieves superior results with regard to other user-interactive 3D modeling tools.Source: Sensors (Basel) 23 (2023). doi:10.3390/s23146456
DOI: 10.3390/s23146456
Project(s): EVOCATION
Metrics:
Siddique A., Banterle F., Corsini M., Cignoni P., Sommerville D., Joffe C.
We present MoReLab, a tool for user-assisted 3D reconstruction. This reconstruction requires an understanding of the shapes of the desired objects. Our experiments demonstrate that existing Structure from Motion (SfM) software packages fail to estimate accurate 3D models in low-quality videos due to several issues such as low resolution, featureless surfaces, low lighting, etc. In such scenarios, which are common for industrial utility companies, user assistance becomes necessary to create reliable 3D models. In our system, the user first needs to add features and correspondences manually on multiple video frames. Then, classic camera calibration and bundle adjustment are applied. At this point, MoReLab provides several primitive shape tools such as rectangles, cylinders, curved cylinders, etc., to model different parts of the scene and export 3D meshes. These shapes are essential for modeling industrial equipment whose videos are typically captured by utility companies with old video cameras (low resolution, compression artifacts, etc.) and in disadvantageous lighting conditions (low lighting, torchlight attached to the video camera, etc.). We evaluate our tool on real industrial case scenarios and compare it against existing approaches. Visual comparisons and quantitative results show that MoReLab achieves superior results with regard to other user-interactive 3D modeling tools.Source: Sensors (Basel) 23 (2023). doi:10.3390/s23146456
DOI: 10.3390/s23146456
Project(s): EVOCATION
Metrics:
See at:
Sensors | ISTI Repository | www.mdpi.com | CNR ExploRA
2022
Journal article
Unknown
AI-Based media coding standards
Basso A, Ribeca P, Bosi M, Pretto N, Chollet G, Guarise M, Choi M, Chiariglione L, Iacoviello R, Banterle F, Artusi A, Gissi F, Fiandrotti A, Ballocca G, Mazzaglia M, Moskowitz S
Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is the first standards organization to develop data coding standards that have artificial intelligence (AI) as their core technology. MPAI believes that universally accessible standards for AI-based data coding can have the same positive effects on AI as standards had on digital media. Elementary components of MPAI standards-AI modules (AIMs)-expose standard interfaces for operation in a standard AI framework (AIF). As their performance may depend on the technologies used, MPAI expects that competing developers providing AIMs will promote horizontal markets of AI solutions that build on and further promote AI innovation. Finally, the MPAI framework licences (FWLs) provide guidelines to intellectual property right (IPR) holders facilitating the availability of compatible licenses to standard users.Source: SMPTE MOTION IMAGING JOURNAL, vol. 131 (issue 4), pp. 10-20
DOI: 10.5594/jmi.2022.3160793
Metrics:
Basso A, Ribeca P, Bosi M, Pretto N, Chollet G, Guarise M, Choi M, Chiariglione L, Iacoviello R, Banterle F, Artusi A, Gissi F, Fiandrotti A, Ballocca G, Mazzaglia M, Moskowitz S
Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is the first standards organization to develop data coding standards that have artificial intelligence (AI) as their core technology. MPAI believes that universally accessible standards for AI-based data coding can have the same positive effects on AI as standards had on digital media. Elementary components of MPAI standards-AI modules (AIMs)-expose standard interfaces for operation in a standard AI framework (AIF). As their performance may depend on the technologies used, MPAI expects that competing developers providing AIMs will promote horizontal markets of AI solutions that build on and further promote AI innovation. Finally, the MPAI framework licences (FWLs) provide guidelines to intellectual property right (IPR) holders facilitating the availability of compatible licenses to standard users.Source: SMPTE MOTION IMAGING JOURNAL, vol. 131 (issue 4), pp. 10-20
DOI: 10.5594/jmi.2022.3160793
Metrics:
See at:
SMPTE Motion Imaging Journal | CNR IRIS | ieeexplore.ieee.org
2022
Open Access
Quantum computing algorithms: getting closer to critical problems in computational biology
Marchetti L., Nifosì R., Martelli P. L., Da Pozzo E., Cappello V., Banterle F., Trincavelli M. L., Martini C., D'Elia M.
The recent biotechnological progress has allowed life scientists and physicians to access an unprecedented, massive amount of data at all levels (molecular, supramolecular, cellular and so on) of biological complexity. So far, mostly classical computational efforts have been dedicated to the simulation, prediction or de novo design of biomolecules, in order to improve the understanding of their function or to develop novel therapeutics. At a higher level of complexity, the progress of omics disciplines (genomics, transcriptomics, proteomics and metabolomics) has prompted researchers to develop informatics means to describe and annotate new biomolecules identified with a resolution down to the single cell, but also with a high-throughput speed. Machine learning approaches have been implemented to both the modelling studies and the handling of biomedical data. Quantum computing (QC) approaches hold the promise to resolve, speed up or refine the analysis of a wide range of these computational problems. Here, we review and comment on recently developed QC algorithms for biocomputing, with a particular focus on multi-scale modelling and genomic analyses. Indeed, differently from other computational approaches such as protein structure prediction, these problems have been shown to be adequately mapped onto quantum architectures, the main limit for their immediate use being the number of qubits and decoherence effects in the available quantum machines. Possible advantages over the classical counterparts are highlighted, along with a description of some hybrid classical/quantum approaches, which could be the closest to be realistically applied in biocomputation.Source: Briefings in bioinformatics 23 (2022). doi:10.1093/bib/bbac437
DOI: 10.1093/bib/bbac437
Metrics:
Marchetti L., Nifosì R., Martelli P. L., Da Pozzo E., Cappello V., Banterle F., Trincavelli M. L., Martini C., D'Elia M.
The recent biotechnological progress has allowed life scientists and physicians to access an unprecedented, massive amount of data at all levels (molecular, supramolecular, cellular and so on) of biological complexity. So far, mostly classical computational efforts have been dedicated to the simulation, prediction or de novo design of biomolecules, in order to improve the understanding of their function or to develop novel therapeutics. At a higher level of complexity, the progress of omics disciplines (genomics, transcriptomics, proteomics and metabolomics) has prompted researchers to develop informatics means to describe and annotate new biomolecules identified with a resolution down to the single cell, but also with a high-throughput speed. Machine learning approaches have been implemented to both the modelling studies and the handling of biomedical data. Quantum computing (QC) approaches hold the promise to resolve, speed up or refine the analysis of a wide range of these computational problems. Here, we review and comment on recently developed QC algorithms for biocomputing, with a particular focus on multi-scale modelling and genomic analyses. Indeed, differently from other computational approaches such as protein structure prediction, these problems have been shown to be adequately mapped onto quantum architectures, the main limit for their immediate use being the number of qubits and decoherence effects in the available quantum machines. Possible advantages over the classical counterparts are highlighted, along with a description of some hybrid classical/quantum approaches, which could be the closest to be realistically applied in biocomputation.Source: Briefings in bioinformatics 23 (2022). doi:10.1093/bib/bbac437
DOI: 10.1093/bib/bbac437
Metrics:
See at:
academic.oup.com | Briefings in Bioinformatics | ISTI Repository | Briefings in Bioinformatics | CNR ExploRA
2021
Contribution to book
Restricted
Virtual clones for cultural heritage applications
Potenziani M, Banterle F, Callieri M, Dellepiane M, Ponchio F, Scopigno R
Digital technologies are now mature for producing high quality digital replicas of Cultural Heritage (CH) artifacts. The research results produced in the last decade have shown an impressive evolution and consolidation of the technologies for acquiring high-quality digital 3D models, encompassing both geometry and color (or, better, surface reflectance properties). Some recent technologies for constructing 3D models enriched by a high-quality encoding of the color attribute will be presented. The focus of this paper is to show and discuss practical solutions, which could be deployed without requiring the installation of a specific or sophisticated acquisition lab setup. In the second part of this paper, we focus on new solutions for the interactive visualization of complex models, adequate for modern communication channels such as the web and the mobile platforms. Together with the algorithms and approaches, we show also some practical examples where high-quality 3D models have been used in CH research, restoration and conservation.
Potenziani M, Banterle F, Callieri M, Dellepiane M, Ponchio F, Scopigno R
Digital technologies are now mature for producing high quality digital replicas of Cultural Heritage (CH) artifacts. The research results produced in the last decade have shown an impressive evolution and consolidation of the technologies for acquiring high-quality digital 3D models, encompassing both geometry and color (or, better, surface reflectance properties). Some recent technologies for constructing 3D models enriched by a high-quality encoding of the color attribute will be presented. The focus of this paper is to show and discuss practical solutions, which could be deployed without requiring the installation of a specific or sophisticated acquisition lab setup. In the second part of this paper, we focus on new solutions for the interactive visualization of complex models, adequate for modern communication channels such as the web and the mobile platforms. Together with the algorithms and approaches, we show also some practical examples where high-quality 3D models have been used in CH research, restoration and conservation.
See at:
CNR IRIS | CNR IRIS | www.lerma.it
2021
Conference article
Open Access
Collaborative visual environments for evidence taking in digital justice: a design concept
Erra U., Capece N., Lettieri N., Fabiani E., Banterle F., Cignoni P., Dazzi P., Aleotti J., Monica R.
In recent years, Spatial Computing (SC) has emerged as a novel paradigm thanks to the advancements in Extended Reality (XR), remote sensing, and artificial intelligence. Computers are nowadays more and more aware of physical environments (i.e. objects shape, size, location and movement) and can use this knowledge to blend technology into reality seamlessly, merge digital and real worlds, and connect users by providing innovative interaction methods. Criminal and civil trials offer an ideal scenario to exploit Spatial Computing. The taking of evidence, indeed, is a complex activity that not only involves several actors (judges, lawyers, clerks, advi- sors) but it often requires accurate topographic surveys of places and objects. Moreover, another essential means of proof, the "judi- cial experiments" - reproductions of real-world events (e.g. a road accident) the judge uses to evaluate if and how a given fact has taken place - could be usefully carried out in virtual environments. In this paper we propose a novel approach for digital justice based on a multi-user, multimodal virtual collaboration platform that enables technology-enhanced acquisition and analysis of trial evidence.Source: FRAME'21 - 1st Workshop on Flexible Resource and Application Management on the Edge, pp. 41–44, Sweden, Virtual Event, 25/06/2021
DOI: 10.1145/3452369.3463820
Project(s): ACCORDION
Metrics:
Erra U., Capece N., Lettieri N., Fabiani E., Banterle F., Cignoni P., Dazzi P., Aleotti J., Monica R.
In recent years, Spatial Computing (SC) has emerged as a novel paradigm thanks to the advancements in Extended Reality (XR), remote sensing, and artificial intelligence. Computers are nowadays more and more aware of physical environments (i.e. objects shape, size, location and movement) and can use this knowledge to blend technology into reality seamlessly, merge digital and real worlds, and connect users by providing innovative interaction methods. Criminal and civil trials offer an ideal scenario to exploit Spatial Computing. The taking of evidence, indeed, is a complex activity that not only involves several actors (judges, lawyers, clerks, advi- sors) but it often requires accurate topographic surveys of places and objects. Moreover, another essential means of proof, the "judi- cial experiments" - reproductions of real-world events (e.g. a road accident) the judge uses to evaluate if and how a given fact has taken place - could be usefully carried out in virtual environments. In this paper we propose a novel approach for digital justice based on a multi-user, multimodal virtual collaboration platform that enables technology-enhanced acquisition and analysis of trial evidence.Source: FRAME'21 - 1st Workshop on Flexible Resource and Application Management on the Edge, pp. 41–44, Sweden, Virtual Event, 25/06/2021
DOI: 10.1145/3452369.3463820
Project(s): ACCORDION
Metrics:
See at:
ISTI Repository | dl.acm.org | CNR ExploRA
2021
Conference article
Open Access
A deep learning method for frame selection in videos for structure from motion pipelines
Banterle F, Gong R, Corsini M, Ganovelli F, Van Gool L, Cignoni P
Structure-from-Motion (SfM) using the frames of a video sequence can be a challenging task because there is a lot of redundant information, the computational time increases quadratically with the number of frames, there would be low-quality images (e.g., blurred frames) that can decrease the final quality of the reconstruction, etc. To overcome all these issues, we present a novel deep-learning architecture that is meant for speeding up SfM by selecting frames using predicted sub-sampling frequency. This architecture is general and can learn/distill the knowledge of any algorithm for selecting frames from a video for generating high-quality reconstructions. One key advantage is that we can run our architecture in real-time saving computations while keeping high-quality results.Source: PROCEEDINGS - INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, pp. 3667-3671. Anchorage, Alaska, USA, 19-22/09/2021
DOI: 10.1109/icip42928.2021.9506227
Project(s): ENCORE
Metrics:
Banterle F, Gong R, Corsini M, Ganovelli F, Van Gool L, Cignoni P
Structure-from-Motion (SfM) using the frames of a video sequence can be a challenging task because there is a lot of redundant information, the computational time increases quadratically with the number of frames, there would be low-quality images (e.g., blurred frames) that can decrease the final quality of the reconstruction, etc. To overcome all these issues, we present a novel deep-learning architecture that is meant for speeding up SfM by selecting frames using predicted sub-sampling frequency. This architecture is general and can learn/distill the knowledge of any algorithm for selecting frames from a video for generating high-quality reconstructions. One key advantage is that we can run our architecture in real-time saving computations while keeping high-quality results.Source: PROCEEDINGS - INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, pp. 3667-3671. Anchorage, Alaska, USA, 19-22/09/2021
DOI: 10.1109/icip42928.2021.9506227
Project(s): ENCORE
Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | ISTI Repository | doi.org | CNR IRIS | CNR IRIS
2021
Book
Open Access
Proceedings - Web3D 2021: 26th ACM International Conference on 3D Web Technology
Ganovelli F, Mc Donald C, Banterle F, Potenziani M, Callieri M, Jung Y
The annual ACM Web3D Conference is a major event which unites researchers, developers, entrepreneurs, experimenters, artists and content creators in a dynamic learning environment. Attendees share and explore methods of using, enhancing and creating new 3D Web and Multimedia technologies such as X3D, VRML, Collada, MPEG family, U3D, Java3D and other technologies. The conference also focuses on recent trends in interactive 3D graphics, information integration and usability in the wide range of Web3D applications from mobile devices to high-end immersive environments.DOI: 10.1145/3485444
Metrics:
Ganovelli F, Mc Donald C, Banterle F, Potenziani M, Callieri M, Jung Y
The annual ACM Web3D Conference is a major event which unites researchers, developers, entrepreneurs, experimenters, artists and content creators in a dynamic learning environment. Attendees share and explore methods of using, enhancing and creating new 3D Web and Multimedia technologies such as X3D, VRML, Collada, MPEG family, U3D, Java3D and other technologies. The conference also focuses on recent trends in interactive 3D graphics, information integration and usability in the wide range of Web3D applications from mobile devices to high-end immersive environments.DOI: 10.1145/3485444
Metrics:
See at:
dl.acm.org | CNR IRIS | ISTI Repository | CNR IRIS
2021
Conference article
Restricted
NoR-VDPNet++: Efficient training and architecture for deep no-reference image quality metrics
Banterle F, Artusi A, Moreo A, Carrara F
Efficiency and efficacy are two desirable properties of the utmost importance for any evaluation metric having to do with Standard Dynamic Range (SDR) imaging or High Dynamic Range (HDR) imaging. However, these properties are hard to achieve simultaneously. On the one side, metrics like HDR-VDP2.2 are known to mimic the human visual system (HVS) very accurately, but its high computational cost prevents its widespread use in large evaluation campaigns. On the other side, computationally cheaper alternatives like PSNR or MSE fail to capture many of the crucial aspects of the HVS. In this work, we try to get the best of the two worlds: we present NoR-VDPNet++, an improved variant of a previous deep learning-based metric for distilling HDR-VDP2.2 into a convolutional neural network (CNN). In this work, we try to get the best of the two worlds: we present NoR-VDPNet++, an improved version of a deep learning-based metric for distilling HDR-VDP2.2 into a convolutional neural network (CNN).DOI: 10.1145/3450623.3464636
Metrics:
Banterle F, Artusi A, Moreo A, Carrara F
Efficiency and efficacy are two desirable properties of the utmost importance for any evaluation metric having to do with Standard Dynamic Range (SDR) imaging or High Dynamic Range (HDR) imaging. However, these properties are hard to achieve simultaneously. On the one side, metrics like HDR-VDP2.2 are known to mimic the human visual system (HVS) very accurately, but its high computational cost prevents its widespread use in large evaluation campaigns. On the other side, computationally cheaper alternatives like PSNR or MSE fail to capture many of the crucial aspects of the HVS. In this work, we try to get the best of the two worlds: we present NoR-VDPNet++, an improved variant of a previous deep learning-based metric for distilling HDR-VDP2.2 into a convolutional neural network (CNN). In this work, we try to get the best of the two worlds: we present NoR-VDPNet++, an improved version of a deep learning-based metric for distilling HDR-VDP2.2 into a convolutional neural network (CNN).DOI: 10.1145/3450623.3464636
Metrics:
See at:
dblp.uni-trier.de | IRIS Cnr | doi.org | IRIS Cnr | CNR IRIS
2021
Conference article
Open Access
Collaborative visual environments for evidence taking in digital justice: a design concept
Erra Ugo, Capece Nicola, Lettieri Nicola, Fabiani Ernesto, Banterle Francesco, Cignoni Paolo, Dazzi Patrizio, Aleotti Jacopo, Monica Riccardo
In recent years, Spatial Computing (SC) has emerged as a novel paradigm thanks to the advancements in Extended Reality (XR), remote sensing, and artificial intelligence. Computers are nowadays more and more aware of physical environments (i.e. objects shape, size, location and movement) and can use this knowledge to blend technology into reality seamlessly, merge digital and real worlds, and connect users by providing innovative interaction methods. Criminal and civil trials offer an ideal scenario to exploit Spatial Computing. The taking of evidence, indeed, is a complex activity that not only involves several actors (judges, lawyers, clerks, advisors) but it often requires accurate topographic surveys of places and objects. Moreover, another essential means of proof, the "judicial experiments"- reproductions of real-world events (e.g. a road accident) the judge uses to evaluate if and how a given fact has taken place - could be usefully carried out in virtual environments. In this paper we propose a novel approach for digital justice based on a multi-user, multimodal virtual collaboration platform that enables technology-enhanced acquisition and analysis of trial evidence.DOI: 10.1145/3452369.3463821
Project(s): ACCORDION, SoBigData-PlusPlus
Metrics:
Erra Ugo, Capece Nicola, Lettieri Nicola, Fabiani Ernesto, Banterle Francesco, Cignoni Paolo, Dazzi Patrizio, Aleotti Jacopo, Monica Riccardo
In recent years, Spatial Computing (SC) has emerged as a novel paradigm thanks to the advancements in Extended Reality (XR), remote sensing, and artificial intelligence. Computers are nowadays more and more aware of physical environments (i.e. objects shape, size, location and movement) and can use this knowledge to blend technology into reality seamlessly, merge digital and real worlds, and connect users by providing innovative interaction methods. Criminal and civil trials offer an ideal scenario to exploit Spatial Computing. The taking of evidence, indeed, is a complex activity that not only involves several actors (judges, lawyers, clerks, advisors) but it often requires accurate topographic surveys of places and objects. Moreover, another essential means of proof, the "judicial experiments"- reproductions of real-world events (e.g. a road accident) the judge uses to evaluate if and how a given fact has taken place - could be usefully carried out in virtual environments. In this paper we propose a novel approach for digital justice based on a multi-user, multimodal virtual collaboration platform that enables technology-enhanced acquisition and analysis of trial evidence.DOI: 10.1145/3452369.3463821
Project(s): ACCORDION
, SoBigData-PlusPlus Metrics:
See at:
IRIS Cnr | IRIS Cnr | IRIS Cnr | dblp.uni-trier.de | dl.acm.org | dl.acm.org | doi.org | Archivio della Ricerca - Università di Pisa | Archivio della Ricerca - Università di Pisa | IRIS Cnr | CNR IRIS | iris.unibas.it | IRIS Cnr
2020
Journal article
Closed Access
Turning a Smartphone Selfie into a Studio Portrait
Capece N., Banterle F., Cignoni P., Ganovelli F., Erra U., Potel M.
We introduce a novel algorithm that turns a flash selfie taken with a smartphone into a studio-like photograph with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in a controlled environment. For each pair, we have one photograph of a subject's face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend lighting artifacts introduced by a close-up camera flash, such as specular highlights, shadows, and skin shine.Source: IEEE computer graphics and applications 40 (2020): 140–147. doi:10.1109/MCG.2019.2958274
DOI: 10.1109/mcg.2019.2958274
Metrics:
Capece N., Banterle F., Cignoni P., Ganovelli F., Erra U., Potel M.
We introduce a novel algorithm that turns a flash selfie taken with a smartphone into a studio-like photograph with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in a controlled environment. For each pair, we have one photograph of a subject's face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend lighting artifacts introduced by a close-up camera flash, such as specular highlights, shadows, and skin shine.Source: IEEE computer graphics and applications 40 (2020): 140–147. doi:10.1109/MCG.2019.2958274
DOI: 10.1109/mcg.2019.2958274
Metrics:
See at:
IEEE Computer Graphics and Applications | ieeexplore.ieee.org | CNR ExploRA
2020
Conference article
Open Access
Nor-Vdpnet: a no-reference high dynamic range quality metric trained on Hdr-Vdp 2
Banterle F, Artusi A, Moreo A, Carrara F
HDR-VDP 2 has convincingly shown to be a reliable metric for image quality assessment, and it is currently playing a remarkable role in the evaluation of complex image processing algorithms. However, HDR-VDP 2 is known to be computationally expensive (both in terms of time and memory) and is constrained to the availability of a ground-truth image (the so-called reference) against to which the quality of a processed imaged is quantified. These aspects impose severe limitations on the applicability of HDR-VDP 2 to realworld scenarios involving large quantities of data or requiring real-time responses. To address these issues, we propose Deep No-Reference Quality Metric (NoR-VDPNet), a deeplearning approach that learns to predict the global image quality feature (i.e., the mean-opinion-score index Q) that HDRVDP 2 computes. NoR-VDPNet is no-reference (i.e., it operates without a ground truth reference) and its computational cost is substantially lower when compared to HDR-VDP 2 (by more than an order of magnitude). We demonstrate the performance of NoR-VDPNet in a variety of scenarios, including the optimization of parameters of a denoiser and JPEG-XT.DOI: 10.1109/icip40778.2020.9191202
Project(s): EVOCATION, ENCORE , RISE
Metrics:
Banterle F, Artusi A, Moreo A, Carrara F
HDR-VDP 2 has convincingly shown to be a reliable metric for image quality assessment, and it is currently playing a remarkable role in the evaluation of complex image processing algorithms. However, HDR-VDP 2 is known to be computationally expensive (both in terms of time and memory) and is constrained to the availability of a ground-truth image (the so-called reference) against to which the quality of a processed imaged is quantified. These aspects impose severe limitations on the applicability of HDR-VDP 2 to realworld scenarios involving large quantities of data or requiring real-time responses. To address these issues, we propose Deep No-Reference Quality Metric (NoR-VDPNet), a deeplearning approach that learns to predict the global image quality feature (i.e., the mean-opinion-score index Q) that HDRVDP 2 computes. NoR-VDPNet is no-reference (i.e., it operates without a ground truth reference) and its computational cost is substantially lower when compared to HDR-VDP 2 (by more than an order of magnitude). We demonstrate the performance of NoR-VDPNet in a variety of scenarios, including the optimization of parameters of a denoiser and JPEG-XT.DOI: 10.1109/icip40778.2020.9191202
Project(s): EVOCATION
, ENCORE , RISE Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | ISTI Repository | ISTI Repository | zenodo.org | doi.org | CNR IRIS | CNR IRIS | CNR IRIS
2020
Conference article
Open Access
ViDA 3D: towards a view-based dataset for aesthetic prediction on 3D models
Angelini M., Ferrulli V., Banterle F., Corsini M., Pascali M. A., Cignoni P., Giorgi D.
We present the ongoing effort to build the first benchmark dataset for aestethic prediction on 3D models. The dataset is built on top of Sketchfab, a popular platform for 3D content sharing. In our dataset, the visual 3D content is aligned with aestheticsrelated metadata: each 3D model is associated with a number of snapshots taken from different camera positions, the number of times the model has been viewed in-between its upload and its retrieval, the number of likes the model got, and the tags and comments received from users. The metadata provide precious supervisory information for data-driven research on 3D visual attractiveness and preference prediction.The paper contribution is twofold. First, we introduce an interactive platform for visualizing data about Sketchfab. We report a detailed qualitative and quantitative analysis of numerical scores (views and likes collected by 3D models) and textual information (tags and comments) for different 3D object categories. The analysis of the content of Sketchfab provided us the base for selecting a reasoned subset of annotated models. The second contribution is the first version of the ViDA 3D dataset, which contains the full set of content required for data-driven approaches to 3D aesthetic analysis. While similar datasets are available for images, to our knowledge this is the first attempt to create a benchmark for aestethic prediction for 3D models. We believe our dataset can be a great resource to boost research on this hot and far-from-solved problem.Source: ITALIAN CHAPTER CONFERENCE, pp. 45-55
DOI: 10.2312/stag.20201239
Metrics:
Angelini M., Ferrulli V., Banterle F., Corsini M., Pascali M. A., Cignoni P., Giorgi D.
We present the ongoing effort to build the first benchmark dataset for aestethic prediction on 3D models. The dataset is built on top of Sketchfab, a popular platform for 3D content sharing. In our dataset, the visual 3D content is aligned with aestheticsrelated metadata: each 3D model is associated with a number of snapshots taken from different camera positions, the number of times the model has been viewed in-between its upload and its retrieval, the number of likes the model got, and the tags and comments received from users. The metadata provide precious supervisory information for data-driven research on 3D visual attractiveness and preference prediction.The paper contribution is twofold. First, we introduce an interactive platform for visualizing data about Sketchfab. We report a detailed qualitative and quantitative analysis of numerical scores (views and likes collected by 3D models) and textual information (tags and comments) for different 3D object categories. The analysis of the content of Sketchfab provided us the base for selecting a reasoned subset of annotated models. The second contribution is the first version of the ViDA 3D dataset, which contains the full set of content required for data-driven approaches to 3D aesthetic analysis. While similar datasets are available for images, to our knowledge this is the first attempt to create a benchmark for aestethic prediction for 3D models. We believe our dataset can be a great resource to boost research on this hot and far-from-solved problem.Source: ITALIAN CHAPTER CONFERENCE, pp. 45-55
DOI: 10.2312/stag.20201239
Metrics:
See at:
diglib.eg.org | CNR IRIS | doi.org | IRIS Cnr | CNR IRIS
2019
Journal article
Open Access
DeepFlash: turning a flash selfie into a studio portrait
Capece N, Banterle F, Cignoni P, Ganovelli F, Scopigno R, Erra U
We present a method for turning a flash selfie taken with a smartphone into a photograph as if it was taken in a studio setting with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in an ad-hoc acquisition campaign. Each pair consists of one photograph of a subject's face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend defects introduced by a close-up camera flash, such as specular highlights, shadows, skin shine, and flattened images.Source: SIGNAL PROCESSING-IMAGE COMMUNICATION, vol. 77 (issue September), pp. 28-39
DOI: 10.1016/j.image.2019.05.013
DOI: 10.48550/arxiv.1901.04252
Metrics:
Capece N, Banterle F, Cignoni P, Ganovelli F, Scopigno R, Erra U
We present a method for turning a flash selfie taken with a smartphone into a photograph as if it was taken in a studio setting with uniform lighting. Our method uses a convolutional neural network trained on a set of pairs of photographs acquired in an ad-hoc acquisition campaign. Each pair consists of one photograph of a subject's face taken with the camera flash enabled and another one of the same subject in the same pose illuminated using a photographic studio-lighting setup. We show how our method can amend defects introduced by a close-up camera flash, such as specular highlights, shadows, skin shine, and flattened images.Source: SIGNAL PROCESSING-IMAGE COMMUNICATION, vol. 77 (issue September), pp. 28-39
DOI: 10.1016/j.image.2019.05.013
DOI: 10.48550/arxiv.1901.04252
Metrics:
See at:
arXiv.org e-Print Archive | Signal Processing Image Communication | CNR IRIS | ISTI Repository | Signal Processing Image Communication | doi.org | CNR IRIS | CNR IRIS