2026
Journal article
Restricted
Zero-Shot infrared-guided HDR video deflickering
Peng Jingchao, Bashford-Rogers Tthomas, Banterle Francesco, Zhao Haitao, Debattista Kurt
High dynamic range (HDR) imaging is crucial for realistic lighting, but image-trained methods often flicker on video due to the absence of temporal consistency constraints, especially for infrared-guided HDR, where aligning HDR and IR video data is challenging. This paper proposes ZS-HDRTVNet, a zero-shot framework that achieves temporally stable IR-guided HDR video without aligned IR–HDR videos for training. A channel-aligned fusion (CAF) module maps RGB/IR features into orthogonal spaces for simple addition-based fusion and operates flexibly with or without IR. A temporal consistency branch couples optical-flow estimation with occlusion handling to model inter-frame motion and suppress flicker. We train CAF on aligned IR–HDR images and the temporal branch on HDR videos without IR, enabling complementary supervision. Experimental results demonstrate that the proposed method achieves state-of-the-art HDR quality and temporal consistency, effectively eliminating flicker and enhancing visual coherence.Source: PATTERN RECOGNITION, vol. 179
DOI: 10.1016/j.patcog.2026.113759
Project(s): NSFC
Metrics:
Peng Jingchao, Bashford-Rogers Tthomas, Banterle Francesco, Zhao Haitao, Debattista Kurt
High dynamic range (HDR) imaging is crucial for realistic lighting, but image-trained methods often flicker on video due to the absence of temporal consistency constraints, especially for infrared-guided HDR, where aligning HDR and IR video data is challenging. This paper proposes ZS-HDRTVNet, a zero-shot framework that achieves temporally stable IR-guided HDR video without aligned IR–HDR videos for training. A channel-aligned fusion (CAF) module maps RGB/IR features into orthogonal spaces for simple addition-based fusion and operates flexibly with or without IR. A temporal consistency branch couples optical-flow estimation with occlusion handling to model inter-frame motion and suppress flicker. We train CAF on aligned IR–HDR images and the temporal branch on HDR videos without IR, enabling complementary supervision. Experimental results demonstrate that the proposed method achieves state-of-the-art HDR quality and temporal consistency, effectively eliminating flicker and enhancing visual coherence.Source: PATTERN RECOGNITION, vol. 179
DOI: 10.1016/j.patcog.2026.113759
Project(s): NSFC
Metrics:
See at:
CNR IRIS 
| CNR IRIS | www.sciencedirect.com
2025
Conference article
Open Access
A color preserving tone mapping framework in the intrinsic domain
Melcarne S., Cyriac P., Dugelay J. L., Artusi A., Banterle F.
Tone mapping is an essential step in an acquisition or a rendering pipeline to map high dynamic range (HDR) content to a reference display range. The simplest tone mapping approach is to apply a function to the luminance channel of an HDR image and then to propagate the change to the red, green, and blue channels. However, this often causes color distortions since luminance and chrominance channels are interdependent, and modifying one affects the other. We propose a novel tone mapping approach that preliminarily decomposes the image into intrinsic components and leverages them to perform the actual operation. This strategy effectively mitigates color distortions, eliminating the need for post-processing color correction required by many state-of-the-art methods, and it also assists tone mapping operators (TMOs), improving the overall image quality. Our method was validated through quantitative metrics and a psychophysical experiment, both demonstrating its effectiveness.Source: JOURNAL OF PHYSICS. CONFERENCE SERIES, vol. 3128 (issue 1). London, UK, 09-10/09/2025
DOI: 10.1088/1742-6596/3128/1/012008
Metrics:
Melcarne S., Cyriac P., Dugelay J. L., Artusi A., Banterle F.
Tone mapping is an essential step in an acquisition or a rendering pipeline to map high dynamic range (HDR) content to a reference display range. The simplest tone mapping approach is to apply a function to the luminance channel of an HDR image and then to propagate the change to the red, green, and blue channels. However, this often causes color distortions since luminance and chrominance channels are interdependent, and modifying one affects the other. We propose a novel tone mapping approach that preliminarily decomposes the image into intrinsic components and leverages them to perform the actual operation. This strategy effectively mitigates color distortions, eliminating the need for post-processing color correction required by many state-of-the-art methods, and it also assists tone mapping operators (TMOs), improving the overall image quality. Our method was validated through quantitative metrics and a psychophysical experiment, both demonstrating its effectiveness.Source: JOURNAL OF PHYSICS. CONFERENCE SERIES, vol. 3128 (issue 1). London, UK, 09-10/09/2025
DOI: 10.1088/1742-6596/3128/1/012008
Metrics:
See at:
Journal of Physics : Conference Series 
| CNR IRIS | iopscience.iop.org | CNR IRIS
2025
Journal article
Open Access
HDRT: a large-scale dataset for infrared-guided HDR imaging
Peng Ji., Bashford-Rogers T., Banterle F., Zhao H., Debattista K.
Capturing images with enough details to solve imaging tasks is a long-standing challenge in imaging, particularly due to the limitations of standard dynamic range (SDR) images which often lose details in underexposed or overexposed regions. Traditional high dynamic range (HDR) methods, like multi-exposure fusion or inverse tone mapping, struggle with ghosting and incomplete data reconstruction. Infrared (IR) imaging offers a unique advantage by being less affected by lighting conditions, providing consistent detail capture regardless of visible light intensity. In this paper, we introduce the HDRT dataset, the first comprehensive dataset that consists of HDR and thermal IR images. The HDRT dataset comprises 50,000 images captured across three seasons over six months in eight cities, providing a diverse range of lighting conditions and environmental contexts. Leveraging this dataset, we propose HDRTNet, a novel deep neural method that fuses IR and SDR content to generate HDR images. Extensive experiments validate HDRTNet against the state-of-the-art, showing substantial quantitative and qualitative quality improvements. The HDRT dataset not only advances IR-guided HDR imaging but also offers significant potential for broader research in HDR imaging, multi-modal fusion, domain transfer, and beyond. The dataset is available at https://huggingface.co/datasets/jingchao-peng/HDRTDataset.Source: INFORMATION FUSION, vol. 120
DOI: 10.1016/j.inffus.2025.103109
DOI: 10.48550/arxiv.2406.05475
Metrics:
Peng Ji., Bashford-Rogers T., Banterle F., Zhao H., Debattista K.
Capturing images with enough details to solve imaging tasks is a long-standing challenge in imaging, particularly due to the limitations of standard dynamic range (SDR) images which often lose details in underexposed or overexposed regions. Traditional high dynamic range (HDR) methods, like multi-exposure fusion or inverse tone mapping, struggle with ghosting and incomplete data reconstruction. Infrared (IR) imaging offers a unique advantage by being less affected by lighting conditions, providing consistent detail capture regardless of visible light intensity. In this paper, we introduce the HDRT dataset, the first comprehensive dataset that consists of HDR and thermal IR images. The HDRT dataset comprises 50,000 images captured across three seasons over six months in eight cities, providing a diverse range of lighting conditions and environmental contexts. Leveraging this dataset, we propose HDRTNet, a novel deep neural method that fuses IR and SDR content to generate HDR images. Extensive experiments validate HDRTNet against the state-of-the-art, showing substantial quantitative and qualitative quality improvements. The HDRT dataset not only advances IR-guided HDR imaging but also offers significant potential for broader research in HDR imaging, multi-modal fusion, domain transfer, and beyond. The dataset is available at https://huggingface.co/datasets/jingchao-peng/HDRTDataset.Source: INFORMATION FUSION, vol. 120
DOI: 10.1016/j.inffus.2025.103109
DOI: 10.48550/arxiv.2406.05475
Metrics:
See at:
arXiv.org e-Print Archive | CNR IRIS | www.sciencedirect.com | Information Fusion | doi.org | CNR IRIS | CNR IRIS
2025
Journal article
Open Access
Deep chroma compression of tone-mapped images
Milidonis X., Artusi A., Banterle F.
Acquisition of High Dynamic Range (HDR) images is thriving due to the increasing use of smart devices and the demand for high-quality output. Extensive research has focused on developing methods for reducing the luminance range in HDR images using conventional and deep learning-based tone mapping operators to enable accurate reproduction on conventional 8-and 10-bit digital displays. However, these methods often fail to account for pixels that may lie outside the target display's gamut, resulting in visible chromatic distortions or color clipping artifacts. Previous studies suggested that a gamut management step ensures that all pixels remain within the target gamut. However, such approaches are computationally expensive and cannot be deployed on devices with limited computational resources. We propose a generative adversarial network for fast and reliable chroma compression of HDR tone-mapped images. We design a loss function that considers the hue property of generated images to improve color accuracy and train the model on an extensive image dataset. Quantitative experiments demonstrate that the proposed model outperforms state-of-the-art image generation and enhancement networks in color accuracy, while a subjective study suggests that the generated images are on par or superior to those produced by conventional chroma compression methods in terms of visual quality. Additionally, the model achieves real-time performance, showing promising results for deployment on devices with limited computational resources.Source: ACM TRANSACTIONS ON MULTIMEDIA COMPUTING, COMMUNICATIONS AND APPLICATIONS, vol. 21 (issue 12)
DOI: 10.1145/3744925
DOI: 10.48550/arxiv.2409.16032
Project(s): Periodic Reporting for period 4 - RISE (Research Center on Interactive Media, Smart System and Emerging Technologies), RISE
Metrics:
Milidonis X., Artusi A., Banterle F.
Acquisition of High Dynamic Range (HDR) images is thriving due to the increasing use of smart devices and the demand for high-quality output. Extensive research has focused on developing methods for reducing the luminance range in HDR images using conventional and deep learning-based tone mapping operators to enable accurate reproduction on conventional 8-and 10-bit digital displays. However, these methods often fail to account for pixels that may lie outside the target display's gamut, resulting in visible chromatic distortions or color clipping artifacts. Previous studies suggested that a gamut management step ensures that all pixels remain within the target gamut. However, such approaches are computationally expensive and cannot be deployed on devices with limited computational resources. We propose a generative adversarial network for fast and reliable chroma compression of HDR tone-mapped images. We design a loss function that considers the hue property of generated images to improve color accuracy and train the model on an extensive image dataset. Quantitative experiments demonstrate that the proposed model outperforms state-of-the-art image generation and enhancement networks in color accuracy, while a subjective study suggests that the generated images are on par or superior to those produced by conventional chroma compression methods in terms of visual quality. Additionally, the model achieves real-time performance, showing promising results for deployment on devices with limited computational resources.Source: ACM TRANSACTIONS ON MULTIMEDIA COMPUTING, COMMUNICATIONS AND APPLICATIONS, vol. 21 (issue 12)
DOI: 10.1145/3744925
DOI: 10.48550/arxiv.2409.16032
Project(s): Periodic Reporting for period 4 - RISE (Research Center on Interactive Media, Smart System and Emerging Technologies), RISE
Metrics:
See at:
arXiv.org e-Print Archive | dl.acm.org | CNR IRIS | ACM Transactions on Multimedia Computing Communications and Applications | doi.org | CNR IRIS
2025
Journal article
Open Access
Craniocaudal mammograms generation using image-to-image translation techniques
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
Breast cancer is the leading cause of cancer death in women worldwide, emphasizing the need for prevention and early detection. Mammography screening plays a crucial role in secondary prevention, but large datasets of referred mammograms from hospital databases are hard to access due to privacy concerns, and publicly available datasets are often unreliable and unbalanced. We propose a novel workflow using a statistical generative model based on generative adversarial networks to generate high-resolution synthetic mammograms. Utilizing a unique 2D parametric model of the compressed breast in craniocaudal projection and image-to-image translation techniques, our approach allows full and precise control over breast features and the generation of both normal and tumor cases. Quality assessment was conducted through visual analysis, and statistical analysis using the first five statistical moments. Additionally a questionnaire was administered to 45 medical experts (radiologists and radiology residents). The results showed that the features of the real mammograms were accurately replicated in the synthetic ones, the image statistics overall correspond reasonably well, and the two groups of images were statistically indistinguishable in almost all cases according to the experts. The proposed workflow generates realistic synthetic mammograms with fine-tuned features. Synthetic mammograms are powerful tools that can create new or balance existing datasets, allowing for the training of machine learning and deep learning algorithms. These algorithms can then assist radiologists in tasks like classification and segmentation, improving diagnostic performance. The code and dataset are available at: https://github.com/cnr-isti-vclab/CC-Mammograms-Generation_GUI.Source: IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS
DOI: 10.1109/jbhi.2025.3599641
Project(s): "Cloud Computing, Big Data & Cybersecurity” and FoReLab
Metrics:
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
Breast cancer is the leading cause of cancer death in women worldwide, emphasizing the need for prevention and early detection. Mammography screening plays a crucial role in secondary prevention, but large datasets of referred mammograms from hospital databases are hard to access due to privacy concerns, and publicly available datasets are often unreliable and unbalanced. We propose a novel workflow using a statistical generative model based on generative adversarial networks to generate high-resolution synthetic mammograms. Utilizing a unique 2D parametric model of the compressed breast in craniocaudal projection and image-to-image translation techniques, our approach allows full and precise control over breast features and the generation of both normal and tumor cases. Quality assessment was conducted through visual analysis, and statistical analysis using the first five statistical moments. Additionally a questionnaire was administered to 45 medical experts (radiologists and radiology residents). The results showed that the features of the real mammograms were accurately replicated in the synthetic ones, the image statistics overall correspond reasonably well, and the two groups of images were statistically indistinguishable in almost all cases according to the experts. The proposed workflow generates realistic synthetic mammograms with fine-tuned features. Synthetic mammograms are powerful tools that can create new or balance existing datasets, allowing for the training of machine learning and deep learning algorithms. These algorithms can then assist radiologists in tasks like classification and segmentation, improving diagnostic performance. The code and dataset are available at: https://github.com/cnr-isti-vclab/CC-Mammograms-Generation_GUI.Source: IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS
DOI: 10.1109/jbhi.2025.3599641
Project(s): "Cloud Computing, Big Data & Cybersecurity” and FoReLab
Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | IEEE Journal of Biomedical and Health Informatics | CNR IRIS
2025
Contribution to conference
Open Access
NOVA-3DGS: NO-reference objective VAlidation for 3D Gaussian Splatting
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
In recent years, radiance field methods, and in particular 3D Gaussian Splatting (3DGS), have distinguished themselves in the field of image-based rendering and scene reconstruction techniques, gaining significant success in academia and being cited in numerous research papers. Like other methods, 3DGS requires a large and diverse dataset of images for network training as a fundamental step to ensure effectiveness and high-quality results. Consequently, the acquisition phase is highly time-consuming, especially considering that a portion of the acquired dataset is not actually used for training but is reserved for testing. This is necessary because all commonly used metrics for evaluating the quality of 3D reconstructions, such as PSNR and SSIM, are reference-based metrics; i.e., requiring a ground truth. In this work, we present NOVA, a study focused on no-reference evaluation of 3DGS renders, based on key metrics in this field: PSNR and SSIM.DOI: 10.2312/egp.20251024
Metrics:
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
In recent years, radiance field methods, and in particular 3D Gaussian Splatting (3DGS), have distinguished themselves in the field of image-based rendering and scene reconstruction techniques, gaining significant success in academia and being cited in numerous research papers. Like other methods, 3DGS requires a large and diverse dataset of images for network training as a fundamental step to ensure effectiveness and high-quality results. Consequently, the acquisition phase is highly time-consuming, especially considering that a portion of the acquired dataset is not actually used for training but is reserved for testing. This is necessary because all commonly used metrics for evaluating the quality of 3D reconstructions, such as PSNR and SSIM, are reference-based metrics; i.e., requiring a ground truth. In this work, we present NOVA, a study focused on no-reference evaluation of 3DGS renders, based on key metrics in this field: PSNR and SSIM.DOI: 10.2312/egp.20251024
Metrics:
See at:
diglib.eg.org | CNR IRIS | CNR IRIS
2025
Conference article
Open Access
Semantic aware diffusion inverse tone mapping
Goswami A., Sing A. R., Banterle F., Debattista K., Bashford-Rogers T.
Capturing the full luminance range of real-world scenes exceeds the capabilities of most digital cameras, often resulting in detail loss, particularly in bright regions. Inverse tone mapping aims to reconstruct High Dynamic Range (HDR) images from Standard Dynamic Range (SDR) inputs, but typically fails to recover clipped details. This paper presents a novel semantic-aware diffusion-based inpainting approach for inverse tone mapping1. Our method introduces two key contributions: (1) a semantic graph-guided diffusion process to inpaint saturated SDR regions, and (2) a principled HDR lifting formulation inspired by traditional HDR bracketing, designed to complement generative inpainting techniques. Experiments demonstrate that our approach outperforms existing methods both quantitatively and qualitatively across multiple datasets.Source: JOURNAL OF PHYSICS. CONFERENCE SERIES, vol. 3128. London, UK, 09-10/09/2025
DOI: 10.1088/1742-6596/3128/1/012009
DOI: 10.48550/arxiv.2405.15468
Metrics:
Goswami A., Sing A. R., Banterle F., Debattista K., Bashford-Rogers T.
Capturing the full luminance range of real-world scenes exceeds the capabilities of most digital cameras, often resulting in detail loss, particularly in bright regions. Inverse tone mapping aims to reconstruct High Dynamic Range (HDR) images from Standard Dynamic Range (SDR) inputs, but typically fails to recover clipped details. This paper presents a novel semantic-aware diffusion-based inpainting approach for inverse tone mapping1. Our method introduces two key contributions: (1) a semantic graph-guided diffusion process to inpaint saturated SDR regions, and (2) a principled HDR lifting formulation inspired by traditional HDR bracketing, designed to complement generative inpainting techniques. Experiments demonstrate that our approach outperforms existing methods both quantitatively and qualitatively across multiple datasets.Source: JOURNAL OF PHYSICS. CONFERENCE SERIES, vol. 3128. London, UK, 09-10/09/2025
DOI: 10.1088/1742-6596/3128/1/012009
DOI: 10.48550/arxiv.2405.15468
Metrics:
See at:
arXiv.org e-Print Archive | Journal of Physics : Conference Series | CNR IRIS | iopscience.iop.org | doi.org | CNR IRIS
2024
Conference article
Open Access
A study on the use of high dynamic range imaging for gaussian splatting methods: are 8 bits enough?
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
The recent rise of Neural Radiance Fields (NeRFs)-like methods has revolutionized high-fidelity scene reconstruction, with 3D Gaussian Splatting (3DGS) standing out for its ability to generate photorealistic images while maintaining fast, efficient rendering. 3DGS delivers high-fidelity representations of complex scenes at any scale (from very small objects to entire cities), accurately capturing geometry, materials, and lighting, while meeting the need for fast and efficient rendering-crucial for applications requiring real-time performance. Although High Dynamic Range (HDR) technology, which enables the capture of comprehensive real-world lighting information, has been used in novel view synthesis, several questions remain unanswered. For example, does HDR improve the overall quality of reconstruction? Are 8 bits enough? Can tone mapped images be a balanced compromise regarding quality and details? To answer such questions, in this work, we study the application of HDR technology on the 3DGS method for acquiring real-world scenes.DOI: 10.2312/stag.20241341
Project(s): Future-Oriented REsearch LABoratory
Metrics:
Piras V., Bonatti A. F., De Maria C., Cignoni P., Banterle F.
The recent rise of Neural Radiance Fields (NeRFs)-like methods has revolutionized high-fidelity scene reconstruction, with 3D Gaussian Splatting (3DGS) standing out for its ability to generate photorealistic images while maintaining fast, efficient rendering. 3DGS delivers high-fidelity representations of complex scenes at any scale (from very small objects to entire cities), accurately capturing geometry, materials, and lighting, while meeting the need for fast and efficient rendering-crucial for applications requiring real-time performance. Although High Dynamic Range (HDR) technology, which enables the capture of comprehensive real-world lighting information, has been used in novel view synthesis, several questions remain unanswered. For example, does HDR improve the overall quality of reconstruction? Are 8 bits enough? Can tone mapped images be a balanced compromise regarding quality and details? To answer such questions, in this work, we study the application of HDR technology on the 3DGS method for acquiring real-world scenes.DOI: 10.2312/stag.20241341
Project(s): Future-Oriented REsearch LABoratory
Metrics:
See at:
diglib.eg.org | CNR IRIS | CNR IRIS
2024
Journal article
Open Access
Perceptual quality assessment of NeRF and neural view synthesis methods for front-facing views
Liang H., Wu T., Hanji P., Banterle F., Gao H., Mantiuk R., Oztireli C.
Neural view synthesis (NVS) is one of the most successful techniques for synthesizing free viewpoint videos, capable of achieving high fidelity from only a sparse set of captured images. This success has led to many variants of the techniques, each evaluated on a set of test views typically using image quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research on how NVS methods perform with respect to perceived video quality. We present the first study on perceptual evaluation of NVS and NeRF variants. For this study, we collected two datasets of scenes captured in a controlled lab environment as well as in-the-wild. In contrast to existing datasets, these scenes come with reference video sequences, allowing us to test for temporal artifacts and subtle distortions that are easily overlooked when viewing only static images. We measured the quality of videos synthesized by several NVS methods in a well-controlled perceptual quality assessment experiment as well as with many existing state-of-the-art image/video quality metrics. We present a detailed analysis of the results and recommendations for dataset and metric selection for NVS evaluation.Source: COMPUTER GRAPHICS FORUM, vol. 43 (issue 2)
DOI: 10.1111/cgf.15036
DOI: 10.17863/cam.106658
DOI: 10.48550/arxiv.2303.15206
Project(s): RealVision, Machine learning methods for rendering on a high-dynamic-range multi-focal-plane display
Metrics:
Liang H., Wu T., Hanji P., Banterle F., Gao H., Mantiuk R., Oztireli C.
Neural view synthesis (NVS) is one of the most successful techniques for synthesizing free viewpoint videos, capable of achieving high fidelity from only a sparse set of captured images. This success has led to many variants of the techniques, each evaluated on a set of test views typically using image quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research on how NVS methods perform with respect to perceived video quality. We present the first study on perceptual evaluation of NVS and NeRF variants. For this study, we collected two datasets of scenes captured in a controlled lab environment as well as in-the-wild. In contrast to existing datasets, these scenes come with reference video sequences, allowing us to test for temporal artifacts and subtle distortions that are easily overlooked when viewing only static images. We measured the quality of videos synthesized by several NVS methods in a well-controlled perceptual quality assessment experiment as well as with many existing state-of-the-art image/video quality metrics. We present a detailed analysis of the results and recommendations for dataset and metric selection for NVS evaluation.Source: COMPUTER GRAPHICS FORUM, vol. 43 (issue 2)
DOI: 10.1111/cgf.15036
DOI: 10.17863/cam.106658
DOI: 10.48550/arxiv.2303.15206
Project(s): RealVision
, Machine learning methods for rendering on a high-dynamic-range multi-focal-plane displayMetrics:
See at:
arXiv.org e-Print Archive | Apollo | Apollo | Apollo | IRIS Cnr | IRIS Cnr | Apollo | Apollo | Apollo | doi.org | CNR IRIS
2024
Journal article
Open Access
Evaluating image-based interactive 3D modeling tools
Siddique A., Cignoni P., Corsini M., Banterle F.
Structure from Motion (SfM) is a computer vision technique used to reconstruct three-dimensional (3D) structures from a series of two-dimensional (2D) images or video frames. However, SfM tools struggle with transparent objects, reflective surfaces, and low-resolution frames. In such situations, image-based interactive 3D modeling software packages are employed to model 3D objects and measure dimensions. Our contributions to this work are twofold. First, we have introduced new tools to improve 3D modeling software packages; such tools are aimed at easing the workload for users. Second, we have conducted a comprehensive user study to evaluate the efficacy of popular 3d modeling software packages. The task is to measure certain dimensions for which ground truth measurements are already known. A relative error is calculated for every measurement. The evaluation of each software tool is done through survey form, event logs, and measurement relative error. The results of this user study clearly show that our approach to 3D modeling using multiple images has a lower relative error and produces higher quality 3D models than other software packages. In addition, it shows our new tools reduce the required time for completing a task.Source: IEEE ACCESS, vol. 12, pp. 104138-104152
DOI: 10.1109/access.2024.3434584
Project(s): "Photogrammetric Method for Determining BWR Internals Dimensions, EVOCATION
Metrics:
Siddique A., Cignoni P., Corsini M., Banterle F.
Structure from Motion (SfM) is a computer vision technique used to reconstruct three-dimensional (3D) structures from a series of two-dimensional (2D) images or video frames. However, SfM tools struggle with transparent objects, reflective surfaces, and low-resolution frames. In such situations, image-based interactive 3D modeling software packages are employed to model 3D objects and measure dimensions. Our contributions to this work are twofold. First, we have introduced new tools to improve 3D modeling software packages; such tools are aimed at easing the workload for users. Second, we have conducted a comprehensive user study to evaluate the efficacy of popular 3d modeling software packages. The task is to measure certain dimensions for which ground truth measurements are already known. A relative error is calculated for every measurement. The evaluation of each software tool is done through survey form, event logs, and measurement relative error. The results of this user study clearly show that our approach to 3D modeling using multiple images has a lower relative error and produces higher quality 3D models than other software packages. In addition, it shows our new tools reduce the required time for completing a task.Source: IEEE ACCESS, vol. 12, pp. 104138-104152
DOI: 10.1109/access.2024.3434584
Project(s): "Photogrammetric Method for Determining BWR Internals Dimensions, EVOCATION
Metrics:
See at:
IEEE Access | IEEE Access | IRIS Cnr | IRIS Cnr | CNR IRIS
2024
Journal article
Open Access
Self-supervised high dynamic range imaging: what can be learned from a single 8-bit video?
Banterle F., Marnerides D., Bashford-Rogers T., Debattista K.
Recently, Deep Learning-based methods for inverse tone mapping standard dynamic range (SDR) images to obtain high dynamic range (HDR) images have become very popular. These methods manage to fill over-exposed areas convincingly both in terms of details and dynamic range. To be effective, deep learning-based methods need to learn from large datasets and transfer this knowledge to the network weights. In this work, we tackle this problem from a completely different perspective. What can we learn from a single SDR 8-bit video? With the presented self-supervised approach, we show that, in many cases, a single SDR video is sufficient to generate an HDR video of the same quality or better than other state-of-the-art methods.Source: ACM TRANSACTIONS ON GRAPHICS, vol. 43 (issue 2), pp. 1-16
DOI: 10.1145/3648570
Metrics:
Banterle F., Marnerides D., Bashford-Rogers T., Debattista K.
Recently, Deep Learning-based methods for inverse tone mapping standard dynamic range (SDR) images to obtain high dynamic range (HDR) images have become very popular. These methods manage to fill over-exposed areas convincingly both in terms of details and dynamic range. To be effective, deep learning-based methods need to learn from large datasets and transfer this knowledge to the network weights. In this work, we tackle this problem from a completely different perspective. What can we learn from a single SDR 8-bit video? With the presented self-supervised approach, we show that, in many cases, a single SDR video is sufficient to generate an HDR video of the same quality or better than other state-of-the-art methods.Source: ACM TRANSACTIONS ON GRAPHICS, vol. 43 (issue 2), pp. 1-16
DOI: 10.1145/3648570
Metrics:
See at:
dl.acm.org | CNR IRIS | IRIS Cnr | ACM Transactions on Graphics | IRIS Cnr | CNR IRIS | CNR IRIS
2024
Conference article
Open Access
Re:Draw - context aware translation as a controllable method for artistic production
Cardoso J. L., Banterle F., Cignoni P., Wimmer M.
We introduce context-aware translation, a novel method that combines the benefits of inpainting and image-to-image translation, respecting simultane- ously the original input and contextual relevance – where existing methods fall short. By doing so, our method opens new avenues for the controllable use of AI within artistic creation, from animation to digital art. As an use case, we apply our method to redraw any hand-drawn animated character eyes based on any design specifications – eyes serve as a focal point that captures viewer attention and conveys a range of emotions; however, the labor-intensive na- ture of traditional animation often leads to compro- mises in the complexity and consistency of eye de- sign. Furthermore, we remove the need for produc- tion data for training and introduce a new charac- ter recognition method that surpasses existing work by not requiring fine-tuning to specific productions. This proposed use case could help maintain consis- tency throughout production and unlock bolder and more detailed design choices without the produc- tion cost drawbacks. A user study shows context- aware translation is preferred over existing work 95.16% of the time.DOI: 10.24963/ijcai.2024/842
Metrics:
Cardoso J. L., Banterle F., Cignoni P., Wimmer M.
We introduce context-aware translation, a novel method that combines the benefits of inpainting and image-to-image translation, respecting simultane- ously the original input and contextual relevance – where existing methods fall short. By doing so, our method opens new avenues for the controllable use of AI within artistic creation, from animation to digital art. As an use case, we apply our method to redraw any hand-drawn animated character eyes based on any design specifications – eyes serve as a focal point that captures viewer attention and conveys a range of emotions; however, the labor-intensive na- ture of traditional animation often leads to compro- mises in the complexity and consistency of eye de- sign. Furthermore, we remove the need for produc- tion data for training and introduce a new charac- ter recognition method that surpasses existing work by not requiring fine-tuning to specific productions. This proposed use case could help maintain consis- tency throughout production and unlock bolder and more detailed design choices without the produc- tion cost drawbacks. A user study shows context- aware translation is preferred over existing work 95.16% of the time.DOI: 10.24963/ijcai.2024/842
Metrics:
See at:
arXiv.org e-Print Archive | CNR IRIS | www.ijcai.org | doi.org | CNR IRIS | CNR IRIS
2023
Journal article
Open Access
NoR-VDPNet++: real-time no-reference image quality metrics
Banterle F, Artusi A, Moreo A, Carrara F, Cignoni P
Efficiency and efficacy are desirable properties for any evaluation metric having to do with Standard Dynamic Range (SDR) imaging or with High Dynamic Range (HDR) imaging. However, it is a daunting task to satisfy both properties simultaneously. On the one side, existing evaluation metrics like HDR-VDP 2.2 can accurately mimic the Human Visual System (HVS), but this typically comes at a very high computational cost. On the other side, computationally cheaper alternatives (e.g., PSNR, MSE, etc.) fail to capture many crucial aspects of the HVS. In this work, we present NoR-VDPNet++, a deep learning architecture for converting full-reference accurate metrics into no-reference metrics thus reducing the computational burden. We show NoR-VDPNet++ can be successfully employed in different application scenarios.Source: IEEE ACCESS, vol. 11, pp. 34544-34553
DOI: 10.1109/access.2023.3263496
Project(s): ENCORE
Metrics:
Banterle F, Artusi A, Moreo A, Carrara F, Cignoni P
Efficiency and efficacy are desirable properties for any evaluation metric having to do with Standard Dynamic Range (SDR) imaging or with High Dynamic Range (HDR) imaging. However, it is a daunting task to satisfy both properties simultaneously. On the one side, existing evaluation metrics like HDR-VDP 2.2 can accurately mimic the Human Visual System (HVS), but this typically comes at a very high computational cost. On the other side, computationally cheaper alternatives (e.g., PSNR, MSE, etc.) fail to capture many crucial aspects of the HVS. In this work, we present NoR-VDPNet++, a deep learning architecture for converting full-reference accurate metrics into no-reference metrics thus reducing the computational burden. We show NoR-VDPNet++ can be successfully employed in different application scenarios.Source: IEEE ACCESS, vol. 11, pp. 34544-34553
DOI: 10.1109/access.2023.3263496
Project(s): ENCORE
Metrics:
See at:
IEEE Access | CNR IRIS | ieeexplore.ieee.org | ISTI Repository | ISTI Repository | CNR IRIS | CNR IRIS
2023
Journal article
Open Access
MoReLab: a software for user-assisted 3D reconstruction
Siddique A, Banterle F, Corsini M, Cignoni P, Sommerville D, Joffe C
We present MoReLab, a tool for user-assisted 3D reconstruction. This reconstruction requires an understanding of the shapes of the desired objects. Our experiments demonstrate that existing Structure from Motion (SfM) software packages fail to estimate accurate 3D models in low-quality videos due to several issues such as low resolution, featureless surfaces, low lighting, etc. In such scenarios, which are common for industrial utility companies, user assistance becomes necessary to create reliable 3D models. In our system, the user first needs to add features and correspondences manually on multiple video frames. Then, classic camera calibration and bundle adjustment are applied. At this point, MoReLab provides several primitive shape tools such as rectangles, cylinders, curved cylinders, etc., to model different parts of the scene and export 3D meshes. These shapes are essential for modeling industrial equipment whose videos are typically captured by utility companies with old video cameras (low resolution, compression artifacts, etc.) and in disadvantageous lighting conditions (low lighting, torchlight attached to the video camera, etc.). We evaluate our tool on real industrial case scenarios and compare it against existing approaches. Visual comparisons and quantitative results show that MoReLab achieves superior results with regard to other user-interactive 3D modeling tools.Source: SENSORS (BASEL), vol. 23 (issue 14)
DOI: 10.3390/s23146456
Project(s): EVOCATION
Metrics:
Siddique A, Banterle F, Corsini M, Cignoni P, Sommerville D, Joffe C
We present MoReLab, a tool for user-assisted 3D reconstruction. This reconstruction requires an understanding of the shapes of the desired objects. Our experiments demonstrate that existing Structure from Motion (SfM) software packages fail to estimate accurate 3D models in low-quality videos due to several issues such as low resolution, featureless surfaces, low lighting, etc. In such scenarios, which are common for industrial utility companies, user assistance becomes necessary to create reliable 3D models. In our system, the user first needs to add features and correspondences manually on multiple video frames. Then, classic camera calibration and bundle adjustment are applied. At this point, MoReLab provides several primitive shape tools such as rectangles, cylinders, curved cylinders, etc., to model different parts of the scene and export 3D meshes. These shapes are essential for modeling industrial equipment whose videos are typically captured by utility companies with old video cameras (low resolution, compression artifacts, etc.) and in disadvantageous lighting conditions (low lighting, torchlight attached to the video camera, etc.). We evaluate our tool on real industrial case scenarios and compare it against existing approaches. Visual comparisons and quantitative results show that MoReLab achieves superior results with regard to other user-interactive 3D modeling tools.Source: SENSORS (BASEL), vol. 23 (issue 14)
DOI: 10.3390/s23146456
Project(s): EVOCATION
Metrics:
See at:
Sensors | CNR IRIS | ISTI Repository | www.mdpi.com | CNR IRIS
2022
Journal article
Unknown
AI-Based media coding standards
Basso A, Ribeca P, Bosi M, Pretto N, Chollet G, Guarise M, Choi M, Chiariglione L, Iacoviello R, Banterle F, Artusi A, Gissi F, Fiandrotti A, Ballocca G, Mazzaglia M, Moskowitz S
Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is the first standards organization to develop data coding standards that have artificial intelligence (AI) as their core technology. MPAI believes that universally accessible standards for AI-based data coding can have the same positive effects on AI as standards had on digital media. Elementary components of MPAI standards-AI modules (AIMs)-expose standard interfaces for operation in a standard AI framework (AIF). As their performance may depend on the technologies used, MPAI expects that competing developers providing AIMs will promote horizontal markets of AI solutions that build on and further promote AI innovation. Finally, the MPAI framework licences (FWLs) provide guidelines to intellectual property right (IPR) holders facilitating the availability of compatible licenses to standard users.Source: SMPTE MOTION IMAGING JOURNAL, vol. 131 (issue 4), pp. 10-20
DOI: 10.5594/jmi.2022.3160793
Metrics:
Basso A, Ribeca P, Bosi M, Pretto N, Chollet G, Guarise M, Choi M, Chiariglione L, Iacoviello R, Banterle F, Artusi A, Gissi F, Fiandrotti A, Ballocca G, Mazzaglia M, Moskowitz S
Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is the first standards organization to develop data coding standards that have artificial intelligence (AI) as their core technology. MPAI believes that universally accessible standards for AI-based data coding can have the same positive effects on AI as standards had on digital media. Elementary components of MPAI standards-AI modules (AIMs)-expose standard interfaces for operation in a standard AI framework (AIF). As their performance may depend on the technologies used, MPAI expects that competing developers providing AIMs will promote horizontal markets of AI solutions that build on and further promote AI innovation. Finally, the MPAI framework licences (FWLs) provide guidelines to intellectual property right (IPR) holders facilitating the availability of compatible licenses to standard users.Source: SMPTE MOTION IMAGING JOURNAL, vol. 131 (issue 4), pp. 10-20
DOI: 10.5594/jmi.2022.3160793
Metrics:
See at:
SMPTE Motion Imaging Journal | CNR IRIS | ieeexplore.ieee.org
2022
Journal article
Open Access
Quantum computing algorithms: getting closer to critical problems in computational biology
Marchetti L., Nifosì R., Martelli P. L., Da Pozzo E., Cappello V., Banterle F., Trincavelli M. L., Martini C., D'Elia M.
The recent biotechnological progress has allowed life scientists and physicians to access an unprecedented, massive amount of data at all levels (molecular, supramolecular, cellular and so on) of biological complexity. So far, mostly classical computational efforts have been dedicated to the simulation, prediction or de novo design of biomolecules, in order to improve the understanding of their function or to develop novel therapeutics. At a higher level of complexity, the progress of omics disciplines (genomics, transcriptomics, proteomics and metabolomics) has prompted researchers to develop informatics means to describe and annotate new biomolecules identified with a resolution down to the single cell, but also with a high-throughput speed. Machine learning approaches have been implemented to both the modelling studies and the handling of biomedical data. Quantum computing (QC) approaches hold the promise to resolve, speed up or refine the analysis of a wide range of these computational problems. Here, we review and comment on recently developed QC algorithms for biocomputing, with a particular focus on multi-scale modelling and genomic analyses. Indeed, differently from other computational approaches such as protein structure prediction, these problems have been shown to be adequately mapped onto quantum architectures, the main limit for their immediate use being the number of qubits and decoherence effects in the available quantum machines. Possible advantages over the classical counterparts are highlighted, along with a description of some hybrid classical/quantum approaches, which could be the closest to be realistically applied in biocomputation.Source: BRIEFINGS IN BIOINFORMATICS, vol. 23 (issue 6)
DOI: 10.1093/bib/bbac437
Metrics:
Marchetti L., Nifosì R., Martelli P. L., Da Pozzo E., Cappello V., Banterle F., Trincavelli M. L., Martini C., D'Elia M.
The recent biotechnological progress has allowed life scientists and physicians to access an unprecedented, massive amount of data at all levels (molecular, supramolecular, cellular and so on) of biological complexity. So far, mostly classical computational efforts have been dedicated to the simulation, prediction or de novo design of biomolecules, in order to improve the understanding of their function or to develop novel therapeutics. At a higher level of complexity, the progress of omics disciplines (genomics, transcriptomics, proteomics and metabolomics) has prompted researchers to develop informatics means to describe and annotate new biomolecules identified with a resolution down to the single cell, but also with a high-throughput speed. Machine learning approaches have been implemented to both the modelling studies and the handling of biomedical data. Quantum computing (QC) approaches hold the promise to resolve, speed up or refine the analysis of a wide range of these computational problems. Here, we review and comment on recently developed QC algorithms for biocomputing, with a particular focus on multi-scale modelling and genomic analyses. Indeed, differently from other computational approaches such as protein structure prediction, these problems have been shown to be adequately mapped onto quantum architectures, the main limit for their immediate use being the number of qubits and decoherence effects in the available quantum machines. Possible advantages over the classical counterparts are highlighted, along with a description of some hybrid classical/quantum approaches, which could be the closest to be realistically applied in biocomputation.Source: BRIEFINGS IN BIOINFORMATICS, vol. 23 (issue 6)
DOI: 10.1093/bib/bbac437
Metrics:
See at:
academic.oup.com | Briefings in Bioinformatics | CNR IRIS | ISTI Repository | CNR IRIS | Briefings in Bioinformatics
2021
Contribution to book
Restricted
Virtual clones for cultural heritage applications
Potenziani M, Banterle F, Callieri M, Dellepiane M, Ponchio F, Scopigno R
Digital technologies are now mature for producing high quality digital replicas of Cultural Heritage (CH) artifacts. The research results produced in the last decade have shown an impressive evolution and consolidation of the technologies for acquiring high-quality digital 3D models, encompassing both geometry and color (or, better, surface reflectance properties). Some recent technologies for constructing 3D models enriched by a high-quality encoding of the color attribute will be presented. The focus of this paper is to show and discuss practical solutions, which could be deployed without requiring the installation of a specific or sophisticated acquisition lab setup. In the second part of this paper, we focus on new solutions for the interactive visualization of complex models, adequate for modern communication channels such as the web and the mobile platforms. Together with the algorithms and approaches, we show also some practical examples where high-quality 3D models have been used in CH research, restoration and conservation.
Potenziani M, Banterle F, Callieri M, Dellepiane M, Ponchio F, Scopigno R
Digital technologies are now mature for producing high quality digital replicas of Cultural Heritage (CH) artifacts. The research results produced in the last decade have shown an impressive evolution and consolidation of the technologies for acquiring high-quality digital 3D models, encompassing both geometry and color (or, better, surface reflectance properties). Some recent technologies for constructing 3D models enriched by a high-quality encoding of the color attribute will be presented. The focus of this paper is to show and discuss practical solutions, which could be deployed without requiring the installation of a specific or sophisticated acquisition lab setup. In the second part of this paper, we focus on new solutions for the interactive visualization of complex models, adequate for modern communication channels such as the web and the mobile platforms. Together with the algorithms and approaches, we show also some practical examples where high-quality 3D models have been used in CH research, restoration and conservation.
See at:
CNR IRIS | CNR IRIS | www.lerma.it
2021
Conference article
Open Access
Collaborative visual environments for evidence taking in digital justice: a design concept
Erra U., Capece N., Lettieri N., Fabiani E., Banterle F., Cignoni P., Dazzi P., Aleotti J., Monica R.
In recent years, Spatial Computing (SC) has emerged as a novel paradigm thanks to the advancements in Extended Reality (XR), remote sensing, and artificial intelligence. Computers are nowadays more and more aware of physical environments (i.e. objects shape, size, location and movement) and can use this knowledge to blend technology into reality seamlessly, merge digital and real worlds, and connect users by providing innovative interaction methods. Criminal and civil trials offer an ideal scenario to exploit Spatial Computing. The taking of evidence, indeed, is a complex activity that not only involves several actors (judges, lawyers, clerks, advi- sors) but it often requires accurate topographic surveys of places and objects. Moreover, another essential means of proof, the "judi- cial experiments" - reproductions of real-world events (e.g. a road accident) the judge uses to evaluate if and how a given fact has taken place - could be usefully carried out in virtual environments. In this paper we propose a novel approach for digital justice based on a multi-user, multimodal virtual collaboration platform that enables technology-enhanced acquisition and analysis of trial evidence.Source: FRAME'21 - 1st Workshop on Flexible Resource and Application Management on the Edge, pp. 41–44, Sweden, Virtual Event, 25/06/2021
DOI: 10.1145/3452369.3463820
Project(s): ACCORDION
Metrics:
Erra U., Capece N., Lettieri N., Fabiani E., Banterle F., Cignoni P., Dazzi P., Aleotti J., Monica R.
In recent years, Spatial Computing (SC) has emerged as a novel paradigm thanks to the advancements in Extended Reality (XR), remote sensing, and artificial intelligence. Computers are nowadays more and more aware of physical environments (i.e. objects shape, size, location and movement) and can use this knowledge to blend technology into reality seamlessly, merge digital and real worlds, and connect users by providing innovative interaction methods. Criminal and civil trials offer an ideal scenario to exploit Spatial Computing. The taking of evidence, indeed, is a complex activity that not only involves several actors (judges, lawyers, clerks, advi- sors) but it often requires accurate topographic surveys of places and objects. Moreover, another essential means of proof, the "judi- cial experiments" - reproductions of real-world events (e.g. a road accident) the judge uses to evaluate if and how a given fact has taken place - could be usefully carried out in virtual environments. In this paper we propose a novel approach for digital justice based on a multi-user, multimodal virtual collaboration platform that enables technology-enhanced acquisition and analysis of trial evidence.Source: FRAME'21 - 1st Workshop on Flexible Resource and Application Management on the Edge, pp. 41–44, Sweden, Virtual Event, 25/06/2021
DOI: 10.1145/3452369.3463820
Project(s): ACCORDION
Metrics:
See at:
ISTI Repository | dl.acm.org | CNR ExploRA
2021
Conference article
Open Access
A deep learning method for frame selection in videos for structure from motion pipelines
Banterle F, Gong R, Corsini M, Ganovelli F, Van Gool L, Cignoni P
Structure-from-Motion (SfM) using the frames of a video sequence can be a challenging task because there is a lot of redundant information, the computational time increases quadratically with the number of frames, there would be low-quality images (e.g., blurred frames) that can decrease the final quality of the reconstruction, etc. To overcome all these issues, we present a novel deep-learning architecture that is meant for speeding up SfM by selecting frames using predicted sub-sampling frequency. This architecture is general and can learn/distill the knowledge of any algorithm for selecting frames from a video for generating high-quality reconstructions. One key advantage is that we can run our architecture in real-time saving computations while keeping high-quality results.Source: PROCEEDINGS - INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, pp. 3667-3671. Anchorage, Alaska, USA, 19-22/09/2021
DOI: 10.1109/icip42928.2021.9506227
Project(s): ENCORE
Metrics:
Banterle F, Gong R, Corsini M, Ganovelli F, Van Gool L, Cignoni P
Structure-from-Motion (SfM) using the frames of a video sequence can be a challenging task because there is a lot of redundant information, the computational time increases quadratically with the number of frames, there would be low-quality images (e.g., blurred frames) that can decrease the final quality of the reconstruction, etc. To overcome all these issues, we present a novel deep-learning architecture that is meant for speeding up SfM by selecting frames using predicted sub-sampling frequency. This architecture is general and can learn/distill the knowledge of any algorithm for selecting frames from a video for generating high-quality reconstructions. One key advantage is that we can run our architecture in real-time saving computations while keeping high-quality results.Source: PROCEEDINGS - INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, pp. 3667-3671. Anchorage, Alaska, USA, 19-22/09/2021
DOI: 10.1109/icip42928.2021.9506227
Project(s): ENCORE
Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | ISTI Repository | doi.org | CNR IRIS | CNR IRIS
2021
Book
Open Access
Proceedings - Web3D 2021: 26th ACM International Conference on 3D Web Technology
Ganovelli F, Mc Donald C, Banterle F, Potenziani M, Callieri M, Jung Y
The annual ACM Web3D Conference is a major event which unites researchers, developers, entrepreneurs, experimenters, artists and content creators in a dynamic learning environment. Attendees share and explore methods of using, enhancing and creating new 3D Web and Multimedia technologies such as X3D, VRML, Collada, MPEG family, U3D, Java3D and other technologies. The conference also focuses on recent trends in interactive 3D graphics, information integration and usability in the wide range of Web3D applications from mobile devices to high-end immersive environments.DOI: 10.1145/3485444
Metrics:
Ganovelli F, Mc Donald C, Banterle F, Potenziani M, Callieri M, Jung Y
The annual ACM Web3D Conference is a major event which unites researchers, developers, entrepreneurs, experimenters, artists and content creators in a dynamic learning environment. Attendees share and explore methods of using, enhancing and creating new 3D Web and Multimedia technologies such as X3D, VRML, Collada, MPEG family, U3D, Java3D and other technologies. The conference also focuses on recent trends in interactive 3D graphics, information integration and usability in the wide range of Web3D applications from mobile devices to high-end immersive environments.DOI: 10.1145/3485444
Metrics:
See at:
dl.acm.org | CNR IRIS | ISTI Repository | CNR IRIS