2024
Conference article
Open Access
Preprocessing of recto-verso printed documents based on neural networks for text analysis
Savino P, Tonazzini A
Among the many and varied damages affectingancient documents, the penetration of ink from one side of thepage to the other is one of the most frequent and invasive. In thiswork, we are interested in binarizing such degraded documents,for the application of OCR or other automatic text analysistools, which can help philologists and palaeographers in texttranscription. We previously proposed a data model that roughlydescribes this damage for front-to-back documents, and used itto generate an artificial training set that can teach a shallowneural network how to classify pixels on both sides into clean orcorrupt. We show that this joint processing of the two sides of thedocument can significantly improve binarization and thereforeOCR and other text analysis tasks, compared to the separateprocessing of the single sides, using the same information.DOI: 10.1109/cist56084.2023.10409970
Metrics:
Savino P, Tonazzini A
Among the many and varied damages affectingancient documents, the penetration of ink from one side of thepage to the other is one of the most frequent and invasive. In thiswork, we are interested in binarizing such degraded documents,for the application of OCR or other automatic text analysistools, which can help philologists and palaeographers in texttranscription. We previously proposed a data model that roughlydescribes this damage for front-to-back documents, and used itto generate an artificial training set that can teach a shallowneural network how to classify pixels on both sides into clean orcorrupt. We show that this joint processing of the two sides of thedocument can significantly improve binarization and thereforeOCR and other text analysis tasks, compared to the separateprocessing of the single sides, using the same information.DOI: 10.1109/cist56084.2023.10409970
Metrics:
See at:
CNR IRIS 
| ieeexplore.ieee.org | doi.org 
| IRIS Cnr | CNR IRIS | CNR IRIS
2024
Journal article
Open Access
Training a shallow NN to erase ink seepage in historical manuscripts based on a degradation model
Savino P, Tonazzini A
In historical recto-verso manuscripts, very often the text written on the opposite page of the folio penetrates through the fiber of the paper, so that the texts on the two sides appear mixed. This is a very impairing damage that cannot be physically removed, and hinders both the work of philologists and palaeographers and the automatic analysis of linguistic contents. A procedure based on neural networks (NN) is proposed here to clean up the complex background of the manuscripts from this interference. We adopt a very simple shallow NN whose learning phase employs a training set generated from the data itself using a theoretical blending model that takes into account ink diffusion and saturation. By virtue of the parametric nature of the model, various levels of damage can be simulated in the training set, favoring a generalization capability of the NN. More explicitly, the network can be trained without the need for a large class of other similar manuscripts, but is still able, at least to some extent, to classify manuscripts with varying degrees of corruption. We compare the performance of this NN and other methods both qualitatively and quantitatively on a reference dataset and heavily damaged historical manuscripts.Source: NEURAL COMPUTING & APPLICATIONS
DOI: 10.1007/s00521-023-09354-7
Metrics:
Savino P, Tonazzini A
In historical recto-verso manuscripts, very often the text written on the opposite page of the folio penetrates through the fiber of the paper, so that the texts on the two sides appear mixed. This is a very impairing damage that cannot be physically removed, and hinders both the work of philologists and palaeographers and the automatic analysis of linguistic contents. A procedure based on neural networks (NN) is proposed here to clean up the complex background of the manuscripts from this interference. We adopt a very simple shallow NN whose learning phase employs a training set generated from the data itself using a theoretical blending model that takes into account ink diffusion and saturation. By virtue of the parametric nature of the model, various levels of damage can be simulated in the training set, favoring a generalization capability of the NN. More explicitly, the network can be trained without the need for a large class of other similar manuscripts, but is still able, at least to some extent, to classify manuscripts with varying degrees of corruption. We compare the performance of this NN and other methods both qualitatively and quantitatively on a reference dataset and heavily damaged historical manuscripts.Source: NEURAL COMPUTING & APPLICATIONS
DOI: 10.1007/s00521-023-09354-7
Metrics:
See at:
CNR IRIS | link.springer.com | ISTI Repository | CNR IRIS | CNR IRIS
2024
Other
Open Access
AIMH Research Activities 2024
Aloia N., Amato G., Bartalesi Lenzi V., Bianchi L., Bolettieri P., Bosio C., Carraglia M., Carrara F., Casarosa V., Cassese M., Ciampi L., Coccomini D. A., Concordia C., Connor R., Corbara S., De Martino C., Di Benedetto M., Esuli A., Falchi F., Fazzari E., Gennaro C., Iannello L., Negi K., Lagani G., Lenzi E., Leocata M., Malvaldi M., Meghini C., Messina N., Moreo Fernandez A., Nardi A., Pacini G., Pedrotti A., Pratelli N., Puccetti G., Rabitti F., Savino P., Scotti F., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C., Versienti L., Volpi L.
The AIMH (Artificial Intelligence for Media and Humanities) laboratory is committed to advancing the field of Artificial Intelligence, with a special emphasis on its applications in digital media and the humanities. The lab aims to improve AI technologies, particularly in areas such as deep learning, text analysis, computer vision, multimedia information retrieval, content analysis, recognition, and retrieval. This report summarizes the laboratory’s achievements and activities over the course of 2024.DOI: 10.32079/isti-ar-2024/001
Metrics:
Aloia N., Amato G., Bartalesi Lenzi V., Bianchi L., Bolettieri P., Bosio C., Carraglia M., Carrara F., Casarosa V., Cassese M., Ciampi L., Coccomini D. A., Concordia C., Connor R., Corbara S., De Martino C., Di Benedetto M., Esuli A., Falchi F., Fazzari E., Gennaro C., Iannello L., Negi K., Lagani G., Lenzi E., Leocata M., Malvaldi M., Meghini C., Messina N., Moreo Fernandez A., Nardi A., Pacini G., Pedrotti A., Pratelli N., Puccetti G., Rabitti F., Savino P., Scotti F., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C., Versienti L., Volpi L.
The AIMH (Artificial Intelligence for Media and Humanities) laboratory is committed to advancing the field of Artificial Intelligence, with a special emphasis on its applications in digital media and the humanities. The lab aims to improve AI technologies, particularly in areas such as deep learning, text analysis, computer vision, multimedia information retrieval, content analysis, recognition, and retrieval. This report summarizes the laboratory’s achievements and activities over the course of 2024.DOI: 10.32079/isti-ar-2024/001
Metrics:
2024
Journal article
Open Access
Mathematical data models and context-based features for enhancing historical degraded manuscripts using neural network classification
Savino P., Tonazzini A.
A common cause of deterioration in historic manuscripts is ink transparency or bleeding from the opposite page. Philologists and paleographers can significantly benefit from minimizing these interferences when attempting to decipher the original text. Additionally, computer-aided text analysis can also gain from such text enhancement. In previous work, we proposed the use of neural networks (NNs) in combination with a data model that characterizes the damage when both sides of a page have been digitized. This approach offers the distinct advantage of allowing the creation of an artificial training set that teaches the NN to differentiate between clean and damaged pixels. We tested this concept using a shallow NN, which proved effective in categorizing texts with varying levels of deterioration. In this study, we adapt the NN design to tackling remaining classification uncertainties caused by areas of text overlap, inhomogeneity, and peaks of degradation. Specifically, we introduce a new output class for pixels within overlapping text areas and incorporate additional features related to the pixel context information to promote the same classification for pixels adjacent to each other. Our experiments demonstrate that these enhancements significantly improve the classification accuracy. This improvement is evident in the quality of both binarization, which aids in text analysis, and virtual restoration, aimed at recovering the manuscript’s original appearance. Tests conducted on a public dataset, using standard quality indices, reveal that the proposed method outperforms both our previous proposals and other notable methods found in the literature.Source: MATHEMATICS, vol. 12 (issue 21), p. 3402
DOI: 10.3390/math12213402
Metrics:
Savino P., Tonazzini A.
A common cause of deterioration in historic manuscripts is ink transparency or bleeding from the opposite page. Philologists and paleographers can significantly benefit from minimizing these interferences when attempting to decipher the original text. Additionally, computer-aided text analysis can also gain from such text enhancement. In previous work, we proposed the use of neural networks (NNs) in combination with a data model that characterizes the damage when both sides of a page have been digitized. This approach offers the distinct advantage of allowing the creation of an artificial training set that teaches the NN to differentiate between clean and damaged pixels. We tested this concept using a shallow NN, which proved effective in categorizing texts with varying levels of deterioration. In this study, we adapt the NN design to tackling remaining classification uncertainties caused by areas of text overlap, inhomogeneity, and peaks of degradation. Specifically, we introduce a new output class for pixels within overlapping text areas and incorporate additional features related to the pixel context information to promote the same classification for pixels adjacent to each other. Our experiments demonstrate that these enhancements significantly improve the classification accuracy. This improvement is evident in the quality of both binarization, which aids in text analysis, and virtual restoration, aimed at recovering the manuscript’s original appearance. Tests conducted on a public dataset, using standard quality indices, reveal that the proposed method outperforms both our previous proposals and other notable methods found in the literature.Source: MATHEMATICS, vol. 12 (issue 21), p. 3402
DOI: 10.3390/math12213402
Metrics:
See at:
Mathematics | IRIS Cnr | IRIS Cnr | IRIS Cnr | CNR IRIS
2023
Conference article
Open Access
Mathematical models and neural networks for the description and the correction of typical distortions of historical manuscripts
Savino P, Tonazzini A
Historical manuscripts are very often degraded by the seeping or transparency of the ink from the page opposite side. Suppressing the interfering text can be of great aid to philologists and paleographers who aim at interpreting the primary text, and nowadays also for the automatic analysis of the text. We formerly proposed a data model, which approximately describes this damage, to generate an artificial training set able to teach a shallow neural network how to classify pixels in clean or corrupted. This NN has proved to be effective in classifying manuscripts where the degradation can be also widely variable. In this paper, we modify the architecture of the NN to better account for ink saturation in text overlay areas, by including a specific class for these pixels. From the experiments, the improvement of the classification and then the restoration is significant.DOI: 10.1007/978-3-031-37117-2_37
Metrics:
Savino P, Tonazzini A
Historical manuscripts are very often degraded by the seeping or transparency of the ink from the page opposite side. Suppressing the interfering text can be of great aid to philologists and paleographers who aim at interpreting the primary text, and nowadays also for the automatic analysis of the text. We formerly proposed a data model, which approximately describes this damage, to generate an artificial training set able to teach a shallow neural network how to classify pixels in clean or corrupted. This NN has proved to be effective in classifying manuscripts where the degradation can be also widely variable. In this paper, we modify the architecture of the NN to better account for ink saturation in text overlay areas, by including a specific class for these pixels. From the experiments, the improvement of the classification and then the restoration is significant.DOI: 10.1007/978-3-031-37117-2_37
Metrics:
See at:
CNR IRIS | link.springer.com | ISTI Repository | CNR IRIS | CNR IRIS
2023
Other
Open Access
AIMH Research Activities 2023
Aloia N., Amato G., Bartalesi Lenzi V., Bianchi L., Bolettieri P., Bosio C., Carraglia M., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., De Martino C., Di Benedetto M., Esuli A., Falchi F., Fazzari E., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Puccetti G., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C., Versienti L.
The AIMH (Artificial Intelligence for Media and Humanities) laboratory is dedicated to exploring and pushing the boundaries in the field of Artificial Intelligence, with a particular focus on its application in digital media and humanities. This lab's objective is to enhance the current state of AI technology particularly on deep learning, text analysis, computer vision, multimedia information retrieval, multimedia content analysis, recognition, and retrieval. This report encapsulates the laboratory's progress and activities throughout the year 2023.DOI: https://doi.org/10.32079/isti-ar-2023/001
DOI: 10.32079/isti-ar-2023/001
Metrics:
Aloia N., Amato G., Bartalesi Lenzi V., Bianchi L., Bolettieri P., Bosio C., Carraglia M., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., De Martino C., Di Benedetto M., Esuli A., Falchi F., Fazzari E., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Puccetti G., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C., Versienti L.
The AIMH (Artificial Intelligence for Media and Humanities) laboratory is dedicated to exploring and pushing the boundaries in the field of Artificial Intelligence, with a particular focus on its application in digital media and humanities. This lab's objective is to enhance the current state of AI technology particularly on deep learning, text analysis, computer vision, multimedia information retrieval, multimedia content analysis, recognition, and retrieval. This report encapsulates the laboratory's progress and activities throughout the year 2023.DOI: https://doi.org/10.32079/isti-ar-2023/001
DOI: 10.32079/isti-ar-2023/001
Metrics:
See at:
CNR IRIS | ISTI Repository | CNR IRIS
2022
Journal article
Open Access
Blind bleed-through removal in color ancient manuscripts
Hanif M, Tonazzini A, Hussain Sf, Habib U, Salerno E, Savino P, Halim Z
Archaic manuscripts are an important part of ancient civilization. Unfortunately, such documents are often affected by various age related degradations, which impinge their legibility and information contents, and destroy their original look. In general, these documents are composed of three layers of information: foreground text, background, and unwanted degradation in the form of patterns interfering with the main text. In this work, we are presenting a color space based image segmentation technique to separate and remove the bleed-through degradation in digital ancient manuscripts. The main theme is to improve their readability and restore their original aesthetic look. For each pixel, a feature vector is created using color spectral and spatial location information. A pixel based segmentation method using Gaussian Mixture Model (GMM) is employed, assuming that each feature vector corresponds to a Gaussian distribution. Based on this assumption, each pixel is supposed to be drawn from a mixture of Gaussian distribution, with unknown parameters. The Expectation-Maximization (EM) approach is then used to estimate the unknown GMM parameters. The appropriate class label for each pixel is then estimated using posterior probability and GMM parameters. Unlike other binarization based document restoration method where the focus is on text extraction, we are more interested in restoring the aesthetically pleasing look of the ancient documents.The experimental results validate the usefulness of proposed method in terms of successful bleed-through identification and removal, while preserving foreground-text and background information.Source: MULTIMEDIA TOOLS AND APPLICATIONS (DORDRECHT. ONLINE), vol. 82, pp. 12321-12335
DOI: 10.1007/s11042-022-13755-6
DOI: https://doi.org/10.1007/s11042-022-13755-6
Metrics:
Hanif M, Tonazzini A, Hussain Sf, Habib U, Salerno E, Savino P, Halim Z
Archaic manuscripts are an important part of ancient civilization. Unfortunately, such documents are often affected by various age related degradations, which impinge their legibility and information contents, and destroy their original look. In general, these documents are composed of three layers of information: foreground text, background, and unwanted degradation in the form of patterns interfering with the main text. In this work, we are presenting a color space based image segmentation technique to separate and remove the bleed-through degradation in digital ancient manuscripts. The main theme is to improve their readability and restore their original aesthetic look. For each pixel, a feature vector is created using color spectral and spatial location information. A pixel based segmentation method using Gaussian Mixture Model (GMM) is employed, assuming that each feature vector corresponds to a Gaussian distribution. Based on this assumption, each pixel is supposed to be drawn from a mixture of Gaussian distribution, with unknown parameters. The Expectation-Maximization (EM) approach is then used to estimate the unknown GMM parameters. The appropriate class label for each pixel is then estimated using posterior probability and GMM parameters. Unlike other binarization based document restoration method where the focus is on text extraction, we are more interested in restoring the aesthetically pleasing look of the ancient documents.The experimental results validate the usefulness of proposed method in terms of successful bleed-through identification and removal, while preserving foreground-text and background information.Source: MULTIMEDIA TOOLS AND APPLICATIONS (DORDRECHT. ONLINE), vol. 82, pp. 12321-12335
DOI: 10.1007/s11042-022-13755-6
DOI: https://doi.org/10.1007/s11042-022-13755-6
Metrics:
See at:
CNR IRIS | ISTI Repository | CNR IRIS | CNR IRIS
2022
Conference article
Open Access
A shallow neural net with model-based learning for the virtual restoration of recto-verso manuscript
Savino P, Tonazzini A
We propose a fast procedure based on neural networks (NN) to correct the typically complex background of recto-verso historical manuscripts, where the texts of the two sides often appear mixed. The purpose is to eliminate the interfering, shining-through text, to facilitate both the work of philologists and paleographers and the automatic analysis of the linguistic contents. We adapt the learning phase of a very simple shallow NN to exploit the information of the registered recto and verso sides of the manuscript without the need for a large class of other similar manuscripts. Hence, the training set is self-generated from the data images based on a theoretical mixing model that accounts for ink spreading through the paper fiber and for ink saturation in the text superposition areas. Operationally, we select pairs of patches containing clean text from the manuscript and then mix them symmetrically using the model with varying parameters that span the allowed range. This makes the NN able to generalize to diverse amounts of ink seeping and then classify different manuscripts. We show comparisons between the results obtained on heavily damaged manuscripts with this NN and other approaches. From a qualitative point of view, the proposed method seems quite promising.Source: CEUR WORKSHOP PROCEEDINGS. Online event, 12/09/2022
Savino P, Tonazzini A
We propose a fast procedure based on neural networks (NN) to correct the typically complex background of recto-verso historical manuscripts, where the texts of the two sides often appear mixed. The purpose is to eliminate the interfering, shining-through text, to facilitate both the work of philologists and paleographers and the automatic analysis of the linguistic contents. We adapt the learning phase of a very simple shallow NN to exploit the information of the registered recto and verso sides of the manuscript without the need for a large class of other similar manuscripts. Hence, the training set is self-generated from the data images based on a theoretical mixing model that accounts for ink spreading through the paper fiber and for ink saturation in the text superposition areas. Operationally, we select pairs of patches containing clean text from the manuscript and then mix them symmetrically using the model with varying parameters that span the allowed range. This makes the NN able to generalize to diverse amounts of ink seeping and then classify different manuscripts. We show comparisons between the results obtained on heavily damaged manuscripts with this NN and other approaches. From a qualitative point of view, the proposed method seems quite promising.Source: CEUR WORKSHOP PROCEEDINGS. Online event, 12/09/2022
See at:
ceur-ws.org | CNR IRIS | ISTI Repository | ISTI Repository | CNR IRIS
2022
Other
Open Access
AIMH research activities 2022
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A. D., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability.This report summarize the 2022 activities of the research group.DOI: https://doi.org/10.32079/isti-ar-2022/002
DOI: 10.32079/isti-ar-2022/002
Metrics:
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Ciampi L., Coccomini D. A., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Lenzi E., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A. D., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability.This report summarize the 2022 activities of the research group.DOI: https://doi.org/10.32079/isti-ar-2022/002
DOI: 10.32079/isti-ar-2022/002
Metrics:
See at:
CNR IRIS | ISTI Repository | CNR IRIS
2021
Journal article
Open Access
A procedure for the correction of back-to-front degradations in archival manuscripts with preservation of the original appearance
Savino P, Tonazzini A
Virtual restoration of digital copies of the human documental heritage is crucial for facilitating both the traditional work of philologists and paleographers and the automatic analysis of the contents. Here we propose a practical and fast procedure for the correction of the typically complex background of recto-verso historical manuscripts. The procedure has two main, distinctive features: it does not need for a preliminary registration of the two page sides, and it is non-invasive, as it does not alter the original appearance of the manuscript. This makes it suitable for the routinary use in the archives, and permits an easier fruition of the manuscripts, without any information being lost. In the ̄rst stage, the detection of both the primary text and the spurious strokes is performed via soft segmentation, based on the statistical decorrelation of the two recto and verso images. In the second stage, the noisy pattern is substituted with pixels that simulate the texture of the clean surrounding background, through an e±cient technique of image inpainting. As shown in the experimental results, evaluated both qualitatively and quantitatively, the proposed procedure is able to perform a ̄ne and selective removal of the degradation, while preserving other informative marks of the manuscript history.Source: VIETNAM JOURNAL OF COMPUTER SCIENCE
DOI: 10.1142/s2196888822500099
Metrics:
Savino P, Tonazzini A
Virtual restoration of digital copies of the human documental heritage is crucial for facilitating both the traditional work of philologists and paleographers and the automatic analysis of the contents. Here we propose a practical and fast procedure for the correction of the typically complex background of recto-verso historical manuscripts. The procedure has two main, distinctive features: it does not need for a preliminary registration of the two page sides, and it is non-invasive, as it does not alter the original appearance of the manuscript. This makes it suitable for the routinary use in the archives, and permits an easier fruition of the manuscripts, without any information being lost. In the ̄rst stage, the detection of both the primary text and the spurious strokes is performed via soft segmentation, based on the statistical decorrelation of the two recto and verso images. In the second stage, the noisy pattern is substituted with pixels that simulate the texture of the clean surrounding background, through an e±cient technique of image inpainting. As shown in the experimental results, evaluated both qualitatively and quantitatively, the proposed procedure is able to perform a ̄ne and selective removal of the degradation, while preserving other informative marks of the manuscript history.Source: VIETNAM JOURNAL OF COMPUTER SCIENCE
DOI: 10.1142/s2196888822500099
Metrics:
See at:
CNR IRIS | ISTI Repository | ISTI Repository | www.worldscientific.com | Vietnam Journal of Computer Science | CNR IRIS | CNR IRIS
2021
Other
Open Access
AIMH research activities 2021
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Coccomini D., Ciampi L., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2021 activities of the research group.DOI: https://doi.org/10.32079/isti-ar-2021/003
DOI: 10.32079/isti-ar-2021/003
Metrics:
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Cafarelli D., Carrara F., Casarosa V., Coccomini D., Ciampi L., Concordia C., Corbara S., Di Benedetto M., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Sperduti G., Thanos C., Trupiano L., Vadicamo L., Vairo C.
The Artificial Intelligence for Media and Humanities laboratory (AIMH) has the mission to investigate and advance the state of the art in the Artificial Intelligence field, specifically addressing applications to digital media and digital humanities, and taking also into account issues related to scalability. This report summarize the 2021 activities of the research group.DOI: https://doi.org/10.32079/isti-ar-2021/003
DOI: 10.32079/isti-ar-2021/003
Metrics:
See at:
CNR IRIS | ISTI Repository | CNR IRIS
2020
Other
Open Access
AIMH research activities 2020
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Carrara F., Casarosa V., Ciampi L., Concordia C., Corbara S., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Thanos C., Trupiano L., Vadicamo L., Vairo C.
Annual Report of the Artificial Intelligence for Media and Humanities laboratory (AIMH) research activities in 2020.DOI: https://doi.org/10.32079/isti-ar-2020/001
DOI: 10.32079/isti-ar-2020/001
Metrics:
Aloia N., Amato G., Bartalesi Lenzi V., Benedetti F., Bolettieri P., Carrara F., Casarosa V., Ciampi L., Concordia C., Corbara S., Esuli A., Falchi F., Gennaro C., Lagani G., Massoli F. V., Meghini C., Messina N., Metilli D., Molinari A., Moreo Fernandez A., Nardi A., Pedrotti A., Pratelli N., Rabitti F., Savino P., Sebastiani F., Thanos C., Trupiano L., Vadicamo L., Vairo C.
Annual Report of the Artificial Intelligence for Media and Humanities laboratory (AIMH) research activities in 2020.DOI: https://doi.org/10.32079/isti-ar-2020/001
DOI: 10.32079/isti-ar-2020/001
Metrics:
See at:
CNR IRIS | ISTI Repository | CNR IRIS
2020
Conference article
Restricted
A procedure for the routinary correction of back-to-front degradations in archival manuscripts
Savino P., Tonazzini A.
Virtual restoration of the nowadays frequently available dig-ital copies of the human documental heritage is crucial for facilitating both the traditional work of philologists and paleographers and the automatic analysis of the contents. Here we propose a fast procedure for the correction of the typically complex background of recto-verso historical manuscripts, which does not need for a preliminary registration of the images of the two page sides. The simplicity of the procedure makes it suitable for the routinary use in the archives. The method consists of two stages, degradation identification and degradation cancellation. The first step performs the detection of both the primary text and the spurious strokes via soft segmentation based on the statistical decorrelation of the two recto and verso images. In the second step, the noisy pattern in the background is substituted with pixels that simulate the texture of the clean surrounding background, through an efficient technique of image inpainting. As shown in the experimental results, the proposed procedure is able to perform a fine and selective removal of the degradation, while preserving the original appearance of the manuscript.Source: LECTURE NOTES IN COMPUTER SCIENCE, vol. 12496, pp. 838-849. Da Nang, Vietnam, 30/11 - 3/12/2020
DOI: 10.1007/978-3-030-63007-2_66
Metrics:
Savino P., Tonazzini A.
Virtual restoration of the nowadays frequently available dig-ital copies of the human documental heritage is crucial for facilitating both the traditional work of philologists and paleographers and the automatic analysis of the contents. Here we propose a fast procedure for the correction of the typically complex background of recto-verso historical manuscripts, which does not need for a preliminary registration of the images of the two page sides. The simplicity of the procedure makes it suitable for the routinary use in the archives. The method consists of two stages, degradation identification and degradation cancellation. The first step performs the detection of both the primary text and the spurious strokes via soft segmentation based on the statistical decorrelation of the two recto and verso images. In the second step, the noisy pattern in the background is substituted with pixels that simulate the texture of the clean surrounding background, through an efficient technique of image inpainting. As shown in the experimental results, the proposed procedure is able to perform a fine and selective removal of the degradation, while preserving the original appearance of the manuscript.Source: LECTURE NOTES IN COMPUTER SCIENCE, vol. 12496, pp. 838-849. Da Nang, Vietnam, 30/11 - 3/12/2020
DOI: 10.1007/978-3-030-63007-2_66
Metrics:
See at:
dblp.uni-trier.de | doi.org | IRIS Cnr | CNR IRIS | IRIS Cnr
2019
Journal article
Open Access
Virtual restoration and content analysis of ancient degraded manuscripts
Tonazzini A, Savino P, Salerno E, Hanif M, Debole F
In recent years, extensive campaigns of digitization of the documental heritage conserved in libraries and archives have been performed, with the primary goal to ensure the preservation and fruition of this important part of the human cultural and historical patrimony. Besides protecting conservation, the availability of high quality digital copies has increasingly stimulated the use of image processing techniques, to perform a number of operations on documents and manuscripts, without harming the often precious and fragile originals. Among those, virtual restoration tasks are crucial, as they facilitate the traditional work of philologists and paleographers, and constitute a first step towards an automatic analysis of the written contents. Here we report our experience in this field, referring, as a case study, to the problem of removing one of the most frequent and impairing degradations affecting ancient manuscripts, i.e., the bleed-through distortion.We show that techniques of blind source separation are versatile tools to either cancel these unwanted interferences or isolate specific features for content analysis goals. Specialized algorithms, based on recto-verso models and sparse image representation, are then shown to be able to perform a fine and selective removal of the degradation, while preserving the original appearance of the manuscript.Source: INTERNATIONAL JOURNAL OF INFORMATION SCIENCE AND TECHNOLOGY, vol. 3 (issue 5), pp. 16-25
DOI: 10.57675/imist.prsm/ijist-v3i5.133
Metrics:
Tonazzini A, Savino P, Salerno E, Hanif M, Debole F
In recent years, extensive campaigns of digitization of the documental heritage conserved in libraries and archives have been performed, with the primary goal to ensure the preservation and fruition of this important part of the human cultural and historical patrimony. Besides protecting conservation, the availability of high quality digital copies has increasingly stimulated the use of image processing techniques, to perform a number of operations on documents and manuscripts, without harming the often precious and fragile originals. Among those, virtual restoration tasks are crucial, as they facilitate the traditional work of philologists and paleographers, and constitute a first step towards an automatic analysis of the written contents. Here we report our experience in this field, referring, as a case study, to the problem of removing one of the most frequent and impairing degradations affecting ancient manuscripts, i.e., the bleed-through distortion.We show that techniques of blind source separation are versatile tools to either cancel these unwanted interferences or isolate specific features for content analysis goals. Specialized algorithms, based on recto-verso models and sparse image representation, are then shown to be able to perform a fine and selective removal of the degradation, while preserving the original appearance of the manuscript.Source: INTERNATIONAL JOURNAL OF INFORMATION SCIENCE AND TECHNOLOGY, vol. 3 (issue 5), pp. 16-25
DOI: 10.57675/imist.prsm/ijist-v3i5.133
Metrics:
See at:
CNR IRIS | ISTI Repository | International Journal of Information Science and Technology | www.innove.org | doi.org | CNR IRIS
2019
Other
Open Access
AIMIR 2019 Research Activities
Amato G, Bolettieri P, Carrara F, Ciampi L, Di Benedetto M, Debole F, Falchi F, Gennaro C, Lagani G, Massoli Fv, Messina N, Rabitti F, Savino P, Vadicamo L, Vairo C
Multimedia Information Retrieval (AIMIR) research group is part of the NeMIS laboratory of the Information Science and Technologies Institute "A. Faedo" (ISTI) of the Italian National Research Council (CNR). The AIMIR group has a long experience in topics related to: Artificial Intelligence, Multimedia Information Retrieval, Computer Vision and Similarity search on a large scale. We aim at investigating the use of Artificial Intelligence and Deep Learning, for Multimedia Information Retrieval, addressing both effectiveness and efficiency. Multimedia information retrieval techniques should be able to provide users with pertinent results, fast, on huge amount of multimedia data. Application areas of our research results range from cultural heritage to smart tourism, from security to smart cities, from mobile visual search to augmented reality. This report summarize the 2019 activities of the research group.
Amato G, Bolettieri P, Carrara F, Ciampi L, Di Benedetto M, Debole F, Falchi F, Gennaro C, Lagani G, Massoli Fv, Messina N, Rabitti F, Savino P, Vadicamo L, Vairo C
Multimedia Information Retrieval (AIMIR) research group is part of the NeMIS laboratory of the Information Science and Technologies Institute "A. Faedo" (ISTI) of the Italian National Research Council (CNR). The AIMIR group has a long experience in topics related to: Artificial Intelligence, Multimedia Information Retrieval, Computer Vision and Similarity search on a large scale. We aim at investigating the use of Artificial Intelligence and Deep Learning, for Multimedia Information Retrieval, addressing both effectiveness and efficiency. Multimedia information retrieval techniques should be able to provide users with pertinent results, fast, on huge amount of multimedia data. Application areas of our research results range from cultural heritage to smart tourism, from security to smart cities, from mobile visual search to augmented reality. This report summarize the 2019 activities of the research group.
See at:
CNR IRIS | ISTI Repository | CNR IRIS
2019
Journal article
Open Access
Bleed-through cancellation in non-rigidly misaligned recto-verso archival manuscripts based on local registration
Savino P, Tonazzini A, Bedini L
Ancient manuscripts written on both pages of the sheet are frequently affected by ink bleeding from the reverse side. This phenomenon produces a significant degradation of both the foreground text and the general appearance of the manuscript. Effective digital image restoration techniques may require the use of the content of both document sides, thus needing their perfect alignment. Although often available, recto and verso are usually not aligned, either for rigid misalignments occurring during acquisition, or for non-rigid deformations of the sheet. In this paper, we propose a novel method to restore color recto-verso manuscript images in a piecewise manner, without the need of a preliminary, global registration of the two sides. We assume that at the local level any deformation can be approximated by a displacement and subdivide the two images into small patches of same size. For each pair of patches at the same location, their relative shift is estimated by cross-correlation, thus allowing their straightforward alignment. A bleed-through removal algorithm is then applied to the registered patches. By spanning the entire images, this procedure returns free-of-interferences versions of the images in their original acquisition layout. The experiments show that the restoration results so obtained are better than those obtained with the classical approach that first registers the whole recto-verso pair and then performs restoration. Further advantages are a much lower computational cost, the possibility to manage non-global and non-rigid deformations, and the unaltered geometry and color appearance of the two restored images.Source: INTERNATIONAL JOURNAL ON DOCUMENT ANALYSIS AND RECOGNITION, vol. 22 (issue 2), pp. 163-176
DOI: 10.1007/s10032-019-00323-2
Metrics:
Savino P, Tonazzini A, Bedini L
Ancient manuscripts written on both pages of the sheet are frequently affected by ink bleeding from the reverse side. This phenomenon produces a significant degradation of both the foreground text and the general appearance of the manuscript. Effective digital image restoration techniques may require the use of the content of both document sides, thus needing their perfect alignment. Although often available, recto and verso are usually not aligned, either for rigid misalignments occurring during acquisition, or for non-rigid deformations of the sheet. In this paper, we propose a novel method to restore color recto-verso manuscript images in a piecewise manner, without the need of a preliminary, global registration of the two sides. We assume that at the local level any deformation can be approximated by a displacement and subdivide the two images into small patches of same size. For each pair of patches at the same location, their relative shift is estimated by cross-correlation, thus allowing their straightforward alignment. A bleed-through removal algorithm is then applied to the registered patches. By spanning the entire images, this procedure returns free-of-interferences versions of the images in their original acquisition layout. The experiments show that the restoration results so obtained are better than those obtained with the classical approach that first registers the whole recto-verso pair and then performs restoration. Further advantages are a much lower computational cost, the possibility to manage non-global and non-rigid deformations, and the unaltered geometry and color appearance of the two restored images.Source: INTERNATIONAL JOURNAL ON DOCUMENT ANALYSIS AND RECOGNITION, vol. 22 (issue 2), pp. 163-176
DOI: 10.1007/s10032-019-00323-2
Metrics:
See at:
CNR IRIS | link.springer.com | ISTI Repository | International Journal on Document Analysis and Recognition (IJDAR) | CNR IRIS | CNR IRIS
2019
Journal article
Open Access
A data model and a cataloguing, storage and retrieval system for ancient document archives
Savino P, Tonazzini A, Debole F
Digitalization of ancient manuscripts is becoming a common practice in many archives and libraries, mainly for preservation purposes. This opens many new opportunities for the diffusion of these precious cultural assets, since several scholars and researchers, as well as the general public, may access and use them for research purposes, for study, and for general information. This is made possible if the documents, their descriptions, and the result of all processing activities performed on them are acquired at a good quality and can be easily accessed by using simple and powerful retrieval mechanisms. Acquired manuscripts suffer of degradations that may require different types of elaborations on the digital images, to improve their visual quality and legibility, or to discover hidden text that is not visible. Natural Language Processing requires the creation of transcriptions of the text contained in the manuscript, as well as encoding of the document structure and creation of user annotations. This paper presents a document management system and a metadata schema that make possible the storage and content-based retrieval of original documents, elaborations performed to improve their readability, textual transcriptions, and linguistic annotations. The archive will offer the possibility of describing, storing and accessing all the available manuscript versions, document transcriptions and annotations, and to search and retrieve documents based on all this information.Source: INTERNATIONAL JOURNAL OF INFORMATION SCIENCE AND TECHNOLOGY, vol. 3 (issue 5), pp. 6-15
DOI: 10.57675/imist.prsm/ijist-v3i5.132
Metrics:
Savino P, Tonazzini A, Debole F
Digitalization of ancient manuscripts is becoming a common practice in many archives and libraries, mainly for preservation purposes. This opens many new opportunities for the diffusion of these precious cultural assets, since several scholars and researchers, as well as the general public, may access and use them for research purposes, for study, and for general information. This is made possible if the documents, their descriptions, and the result of all processing activities performed on them are acquired at a good quality and can be easily accessed by using simple and powerful retrieval mechanisms. Acquired manuscripts suffer of degradations that may require different types of elaborations on the digital images, to improve their visual quality and legibility, or to discover hidden text that is not visible. Natural Language Processing requires the creation of transcriptions of the text contained in the manuscript, as well as encoding of the document structure and creation of user annotations. This paper presents a document management system and a metadata schema that make possible the storage and content-based retrieval of original documents, elaborations performed to improve their readability, textual transcriptions, and linguistic annotations. The archive will offer the possibility of describing, storing and accessing all the available manuscript versions, document transcriptions and annotations, and to search and retrieve documents based on all this information.Source: INTERNATIONAL JOURNAL OF INFORMATION SCIENCE AND TECHNOLOGY, vol. 3 (issue 5), pp. 6-15
DOI: 10.57675/imist.prsm/ijist-v3i5.132
Metrics:
See at:
CNR IRIS | innove.org | ISTI Repository | International Journal of Information Science and Technology | doi.org | CNR IRIS
2018
Journal article
Open Access
Sparse representation based inpainting for the restoration of document images affected by bleed-through
Hanif M, Tonazzini A, Savino P, Salerno E
Bleed-through is a commonly encountered degradation in ancient printed documents and manuscripts, which severely impair their readability. Digital image restoration techniques can be effective to remove or significantly reduce this degradation. In bleed-through document image restoration the main issue is to identify the bleed-through pixels and replace them with appropriate values, in accordance to their surroundings. In this paper, we propose a two stage method, where a pair of properly registered images of the document recto and verso is first used to locate the bleed-through pixels in each side, and then a sparse representation based image inpainting technique is used to fill-in the bleed-through areas according to the neighbourhood, in such a way to preserve the original appearance of the document. The advantages of the proposed inpainting technique over state-of-the-art methods are illustrated by the improvement in the visual results.Source: PROCEEDINGS (MDPI), vol. 2 (issue 2)
DOI: 10.3390/proceedings2020093
Metrics:
Hanif M, Tonazzini A, Savino P, Salerno E
Bleed-through is a commonly encountered degradation in ancient printed documents and manuscripts, which severely impair their readability. Digital image restoration techniques can be effective to remove or significantly reduce this degradation. In bleed-through document image restoration the main issue is to identify the bleed-through pixels and replace them with appropriate values, in accordance to their surroundings. In this paper, we propose a two stage method, where a pair of properly registered images of the document recto and verso is first used to locate the bleed-through pixels in each side, and then a sparse representation based image inpainting technique is used to fill-in the bleed-through areas according to the neighbourhood, in such a way to preserve the original appearance of the document. The advantages of the proposed inpainting technique over state-of-the-art methods are illustrated by the improvement in the visual results.Source: PROCEEDINGS (MDPI), vol. 2 (issue 2)
DOI: 10.3390/proceedings2020093
Metrics:
See at:
doi.org | CNR IRIS | ISTI Repository | www.mdpi.com | www.mdpi.com | CNR IRIS
2018
Journal article
Open Access
Non-local sparse image inpainting for document bleed-through removal
Hanif M, Tonazzini A, Savino P, Salerno E
Bleed-through is a frequent, pervasive degradation in ancient manuscripts, which is caused by ink seeped from the opposite side of the sheet. Bleed-through, appearing as an extra interfering text, hinders document readability and makes it difficult to decipher the information contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. This paper proposes a two-step restoration method for documents affected by bleed-through, exploiting information from the recto and verso images. First, the bleed-through pixels are identified, based on a non-stationary, linear model of the two texts overlapped in the recto-verso pair. In the second step, a dictionary learning-based sparse image inpainting technique, with non-local patch grouping, is used to reconstruct the bleed-through-contaminated image information. An overcomplete sparse dictionary is learned from the bleed-through-free image patches, which is then used to estimate a befitting fill-in for the identified bleed-through pixels. The non-local patch similarity is employed in the sparse reconstruction of each patch, to enforce the local similarity. Thanks to the intrinsic image sparsity and non-local patch similarity, the natural texture of the background is well reproduced in the bleed-through areas, and even a possible overestimation of the bleed through pixels is effectively corrected, so that the original appearance of the document is preserved. We evaluate the performance of the proposed method on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.Source: JOURNAL OF IMAGING, vol. 4 (issue 5)
DOI: 10.3390/jimaging4050068
Metrics:
Hanif M, Tonazzini A, Savino P, Salerno E
Bleed-through is a frequent, pervasive degradation in ancient manuscripts, which is caused by ink seeped from the opposite side of the sheet. Bleed-through, appearing as an extra interfering text, hinders document readability and makes it difficult to decipher the information contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. This paper proposes a two-step restoration method for documents affected by bleed-through, exploiting information from the recto and verso images. First, the bleed-through pixels are identified, based on a non-stationary, linear model of the two texts overlapped in the recto-verso pair. In the second step, a dictionary learning-based sparse image inpainting technique, with non-local patch grouping, is used to reconstruct the bleed-through-contaminated image information. An overcomplete sparse dictionary is learned from the bleed-through-free image patches, which is then used to estimate a befitting fill-in for the identified bleed-through pixels. The non-local patch similarity is employed in the sparse reconstruction of each patch, to enforce the local similarity. Thanks to the intrinsic image sparsity and non-local patch similarity, the natural texture of the background is well reproduced in the bleed-through areas, and even a possible overestimation of the bleed through pixels is effectively corrected, so that the original appearance of the document is preserved. We evaluate the performance of the proposed method on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.Source: JOURNAL OF IMAGING, vol. 4 (issue 5)
DOI: 10.3390/jimaging4050068
Metrics:
See at:
Journal of Imaging | CNR IRIS | ISTI Repository | DOAJ-Articles | Journal of Imaging | CNR IRIS
2018
Conference article
Open Access
Document bleed-through removal using sparse image inpainting
Hanif M, Tonazzini A, Savino P, Salerno E, Tsagkatakis G
Bleed-through is a pervasive degradation in ancient documents, caused by the ink of the opposite side of the sheet that has seeped through the paper fiber, and appears as an extra, interfering text. Bleed-through severely impairs document readability and makes it difficult to decipher the contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. The main theme is to identify the bleedthrough pixels and estimate an appropriate replacement for them, in accordance to their surrounding. This paper proposes a two-step image restoration method, exploiting information from the recto and verso images. First, based on a non-stationary linear model of the two texts overlapped in the recto-verso pair, the bleed-through pixels are identified. In the second step, a sparse representation based image inpainting technique, with a non-negative sparsity constraint, is used to find an appropriate replacement for the bleedthough pixels. Thanks to the power of dictionary learning and sparse image reconstruction methods, the natural texture of the background is well reproduced in the bleed-through areas, and even a their possible overestimation is effectively corrected, so that the original appearance of the document is preserved. The experiments are conducted on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.DOI: 10.1109/das.2018.21
Metrics:
Hanif M, Tonazzini A, Savino P, Salerno E, Tsagkatakis G
Bleed-through is a pervasive degradation in ancient documents, caused by the ink of the opposite side of the sheet that has seeped through the paper fiber, and appears as an extra, interfering text. Bleed-through severely impairs document readability and makes it difficult to decipher the contents. Digital image restoration techniques have been successfully employed to remove or significantly reduce this distortion. The main theme is to identify the bleedthrough pixels and estimate an appropriate replacement for them, in accordance to their surrounding. This paper proposes a two-step image restoration method, exploiting information from the recto and verso images. First, based on a non-stationary linear model of the two texts overlapped in the recto-verso pair, the bleed-through pixels are identified. In the second step, a sparse representation based image inpainting technique, with a non-negative sparsity constraint, is used to find an appropriate replacement for the bleedthough pixels. Thanks to the power of dictionary learning and sparse image reconstruction methods, the natural texture of the background is well reproduced in the bleed-through areas, and even a their possible overestimation is effectively corrected, so that the original appearance of the document is preserved. The experiments are conducted on the images of a popular database of ancient documents, and the results validate the performance of the proposed method compared to the state of the art.DOI: 10.1109/das.2018.21
Metrics:
See at:
CNR IRIS | ieeexplore.ieee.org | ISTI Repository | doi.org | CNR IRIS | CNR IRIS