Efficient and user-friendly visualization of neural relightable images for cultural heritage applications

Righetto L., Khademizadeh M., Giachetti A., Ponchio F., Gigilashvili D., Bettio F., Gobbetti E.

Visualization  Multi-light image collections  RTI  Reflectance Transformation Imaging  neural networks  neural representations  relighting 

We introduce an innovative multi-resolution framework for encoding and interactively visualizing large relightable images using a neural reflectance model derived from a state-of-the-art technique. The framework is seamlessly integrated into a scalable multi-platform framework that supports adaptive streaming and exploration of multi-layered relightable models in web settings. To enhance efficiency, we optimized the neural model, simplified decoding, and implemented a custom WebGL shader specific to the task, eliminating the need for deep-learning library integration in the code. Additionally, we introduce an efficient level-of-detail management system supporting fine-grained adaptive rendering through on-the-fly resampling in latent feature space. The resulting viewer facilitates interactive neural relighting of large images. Its modular design allows the incorporation of functionalities for cultural heritage analysis, such as loading and simultaneous visualization of multiple relightable layers with arbitrary rotations.

Source: ACM JOURNAL ON COMPUTING AND CULTURAL HERITAGE, vol. 17 (issue 4), pp. 1-24

Metrics