Can embeddings analysis explain large language model ranking?

Lucchese C., Minello G., Nardini F. M., Orlando S., Perego R., Veneri A.

Text ranking  Explainable artificial intelligence  Embeddings analysis  Large language models 

Understanding the behavior of deep neural networks for Information Retrieval (IR) is crucial to improve trust in these effective models. Current popular approaches to diagnose the predictions made by deep neural networks are mainly based on: i) the adherence of the retrieval model to some axiomatic property of the IR system, ii) the generation of free-text explanations, or iii) feature importance attributions. In this work, we propose a novel approach that analyzes the changes of document and query embeddings in the latent space and that might explain the inner workings of IR large pre-trained language models. In particular, we focus on predicting query/document relevance, and we characterize the predictions by analyzing the topological arrangement of the embeddings in their latent space and their evolution while passing through the layers of the network. We show that there exists a link between the embedding adjustment and the predicted score, based on how tokens cluster in the embedding space. This novel approach, grounded in the query and document tokens interplay over the latent space, provides a new perspective on neural ranker explanation and a promising strategy for improving the efficiency of the models and Query Performance Prediction (QPP).

Source: CIKM '23 - 32nd ACM International Conference on Information and Knowledge Management, pp. 4150–4154, Birmingham, UK, 21-25/10/2023

Metrics