The present study deals with word frequencies distributions and their relation to probabilistic Information Retrieval (IR) models. We examine the burstiness phenomenon of word frequencies in textual collections. We propose to model this phenomenon as a property of probability distributions and we study the Beta Negative Binomial and Log-Logistic distributions to model word frequencies. We then focus on probabilistic IR models and their fundamental properties. Our analysis reveals that probability distributions underlying most state-of-the-art models do not take this phenomenon into account , even if fundamental properties of IR models such as concavity enable implicitly to take it into account. We then introduce a novel family of probabilistic IR model, based on Shannon information. These new models bridge the gap between significant properties of IR models and the burstiness phenomenon of word frequencies. Lastly, we study empirically and theoretically pseudo relevance feedback models. We propose a theoretical framework which explain well the empirical behaviour and performance of pseudo relevance feedback models. Overall, this highlights interesting properties for pseudo relevance feedback and shows that some state-of-the-art model are inadequate.
