Learning Majority Vote for Supervised Classification and Domain Adaptation: PAC-Bayesian Approaches and Similarity Combination

Nowadays, due to the expansion of the web a plenty of data are available and many applications need to make use of supervised machine learning methods able to take into account different information sources. For instance, for multimedia semantic indexing applications, one have to efficiently take advantage of information about color, textual, texture or sound sources of the document. Most of the existing methods try to combine these multimodal informations, either by directly fusionning the descriptors or by combining similarities or classifiers, in order to produce a classification model more reliable for the considered task. Usually, these multimodal facets imply two main issues. On the one hand, one have to be able to correctly make use of all the a priori information available. On the other hand, the data, on which the model will be applied, does not come from the same probability distribution than the data used during the learning step. In this context, we have to adapt the model on new data, which is known as domain adaptation. In this thesis, we propose several theoretically-founded contributions for tackle these issues. A first serie of contributions studies the problem of learning a weighted majority vote over a set of voters in a supervised classification setting.These results fall within the context of the PAC-Bayesian theory allowing to derive generalization abilities for such a vote by assuming an a priori on the relevance of the voters. Our first contribution aims at extending a recent algorithm, MinCq, minimizing a bound over the error of the majority vote in binary classification. This extension can take into account an a priori belief on the performances of the voters. This belief is expressed as an aligned distribution. We illustrate its usefulness for combining nearest neighbor classifiers, and for classifier fusion on a multimedia semantic indexing task. Then, we propose a theoretical contribution for multiclass classification tasks. Our approach is based on an original PAC-Bayesian analysis considering the operator norm of the confusion matrix as an error measure. Our second series of contributions relates to domain adaptation. In this situation we present our third result for combining similarities in order to infer a representation space for moving closer the learning distribution and the testing distribution. This contribution is based on the theory of learning from (epsilon,gamma,tau)-good similarity functions and is justified by the minimization of an usual bound in domain adaptation. For our last contribution, we propose the first PAC-Bayesian analysis for domain adaptation. This analysis is based on a consistent divergence measure between distributions allowing us to derive a generalization bound for learning majority votes in binary classification. Moreover, we propose a first algorithm specialized to linear classifiers and able to directly minimize our bound.

Data and Resources

Additional Info

Field Value
Source https://theses.hal.science/tel-00879072
Author Morvant, Emilie
Maintainer CCSD
Last Updated May 9, 2026, 04:28 (UTC)
Created May 9, 2026, 04:28 (UTC)
Identifier tel-00879072
Language fr
Rights https://about.hal.science/hal-authorisation-v1/
contributor éQuipe AppRentissage et MultimediA [Marseille] (QARMA) ; Laboratoire d'informatique Fondamentale de Marseille (LIF) ; Aix Marseille Université (AMU)-École Centrale de Marseille (ECM)-Centre National de la Recherche Scientifique (CNRS)-Aix Marseille Université (AMU)-École Centrale de Marseille (ECM)-Centre National de la Recherche Scientifique (CNRS)
creator Morvant, Emilie
date 2013-09-18T00:00:00
harvest_object_id ccd029e9-a80a-4cf3-bab3-c0cf2899bb7f
harvest_source_id 3374d638-d20b-4672-ba96-a23232d55657
harvest_source_title test moissonnage SELUNE
metadata_modified 2026-04-23T00:00:00
set_spec type:THESE