Learning Deep Representations : Toward a better new understanding of the deep learning paradigm
type: Distribution
access URL: https://theses.hal.science/tel-00842447
format: HTML
issued: 2026-05-10T10:31:40.450683
modified: 2026-05-10T10:31:40.420077

test_moissonnage_selune
type: Agent

Learning Deep Representations : Toward a better new understanding of the deep learning paradigm
type: Dataset
contact point: Organization
  fn: CCSD
keyword: theses
keyword: infoinfo-ohcomputer-science-csother-csoh
keyword: machine-learning
keyword: infoeu-reposemanticsdoctoralthesis
keyword: apprentissage-profond
keyword: apprentissage-artificiel
keyword: reseaux-de-neurones
keyword: deep-learning
keyword: neural-networks
landing page: https://theses.hal.science/tel-00842447
description: Since 2006, deep learning algorithms which rely on deep architectures with several layers of increasingly complex representations have been able to outperform state-of-the-art methods in several settings. Deep architectures can be very efficient in terms of the number of parameters required to represent complex operations which makes them very appealing to achieve good generalization with small amounts of data. Although training deep architectures has traditionally been considered a difficult problem, a successful approach has been to employ an unsupervised layer-wise pre-training step to initialize deep supervised models. First, unsupervised learning has many benefits w.r.t. generalization because it only relies on unlabeled data which is easily found. Second, the possibility to learn representations layer by layer instead of all layers at once improves generalization further and reduces computational time. However, deep learning is a very recent approach and still poses a lot of theoretical and practical questions concerning the consistency of layer-wise learning with many layers and difficulties such as evaluating performance, performing model selection and optimizing layers. In this thesis we first discuss the limitations of the current variational justification for layer-wise learning which does not generalize well to many layers. We ask if a layer-wise method can ever be truly consistent, i.e. capable of finding an optimal deep model by training one layer at a time without knowledge of the upper layers. We find that layer-wise learning can in fact be consistent and can lead to optimal deep generative models. To do this, we introduce the Best Latent Marginal (BLM) upper bound, a new criterion which represents the maximum log-likelihood of a deep generative model where the upper layers are unspecified. We prove that maximizing this criterion for each layer leads to an optimal deep architecture, provided the rest of the training goes well. Although this criterion…
identifier: NNT: 2013PA112103
issued: 2026-05-10T10:31:40.433512
language: en
modified: 2026-05-10T10:31:40.433516