Continuous space models with neural networks in natural language processing
type: Dataset
description: The purpose of language models is in general to capture and to model regularities of language, thereby capturing morphological, syntactical and distributional properties of word sequences in a given language. They play an important role in many successful applications of Natural Language Processing, such as Automatic Speech Recognition, Machine Translation and Information Extraction. The most successful approaches to date are based on n-gram assumption and the adjustment of statistics from the training data by applying smoothing and back-off techniques, notably Kneser-Ney technique, introduced twenty years ago. In this way, language models predict a word based on its n-1 previous words. In spite of their prevalence, conventional n-gram based language models still suffer from several limitations that could be intuitively overcome by consulting human expert knowledge. One critical limitation is that, ignoring all linguistic properties, they treat each word as one discrete symbol with no relation with the others. Another point is that, even with a huge amount of data, the data sparsity issue always has an important impact, so the optimal value of n in the n-gram assumption is often 4 or 5 which is insufficient in practice. This kind of model is constructed based on the count of n-grams in training data. Therefore, the pertinence of these models is conditioned only on the characteristics of the training text (its quantity, its representation of the content in terms of theme, date). Recently, one of the most successful attempts that tries to directly learn word similarities is to use distributed word representations in language modeling, where distributionally words, which have semantic and syntactic similarities, are expected to be represented as neighbors in a continuous space. These representations and the associated objective function (the likelihood of the training data) are jointly learned using a multi-layer neural network architecture. In this way, word similarit…
landing page: https://theses.hal.science/tel-00776704
identifier: NNT: 2012PA112407
keyword: automatic-speech-recognition
keyword: continuous-space-model
keyword: espace-continu-de-representation
keyword: infoeu-reposemanticsdoctoralthesis
keyword: infoinfo-ohcomputer-science-csother-csoh
keyword: modele-de-langage-statistique
keyword: natural-language-processing
keyword: neural-network
keyword: reconnaissance-automatique-de-la-parole
keyword: reseau-de-neurones
keyword: statistical-language-model
keyword: statistical-machine-translation
keyword: theses
keyword: traduction-automatique-par-approche-statistique
keyword: traitement-automatique-du-langage-naturel
issued: 2026-05-15T06:23:38.447985
modified: 2026-05-15T06:23:38.447991
language: en
contact point: Organization
  fn: CCSD

test_moissonnage_selune
type: Agent

Continuous space models with neural networks in natural language processing
type: Distribution
format: HTML
access URL: https://theses.hal.science/tel-00776704
issued: 2026-05-15T06:23:38.496301
modified: 2026-05-15T06:23:38.412825