The aim of this habilitation thesis is to give an overview of my works on high-dimensional statistics and statistical learning, under various sparsity assumptions. In a first part, I will describe the major challenges of high-dimensional statistics in the context of the generic linear regression model. After a brief review of existing results, I will present the theoretical study of aggregated estimators that was done in (Alquier & Lounici 2011). The second part essentially aims at providing extensions of the various theories presented in the first part to the estimation of time series models (Alquier & Doukhan 2011, Alquier & Wintenberger 2013, Alquier & Li 2012, Alquier, Wintenberger & Li 2012). Finally, the third part presents various extensions to nonparametric models, or to specific applications such as quantum statistics (Alquier & Biau 2013, Guedj & Alquier 2013, Alquier, Meziani & Peyré 2013, Alquier, Butucea, Hebiri, Meziani & Morimae 2013, Alquier 2013, Alquier 2008). In each section, we provide explicitely the estimators used and, as much as possible, optimal oracle inequalities satisfied by these estimators.
