The Smooth-Lasso and other $\ell_1+\ell_2$-penalized methods

We consider a linear regression problem in a high dimensional setting where the number of covariates $p$ can be much larger than the sample size $n$. In such a situation, one often assumes sparsity of the regression vector, \textit{i.e.}, the regression vector contains many zero components. We propose a Lasso-type estimator $\hat{\beta}^{Quad}$ (where '$Quad$' stands for quadratic) which is based on two penalty terms. The first one is the $\ell_1$ norm of the regression coefficients used to exploit the sparsity of the regression as done by the Lasso estimator, whereas the second is a quadratic penalty term introduced to capture some additional information on the setting of the problem. We detail two special cases: the Elastic-Net $\hat{\beta}^{EN}$, which deals with sparse problems where correlations between variables may exist; and the Smooth-Lasso $\hat{\beta}^{SL}$, which responds to sparse problems where successive regression coefficients are known to vary slowly (in some situations, this can also be interpreted in terms of correlations between successive variables). From a theoretical point of view, we establish variable selection consistency results and show that $\hat{\beta}^{Quad}$ achieves a Sparsity Inequality, \textit{i.e.}, a bound in terms of the number of non-zero components of the 'true' regression vector. These results are provided under a weaker assumption on the Gram matrix than the one used by the Lasso. In some situations this guarantees a significant improvement over the Lasso. Furthermore, a simulation study is conducted and shows that the S-Lasso $\hat{\beta}^{SL}$ performs better than known methods as the Lasso, the Elastic-Net $\hat{\beta}^{EN}$, and the Fused-Lasso with respect to the estimation accuracy. This is especially the case when the regression vector is 'smooth', \textit{i.e.}, when the variations between successive coefficients of the unknown parameter of the regression are small. The study also reveals that the theoretical calibration of the tuning parameters and the one based on $10$ fold cross validation imply two S-Lasso solutions with close performance.

Data and Resources

Additional Info

Field Value
Source https://hal.science/hal-00462882
Author Hebiri, Mohamed, van de Geer, Sara A.
Maintainer CCSD
Last Updated November 12, 2026, 08:00 (UTC)
Created November 12, 2026, 08:00 (UTC)
Identifier hal-00462882
Language en
Rights https://about.hal.science/hal-authorisation-v1/
contributor Seminar für Statistik ; Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich] (ETH Zürich)
creator Hebiri, Mohamed
date 2011-10-07T00:00:00
harvest_object_id d25dd3ec-8d52-4074-b853-5805cad742bc
harvest_source_id 3374d638-d20b-4672-ba96-a23232d55657
harvest_source_title test moissonnage SELUNE
metadata_modified 2026-04-02T00:00:00
relation info:eu-repo/semantics/altIdentifier/arxiv/1003.4885
set_spec type:UNDEFINED