On group-wise ℓp regularization: Theory and efficient algorithms
MetadataShow full item record
Following advances in compressed sensing and high-dimensional statistics, many pattern recognition methods have been developed with ℓ1 regularization, which promotes sparse solutions. In this work, we instead advocate the use of ℓp (2 ≥ p > 1) regularization in a group setting which provides a better trade-off between sparsity and algorithmic stability. We focus on the simplest case with squared loss, which is known as group bridge regression. On the theoretical side, we prove that group bridge regression is uniformly stable and thus generalizes, which is an important property of a learning method. On the computational side, we make group bridge regression more practically attractive by deriving provably convergent and computationally efficient optimization algorithms. We show that there are at least several values of p over (1,2) at which the iterative update is analytical, thus it is even suitable for large-scale settings. We demonstrate the clear advantage of group bridge regression with the proposed algorithms over other competitive alternatives on several datasets. As ℓp-regularization allows one to achieve flexibility in sparseness/denseness of the solution, we hope that the algorithms will be useful for future applications of this regularization.
Showing items related by title, author, creator and subject.
Pham, DucSon (2015)Following advances in compressed sensing and high-dimensional statistics, many pattern recognition methods have been developed with l1 regularization, which promotes sparse solutions. In this work, we instead advocate ...
Nguyen, Nam; Liu, Wan-Quan; Venkatesh, Svetha (2008)Two dimensional linear discriminant analysis (2DLDA) has received much interest in recent years. However, 2DLDA could make pairwise distances between any two classes become significantly unbalanced, which may affect its ...
Meng, Shuanghui (2007)Glaucoma is a common eye disease that affects the optic nerve. It is the second leading cause of visual loss globally and while it can occur in all age groups, it is most common in the elderly. The main symptom of glaucoma ...