登录 | 注册 | 退出 | 公司首页 | 繁体中文 | 满意度调查
综合馆
一种求解截断L1正则化项问题的坐标下降算法
  • 摘要

    L1正则化在稀疏学习的研究中起关键作用,使用截断L1正则化项往往可以获得更好的准确率,但却导致了非凸优化问题.目前,主要采用多阶段凸松弛(multi-stage convex relaxation,MSCR)算法进行求解,由于每一阶段都需要求解一个凸优化问题,计算代价较大.为了弥补上述不足,提出了一种求解截断L1正则化项非凸学习问题的坐标下降算法(Non-convex CD).该算法只需在多阶段凸松弛算法的每一阶段执行单步的坐标下降算法,有效降低了计算复杂性.理论分析表明所提出的算法是收敛的.针对Lasso问题,在大规模真实数据库作了实验,实验结果表明,Non-convex CD在取得和MSCR几乎相同准确率的基础上,求解的CPU时间甚至优于求解凸问题的坐标下降方法.为了进一步说明所提算法的性能,进一步研究了Non-convex CD在图像去模糊化中的应用问题.

  • 作者

    王玉军  高乾坤  章显  陶卿  Wang Yujun  Gao Qiankun  Zhang Xian  Tao Qing 

  • 作者单位

    中国人民解放军陆军军官学院 合肥 230031

  • 刊期

    2014年6期 ISTIC EI PKU

  • 关键词

    截断L1正则化项  非凸优化  多阶段凸松弛  坐标下降  图像去模糊化  capped-L1  regularization  Non-convex  optimization  multi-stage  convex  relaxation  coordinate  descent  image  deblurring 

参考文献
  • [1] Zhang Tong.Multi-stage convex relaxation for learning with sparse regularization[J].Neural Information Processing Systems,2008,37(6):763-771. 2008
  • [2] Tibshirani R.Regression shrinkage and selection via the lasso[J].Journal of the Royal Statistical Society,1996,58(1):267-288. 1996
  • [3] Luo Z Q,Tseng P.On the convergence of coordinate descent method for convex differentiable minimization[J].Journal of Optimization Theory and Applications,1992,72(1):7-35. 1992
  • [4] Luo Z Q,Tseng P.On the linear convergence of descent methods for convex essentially smooth minimization[J].SIAM Journal on Control and Optimization,1992,30(2):408-425. 1992
  • [5] Tseng P.Convergence of a block coordinate descent method for non-differentiable minimization[J].Journal of Optimization Theory and Applications,2001,109(3):475-494. 2001
  • [6] Fu W J.Penalized regressions:The bridge versus the lasso[J].Journal of Computational and Graphical Statistics,1998,7(3):397-416. 1998
  • [7] Friedman J,Hastie T,Hudotofling H,et al.Pathwise coordinate optimization[J].The Annals of Applied Statistics,2007,1(2):302-332. 2007
  • [8] Wu T T,Lange K.Coordinate descent algorithms for lasso penalized regression[J].The Annals of Applied Statistics,2008:224-244. 2008
  • [9] Chang Kaiwei,Hsieh C J,Lin C J.Coordinate descent method for large-scale L2-loss linear support vector machines[J].Journal of Machine Learning Research,2008,49 (7):1369-1398. 2008
  • [10] Lin C J,Ruby C W,Keerthi S S.Trust region Newton method for large-scale logistic regression[J].Journal of Machine Learning Research,2008,9(10):627-650. 2008
  • [11] Franc V,Sonnenburg S.Optimized cutting plane algorithmfor support vector machines[C]//Proc of the 25th Int Conf on Machine Learning.New York:ACM,2008:320-327. 2008
  • [12] Shalev-Shwartz S,Singer Y,Srebro N.Pegasos:Primal estimated sub-gradient solver for SVM[C]//Proc of the 24th Int Conf on Machine Learning.New York:ACM,2007:807-814. 2007
  • [13] Sriperumbudur,Lanckriet G.On the convergence of the concave-convex procedure[C]//Proc of Advances in Neural Information Processing Systems.New York:ACM,2009:1759-1767. 2009
  • [14] Chang Kaiwei,Hsieh C J,Lin C J.Coordinate descent method for large-scale L2-loss linear support vector machines[J].Journal of Machine Learning Research,2008,49 (7):1369-1398. 2008
  • [15] Yuan G X,Chang Kaiwei,Hsieh C J,et al.A comparison of optimization methods and software for large-scale L1regularized linear classification[J].Journal of Machine Learning Research,2010,9999:3183-3234. 2010
  • [16] Xiao Ling.Dual averaging methods for regularized stochastic learning and online optimization[J].The Journal of Machine Learning Research,2010,3(11):2543-2569. 2010
  • [17] Kong Kang,Tao Qing,Wang Qunshan,et al.A sub-gadient based solver for L1-rgularization+hing problem[J].Journal of Computer Research and Development,2012,49(7):1494-1499 (in Chinese)(孔康,陶卿,汪群山,等.基于次梯度的L1正则化Hing损失问题求解研究[J].计算机研究与发展,2012,49(7):1494-1499). 2012
  • [18] Collobert R,Sinz F,Weston J,et al.Trading convexity for scalability[C]//Proc of the 23th Int Conf on Machine Learning.New York:ACM,2006:201-208. 2006
  • [19] Yuille A L,Rangarajan A.The concave-convex procedure (CCCP)[J].Neural Computation,2003,15(4):935-936. 2003
  • [20] http://www.csie.ntu.edu.tw/~cjlin/.
  • [21] http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/.
  • [22] Sun Zhengya,Tao Qing.Statistical machine learning:A review of the loss function and optimization[J].Communications of The China Computer Federation,2009,5(8):7-14 (in Chinese)(孙正雅,陶卿.统计机器学习综述:损失函数与优化求解[J].中国计算机学会通讯,2009,5(8):7-14). 2009
  • [23] Hsieh C J,Chang Kaiwei,Lin C J,et al.A dual coordinate descent method for large-scale linear SVM[C]//Proc of the 25th Int Conf on Machine Learning.New York:ACM,2008:408-415. 2008
  • [24] Hansen P C,Nagy J G,et al.Deblurring Images Matrices,Spectra,and Fltering[M].Philadelphia:Society for Industrial and Applied Mathematics,2006:173-192. 2006
查看更多︾
相似文献 查看更多>>
18.207.130.162