Linear Programming-Based Sparse Kernel Regression with L1-Norm Minimization for Nonlinear System Modeling

被引:0
|
作者
Liu, Xiaoyong [1 ]
Yan, Genglong [1 ]
Zhang, Fabin [1 ]
Zeng, Chengbin [1 ]
Tian, Peng [1 ]
机构
[1] Moutai Inst, Automat Dept Brewing Engn, Renhuai 564507, Peoples R China
基金
中国国家自然科学基金;
关键词
linear programming; structural risk minimization; L1-norm on approximation error; sparse kernel regression; nonlinear system modeling; IDENTIFICATION;
D O I
10.3390/pr12112358
中图分类号
TQ [化学工业];
学科分类号
0817 ;
摘要
This paper integrates L1-norm structural risk minimization with L1-norm approximation error to develop a new optimization framework for solving the parameters of sparse kernel regression models, addressing the challenges posed by complex model structures, over-fitting, and limited modeling accuracy in traditional nonlinear system modeling. The first L1-norm regulates the complexity of the model structure to maintain its sparsity, while another L1-norm is essential for ensuring modeling accuracy. In the optimization of support vector regression (SVR), the L2-norm structural risk is converted to an L1-norm framework through the condition of non-negative Lagrange multipliers. Furthermore, L1-norm optimization for modeling accuracy is attained by minimizing the maximum approximation error. The integrated L1-norm of structural risk and approximation errors creates a new, simplified optimization problem that is solved using linear programming (LP) instead of the more complex quadratic programming (QP). The proposed sparse kernel regression model has the following notable features: (1) it is solved through relatively simple LP; (2) it effectively balances the trade-off between model complexity and modeling accuracy; and (3) the solution is globally optimal rather than just locally optimal. In our three experiments, the sparsity metrics of SVs% were 2.67%, 1.40%, and 0.8%, with test RMSE values of 0.0667, 0.0701, 0.0614 (sinusoidal signal), and 0.0431 (step signal), respectively. This demonstrates the balance between sparsity and modeling accuracy.
引用
收藏
页数:17
相关论文
共 50 条
  • [1] Linear programming and l1-norm minimization problems with convolution constraints
    Hill, Robin D.
    2005 44TH IEEE CONFERENCE ON DECISION AND CONTROL & EUROPEAN CONTROL CONFERENCE, VOLS 1-8, 2005, : 4404 - 4409
  • [2] Comparison of l1-Norm SVR and Sparse Coding Algorithms for Linear Regression
    Zhang, Qingtian
    Hu, Xiaolin
    Zhang, Bo
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2015, 26 (08) : 1828 - 1833
  • [3] BEYOND l1-NORM MINIMIZATION FOR SPARSE SIGNAL RECOVERY
    Mansour, Hassan
    2012 IEEE STATISTICAL SIGNAL PROCESSING WORKSHOP (SSP), 2012, : 337 - 340
  • [4] Design of Sparse FIR Filters Based on Reweighted l1-Norm Minimization
    Yang, Yuhua
    Zhu, Wei-Ping
    Wu, Dalei
    2015 IEEE INTERNATIONAL CONFERENCE ON DIGITAL SIGNAL PROCESSING (DSP), 2015, : 858 - 862
  • [5] A Novel Sparse System Estimation Method based on Least Squares, l1-Norm Minimization and Shrinkage
    Torturela, Alexandre de M.
    de Lamare, Rodrigo C.
    Medina, Cesar A.
    Sampaio-Neto, Raimundo
    2015 INTERNATIONAL CONFERENCE ON COMPUTING, NETWORKING AND COMMUNICATIONS (ICNC), 2015, : 895 - 899
  • [6] NONLINEAR L1-NORM MINIMIZATION LEARNING FOR HUMAN DETECTION
    Xu, Ran
    Jiao, Jianbin
    Ye, Qixiang
    2011 18TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2011,
  • [7] A NEW ALGORITHM FOR NONLINEAR L1-NORM MINIMIZATION WITH NONLINEAR EQUALITY CONSTRAINTS
    SOLIMAN, SA
    CHRISTENSEN, GS
    ROUHI, AH
    COMPUTATIONAL STATISTICS & DATA ANALYSIS, 1991, 11 (01) : 97 - 109
  • [8] Lagrange Programming Neural Network for the l1-norm Constrained Quadratic Minimization
    Lee, Ching Man
    Feng, Ruibin
    Leung, Chi-Sing
    NEURAL INFORMATION PROCESSING, PT III, 2015, 9491 : 119 - 126
  • [9] Robust Plane Clustering Based on L1-Norm Minimization
    Yang, Hongxin
    Yang, Xubing
    Zhang, Fuquan
    Ye, Qiaolin
    IEEE ACCESS, 2020, 8 : 29489 - 29500
  • [10] Brain abnormality segmentation based on l1-norm minimization
    Zeng, Ke
    Erus, Guray
    Tanwar, Manoj
    Davatzikos, Christos
    MEDICAL IMAGING 2014: IMAGE PROCESSING, 2014, 9034