Risk-Sensitive Optimal Feedback Control Accounts for Sensorimotor Behavior under Uncertainty

被引:59
|
作者
Nagengast, Arne J. [1 ,2 ]
Braun, Daniel A. [1 ]
Wolpert, Daniel M. [1 ]
机构
[1] Univ Cambridge, Dept Engn, Computat & Biol Learning Lab, Cambridge CB2 1PZ, England
[2] Univ Cambridge, Dept Expt Psychol, Cambridge CB2 3EB, England
基金
英国惠康基金;
关键词
ADAPTATION; SPECIFICITY; STRATEGIES; MOVEMENTS;
D O I
10.1371/journal.pcbi.1000857
中图分类号
Q5 [生物化学];
学科分类号
071010 ; 081704 ;
摘要
Many aspects of human motor behavior can be understood using optimality principles such as optimal feedback control. However, these proposed optimal control models are risk-neutral; that is, they are indifferent to the variability of the movement cost. Here, we propose the use of a risk-sensitive optimal controller that incorporates movement cost variance either as an added cost (risk-averse controller) or as an added value (risk-seeking controller) to model human motor behavior in the face of uncertainty. We use a sensorimotor task to test the hypothesis that subjects are risk-sensitive. Subjects controlled a virtual ball undergoing Brownian motion towards a target. Subjects were required to minimize an explicit cost, in points, that was a combination of the final positional error of the ball and the integrated control cost. By testing subjects on different levels of Brownian motion noise and relative weighting of the position and control cost, we could distinguish between risk-sensitive and risk-neutral control. We show that subjects change their movement strategy pessimistically in the face of increased uncertainty in accord with the predictions of a risk-averse optimal controller. Our results suggest that risk-sensitivity is a fundamental attribute that needs to be incorporated into optimal feedback control models.
引用
收藏
页数:15
相关论文
共 50 条
  • [31] Quantum risk-sensitive control
    D'Helon, C.
    Doherty, A. C.
    James, M. R.
    Wilson, S. D.
    PROCEEDINGS OF THE 45TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-14, 2006, : 3132 - +
  • [32] RISK-SENSITIVE LQG DISCOUNTED CONTROL PROBLEMS AND THEIR ASYMPTOTIC BEHAVIOR
    Hernandez-Hernandez, Daniel
    Salazar-Sanchez, Pedro
    SIAM JOURNAL ON CONTROL AND OPTIMIZATION, 2023, 61 (03) : 1136 - 1161
  • [33] Risk-sensitive reinforcement learning applied to control under constraints
    Geibel, P. (PGEIBEL@UOS.DE), 1600, American Association for Artificial Intelligence (24):
  • [34] Risk-sensitive reinforcement learning applied to control under constraints
    Geibel, P
    Wysotzki, F
    JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH, 2005, 24 : 81 - 108
  • [35] The Stochastic Maximum Principle for Risk-Sensitive Optimal Control with Delay and Applications
    Lee, Myoung Hoon
    Moon, Jun
    2018 IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2018, : 7052 - 7057
  • [36] Risk-sensitive optimal control of hidden Markov models: Structural results
    FernandezGaucherand, E
    Marcus, SI
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 1997, 42 (10) : 1418 - 1422
  • [37] Risk-sensitive optimal control for Markov decision processes with monotone cost
    Borkar, VS
    Meyn, SP
    MATHEMATICS OF OPERATIONS RESEARCH, 2002, 27 (01) : 192 - 209
  • [38] RISK-SENSITIVE APPROACH TO OPTIMAL FILTERING AND CONTROL FOR LINEAR STOCHASTIC SYSTEMS
    Aracelia Alcorta-Garcia, Ma.
    Basin, Michael
    Acosta Sanchez, Yazmin Gpe
    INTERNATIONAL JOURNAL OF INNOVATIVE COMPUTING INFORMATION AND CONTROL, 2009, 5 (06): : 1599 - 1613
  • [39] Linear Quadratic Optimal and Risk-Sensitive Control for Vehicle Active Suspensions
    Brezas, Panos
    Smith, Malcolm C.
    IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, 2014, 22 (02) : 543 - 556
  • [40] Markov risk mappings and risk-sensitive optimal prediction
    Kosmala, Tomasz
    Martyr, Randall
    Moriarty, John
    MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 2023, 97 (01) : 91 - 116