Optimal model inference for Bayesian mixture of experts

被引：0

作者：

Ueda, N ^{[1
]}

Ghahramani, Z ^{[1
]}

机构：

[1] NTT, Commun Sci Labs, Kyoto 6190237, Japan

来源：

NEURAL NETWORKS FOR SIGNAL PROCESSING X, VOLS 1 AND 2, PROCEEDINGS | 2000年

关键词：

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We present an algorithm for inferring the parameter and model structure of a mixture of experts model (MoE) based on the variational Bayesian (VB) framework. First, in the VB framework, we show that the model parameter and structure of a MoE can be simultaneously optimized by maximizing an objective funtion derived in this paper. Next, we present a deterministic algorithm to find the optimal number of experts of a MoE while avoiding local maxima. Our experimental results demonstrate the practical usefulness of the method.

引用

页码：145 / 154

页数：10

共 50 条

[31] Full Bayesian inference with hazard mixture models
Arbel, Julyan
Lijoi, Antonio
Nipoti, Bernardo
COMPUTATIONAL STATISTICS & DATA ANALYSIS, 2016, 93 : 359 - 372
[32] Scale mixture models with applications to Bayesian inference
Qin, ZS
Damien, P
Walker, S
MONTE CARLO METHOD IN THE PHYSICAL SCIENCES, 2003, 690 : 394 - 395
[33] Efficient Bayesian inference for dynamic mixture models
Gerlach, R
Carter, C
Kohn, R
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2000, 95 (451) : 819 - 828
[34] Optimal transport and variational Bayesian inference
Bahraini, Alireza
Sadeghi, Saeed
INTERNATIONAL JOURNAL OF APPROXIMATE REASONING, 2023, 162
[35] Optimal approximation algorithm for Bayesian inference
Stanford Univ Sch of Medicine, Stanford, United States
Artif Intell, 1-2 (1-27):
[36] An optimal approximation algorithm for Bayesian inference
Dagum, P
Luby, M
ARTIFICIAL INTELLIGENCE, 1997, 93 (1-2) : 1 - 27
[37] Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference
Yao, Jinghan
Anthony, Quentin
Shafi, Aamir
Subramoni, Hari
Panda, Dhabaleswar K.
PROCEEDINGS 2024 IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM, IPDPS 2024, 2024, : 915 - 925
[38] A mixture-of-experts approach for gene regulatory network inference
Shao, Borong
Lavesson, Niklas
Boeva, Veselka
Shahzad, Raja Khurram
INTERNATIONAL JOURNAL OF DATA MINING AND BIOINFORMATICS, 2016, 14 (03) : 258 - 275
[39] Non-parametric Bayesian inference for continuous density hidden Markov mixture model
Bathaee, Najmeh
Sheikhzadeh, Hamid
STATISTICAL METHODOLOGY, 2016, 33 : 256 - 275
[40] Markov chain Monte Carlo simulation of a Bayesian mixture model for gene network inference
Younhee Ko
Jaebum Kim
Sandra L. Rodriguez-Zas
Genes & Genomics, 2019, 41 : 547 - 555

← 1 2 3 4 5 →