Optimal model inference for Bayesian mixture of experts

被引：0

作者：

Ueda, N ^{[1
]}

Ghahramani, Z ^{[1
]}

机构：

[1] NTT, Commun Sci Labs, Kyoto 6190237, Japan

来源：

NEURAL NETWORKS FOR SIGNAL PROCESSING X, VOLS 1 AND 2, PROCEEDINGS | 2000年

关键词：

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We present an algorithm for inferring the parameter and model structure of a mixture of experts model (MoE) based on the variational Bayesian (VB) framework. First, in the VB framework, we show that the model parameter and structure of a MoE can be simultaneously optimized by maximizing an objective funtion derived in this paper. Next, we present a deterministic algorithm to find the optimal number of experts of a MoE while avoiding local maxima. Our experimental results demonstrate the practical usefulness of the method.

引用

页码：145 / 154

页数：10

共 50 条

[11] Community Embeddings with Bayesian Gaussian Mixture Model and Variational Inference
Begehr, Anton I. N.
Panfilov, Peter B.
2022 IEEE 24TH CONFERENCE ON BUSINESS INFORMATICS (CBI 2022), VOL 2, 2022, : 88 - 96
[12] Classical and Bayesian Inference of a Mixture of Bivariate Exponentiated Exponential Model
Alotaibi, Refah
Khalifa, Mervat
Almetwally, Ehab M.
Ghosh, Indranil
Rezk, H.
JOURNAL OF MATHEMATICS, 2021, 2021
[13] Bayesian rough set and mixture experts model based CBR system
Han, M. (minhan@dlut.edu.cn), 1600, Northeast University (28):
[14] Merging experts' opinions: A Bayesian hierarchical model with mixture of prior distributions
Rufo, M. J.
Perez, C. J.
Martin, J.
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, 2010, 207 (01) : 284 - 289
[15] Bayesian inference and optimal design for the sparse linear model
Seeger, Matthias W.
JOURNAL OF MACHINE LEARNING RESEARCH, 2008, 9 : 759 - 813
[16] Mixture of Prompt Experts for Natural Language Inference
Zheng, Ziou
Zhu, Xiaodan
2024 IEEE CANADIAN CONFERENCE ON ELECTRICAL AND COMPUTER ENGINEERING, CCECE 2024, 2024, : 43 - 48
[17] Bayesian nonparametric mixture of experts for inverse problems
Nguyen, Trungtin
Forbes, Florence
Arbel, Julyan
Nguyen, Hien Duy
JOURNAL OF NONPARAMETRIC STATISTICS, 2024,
[18] Complex Sequential Tasks Learning with Bayesian Inference and Gaussian Mixture Model
Zhang, Huiwen
Han, Xiaoning
Zhang, Wei
Zhou, Weijia
2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO), 2018, : 1927 - 1934
[19] A Bayesian mixture of experts approach to covariate misclassification
Xia, Michelle
Hahn, P. Richard
Gustafson, Paul
CANADIAN JOURNAL OF STATISTICS-REVUE CANADIENNE DE STATISTIQUE, 2020, 48 (04): : 731 - 750
[20] Learning ambiguities using Bayesian mixture of experts
Kanaujia, Atul
Metaxas, Dimitris
ICTAI-2006: EIGHTEENTH INTERNATIONAL CONFERENCE ON TOOLS WITH ARTIFICIAL INTELLIGENCE, PROCEEDINGS, 2006, : 436 - +

← 1 2 3 4 5 →