Entropic Fictitious Play for Mean Field Optimization Problem - Institut Polytechnique de Paris Access content directly
Journal Articles Journal of Machine Learning Research Year : 2023

Entropic Fictitious Play for Mean Field Optimization Problem

Abstract

We study two-layer neural networks in the mean field limit, where the number of neurons tends to infinity. In this regime, the optimization over the neuron parameters becomes the optimization over the probability measures, and by adding an entropic regularizer, the minimizer of the problem is identified as a fixed point. We propose a novel training algorithm named entropic fictitious play, inspired by the classical fictitious play in game theory for learning Nash equilibriums, to recover this fixed point, and the algorithm exhibits a two-loop iteration structure. Exponential convergence is proved in this paper and we also verify our theoretical results by simple numerical examples.
Fichier principal
Vignette du fichier
Chen et al. - 2023 - Entropic Fictitious Play for Mean Field Optimizati.pdf (1006.82 Ko) Télécharger le fichier
Origin : Publisher files allowed on an open archive

Dates and versions

hal-04508395 , version 1 (18-03-2024)

Identifiers

Cite

Fan Chen, Zhenjie Ren, Songbo Wang. Entropic Fictitious Play for Mean Field Optimization Problem. Journal of Machine Learning Research, 2023, 24, pp.no. 211. ⟨hal-04508395⟩
14 View
1 Download

Altmetric

Share

Gmail Facebook X LinkedIn More