Learning under singularity: an information criterion improving WBIC and sBIC

Liu, Lirui; Suzuki, Joe

doi:https://doi.org/10.1007/s42081-024-00262-1

Learning under singularity: an information criterion improving WBIC and sBIC

Liu, Lirui; Suzuki, Joe

Japanese Journal of Statistics and Data Science, 2024

Number of Access:159（2025-08-01 19:36 Counts）

Identifier to cite or link to this item: https://hdl.handle.net/11094/98103

Link to primary information

File	Format	Terms of use	Size	Views	Date.Available	Description	information
JpnJStatDataSci_s42081-024-00262-1	pdf	None	306 KB	45	2024.09.19

Creative Commons : Attribution

Item Information

Output File Export EndNote Basic Export Mendeley

Title	Learning under singularity: an information criterion improving WBIC and sBIC
Creator	CreatorName Liu, Lirui
Creator	NRID 1000050216397 1000050216397 ORCID 0000-0002-3195-9922 0000-0002-3195-9922 researchmap permalink read0185200 read0185200 Scopus Author ID 7401633246 7401633246 CreatorName Suzuki, Joe
Subject	Information criterion
	Learning under singularity
	Model selection
	Regularity
	Singular Bayesian information criterion
	Widely applicable Bayes information criterion
Description	The version of record of this article, first published in Japanese Journal of Statistics and Data Science, is available online at Publisher’s website: https://doi.org/10.1007/s42081-024-00262-1.
Abstract	We introduce a novel information criterion, termed learning under singularity (LS), designed to enhance the functionality of the widely applicable Bayes information criterion (WBIC) and the singular Bayesian information criterion (sBIC). LS is effective without regularity constraints and demonstrates stability. Watanabe defined a statistical model or a learning machine as regular if the mapping from a parameter to a probability distribution is one-to-one and its Fisher information matrix is positive definite. In contrast, models not meeting these conditions are termed singular. Over the past decade, several information criteria for singular cases have been proposed, including WBIC and sBIC. WBIC is applicable in non-regular scenarios but faces challenges with large sample sizes and redundant estimation of known learning coefficients. Conversely, sBIC is limited in its broader application due to its dependence on maximum likelihood estimates. LS addresses these limitations by enhancing the utility of both WBIC and sBIC. It incorporates the empirical loss from the widely applicable information criterion (WAIC) to represent the goodness of fit to the statistical model, along with a penalty term similar to that of sBIC. This approach offers a flexible and robust method for model selection, free from regularity constraints.
Publisher	Springer
Source Title	Japanese Journal of Statistics and Data Science
Date of Issued	2024-08-21
Language	English
Handle URL	https://hdl.handle.net/11094/98103
eISSN	25208764
Relation.isIdenticalTo	DOI (Publisher Version) https://doi.org/10.1007/s42081-024-00262-1
Access Rights	open access
CopyRight	This article is licensed under a Creative Commons Attribution 4.0 International License.
oaire:version	Version of Record
Category	学術雑誌論文 Journal Article

Resource Type	journal article
Local Type	Journal Article
DCTERMS.bibliographicCitation	Japanese Journal of Statistics and Data Science. 2024
DC.title	Learning under singularity: an information criterion improving WBIC and sBIC
DC.creator	Liu, Lirui
DC.creator	Suzuki, Joe
DC.publisher	Springer
DC.language' scheme='DCTERMS.RFC1766	English
DCTERMS.issued' scheme='DCTERMS.W3CDTF	2024-08-21
DC.identifier	https://doi.org/10.1007/s42081-024-00262-1
DC.subject	Information criterion
	Learning under singularity
	Model selection
	Regularity
	Singular Bayesian information criterion
	Widely applicable Bayes information criterion
DC.description	The version of record of this article, first published in Japanese Journal of Statistics and Data Science, is available online at Publisher’s website: https://doi.org/10.1007/s42081-024-00262-1.
DCTERMS.abstract	We introduce a novel information criterion, termed learning under singularity (LS), designed to enhance the functionality of the widely applicable Bayes information criterion (WBIC) and the singular Bayesian information criterion (sBIC). LS is effective without regularity constraints and demonstrates stability. Watanabe defined a statistical model or a learning machine as regular if the mapping from a parameter to a probability distribution is one-to-one and its Fisher information matrix is positive definite. In contrast, models not meeting these conditions are termed singular. Over the past decade, several information criteria for singular cases have been proposed, including WBIC and sBIC. WBIC is applicable in non-regular scenarios but faces challenges with large sample sizes and redundant estimation of known learning coefficients. Conversely, sBIC is limited in its broader application due to its dependence on maximum likelihood estimates. LS addresses these limitations by enhancing the utility of both WBIC and sBIC. It incorporates the empirical loss from the widely applicable information criterion (WAIC) to represent the goodness of fit to the statistical model, along with a penalty term similar to that of sBIC. This approach offers a flexible and robust method for model selection, free from regularity constraints.
DC.rights	This article is licensed under a Creative Commons Attribution 4.0 International License.
DC.format	98103
citation_title	Learning under singularity: an information criterion improving WBIC and sBIC
citation_author	Liu, Lirui
citation_author	Suzuki, Joe
citation_publisher	Springer
citation_language	English
citation_date	2024-08-21
citation_journal_title	Japanese Journal of Statistics and Data Science
citation_doi	https://doi.org/10.1007/s42081-024-00262-1
citation_public_url	https://hdl.handle.net/11094/98103

Learning under singularity: an information criterion improving WBIC and sBIC

Link to primary information

Item Information

Show details