Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study

Yamamoto, Takuto; Akahoshi, Hirosato; Kitazawa, Shigeru

doi:https://doi.org/10.1016/j.neunet.2025.107595

Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study

Yamamoto, Takuto; Akahoshi, Hirosato; Kitazawa, Shigeru

Neural Networks, 2025, 189, 107595

アクセス数:3件（2025-07-28 07:56 集計）

固定URL: https://hdl.handle.net/11094/102209

閲覧可能ファイル

ファイル	フォーマット	利用条件	サイズ	閲覧回数	利用開始日	説明	information
NeuralNetw_189	pdf	なし	5.98 MB	4	2025.07.10

Creative Commons : 表示

論文情報

ファイル出力 EndNote Basic出力 Mendeley出力

タイトル	Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study
著者	著者名 Yamamoto, Takuto
	著者名 Akahoshi, Hirosato
	NRID 1000000251231 1000000251231 researchmap permalink shigerukitazawa shigerukitazawa Scopus Author ID 36865431600 36865431600 著者名 Kitazawa, Shigeru
キーワード等	Attention
	DINO
	Eye-tracking
	Figure ground separation
	Self-supervised learning
	Vision transformer
内容	Yamamoto T., Akahoshi H., Kitazawa S.. Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study. Neural Networks 189, 107595 (2025); https://doi.org/10.1016/j.neunet.2025.107595.
抄録	Visual attention models aim to predict human gaze behavior, yet traditional saliency models and deep gaze prediction networks face limitations. Saliency models rely on handcrafted low-level visual features, often failing to capture human gaze dynamics, while deep learning-based gaze prediction models lack biological plausibility. Vision Transformers (ViTs), which use self-attention mechanisms, offer an alternative, but when trained with conventional supervised learning, their attention patterns tend to be dispersed and unfocused. This study demonstrates that ViTs trained with self-supervised DINO (self-Distillation with NO labels) develop structured attention that closely aligns with human gaze behavior when viewing videos. Our analysis reveals that self-attention heads in later layers of DINO-trained ViTs autonomously differentiate into three distinct clusters: (1) G1 heads (20%), which focus on key points within figures (e.g., the eyes of the main character) and resemble human gaze; (2) G2 heads (60%), which distribute attention over entire figures with sharp contours (e.g., the bodies of all characters); and (3) G3 heads (20%), which primarily attend to the background. These findings provide insights into how human overt attention and figure-ground segregation emerge in visual perception. Our work suggests that self-supervised learning enables ViTs to develop attention mechanisms that are more aligned with biological vision than traditional supervised training.
出版者	Elsevier Ltd
収録物名	Neural Networks
巻(号)	189
ページ	107595
刊行年月	2025-05-26
言語	英語
ハンドルURL	https://hdl.handle.net/11094/102209
PISSN	08936080
eISSN	18792782
関連情報 (isIdenticalTo)	DOI (出版社版) https://doi.org/10.1016/j.neunet.2025.107595
アクセス権	open access
権利情報	This article is licensed under a Creative Commons Attribution 4.0 International License.
出版タイプ	Version of Record
カテゴリ	学術雑誌論文 Journal Article

資源タイプ	学術雑誌論文
ローカル資源タイプ	学術雑誌論文
DCMI資源タイプ	text
DCTERMS.bibliographicCitation	Neural Networks. 2025, 189, p. 107595
DC.title	Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study
DC.creator	Yamamoto, Takuto
	Akahoshi, Hirosato
	Kitazawa, Shigeru
DC.publisher	Elsevier Ltd
DC.language' scheme='DCTERMS.RFC1766	英語
DC.type' scheme='DCTERMS.DCMIType	text
DCTERMS.issued' scheme='DCTERMS.W3CDTF	2025-05-26
DC.identifier	https://doi.org/10.1016/j.neunet.2025.107595
DC.subject	Attention
	DINO
	Eye-tracking
	Figure ground separation
	Self-supervised learning
	Vision transformer
DC.description	Yamamoto T., Akahoshi H., Kitazawa S.. Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study. Neural Networks 189, 107595 (2025); https://doi.org/10.1016/j.neunet.2025.107595.
DCTERMS.abstract	Visual attention models aim to predict human gaze behavior, yet traditional saliency models and deep gaze prediction networks face limitations. Saliency models rely on handcrafted low-level visual features, often failing to capture human gaze dynamics, while deep learning-based gaze prediction models lack biological plausibility. Vision Transformers (ViTs), which use self-attention mechanisms, offer an alternative, but when trained with conventional supervised learning, their attention patterns tend to be dispersed and unfocused. This study demonstrates that ViTs trained with self-supervised DINO (self-Distillation with NO labels) develop structured attention that closely aligns with human gaze behavior when viewing videos. Our analysis reveals that self-attention heads in later layers of DINO-trained ViTs autonomously differentiate into three distinct clusters: (1) G1 heads (20%), which focus on key points within figures (e.g., the eyes of the main character) and resemble human gaze; (2) G2 heads (60%), which distribute attention over entire figures with sharp contours (e.g., the bodies of all characters); and (3) G3 heads (20%), which primarily attend to the background. These findings provide insights into how human overt attention and figure-ground segregation emerge in visual perception. Our work suggests that self-supervised learning enables ViTs to develop attention mechanisms that are more aligned with biological vision than traditional supervised training.
DC.rights	This article is licensed under a Creative Commons Attribution 4.0 International License.
DC.format	102209
citation_title	Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study
citation_author	Yamamoto, Takuto
	Akahoshi, Hirosato
	Kitazawa, Shigeru
citation_publisher	Elsevier Ltd
citation_language	英語
citation_date	2025-05-26
citation_journal_title	Neural Networks
citation_volume	189
citation_firstpage	107595
citation_doi	https://doi.org/10.1016/j.neunet.2025.107595
citation_public_url	https://hdl.handle.net/11094/102209

Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study

閲覧可能ファイル

論文情報

論文詳細を表示