Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study

Yamamoto, Takuto; Akahoshi, Hirosato; Kitazawa, Shigeru

doi:https://doi.org/10.1016/j.neunet.2025.107595

Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study

Yamamoto, Takuto; Akahoshi, Hirosato; Kitazawa, Shigeru

Neural Networks, 2025, 189, 107595

Number of Access:5（2025-08-16 05:20 Counts）

Identifier to cite or link to this item: https://hdl.handle.net/11094/102209

Link to primary information

File	Format	Terms of use	Size	Views	Date.Available	Description	information
NeuralNetw_189	pdf	None	5.98 MB	8	2025.07.10

Creative Commons : Attribution

Item Information

Output File Export EndNote Basic Export Mendeley

Title	Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study
Creator	CreatorName Yamamoto, Takuto
	CreatorName Akahoshi, Hirosato
	NRID 1000000251231 1000000251231 researchmap permalink shigerukitazawa shigerukitazawa Scopus Author ID 36865431600 36865431600 CreatorName Kitazawa, Shigeru
Subject	Attention
	DINO
	Eye-tracking
	Figure ground separation
	Self-supervised learning
	Vision transformer
Description	Yamamoto T., Akahoshi H., Kitazawa S.. Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study. Neural Networks 189, 107595 (2025); https://doi.org/10.1016/j.neunet.2025.107595.
Abstract	Visual attention models aim to predict human gaze behavior, yet traditional saliency models and deep gaze prediction networks face limitations. Saliency models rely on handcrafted low-level visual features, often failing to capture human gaze dynamics, while deep learning-based gaze prediction models lack biological plausibility. Vision Transformers (ViTs), which use self-attention mechanisms, offer an alternative, but when trained with conventional supervised learning, their attention patterns tend to be dispersed and unfocused. This study demonstrates that ViTs trained with self-supervised DINO (self-Distillation with NO labels) develop structured attention that closely aligns with human gaze behavior when viewing videos. Our analysis reveals that self-attention heads in later layers of DINO-trained ViTs autonomously differentiate into three distinct clusters: (1) G1 heads (20%), which focus on key points within figures (e.g., the eyes of the main character) and resemble human gaze; (2) G2 heads (60%), which distribute attention over entire figures with sharp contours (e.g., the bodies of all characters); and (3) G3 heads (20%), which primarily attend to the background. These findings provide insights into how human overt attention and figure-ground segregation emerge in visual perception. Our work suggests that self-supervised learning enables ViTs to develop attention mechanisms that are more aligned with biological vision than traditional supervised training.
Publisher	Elsevier Ltd
Source Title	Neural Networks
Volume (Issue)	189
Page	107595
Date of Issued	2025-05-26
Language	English
Handle URL	https://hdl.handle.net/11094/102209
PISSN	08936080
eISSN	18792782
Relation.isIdenticalTo	DOI (Publisher Version) https://doi.org/10.1016/j.neunet.2025.107595
Access Rights	open access
CopyRight	This article is licensed under a Creative Commons Attribution 4.0 International License.
oaire:version	Version of Record
Category	学術雑誌論文 Journal Article

Resource Type	journal article
Local Type	Journal Article
DCMI Type	Text
DCTERMS.bibliographicCitation	Neural Networks. 2025, 189, p. 107595
DC.title	Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study
DC.creator	Yamamoto, Takuto
	Akahoshi, Hirosato
	Kitazawa, Shigeru
DC.publisher	Elsevier Ltd
DC.language' scheme='DCTERMS.RFC1766	English
DC.type' scheme='DCTERMS.DCMIType	Text
DCTERMS.issued' scheme='DCTERMS.W3CDTF	2025-05-26
DC.identifier	https://doi.org/10.1016/j.neunet.2025.107595
DC.subject	Attention
	DINO
	Eye-tracking
	Figure ground separation
	Self-supervised learning
	Vision transformer
DC.description	Yamamoto T., Akahoshi H., Kitazawa S.. Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study. Neural Networks 189, 107595 (2025); https://doi.org/10.1016/j.neunet.2025.107595.
DCTERMS.abstract	Visual attention models aim to predict human gaze behavior, yet traditional saliency models and deep gaze prediction networks face limitations. Saliency models rely on handcrafted low-level visual features, often failing to capture human gaze dynamics, while deep learning-based gaze prediction models lack biological plausibility. Vision Transformers (ViTs), which use self-attention mechanisms, offer an alternative, but when trained with conventional supervised learning, their attention patterns tend to be dispersed and unfocused. This study demonstrates that ViTs trained with self-supervised DINO (self-Distillation with NO labels) develop structured attention that closely aligns with human gaze behavior when viewing videos. Our analysis reveals that self-attention heads in later layers of DINO-trained ViTs autonomously differentiate into three distinct clusters: (1) G1 heads (20%), which focus on key points within figures (e.g., the eyes of the main character) and resemble human gaze; (2) G2 heads (60%), which distribute attention over entire figures with sharp contours (e.g., the bodies of all characters); and (3) G3 heads (20%), which primarily attend to the background. These findings provide insights into how human overt attention and figure-ground segregation emerge in visual perception. Our work suggests that self-supervised learning enables ViTs to develop attention mechanisms that are more aligned with biological vision than traditional supervised training.
DC.rights	This article is licensed under a Creative Commons Attribution 4.0 International License.
DC.format	102209
citation_title	Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study
citation_author	Yamamoto, Takuto
	Akahoshi, Hirosato
	Kitazawa, Shigeru
citation_publisher	Elsevier Ltd
citation_language	English
citation_date	2025-05-26
citation_journal_title	Neural Networks
citation_volume	189
citation_firstpage	107595
citation_doi	https://doi.org/10.1016/j.neunet.2025.107595
citation_public_url	https://hdl.handle.net/11094/102209

Emergence of human-like attention and distinct head clusters in self-supervised vision transformers: A comparative eye-tracking study

Link to primary information

Item Information

Show details