Evaluating Dialogue Systems from the System Owners’Perspectives

Nakano, Mikio; Mukai, Hisahiro; Matsuyama, Yoichi; Komatani, Kazunori

Evaluating Dialogue Systems from the System Owners’Perspectives

Nakano, Mikio; Mukai, Hisahiro; Matsuyama, Yoichi et al.

2024

Number of Access:427（2025-08-13 22:21 Counts）

Identifier to cite or link to this item: https://hdl.handle.net/11094/95314

Link to primary information

File	Format	Terms of use	Size	Views	Date.Available	Description	information
nakano-IWSDS24_Evaluation-camera-ready-final	pdf	None	47.3 KB	111	2024.04.23

Item Information

Output File Export EndNote Basic Export Mendeley

Title	Evaluating Dialogue Systems from the System Owners’Perspectives
Creator	CreatorName Nakano, Mikio
	CreatorName Mukai, Hisahiro
	CreatorName Matsuyama, Yoichi
	NRID 1000040362579 1000040362579 ORCID 0000-0002-6052-600X 0000-0002-6052-600X researchmap permalink komatani komatani Scopus Author ID 35577813100 35577813100 CreatorName Komatani, Kazunori
Abstract	This position paper proposes to evaluate dialogue systems from the perspectives of the system owners, who decide funding the system development and operation, unlike evaluating based only on the users’ satisfaction and experiences asusually done in academic research.We suspect this difference causes a gap between conducting academic research and providing services using dialogue systems. This paper presents an initial list of evaluation criteria from the system owners’ perspectives. They consist of three groups, namely the system owners’ benefits, costs, and risks. This paper also indicates some of them have been overlooked in academic research, and that considering them will lead to novel research topics.
Conference Information	Conference Name International Workshop on Spoken Dialogue Systems Technology (IWSDS 2024) Conference Sequence 14 Conference Date March 4th to March 6th, 2024 Conference Venue Premiere Hotel Nakajima Park Sapporo Conference Place Sapporo Conference Country Japan
Date of Issued	2024-03-04
Language	English
Handle URL	https://hdl.handle.net/11094/95314
Access Rights	open access
oaire:version	Author’s Original
Category	会議発表論文 Conference Paper

Resource Type	conference paper
Local Type	Conference Paper
DCMI Type	Text
DC.title	Evaluating Dialogue Systems from the System Owners’Perspectives
DC.creator	Nakano, Mikio
	Mukai, Hisahiro
	Matsuyama, Yoichi
	Komatani, Kazunori
DC.language' scheme='DCTERMS.RFC1766	English
DC.type' scheme='DCTERMS.DCMIType	Text
DCTERMS.issued' scheme='DCTERMS.W3CDTF	2024-03-04
DCTERMS.abstract	This position paper proposes to evaluate dialogue systems from the perspectives of the system owners, who decide funding the system development and operation, unlike evaluating based only on the users’ satisfaction and experiences asusually done in academic research.We suspect this difference causes a gap between conducting academic research and providing services using dialogue systems. This paper presents an initial list of evaluation criteria from the system owners’ perspectives. They consist of three groups, namely the system owners’ benefits, costs, and risks. This paper also indicates some of them have been overlooked in academic research, and that considering them will lead to novel research topics.
DC.format	95314
citation_title	Evaluating Dialogue Systems from the System Owners’Perspectives
citation_author	Nakano, Mikio
	Mukai, Hisahiro
	Matsuyama, Yoichi
	Komatani, Kazunori
citation_language	English
citation_date	2024-03-04
citation_public_url	https://hdl.handle.net/11094/95314

Evaluating Dialogue Systems from the System Owners’Perspectives

Link to primary information

Item Information

Show details