Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units

Ikoma, Akishige; Ohsita, Yuichi; Murata, Masayuki

doi:https://doi.org/10.1109/TCC.2025.3626959

Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units

Ikoma, Akishige; Ohsita, Yuichi; Murata, Masayuki

IEEE Transactions on Cloud Computing, 2025, 13(4), 1105-1118

アクセス数:26件（2026-02-16 02:10 集計）

固定URL: https://hdl.handle.net/11094/103688

閲覧可能ファイル

ファイル	フォーマット	利用条件	サイズ	閲覧回数	利用開始日	説明	information
IEEETransCloudComput_13_4_1105	pdf	なし	4.74 MB	15	2026.01.30

Creative Commons : 表示

論文情報

ファイル出力 EndNote Basic出力 Mendeley出力

タイトル	Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units
著者	著者名 Ikoma, Akishige
	NRID 1000080432425 1000080432425 ORCID 0000-0001-9784-627X 0000-0001-9784-627X researchmap permalink read0210112 read0210112 Scopus Author ID 8216144600 8216144600 著者名 Ohsita, Yuichi
	NRID 1000080200301 1000080200301 ORCID 0000-0002-4168-2875 0000-0002-4168-2875 researchmap permalink Masayuki-Murata Masayuki-Murata Scopus Author ID 55710952700 55710952700 著者名 Murata, Masayuki
キーワード等	Deep learning (DL) model partitioning
	GPU cluster
	pipeline parallelism
	resource allocation
内容	Ikoma A., Ohsita Y., Murata M.. Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units. IEEE Transactions on Cloud Computing 13, 1105 (2025); https://doi.org/10.1109/TCC.2025.3626959.
抄録	Deep learning (DL) models have rapidly evolved, and their scales have become larger. Pipeline parallelism is used to execute a large-scale DL model. In pipeline parallelism, DL models are partitioned and allocated graphics processing units (GPUs) to execute each partition. However, to execute numerous DL services in clusters with heterogeneous GPUs, a DL model partitioning that considers a specific type and number of GPUs available and the GPUs allocated for the service is required. We propose resource aware model partitioning and allocation (RAMPA) to execute more DL services while satisfying the performance requirements. RAMPA minimizes the allocation of important resources for future DL service execution to avoid inhibiting future DL service execution. We define the resource allocation cost based on resource importance. Furthermore, we formulate the impact of model partitioning and allocated resources on service performance. We define an optimization problem to minimize resource allocation costs while satisfying service performance requirements. We evaluated the effectiveness of RAMPA by simulating the execution of DL services in clusters with heterogeneous GPUs. The results demonstrate that more services can be executed while satisfying performance requirements compared to the conventional method. RAMPA enabled efficient GPU utilization to deliver many DL services.
出版者	Institute of Electrical and Electronics Engineers Inc.
収録物名	IEEE Transactions on Cloud Computing
巻(号)	13(4)
ページ	1105-1118
刊行年月	2025-10-30
言語	英語
ハンドルURL	https://hdl.handle.net/11094/103688
eISSN	21687161
関連情報 (isIdenticalTo)	DOI (出版社版) https://doi.org/10.1109/TCC.2025.3626959
アクセス権	open access
権利情報	This article is licensed under a Creative Commons Attribution 4.0 International License.
出版タイプ	Version of Record
カテゴリ	学術雑誌論文 Journal Article

資源タイプ	学術雑誌論文
ローカル資源タイプ	学術雑誌論文
DCMI資源タイプ	text
DCTERMS.bibliographicCitation	IEEE Transactions on Cloud Computing. 2025, 13(4), p. 1105-1118
DC.title	Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units
DC.creator	Ikoma, Akishige
	Ohsita, Yuichi
	Murata, Masayuki
DC.publisher	Institute of Electrical and Electronics Engineers Inc.
DC.language' scheme='DCTERMS.RFC1766	英語
DC.type' scheme='DCTERMS.DCMIType	text
DCTERMS.issued' scheme='DCTERMS.W3CDTF	2025-10-30
DC.identifier	https://doi.org/10.1109/TCC.2025.3626959
DC.subject	Deep learning (DL) model partitioning
	GPU cluster
	pipeline parallelism
	resource allocation
DC.description	Ikoma A., Ohsita Y., Murata M.. Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units. IEEE Transactions on Cloud Computing 13, 1105 (2025); https://doi.org/10.1109/TCC.2025.3626959.
DCTERMS.abstract	Deep learning (DL) models have rapidly evolved, and their scales have become larger. Pipeline parallelism is used to execute a large-scale DL model. In pipeline parallelism, DL models are partitioned and allocated graphics processing units (GPUs) to execute each partition. However, to execute numerous DL services in clusters with heterogeneous GPUs, a DL model partitioning that considers a specific type and number of GPUs available and the GPUs allocated for the service is required. We propose resource aware model partitioning and allocation (RAMPA) to execute more DL services while satisfying the performance requirements. RAMPA minimizes the allocation of important resources for future DL service execution to avoid inhibiting future DL service execution. We define the resource allocation cost based on resource importance. Furthermore, we formulate the impact of model partitioning and allocated resources on service performance. We define an optimization problem to minimize resource allocation costs while satisfying service performance requirements. We evaluated the effectiveness of RAMPA by simulating the execution of DL services in clusters with heterogeneous GPUs. The results demonstrate that more services can be executed while satisfying performance requirements compared to the conventional method. RAMPA enabled efficient GPU utilization to deliver many DL services.
DC.rights	This article is licensed under a Creative Commons Attribution 4.0 International License.
DC.format	103688
citation_title	Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units
citation_author	Ikoma, Akishige
	Ohsita, Yuichi
	Murata, Masayuki
citation_publisher	Institute of Electrical and Electronics Engineers Inc.
citation_language	英語
citation_date	2025-10-30
citation_journal_title	IEEE Transactions on Cloud Computing
citation_volume	13
citation_issue	4
citation_firstpage	1105
citation_lastpage	1118
citation_doi	https://doi.org/10.1109/TCC.2025.3626959
citation_public_url	https://hdl.handle.net/11094/103688

Resource Aware Deep Learning Model Partitioning and Allocation for Inference Task in Clusters With Heterogeneous Graphics Processing Units

閲覧可能ファイル

論文情報

論文詳細を表示