学術雑誌論文

学術雑誌論文

text

IEEE Robotics and Automation Letters. 2025, 10(10), p. 10122-10129

TARAD: Task-Aware Robot Affordance-Centric Diffusion Policy Learned From LLM-Generated Demonstrations

Institute of Electrical and Electronics Engineers Inc.

英語

text

2025-08-14

https://doi.org/10.1109/LRA.2025.3598998

Hu S., Nagai T., Horii T.. TARAD: Task-Aware Robot Affordance-Centric Diffusion Policy Learned From LLM-Generated Demonstrations. IEEE Robotics and Automation Letters 10, 10122 (2025); https://doi.org/10.1109/LRA.2025.3598998.

In open-ended task settings, the ability of a robot to execute diverse tasks accurately by following language instructions is critical. Methods based on traditional imitation learning typically depend on extensive expert demonstrations and often struggle to generalize in the case of unseen scenarios or tasks. Recently, approaches leveraging large foundational models have demonstrated improved generalization by enhancing task comprehension in novel scenarios based on the intrinsic world knowledge embedded in these models. However, these methods rely on predefined motion primitives and lack a detailed understanding of the environment, which is essential for successful execution. Herein we introduce Task-Aware Robot Affordance-Centric Diffusion Policy (TARAD), a novel framework for robot manipulation. TARAD leverages large language models and vision-language models to perform high-level planning from natural language instructions and extract affordance information from the robot's observations. A heuristic motion planner is employed for low-level motion planning, enabling zero-shot trajectory synthesis and the fully automatic generation of a dataset with language labels and affordances. By incorporating affordances into the observation space, our approach integrates the intrinsic commonsense and reasoning capabilities of foundation models into imitation learning, enabling the training of an affordance-centric, multi-task three-dimensional (3D) diffusion policy. Empirical evaluations in both the RLBench simulated environments and real-world experiments with UR5e demonstrate that TARAD effectively combines the precise control of imitation learning with the strong generalization capabilities of foundation models, all without relying on expert demonstrations or predefined motion primitives.

This article is licensed under a Creative Commons Attribution 4.0 International License.

103259

TARAD: Task-Aware Robot Affordance-Centric Diffusion Policy Learned From LLM-Generated Demonstrations

Institute of Electrical and Electronics Engineers Inc.

英語

2025-08-14

IEEE Robotics and Automation Letters

10

10

10122

10129

https://doi.org/10.1109/LRA.2025.3598998

https://hdl.handle.net/11094/103259