Real-Time Charging Scheduling of Automated Guided Vehicles in Cyber-Physical Smart Factories Using Feature-Based Reinforcement Learning

Chun Cheng Lin*, Kun Yang Chen, Li Tsung Hsieh

*此作品的通信作者

研究成果: Article同行評審

5 引文 斯高帕斯(Scopus)

摘要

In smart factories, a variety of automated guided vehicles (AGVs) communicate with cyber-physical systems (CPSs) to autonomously deliver raw materials and workpieces among smart production facilities. In practice, instead of acquiring more costly AGVs to cause congestion in existing working space, most factories develop rule-based and model-based approaches to improve the AGV utilization rate and further the production efficiency. However, these charging strategies require predefined rules or models for estimating the internal information of batteries, so that they lead to huge computational costs and estimation errors. As a consequence, this work creates a Markov decision process problem for real-time charging scheduling of AGVs to fulfill uncertain AGV dispatching requests from the CPS for production lines, in which four bounds for charging heterogeneous AGVs are considered from practical experiences for increasing the AGV utilization rate. This work further improves a feature-based reinforcement learning approach, in which the state and action space can be effectively reduced through approximating the state-value function by five feature functions, including the estimated revenue for improving the utilization time, the total AGV charging cost, the cost of penalizing unfulfilled dispatching requests, the priority of charging newer batteries, and the priority of charging the batteries close to be fully charged, respectively. Experimental results show that the proposed algorithm obtains better benefits than the current practical approach, and improves the AGV utilization rate.

原文English
頁(從 - 到)4016-4026
頁數11
期刊IEEE Transactions on Intelligent Transportation Systems
24
發行號4
DOIs
出版狀態Published - 1 4月 2023

指紋

深入研究「Real-Time Charging Scheduling of Automated Guided Vehicles in Cyber-Physical Smart Factories Using Feature-Based Reinforcement Learning」主題。共同形成了獨特的指紋。

引用此