Real-Time Charging Scheduling of Automated Guided Vehicles in Cyber-Physical Smart Factories Using Feature-Based Reinforcement Learning

Chun Cheng Lin*, Kun Yang Chen, Li Tsung Hsieh

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

In smart factories, a variety of automated guided vehicles (AGVs) communicate with cyber-physical systems (CPSs) to autonomously deliver raw materials and workpieces among smart production facilities. In practice, instead of acquiring more costly AGVs to cause congestion in existing working space, most factories develop rule-based and model-based approaches to improve the AGV utilization rate and further the production efficiency. However, these charging strategies require predefined rules or models for estimating the internal information of batteries, so that they lead to huge computational costs and estimation errors. As a consequence, this work creates a Markov decision process problem for real-time charging scheduling of AGVs to fulfill uncertain AGV dispatching requests from the CPS for production lines, in which four bounds for charging heterogeneous AGVs are considered from practical experiences for increasing the AGV utilization rate. This work further improves a feature-based reinforcement learning approach, in which the state and action space can be effectively reduced through approximating the state-value function by five feature functions, including the estimated revenue for improving the utilization time, the total AGV charging cost, the cost of penalizing unfulfilled dispatching requests, the priority of charging newer batteries, and the priority of charging the batteries close to be fully charged, respectively. Experimental results show that the proposed algorithm obtains better benefits than the current practical approach, and improves the AGV utilization rate.

Original languageEnglish
Pages (from-to)4016-4026
Number of pages11
JournalIEEE Transactions on Intelligent Transportation Systems
Volume24
Issue number4
DOIs
StatePublished - 1 Apr 2023

Keywords

  • Automated guided vehicle (AGV)
  • charging scheduling
  • cyber physical system
  • reinforcement learning
  • smart manufacturing
  • state-action-reward-state-action (SARSA)

Fingerprint

Dive into the research topics of 'Real-Time Charging Scheduling of Automated Guided Vehicles in Cyber-Physical Smart Factories Using Feature-Based Reinforcement Learning'. Together they form a unique fingerprint.

Cite this