Distributed Flexible Job Shop Scheduling through Deploying Fog and Edge Computing in Smart Factories Using Dual Deep Q Networks

Flexible job shop scheduling (FJSP) has garnered enormous attention within the realm of smart manufacturing, where, beyond job sequencing, the selection of machines holds considerable importance. As smart factories progress with the Internet of things (IoT) and cyber-physical systems (CPS), scheduling methodologies are advancing towards intelligent decentralization. However, with the expansion of factories, conventional cloud computing struggles to manage the substantial influx of data. To tackle this issue, this work incorporates a fog computing and edge computing framework into the distributed FJSP workstations. In this framework, the workstations each of which consists of multiple machines are categorized based on the different nature of the accommodated machines, and operate independently to reduce unnecessary information transmission, in which each machine is equipped with edge computing capacity. The fusion of fog computing and edge computing allows for the offloading of computational tasks from cloud computing, effectively reducing latency. While previous solutions for FJSP have predominantly relied on linear programming or metaheuristic algorithms, this work proposed a novel distributed approach based on a dual deep Q networks (dual DQN) architecture, integrating deep learning (DL) with reinforcement learning (RL). Within the cloud center, the initial neural network determines the machine selection rules for fog computing, while the secondary neural network decides the job dispatching rules for edge computing devices. Edge computing devices execute the schedule and provide feedback to the cloud, which refines the results through an iterative training process, so that to minimize the makespan. The experimental findings indicate that employing dual DQNs outperforms the methods of utilizing only one single machine selection rule.