A Deep Reinforcement Learning Approach for Crowdshipping Vehicle Routing Problem

Extending the vehicle routing problem (VRP), the crowdshipping VRP (CVRP) considers crowdsourcing logistics. Crowdsourcing is flexible and convenient to reduce transportation costs and carbon emissions. However, crowdshipping requires to adapt to real-time changes such as road conditions and customer demands, which heuristic algorithms are not suitable for addressing these issues. Therefore, this study proposes a deep reinforcement learning (DRL) approach to react to real-time environmental changes to solve the CVRP. The CVRP considers a single depot and multiple transfer points to serve multiple customers, in which cargos can be delivered by either the vehicle directly, or crowdworkers after the vehicle stores cargos at transfer points. In the proposed DRL, the agent explores feasible decisions, and revises the path that it should take based on feedbacks. The cost effectiveness that affects crowdshipping includes the vehicle routing, and whether the concerned customer is suitable for crowdshipping. The experimental results show the efficiency and accuracy of the trained model for medium-sized VRPs are much higher than classical heuristic algorithms.