TY - JOUR
T1 - A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting
AU - Li, Guanyao
AU - Zhong, Shuhan
AU - Deng, Xingdong
AU - Xiang, Letian
AU - Gary Chan, S. H.
AU - Li, Ruiyuan
AU - Liu, Yang
AU - Zhang, Ming
AU - Hung, Chih Chieh
AU - Peng, Wen Chih
N1 - Publisher Copyright:
© 1989-2012 IEEE.
PY - 2023/11/1
Y1 - 2023/11/1
N2 - We study the forecasting problem for traffic with dynamic, possibly periodical, and joint spatial-temporal dependency between regions. Given the aggregated inflow and outflow traffic of regions in a city from time slots 0 to t-1, we predict the traffic at time t for any region. Prior arts in the area often considered the spatial and temporal dependencies in a decoupled manner, or were rather computationally intensive in training with a large number of hyper-parameters which needed tuning. We propose ST-TIS, a novel, lightweight and accurate Spatial-Temporal Transformer with information fusion and region sampling for traffic forecasting. ST-TIS extends the canonical Transformer with information fusion and region sampling. The information fusion module captures the complex spatial-temporal dependency between regions. The region sampling module is to improve the efficiency and prediction accuracy, cutting the computation complexity for dependency learning from O(n2) to O(nn), where n is the number of regions. With far fewer parameters than state-of-the-art deep learning models, ST-TIS's offline training is significantly faster in terms of tuning and computation (with a reduction of up to 90% on training time and network parameters). Notwithstanding such training efficiency, extensive experiments show that ST-TIS is substantially more accurate in online prediction than state-of-the-art approaches (with an average improvement of 9.5% on RMSE, and 12.4% on MAPE compared to STDN and DSAN).
AB - We study the forecasting problem for traffic with dynamic, possibly periodical, and joint spatial-temporal dependency between regions. Given the aggregated inflow and outflow traffic of regions in a city from time slots 0 to t-1, we predict the traffic at time t for any region. Prior arts in the area often considered the spatial and temporal dependencies in a decoupled manner, or were rather computationally intensive in training with a large number of hyper-parameters which needed tuning. We propose ST-TIS, a novel, lightweight and accurate Spatial-Temporal Transformer with information fusion and region sampling for traffic forecasting. ST-TIS extends the canonical Transformer with information fusion and region sampling. The information fusion module captures the complex spatial-temporal dependency between regions. The region sampling module is to improve the efficiency and prediction accuracy, cutting the computation complexity for dependency learning from O(n2) to O(nn), where n is the number of regions. With far fewer parameters than state-of-the-art deep learning models, ST-TIS's offline training is significantly faster in terms of tuning and computation (with a reduction of up to 90% on training time and network parameters). Notwithstanding such training efficiency, extensive experiments show that ST-TIS is substantially more accurate in online prediction than state-of-the-art approaches (with an average improvement of 9.5% on RMSE, and 12.4% on MAPE compared to STDN and DSAN).
KW - Efficient Transformer
KW - joint spatial-temporal dependency
KW - region sampling
KW - spatial-temporal data mining
KW - spatial-temporal forecasting
UR - http://www.scopus.com/inward/record.url?scp=85147217804&partnerID=8YFLogxK
U2 - 10.1109/TKDE.2022.3233086
DO - 10.1109/TKDE.2022.3233086
M3 - Article
AN - SCOPUS:85147217804
SN - 1041-4347
VL - 35
SP - 10967
EP - 10980
JO - IEEE Transactions on Knowledge and Data Engineering
JF - IEEE Transactions on Knowledge and Data Engineering
IS - 11
ER -