TY - JOUR
T1 - The Optimal Machine Learning-Based Missing Data Imputation for the Cox Proportional Hazard Model
AU - Guo, Chao Yu
AU - Yang, Ying Chen
AU - Chen, Yi Hau
N1 - Publisher Copyright:
© Copyright © 2021 Guo, Yang and Chen.
PY - 2021/7/5
Y1 - 2021/7/5
N2 - An adequate imputation of missing data would significantly preserve the statistical power and avoid erroneous conclusions. In the era of big data, machine learning is a great tool to infer the missing values. The root means square error (RMSE) and the proportion of falsely classified entries (PFC) are two standard statistics to evaluate imputation accuracy. However, the Cox proportional hazards model using various types requires deliberate study, and the validity under different missing mechanisms is unknown. In this research, we propose supervised and unsupervised imputations and examine four machine learning-based imputation strategies. We conducted a simulation study under various scenarios with several parameters, such as sample size, missing rate, and different missing mechanisms. The results revealed the type-I errors according to different imputation techniques in the survival data. The simulation results show that the non-parametric “missForest” based on the unsupervised imputation is the only robust method without inflated type-I errors under all missing mechanisms. In contrast, other methods are not valid to test when the missing pattern is informative. Statistical analysis, which is improperly conducted, with missing data may lead to erroneous conclusions. This research provides a clear guideline for a valid survival analysis using the Cox proportional hazard model with machine learning-based imputations.
AB - An adequate imputation of missing data would significantly preserve the statistical power and avoid erroneous conclusions. In the era of big data, machine learning is a great tool to infer the missing values. The root means square error (RMSE) and the proportion of falsely classified entries (PFC) are two standard statistics to evaluate imputation accuracy. However, the Cox proportional hazards model using various types requires deliberate study, and the validity under different missing mechanisms is unknown. In this research, we propose supervised and unsupervised imputations and examine four machine learning-based imputation strategies. We conducted a simulation study under various scenarios with several parameters, such as sample size, missing rate, and different missing mechanisms. The results revealed the type-I errors according to different imputation techniques in the survival data. The simulation results show that the non-parametric “missForest” based on the unsupervised imputation is the only robust method without inflated type-I errors under all missing mechanisms. In contrast, other methods are not valid to test when the missing pattern is informative. Statistical analysis, which is improperly conducted, with missing data may lead to erroneous conclusions. This research provides a clear guideline for a valid survival analysis using the Cox proportional hazard model with machine learning-based imputations.
KW - cox proportional hazard model
KW - k-nearest neighbors imputation
KW - machine learning
KW - random forest imputation
KW - survival data simulation
UR - http://www.scopus.com/inward/record.url?scp=85110619131&partnerID=8YFLogxK
U2 - 10.3389/fpubh.2021.680054
DO - 10.3389/fpubh.2021.680054
M3 - Article
C2 - 34291028
AN - SCOPUS:85110619131
SN - 2296-2565
VL - 9
JO - Frontiers in Public Health
JF - Frontiers in Public Health
M1 - 680054
ER -