Deep learning development nowadays has attracted a lot of attention because of its effectiveness and good performance. The performance of deep learning in medical images analysis already can compete with medical image experts. However, there are experts that still believe deep learning only efficient for the big datasets, because of deep learning performance in small datasets still not satisfying enough. In this study, it is aimed to build a deep learning model for image classification that can achieve high accuracy using chest X-ray images with a relatively small dataset. We classify chest X-ray into a binary classification which is a normal image and image with abnormalities. We built and experimented our model using the public dataset of Shenzen Hospital dataset. We also use a different type of input based on different images preprocessing so that the model can perform accurate classification. Based on the result, pre-trained CheXNet with a newly trained fully connected network on the cropped dataset can achieve the accuracy 0.8761, the sensitivity 0.8909, and the specificity 0.8621. The performance of the model also influenced by the certain region inside the images, such as other regions outside the lung region and black colored region outside the body region.