In this paper, an energy-efficient design for massive multiple-input multiple-output (MIMO) systems is studied with the consideration of hardware impairments. The objective is to maximize the system energy efficiency of both uplink and downlink transmissions by jointly optimizing the number of antennas at the base station, the number of served users, and the transmit power. Firstly, by considering the linear distortion due to hardware impairments, the resultant channel estimation error from both distortion and noise is analyzed, following which closed-form approximations for the average achievable uplink/downlink rates are derived. Then, a massive MIMO energy efficiency optimization problem considering hardware impairments is formulated. By applying the techniques of relaxation and change of variables, an alternative optimization with build-in bisection search (AO-BS) algorithm is proposed with the quasi-concavity of the transformed objective function theoretically proved. The performance of the proposed AO-BS algorithm is validated through numerical simulations, which shows fast convergence to near-optimal solutions. Compared with existing approaches, the proposed scheme improves the system energy efficiency greatly when the hardware impairments are considered. Furthermore, the system design guideline in terms of the number of antennas, the number of served users, and the transmit power is provided.