The inevitable time delay in fault detection, i.e. the time between the occurrence and detection of the fault, can cause stability problems in feedback control systems. The state of the art technologies, however, provide neither an analysis tool to evaluate the adverse influence of the detection delay on the closed-loop system's stability and performance nor guidelines for designers to overcome the instability introduced by the detection delay. In this paper we conduct a theoretical analysis of timely fault detection problem which concerns the detection of faults before the closed-loop system's performance deteriorates to an unacceptable extent. We first give a formal definition of the "timely" fault detection problem. Then we derive the upper and lower bounds of thresholds of fault detectors as well as a set of detectable faults. If the threshold is selected within the bounds, faults belonging to the detectable set can be detected in a timely manner and an acceptable level of performance is guaranteed.