In this paper, we propose a fast mode decision algorithm with macroblock-adaptive rate-distortion (R-D) estimation for intra-only scalable video coding (SVC). We make use of the log-linear R-D relationship of inter-dependent layers to predict the better performer among the Intra 4×4 and Intra 8×8 prediction types at the enhancement layers. Based upon the base-layer chosen prediction type, we can further reduce the number of candidate modes. In addition, to ensure the best trade-off between complexity and coding efficiency, the Intra16×16 prediction is retained and enabled only for coding high-resolution videos with smooth image contents. Comparing to the Joint Scalable Video Model v.8 (JSVM 8), an encoder time saving from 49% to 64% depending on the encoder configurations, is achieved with negligible penalty in coding efficiency.