This paper proposes a Markov chain based hierarchical method to efficiently analyze the power delivery network. After the network being partitioned into several subnetworks, each subnetwork is transformed into a local Markov chain. Then, the connective relations between all subnetworks are modeled as a global Markov chain. Finally, those local and the global Markov chains are incorporated to build a hierarchical bipartite Markov chain engine to analyze the power delivery network. The experimental results not only demonstrate the accuracy of proposed method compared with a very accurate time domain solver , but also show its significant runtime improvement, over 200 times faster than the InductWise  and over 10 times faster than the IEKS method , and less memory usage.