A phylogenetic tree is used to present the evolutionary relationships among the interesting biological species based on the similarities in their genetic sequences. The UPGMA is one of the popular algorithms to construct a phylogenetic tree according to the distance matrix created by the pairwise distances among taxa. To solve the performance issue of the UPGMA, the implementation of the UPGMA method on a single GPU has been proposed. However, it is not capable of handling the large taxa set. This work describes a novel parallel UPGMA approach on multiple GPUs that is able to build a tree from extremely large datasets. The experimental results show that the proposed approach with 4 NVIDIA GTX 980 achieves an approximately × fold speedup over the implementation of UPGMA on CPU and GPU, respectively.