A growing body of evidence demonstrates obesity-induced insulin resistance is associated with the development of metabolic diseases. This study was designed to investigate ethyl acetate fraction of Chinese olive fruit extract (CO-EtOAc)-mediated attenuation of obesity and hyperglycemia in a mouse model. About 60% HFD-fed mice were treated intragastrically with CO-EtOAc for last 6 weeks, and body weight, blood biochemical parameters as well as hepatic inflammation response were investigated. Our results showed that CO-EtOAc treatment significantly reduced the formation of hepatic lipid droplets, body weight gain, blood glucose, and improved serum biochemical parameters in HFD-induced obese and insulin resistant mice. We further explored the molecular mechanism underlying the blood glucose modulating effect of CO-EtOAc using L6 myotubes model. We conclude that CO-EtOAc effectively increases the glycogen content and glucose uptake by stimulating the membrane translocation of glucose transporter 4. In addition, CO-EtOAc depolarizes the mitochondrial membrane and decreases the mitochondrial oxygen consumption, which may result in AMPK activation and the consequent mitochondrial fission. This study shows that CO-EtOAc prevents the development of obesity in mice fed with HFD and is also capable of stimulating glucose uptake. The possible mechanism might be due to the effects of CO-EtOAc on activation of AMPK and promotion of mitochondrial fission.