In queueing systems, achieving a reasonable balance between system performance and service quality requires strict control over arrivals. F-policy is a control policy that forbids the entry of new customers when the system is full, only allowing entry when the size of the system is reduced to a predetermined value F. Enhancing server efficiency is another effective approach to improve service quality. This paper considers an F-policy GI/M/1 queue with alternating service rates, which can be upgraded in accordance with the state of the system. Steady-state analysis is performed using a recursive method in conjunction with the supplementary variable technique. Steady-state probability is used to evaluate critical system characteristics and perform sensitivity analysis. A bi-objective optimization scheme is then formulated using the NSGA-II and two multi-objective evolutionary algorithms (MOEAs) in accordance with the performance measures to minimize the expected cost function per unit time as well as the expected waiting time. Numerical results demonstrate that the marginal utility of increasing the budget decreases when the expected waiting time is low.