Conductive bridge random access memories (CBRAMs) are one of the promising nonvolatile memories for the next generation of technology. The resistive switching behavior of CBRAMs relies on the formation and the rupture of the filaments in the switching layer by the cation injection from the active metal electrode. However, CBRAMs are suffered from an overinjection behavior of the cation, deteriorating the switching performance. In this regard, we developed the vertical alignment of the MoSe2 (v-MoSe2) layer by a plasma-assisted chemical vapor reaction to prevent the overmigration of the Cu species. With the insertion of the v-MoSe2 layer, the Cu/v-MoSe2/TiO2/Pt device shows a Forming-free characteristic and low-operation voltage range, avoiding the overinjection of the Cu species and hard breakdown of the device. Besides, the identical pulse scheme was applied to characterize the synaptic function of devices, with which the improved endurance and the enhanced linearity of the resistance change during potentiation and depression can be achieved.