To facilitate the development of low-cost, low-power, and high-density hardware neural networks, we have successfully developed a Ta/TaOx/TiO2/Ti RRAM-based synaptic device. The device exhibits numerous synaptic functions resembling those in biological synapses, including synaptic plasticity of potentiation and depression, spike-timing dependent plasticity, paired-pulse facilitation and a transition from short-term to long-term memory. We further demonstrate 3D high-density, high-connectivity integration of the Ta/TaOx/TiO2/Ti device, and the device exhibits excellent uniformity among interlayer and intralayer cells in a 4 × 4 3D two-layer cross-point array. Finally, we investigate the influence of nonlinearity of synaptic weight updates on neuromorphic computing. A state-independent training scheme is proposed to improve linearity and fault tolerance of training accuracy.