This paper investigates the adaptive formation control problem for a group of unmanned surface vehicles (USVs) with collision avoidance. Firstly, by utilizing the approximation ability of radial basis function-neural networks (RBF-NNs), the extended state observer (ESO) is constructed to obtain the model uncertainties and external disturbances. And then, a novel continuous differentiable potential function is proposed to achieve the collision avoidance. Furthermore, based on the proposed potential function and ESO, an adaptive nonsingular sliding mode control (NSMC) scheme is designed, which guarantees all signals in the closed-loop system are bounded. At last, the simulation results indicate the effectiveness of the developed control algorithm.