Bi-copter UAVs have similar levels of versatility and are more energetically efficient than quadrotors. However, they are more difficult to control as the servo will lead to a non-minimum phase and are sensitive to shifts in the center of gravity. In this paper, we construct the incremental nonlinear dynamic inversion (INDI) attitude controller to linearize the nonlinear dynamics into a linear input-output map in an incremental form. We start by deriving the incremental dynamic of the bi-copter and combining it with the geometric controller to avoid the singularities. Then, the state estimator is designed based on the control allocator to provide angular acceleration estimation. Finally, comparative experiments demonstrate the improvements over a conventional geometry controller.