For the pose control of manipulators, rapid obstacle avoidance remains a critical research challenge. Among the various methods, the Control Barrier Function (CBF) stands out as an effective approach for ensuring safety and enhancing the robot’s maneuverability in dynamic environments. The effectiveness of CBF, however, heavily relies on the ability to avoid collisions with high precision in real-time, and addressing the computational efficiency and robustness of obstacle avoidance is crucial for advancing pose control strategies. In this paper, a pose control scheme of robotic manipulators is proposed, consisting of a servo controller and a CBF-based safety filter with velocity estimation (CBF-VE). A unified description of both translation and orientation is formulated as Lie groups, and the servo strategy based on Lie algebra enables to achieve precise pose control of robotic manipulators. Furthermore, to prevent collisions, optimization problems incorporating CBF constraints is solved to ensure the robot’s safety and efficiency. In addition, a velocity estimator that uses the integral method is developed to detect velocities of obstacles accurately, improving the precision of collisions avoidance. The feasibility of the proposed controller is verified through simulation experiments on a Kinova Gen3 arm.