With the increase of age, some elderly people's mobility problems have caused certain obstacles to daily life, so intelligent devices for helping the elderly are becoming more and more popular. The obstacle avoidance and tracking system of the elderly assistance intelligent car in this article adopts computer vision and multi-sensor fusion technology, which can achieve autonomous driving. The tracking module in the system consists of 5 TCRT5000 infrared detectors. The signal collected by the TCRT5000 infrared detector is amplified through an integrated circuit and sent to the microcontroller. The white and black lines are then determined to correspond to the five TCRT5000 infrared detectors, thereby determining the current position of the car. By adjusting the size of the resistance, an appropriate voltage can be obtained compared to the output signal. The minimum number of successful attempts in tracking and obstacle avoidance tests is 37, but overall it still maintains a high level and can meet basic needs.