Current video captioning rely on pre-training on massive video-text data to align spatiotemporal and linguistic information, often ignoring fine-grained object interactions and temporal ordering because of high data costs. To address these issues in a resource-friendly way, a novel framework BAT-VLP is proposed, which adapts image-based Vision-Language Pre-training (VLP) models for video captioning by introducing three Bootstrapped Auxiliary Tasks into fine-tuning, without modification on network structure or extra video-text data. The three tasks are separately: 1) Temporal-Ordering Question Answering for perceiving orders between events, 2) Multi-Frame Multiple-Object Tracking for capturing fine-grained object interactions, and 3) Information-Decomposed Conversation for mining dynamic and static words and then composing them. To be compatible with image-based VLP models’ input, image-grid format is further proposed to represent video frames, and various spatiotemporal resolutions can be flexibly obtained for generalization. All data for the BAT are automatically constructed from existing video captioning datasets, by taking advantage of the VLP model itself, Part-of-Speech Tagger and designed programs. Experiments show that BAT-VLP achieves state-of-the-art performance, and extensive ablation studies validate the effectiveness of the BAT and image-grid format.