Decision tree is widely used as a classification model, but it is susceptible to a significant privacy issue. Researchers have integrated differential privacy into decision trees, yielding positive outcomes. However, to satisfy differential privacy, the noise will inevitably have a negative effect on its performance. We proposes a differential privacy decision forest algorithm (DPDF-REN). First, the termination division criteria of nodes are established. A node will terminate the division, when its division does not yield better classification results, but instead increases the effect of noise. Second, a node depth-based privacy budget allocation strategy (NDPS) allocates more privacy budget to deeper nodes. This strategy gradually reduces the noise in order to alleviating the signal-to-noise ratio imbalance. Finally, a leaf node reliability-based ensemble (LR-En) is proposed. Leaf node reliability measures the likelihood of being in the maximum instance count class. The ensemble reduces the effect of noise by assigning weights to each decision tree based on the leaf node reliability. We conducted a series of experiments on several public datasets. The experimental results show that the DPDF-REN algorithm has better classification performance compared to other algorithms.