Automated fruit counting, through the use of computer vision methods, is a very important step towards the development of precision agriculture, and, as a consequence, has been object of study by multiple authors. In this work, we present a novel method to detect, track and count fruits that is scalable and robust towards problems existing in other fruit counting methods. Namely, our approach consists in a method that uses (1) a stereo camera that can be used reliably under direct sunlight, (2) a fast detection algorithm, and (3) an algorithmic approach to track fruits which is robust to fruit occlusions. We applied our proposed solution in an apple orchard and were able to provide apple counts with an error ranging from 15% to 57%. The characteristics of this novel approach and the preliminary results achieved seem promising in order to tackle the problem of fruit counting with occlusions at the large scale of a whole orchard, since it addresses the problem of intermittent occlusions that is overlooked by other approaches. Even with errors, the overview of the entire orchard, resulting from the scalability of the process, which can be implemented with a camera in a tractor, makes real-time mapping possible, supporting technical decisions with major economical impact, such as supplementary fruit thinning, water or nutrient adjustments or harvest management.