Temporal Knowledge Graph (TKG) reasoning aims at predicting future facts based on occurred facts. With the development of large language models (LLMs), it has shown that in-context learning based approaches can provide state-of-the-art performance. However, they tend to use historically relevant examples (i.e., historical information) as demonstrations, which brings challenges in predicting facts that have no historical interaction. In reality, the current fact is often the result of historical information combined with underlying factors (i.e., non-historical information). In this paper, we present CALENDAR (i.e., in-context ContrAstive Learning tEmporal kNowleDge grAph Reasoning), an approach that leverages both histories and non-histories to create contrastive demonstrations, thereby reducing the bias toward predicting only historically occurred facts. In CALENDAR, we propose a demonstration candidate generation with high-order information method, which leverages high-order information from histories and non-histories to generate demonstration candidates. Moreover, we devise a contrastive importance based demonstration selection method which selects demonstrations by the sum frequency of entities in histories and non-histories of demonstration candidates. Furthermore, we design a contrastive chain-of-history based demonstration format which generates the negative principle for guiding the LLMs on why they should not rely only on the recurrence and periodicity histories. Experimental results on three benchmark datasets confirm the effectiveness of our approach.