Tabular data are organized in a structured format that contains rich information. However, challenges exist in interpreting this style of data in an easily understandable way. The annotation of semantic and atomic types of the columns, as well as the relationships between different columns, can assist both users and machines in comprehending tabular data across various scenarios. Existing deep learning methods rely on large amounts of training samples per type and suffer from long running times. In this paper, we explore the utilization of large language models (LLMs) and Knowledge Graph (KG) for column type annotation (CTA) and column property annotation (CPA). External knowledge resources, such as DBpedia and CaliGraph, are used to preliminarily refine the label set for selection by the LLMs, while noisy labels in the candidates are removed. A hierarchical search is performed by retrieving concrete entities matched to resources from the subgraph of the golden labels. Moreover, we evaluate both zero-shot and few-shot scenarios using a well-crafted prompt. Specifically, we compare the approaches of selecting few-shot demonstrations either randomly selected from the training set or by leveraging vector similarities between test and few-shot samples. As the experimental results show, our proposed method outperforms the baseline on the SOTABv2 benchmark for both CTA and CPA tasks.