Alzheimer’s Disease (AD) is the most common form of dementia in the world. It is characterized by loss of cognition, lack of memory, and cognitive impairment. For maximizing the effectiveness of the treatment early detection is essential. Accordingly, in this paper, we explore the use of two sets of features over speech transcriptions aiming to automatically identify people suffering from AD. Both characterizations are designed to capture how a patient expresses herself or himself by considering different linguistic phenomena related to speech impairment. The obtained results outperform the state-of-the-art in this task. As a first attempt to contribute to the study of detecting the early stages of AD, we further analyze how the proposed characterizations perform over people with Mild Cognitive Impairment (MCI).