The increasing relevance of programming skills in higher education and the growing use of Large Language Models (LLMs) have inspired the application of these technologies to support learning. This study evaluates the use of the Llama 3 model with 8 billion parameters to generate feedback on student answers in the programming introductory undergraduate course at UFRJ (Universidade Federal do Rio de Janeiro) in Brazil. Data from Machine Teaching, a platform used since 2018 as an online tool to support the resolution of programming exercises, were analyzed. This platform accumulates instructor and teaching assistant comments to investigate LLMs effectiveness as feedback tools. Various prompt engineering strategies and configurations were tested for two programming problems – a very simple one and another more complex. An evaluation of the feedback generated was conducted by instructors, considering readability, structuring, and correctness. A total of 72 evaluations by 12 instructors compared feedback from zero-shot, one-shot, and few-shot prompting. Zero-shot prompting was most effective for the difficult problem, while one-shot excelled in the easy problem. Few-shot performed worst, with many instructors disagreeing with the feedback generated by this approach. Results highlight LLM’s potential to enhance learning in introductory programming courses.