A core challenge in developing conversational agents with Large Language Models (LLMs) is their inability to retain information across interactions. This study examines the impact of two memory approaches, in-context memory and episodic memory, utilizing Random Write with K-Nearest Neighbors (K-NN) retrieval for multi-turn interactions, where coherence and context continuity are crucial for the effectiveness of LLM-based agents. We evaluate these approaches individually and in combination using the OpenAssistant Conversations (OASST1) dataset, measuring answer correctness, latency, cost, and memory usage. Our results show that memory-augmented agents significantly outperform the baseline, with in-context memory reducing incorrect responses by 57%. The hybrid strategy achieves the highest overall correctness with minimal impact on latency and cost. These findings underscore the relevance of memory mechanisms in building more adaptive, context-aware conversational LLM agents.