As AI-based technologies become more prevalent in school classrooms, ensuring that students engage with AI technologies while maintaining balanced levels of agency is more crucial than ever. Past work often assumes student agency as a single-dimensional concept, focusing on finding the right balance between giving and withholding system-led assistance during learning. Considering the increased complexity of understanding student agency when AI systems are used in the classroom, we propose that student agency encompasses multiple critical dimensions of student-AI interactions, namely, Content Agency, Feedback & Help Agency, Co-Orchestration Agency, and Data Agency. We present the Student-AI Agency Framework—a reference model for designers and researchers to examine key dimensions of student-AI interactions within the complex dynamics of classroom environments. We also explore implications for the design of and research on AI-based learning systems through the lens of multi-dimensional student agency.