Big Data Processing and Transformation with AWS Glue and AI Agents
摘要
The core value that the data engineering practice delivers is the expertise and competence to convert raw data into business value, whether that takes the form of a digital product, insights, or the training data that produces ML or AI models that perform high-value business functions. In this chapter, I review the fundamental theories of big data processing in the cloud and the AWS services and tools at your disposal to support and accelerate data ingestion, profiling, and big data processing and transformation. I also present data engineering use cases for applying generative and agentic AI to accelerate time-to-value.