It is likely that at some point during your time working with data, you have come across a dataset that changes, but the history isn’t retained. This can become a nightmare to deal with because, quite often, data professionals need to query the data from a point in time. During this chapter, we will talk about the ways that dbt can help track row changes and maintain change history using snapshots. If you are familiar with dimensional data modeling, you may be familiar with Slowly Changing Dimensions (SCDs), which allow you to track historical data in various ways (or Types). Snapshots are identical to a Type-2 SCD, where a record is stored for both the current and changed rows in the same table and differentiated based on status and timestamp columns. In other data model designs, the concept exists but may be referenced by a different name.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Snapshots

  • Dustin Dorsey,
  • Cameron Cyr

摘要

It is likely that at some point during your time working with data, you have come across a dataset that changes, but the history isn’t retained. This can become a nightmare to deal with because, quite often, data professionals need to query the data from a point in time. During this chapter, we will talk about the ways that dbt can help track row changes and maintain change history using snapshots. If you are familiar with dimensional data modeling, you may be familiar with Slowly Changing Dimensions (SCDs), which allow you to track historical data in various ways (or Types). Snapshots are identical to a Type-2 SCD, where a record is stored for both the current and changed rows in the same table and differentiated based on status and timestamp columns. In other data model designs, the concept exists but may be referenced by a different name.