What is Time Travel?
Time Travel is a feature of modern Open Table Formats (like Apache Iceberg, Apache Hudi, and Delta Lake). It lets a data engineer, data scientist, or business analyst run a SQL query against a data lakehouse table and see the table exactly as it was at an earlier snapshot or point in time.
Before the invention of Open Table Formats, data lakes were built directly on raw cloud object storage (Amazon S3) or HDFS. If an automated pipeline accidentally ran a DELETE or overwrite that removed 500,000 legitimate customer records, there was no table-level history to return to. Recovery meant restoring from backups or object storage versioning, which was slow and manual. Time Travel reduces this risk by keeping recent table versions queryable and restorable.
The Architecture of Snapshot Isolation
Time Travel is a direct result of how modern table formats manage metadata through Snapshot Isolation.
When a pipeline inserts, updates, or deletes data in an Apache Iceberg table, it does not modify the existing Apache Parquet files in place. Instead, it writes new data files (and, for some deletes and updates, delete files). Iceberg then writes new metadata that defines a new “Snapshot”. This snapshot references all the new files and the surviving old files, and no longer references files that were logically removed.
Because the old, logically deleted files remain in object storage until they are cleaned up, earlier versions of the table stay readable. The table’s metadata file keeps a list of its snapshots (until they are expired), and the catalog points to the current metadata file.
Executing a Time Travel Query
Executing a Time Travel query requires no infrastructure restoration; it uses SQL extensions in engines like Spark, Trino, or Dremio. The examples below use Spark SQL syntax; Trino uses FOR TIMESTAMP AS OF and FOR VERSION AS OF, and Dremio uses AT TIMESTAMP and AT SNAPSHOT.
An analyst can query the past using two primary mechanisms:
1. Timestamp-Based Travel
If an analyst knows a faulty pipeline ran at 2:00 AM, they can query the table as it existed at 1:59 AM:
SELECT * FROM customers FOR SYSTEM_TIME AS OF '2026-05-14 01:59:00';
The query engine reads the timestamp, searches the metadata history, locates the Snapshot that was current at that time, and reads only the data files referenced by that snapshot.
2. Snapshot ID-Based Travel
Every commit generates a unique numeric snapshot ID. If a data engineer wants to explicitly compare the current data against the state of the table exactly three commits ago to debug a subtle math error, they query the specific ID:
SELECT * FROM customers FOR SYSTEM_VERSION AS OF 10984384938;
Critical Use Cases
Beyond recovery from bad writes, Time Travel supports several analytical uses:
- Machine Learning Reproducibility: If a data scientist trains a fraud detection model on May 1st, and the model suddenly begins failing on June 1st, they must determine why. Using Time Travel, they can point the training algorithm at the exact historical Snapshot from May 1st. This lets them reproduce the original training data, which is essential for debugging the model (as long as that snapshot has not been expired).
- Audit and Compliance: In regulated industries (like banking), auditors may ask what data a company held on a specific past date. Time Travel lets analysts rerun reports against the table as it was then, provided the retention policy kept that snapshot. Multi-year retention is usually handled with tags or separate archive tables, because keeping every snapshot for years is costly.
Managing Storage Costs (Vacuuming)
Because Time Travel requires preserving old, logically deleted files, storage use grows over time. Open Table Formats manage this using “Vacuum” or “Expire Snapshots” maintenance routines. For example, a data engineer might keep 30 days of history. A scheduled job expires snapshots older than 30 days, removing them from metadata, and deletes the data files that are no longer referenced by any remaining snapshot. Expired snapshots can no longer be used for time travel, so the retention window is a tradeoff between history and storage cost.
Summary of Technical Value
Time Travel improves the reliability of the Data Lakehouse. By building on immutable files and snapshot metadata, it provides fast recovery from bad writes, reproducible training data for machine learning, and simpler historical auditing, all within the retention window the team configures. It gives data engineering teams a practical safety net against accidental data changes.
Learn More
To learn more about the Data Lakehouse, read The Lakehouse Built for Everyone by Alex Merced. You can find this and other books by Alex Merced at books.alexmerced.com.