Latest Articles

Read and Write Parquet Files in Python (PyArrow, Pandas)

Read and Write Parquet Files in Python (PyArrow, Pandas)

How to read and write Parquet files in Python with PyArrow, FastParquet, and Pandas, including column selection, compression, and partitioned datasets.

Read more
All About Parquet Part 09 - Parquet in Data Lake Architectures

All About Parquet Part 09 - Parquet in Data Lake Architectures

Why Parquet is the default file format for data lakes, how it underpins Iceberg, Delta Lake, and Hudi, and best practices for partitioning and pushdown.

Read more
Parquet Performance Tuning: 8 Best Practices

Parquet Performance Tuning: 8 Best Practices

Tune Apache Parquet for speed and cost: row group and file sizes, partitioning, Snappy vs ZSTD compression, encodings, sorting, and predicate pushdown.

Read more
Orchestrating Airflow DAGs with GitHub Actions - A Lightweight Approach to Data Curation Across Spark, Dremio, and Snowflake

Orchestrating Airflow DAGs with GitHub Actions - A Lightweight Approach to Data Curation Across Spark, Dremio, and Snowflake

Trigger Airflow DAGs from GitHub Actions, ingest with Spark, build bronze, silver, and gold views in Dremio with dbt, and load the results into Snowflake.

Read more

Work with Alex