Knowledge Base
Plain-English definitions for the data lakehouse: 401 terms spanning Apache Iceberg, catalogs, table formats, query engines, and agentic AI.
Showing all 401 terms
ACID Transactions in the Data Lakehouse: How Iceberg, Delta Lake, and Hudi Commit
ACID transactions guarantee atomicity, consistency, isolation, and durability. See how Iceberg, Delta Lake, and Hudi provide them on object storage.
Read full definitionAirbyte for the lakehouse: open-source ingestion into Iceberg, and Airbyte vs Fivetran
Airbyte is an open-source data integration platform that moves data from APIs and databases into warehouses and lakes. See how it writes Iceberg vs Fivetran.
Read full definitionAlation: Enterprise Data Catalog and How It Fits Next to Lakehouse Catalogs
Alation is a commercial enterprise data catalog for discovery and governance. Learn how it differs from Iceberg catalogs like Polaris, Unity Catalog, and Glue.
Read full definitionAmazon S3 for Data Lakehouses: Iceberg, S3 Tables, and Storage Costs
Amazon S3 is AWS object storage that keeps files as objects in buckets behind an HTTP API. See how Iceberg runs on S3, what S3 Tables adds, and what it costs.
Read full definitionAmundsen data catalog: archived in 2026, and how it compares to DataHub and OpenMetadata
Amundsen is Lyft's open-source data discovery catalog, archived in September 2026. See how it compares to DataHub and OpenMetadata and fits Iceberg catalogs.
Read full definitionAnswer Engine Optimization (AEO) vs SEO and GEO, and What Data Teams Can Use
Answer Engine Optimization (AEO) structures content so AI and search tools quote it as a direct answer. See how it compares to SEO and GEO and applies to data.
Read full definitionApache Airflow for Lakehouse Pipelines: DAGs, Airflow 3, and Iceberg
Apache Airflow is an open-source platform for authoring, scheduling, and monitoring workflows in Python. See how Airflow 3 orchestrates Iceberg pipelines.
Read full definitionApache Avro in the Lakehouse: Iceberg Manifests, Kafka, and Parquet
Apache Avro is a row-oriented binary serialization format with schemas. In a lakehouse it carries Kafka events and stores Apache Iceberg manifest files.
Read full definitionApache DataFusion and Iceberg: The Embeddable Rust Query Engine
Apache DataFusion is an embeddable SQL query engine written in Rust on Apache Arrow. See how it reads Iceberg tables and compares to DuckDB, Spark, and Trino.
Read full definitionApache Doris and Iceberg: How It Compares to StarRocks
Apache Doris is an open source MPP database for real-time, high-concurrency analytics. See how it reads and writes Iceberg tables and compares to StarRocks.
Read full definitionApache Druid vs an Iceberg Lakehouse for Real-Time Analytics
Apache Druid is an open-source real-time analytics database for fast queries on event data. Here is how it fits next to, and ingests from, Iceberg tables.
Read full definitionApache Flink and Apache Iceberg: Streaming into the Lakehouse
Apache Flink is an open-source stateful stream processing engine. See how it writes to Iceberg, why commits follow checkpoints, and how it compares to Spark.
Read full definitionApache Fluss vs Kafka: Streaming Storage for Iceberg Lakehouses
Apache Fluss is open source streaming storage for real-time analytics that tiers data into lake tables. See how it compares to Kafka for Iceberg tables.
Read full definitionApache Gravitino vs Polaris and Unity Catalog OSS as a Metadata Layer
Apache Gravitino is an open source metadata lake that federates many catalogs behind one API. See how it serves Iceberg and compares to Polaris and Unity.
Read full definitionApache Hive and the Hive Metastore vs Apache Iceberg
Apache Hive is a SQL data warehouse system for HDFS and object storage, known for its Metastore. Compare Hive tables with Iceberg and learn how to migrate.
Read full definitionApache Kafka Connect: Streaming Kafka Topics into Iceberg Tables
Kafka Connect is Apache Kafka's framework for moving data in and out of Kafka with connectors. See how its Iceberg sink lands topics in lakehouse tables.
Read full definitionApache ORC vs Parquet: Which to Use for Iceberg Tables
Apache ORC is an open source columnar file format created in 2013 to speed up Apache Hive. See how it differs from Parquet and when to use it in Iceberg.
Read full definitionApache Parquet: How the File Format Works Under Iceberg Tables
Apache Parquet is an open source columnar file format for analytics. See how its row groups, encodings, and statistics work and how it relates to Iceberg.
Read full definitionApache Pulsar: Architecture, Kafka Comparison, and Iceberg Pipelines
Apache Pulsar is a messaging and streaming platform that separates brokers from BookKeeper storage. See how it compares to Kafka for feeding Iceberg tables.
Read full definitionApache Spark for the Lakehouse: Architecture, Iceberg Writes, and When to Use Other Engines
Apache Spark is an open source distributed engine for batch, SQL, and streaming. Learn how it writes Iceberg tables and when Trino, Dremio, or Flink fit better.
Read full definitionApache Spark SQL on Iceberg: Catalyst, Tungsten, and MERGE INTO
Apache Spark SQL is Spark's module for structured data, SQL, and DataFrames. Learn how Catalyst plans queries and how Spark SQL writes Iceberg tables.
Read full definitionApache Superset on a Data Lakehouse: Setup, Caching, and BI Alternatives
Apache Superset is an open source BI and data visualization platform. Learn how it queries Iceberg tables via Trino or Dremio and how it compares to rival BI.
Read full definitionApache XTable (Incubating): Translating Metadata Between Iceberg, Delta Lake, and Hudi
Apache XTable (incubating) translates table metadata between Iceberg, Delta Lake, and Hudi without copying data. See its limits and how it compares to UniForm.
Read full definitionApache ZooKeeper: What It Does and Whether Your Lakehouse Needs It
Apache ZooKeeper is a coordination service for distributed systems. See what it does, why Kafka replaced it with KRaft, and which lakehouse tools still use it.
Read full definitionArrow Flight
Arrow Flight is an RPC framework for sending Apache Arrow columnar data over the network without row conversion, moving query results from engines to clients.
Read full definitionArrow Flight SQL vs JDBC and ODBC for Lakehouse Queries
Arrow Flight SQL is a protocol for running SQL queries over Apache Arrow Flight. Learn how it compares to JDBC, ODBC, and ADBC when pulling lakehouse data.
Read full definitionAWS Glue for Iceberg Lakehouses: Data Catalog, ETL, and Engines
AWS Glue is Amazon's serverless data integration service with a shared Data Catalog. See how Glue serves Iceberg tables to Spark, Athena, Dremio, and Trino.
Read full definitionBloom Filters in Parquet and Apache Iceberg Tables
A Bloom filter is a compact probabilistic structure that tells an engine a value is definitely absent. Learn how Parquet and Iceberg use them to skip data.
Read full definitionBroadcast Hash Join: How Spark, Trino, and Dremio Join Iceberg Tables
A broadcast hash join copies a small table to every worker so a large table joins without a shuffle. See how Spark and Trino pick it and how Iceberg stats help.
Read full definitionCensus (Now Fivetran Activations): Reverse ETL From the Warehouse and Lakehouse
Census is a reverse ETL tool that syncs modeled warehouse data into apps like Salesforce, now sold as Fivetran Activations. See how it fits a lakehouse.
Read full definitionChange Data Capture (CDC) into Apache Iceberg: Methods, Tools, and Trade-offs
Change data capture streams inserts, updates, and deletes from a source database to other systems. Compare query and log-based CDC and the paths into Iceberg.
Read full definitionChunking for RAG: Strategies and How to Store Chunks in Lakehouse Tables
Chunking splits long documents into smaller segments before embedding them for RAG. Compare chunking strategies and see how to manage chunks in Iceberg tables.
Read full definitionCloud Data Warehouse vs Open Lakehouse: Storage, Formats, and Lock-In
A cloud data warehouse is a managed analytical database such as Snowflake or BigQuery. Compare its storage, formats, and lock-in with an open Iceberg lakehouse.
Read full definitionCollibra and the Open Lakehouse: How It Compares to DataHub and Polaris
Collibra is a commercial data governance and catalog platform for glossaries, stewardship, and policy. See how it fits an Iceberg lakehouse next to DataHub.
Read full definitionConfluent and the Lakehouse: Kafka, Tableflow, and Streaming into Iceberg
Confluent is the Kafka company, now part of IBM, behind Confluent Cloud and Platform. Its Tableflow feature turns Kafka topics into Iceberg and Delta tables.
Read full definitionContinuous Streaming vs Micro-Batching, and What Each Means for Iceberg
Continuous streaming processes each event as it arrives instead of in micro-batches. See how Flink and Spark compare and what latency Iceberg tables add.
Read full definitionCron: Time-Based Scheduling and When Lakehouse Pipelines Need an Orchestrator
Cron is the Unix scheduler that runs commands at set times using five-field expressions. Learn where cron works for lakehouse jobs and where an orchestrator wins.
Read full definitionCSV Files in a Lakehouse: Why and How to Convert Them to Iceberg
A CSV file is plain text where each line is a row and commas separate columns. Learn its limits for analytics and how to load CSV into Iceberg tables.
Read full definitionDagster for lakehouse pipelines: how it compares to Airflow and Prefect
Dagster is an open-source orchestrator that models pipelines as data assets instead of tasks. See how it compares to Airflow and Prefect on Iceberg lakehouses.
Read full definitionData Discovery in the Lakehouse: Catalogs, Profiling, and Trust
Data discovery is how teams find, understand, and trust data before using it. See how discovery tools sit on top of Iceberg catalogs in a lakehouse.
Read full definitionData Federation vs the Lakehouse: When to Query Data in Place
Data federation lets one query engine read and join data across separate systems without copying it. Learn when to federate and when to load into Iceberg.
Read full definitionDataHub and the Iceberg Lakehouse: How It Compares to OpenMetadata and Collibra
DataHub is an open-source metadata platform for data discovery, lineage, and governance. See how it fits an Iceberg lakehouse and compares to the alternatives.
Read full definitiondbt on Apache Iceberg: how SQL transformations work in a lakehouse
dbt is an open-source framework that turns SQL SELECT statements into tested, versioned tables and views. Here is how dbt builds Iceberg tables in a lakehouse.
Read full definitionDead Letter Queue (DLQ): Handling Bad Records in Lakehouse Pipelines
A dead letter queue holds messages a consumer could not process so the rest keep flowing. See how DLQs work in Kafka Connect, SQS, and Iceberg pipelines.
Read full definitionDebezium CDC to Apache Iceberg: How Log-Based Capture Feeds a Lakehouse
Debezium is an open source change data capture platform that streams row changes from database logs. Learn how its events land in Iceberg tables and tradeoffs.
Read full definitionDictionary Encoding: How Parquet, Arrow, and Iceberg Tables Compress Repeated Values
Dictionary encoding stores each distinct value once and replaces repeats with small integer IDs. See how Parquet, Arrow, and Iceberg writers use it and tune it.
Read full definitiondlt (Data Load Tool): Python Ingestion Into Iceberg, DuckDB, and Warehouses
dlt is an open-source Python library that loads API and database data into warehouses and lakes. See how it writes Iceberg tables and compares to Airbyte.
Read full definitionEncryption in Transit for the Lakehouse: TLS for S3, Catalogs, and Engines
Encryption in transit protects data moving over a network, usually with TLS. Learn where TLS applies in a lakehouse: storage, catalogs, engines, and clients.
Read full definitionFact Table vs Dimension Table: Types, Examples, and Iceberg Design
A fact table stores measurable business events at a fixed grain in a star schema. See how it differs from a dimension table and how to build one in Iceberg.
Read full definitionFeature Store on a Lakehouse: Offline Iceberg Tables, Online Serving
A feature store manages ML features for both training and real-time serving. See how its offline store maps to Iceberg tables and how Feast and others compare.
Read full definitionFilter Pushdown: How Iceberg, Parquet, and Lakehouse Engines Skip Data
Filter pushdown moves a query's WHERE conditions down to the storage or source layer so less data is read. Here is how it works with Iceberg and Parquet.
Read full definitionFivetran for the lakehouse: loading Iceberg tables, and Fivetran vs Airbyte
Fivetran is a managed ELT service that copies data from SaaS apps and databases into warehouses and lakes. See how it loads Iceberg and compares to Airbyte.
Read full definitionGold Layer in a Medallion Architecture: Design and Iceberg Examples
The gold layer is the business-ready tier of a medallion architecture. Learn how to model gold tables on Apache Iceberg and when a semantic layer replaces them.
Read full definitionGoogle BigQuery: Architecture, Iceberg Tables, and Lakehouse Fit
Google BigQuery is a serverless cloud data warehouse built on the Dremel engine. See how it reads and writes Apache Iceberg tables in an open lakehouse.
Read full definitionGraph Databases and Graph Queries on Lakehouse Iceberg Tables
A graph database stores data as nodes and edges to traverse relationships quickly. Learn how it works and when to run graph queries on Iceberg tables instead.
Read full definitionGreat Expectations (GX): Data Quality Testing and How It Compares to Soda and dbt Tests
Great Expectations (GX Core) is an open-source Python framework for data quality tests. See how it compares to Soda and dbt tests on an Iceberg lakehouse.
Read full definitionHDFS Explained: Architecture, Limits, and Moving to an Iceberg Lakehouse
HDFS is the Hadoop Distributed File System, which spreads replicated file blocks across a cluster. See how it compares to object storage for Iceberg tables.
Read full definitionHigh Availability (HA) for a Data Lakehouse: Storage, Catalog, and Engines
High availability means designing a system to stay up through failures by removing single points of failure. See how HA applies to storage, catalogs, engines.
Read full definitionHightouch and the Lakehouse: Reverse ETL vs a Traditional CDP
Hightouch is a Reverse ETL and data activation platform that syncs warehouse data into SaaS tools. See how it reads lakehouse data and how it compares to CDPs.
Read full definitionHive Metastore (HMS): How It Works and How It Compares to Iceberg Catalogs
The Hive Metastore (HMS) is the catalog service that maps Hive tables to files. See how it compares to Iceberg REST catalogs like Polaris, Glue, and Nessie.
Read full definitionHTAP vs the data lakehouse: when one database can replace a pipeline
HTAP (hybrid transactional/analytical processing) runs transactions and analytics in one database. See how HTAP systems work and when a lakehouse fits better.
Read full definitionIceberg Hidden Partitioning vs Hive Partitioning, Explained
Hidden partitioning is how Apache Iceberg derives partition values from column transforms. See how it compares to Hive directories and Delta generated columns.
Read full definitionIceberg Manifest List: What It Stores and How Engines Use It
An Iceberg manifest list is the Avro file that lists every manifest in one snapshot. See its fields, how engines prune with it, and how v4 may replace it.
Read full definitionIdempotency in Data Pipelines: Safe Retries on Apache Iceberg Tables
Idempotency means running a pipeline twice gives the same result as running it once. See how MERGE, overwrites, and Iceberg snapshots make retries safe.
Read full definitionIn-Memory Databases vs Lakehouse Caching: When RAM Pays Off
An in-memory database keeps its primary data in RAM for very low latency. See how it differs from the caching that lakehouse engines like Dremio use.
Read full definitionKappa Architecture vs Lambda on an Apache Iceberg Lakehouse
Kappa architecture processes all data, live and historical, through one streaming pipeline. Here is how it compares to Lambda and works with Iceberg tables.
Read full definitionKey Management Service (KMS): How Lakehouse Encryption Keys Work
A key management service (KMS) creates, stores, and controls the keys that encrypt data. See how S3, Parquet, and Iceberg encryption use KMS envelope keys.
Read full definitionLambda Architecture vs Kappa and the Streaming Lakehouse
Lambda architecture splits data into a batch layer and a real-time speed layer merged at query time. See how it compares to Kappa and Iceberg streaming.
Read full definitionLance File Format: How It Compares to Parquet and Apache Iceberg for AI Data
Lance is an open columnar format for AI data with fast random access and built-in vector indexes. See how it compares to Parquet and where it fits with Iceberg.
Read full definitionMage: Open Source Data Pipelines Compared to Airflow, Dagster, and Prefect
Mage is an open-source tool for building, running, and scheduling data pipelines in a notebook-style editor. See how it compares to Airflow, Dagster, and Prefect.
Read full definitionMarquez: The OpenLineage Reference Server for Lakehouse Lineage
Marquez is an open source metadata server that collects OpenLineage events and shows data lineage. See how it compares to DataHub and catalog lineage.
Read full definitionMaster Data Management (MDM) on a Data Lakehouse
Master data management (MDM) builds one trusted golden record for entities like customers and products. See MDM styles and how to build them on Iceberg.
Read full definitionMetabase: Open-Source BI on an Apache Iceberg Lakehouse
Metabase is an open-source BI tool with a visual query builder, SQL editor, and dashboards. See how it queries Iceberg tables through Trino, Athena, or Dremio.
Read full definitionMicro-Batching vs Continuous Streaming for Iceberg Ingestion
Micro-batching processes a data stream as a series of small batch jobs. Learn how Spark, Flink, and Kafka Connect use it to write Iceberg tables and tradeoffs.
Read full definitionMicroservices Data in a Lakehouse: CDC, Outbox Events, and Iceberg
Microservices split an application into small services that deploy and scale on their own. Learn how to move data from many service databases into Iceberg.
Read full definitionMilvus and the Lakehouse: When to Use a Vector Database vs Vector Search on Iceberg
Milvus is an open-source distributed vector database for similarity search on embeddings. See how it works and when a lakehouse needs it over vectors in tables.
Read full definitionMinIO and Apache Iceberg: S3-Compatible Storage After the Community Edition
MinIO is an S3-compatible object storage server often used for local Iceberg lakehouses. Here is its 2026 status and what to run for dev and production.
Read full definitionModel Inference: Batch vs Real-Time Scoring on a Lakehouse
Model inference is running a trained model on new data to get predictions. See how batch, streaming, and real-time inference differ and where Iceberg fits.
Read full definitionNatural Keys vs Surrogate Keys in a Data Lakehouse
A natural key identifies a row with real business data like an email or VIN. Learn when to use it vs a surrogate key, and how Iceberg tables handle both.
Read full definitionNoSQL Databases: Types, Trade-offs, and Analytics on a Lakehouse
A NoSQL database stores data in non-relational models such as documents or key-value pairs. See how NoSQL data gets analyzed on an Iceberg lakehouse.
Read full definitionOLAP vs OLTP on a Data Lakehouse: Columnar Engines on Iceberg
OLAP (Online Analytical Processing) covers systems built for large aggregation queries over historical data. See how OLAP and OLTP map onto a data lakehouse.
Read full definitionOLTP Explained: How Transactional Data Reaches the Lakehouse
OLTP is the class of row-oriented databases that run live business transactions. See how OLTP data reaches Iceberg tables through CDC and federation.
Read full definitionPII in the Data Lakehouse: How to Find, Mask, and Truly Delete It in Iceberg
PII is any data that can identify a specific person, alone or combined. Learn how lakehouse teams detect, mask, and fully delete PII from Iceberg tables.
Read full definitionPipeline Orchestration for the Lakehouse: Airflow, Dagster, Prefect, and Event Triggers
Pipeline orchestration schedules and coordinates the steps of a data pipeline. See what it does for Iceberg tables and how Airflow, Dagster, and Prefect differ.
Read full definitionPrefect for Lakehouse Pipelines: How It Compares to Airflow and Dagster
Prefect is an open-source Python workflow orchestrator built on flows, tasks, and events. See how it runs Iceberg pipelines and compares to Airflow and Dagster.
Read full definitionProject Nessie: Git-Like Branches for Iceberg and How It Compares to Polaris
Project Nessie is an open source Iceberg catalog with Git-like branches, tags, and multi-table commits. See how it works and how it compares to Apache Polaris.
Read full definitionPushdown Execution: How Trino, Dremio, and Spark Push Work to Sources and Iceberg Files
Pushdown execution sends filters, projections, and aggregations to the data source instead of moving rows. See how it works for federated sources and Iceberg.
Read full definitionRedpanda: Kafka-Compatible Streaming and Iceberg Topics for the Lakehouse
Redpanda is a Kafka API compatible streaming platform written in C++. See how it compares to Apache Kafka and how Iceberg Topics write streams to tables.
Read full definitionRelational Databases (RDBMS) and How They Feed the Lakehouse
A relational database stores data in tables of rows and columns linked by keys and changed by ACID transactions. See how RDBMS tables compare to Iceberg tables.
Read full definitionRow-Level Security on Iceberg Lakehouse Tables: Where It Is Enforced
Row-level security filters which table rows each user can see. Learn where RLS is enforced on Iceberg tables and how vended credentials can bypass it.
Read full definitionSilver Layer: The Cleaned, Conformed Tier of the Medallion Architecture
The silver layer is the medallion tier where raw bronze data is cleaned, deduplicated, and conformed. See what belongs there and how to build it on Iceberg.
Read full definitionSlowly Changing Dimensions (SCD) on Apache Iceberg: Types 1, 2, and 3
Slowly changing dimensions are patterns for tracking changes to dimension attributes. See Types 1, 2, and 3 on Iceberg and why time travel is no substitute.
Read full definitionSnowflake and Apache Iceberg: How the Warehouse Fits an Open Lakehouse
Snowflake is a managed cloud data platform that separates storage from compute. See how its Iceberg tables and catalogs fit an open, multi-engine lakehouse.
Read full definitionSort-Merge Join vs Hash Join in Spark, Trino, and Iceberg
A sort-merge join shuffles and sorts both inputs on the join key, then merges. Learn when Spark picks it over hash joins and how Iceberg can skip the shuffle.
Read full definitionSpill to Disk: How Spark, Trino, Dremio, and DuckDB Handle Memory Pressure
Spill to disk is when a query engine writes intermediate data to local disk once memory runs out. Compare how Spark, Trino, Dremio, and DuckDB handle it.
Read full definitionStarRocks and Iceberg: How It Compares to Apache Doris
StarRocks is an open source MPP engine for sub-second analytics on its own tables and lakehouse data. See how it queries Iceberg and compares to Apache Doris.
Read full definitionSurrogate Keys vs Natural Keys: How to Generate Them on Iceberg
A surrogate key is a system-generated identifier with no business meaning. See how it differs from a natural key and how to generate one on Iceberg tables.
Read full definitionSystem Architecture: Principles and How They Shape a Data Lakehouse
System architecture is the high-level design of a system's components and how they interact. See its core tradeoffs and how they map to lakehouse layers.
Read full definitionThird normal form (3NF) explained, and where it still belongs in a lakehouse
Third normal form (3NF) is a schema design rule: every non-key column depends only on the key. See 1NF to 3NF examples and where 3NF fits in a data lakehouse.
Read full definitionTime-Series Database vs Apache Iceberg for Time-Series Data
A time-series database stores and queries timestamped data such as metrics and sensor readings. Learn when a TSDB fits and when Iceberg tables are better.
Read full definitionTrino and Apache Iceberg: How It Compares to Spark and Dremio
Trino is an open source distributed SQL engine that queries data where it lives. See how it reads and writes Iceberg tables and compares to Spark and Dremio.
Read full definitionType 1 SCD: Overwriting Dimension Changes with Iceberg MERGE INTO
A Type 1 SCD overwrites a dimension attribute in place and keeps no history. See how to implement it on Iceberg and why old snapshots still hold the old values.
Read full definitionType 2 SCD on Apache Iceberg: MERGE INTO Example and Time Travel
A Type 2 slowly changing dimension keeps history by adding a new row for each change. See a MERGE INTO example on Iceberg and how it differs from time travel.
Read full definitionVortex File Format vs Parquet: What Changes for the Lakehouse
Vortex is an open source columnar file format and toolkit built for fast scans and random access on object storage. See how it compares to Parquet and Iceberg.
Read full definitionWhat are Agentic Frameworks?
Agentic frameworks are libraries such as LangChain, AutoGen, and CrewAI that give LLMs memory, tool binding, and multi-agent orchestration so they can act.
Read full definitionWhat are Aggregation Reflections?
Aggregation Reflections are Dremio's precomputed, grouped aggregates stored as Iceberg or Parquet, which the optimizer substitutes into matching queries for BI.
Read full definitionWhat are AI Agents?
AI agents are software systems that use an LLM to reason and call external tools in a loop toward a goal, covering tool registries, ReAct, and human oversight.
Read full definitionWhat are Audit Logs?
Audit logs are chronological records of who did what, when, and from where in a system, covering WORM storage, structured event payloads, and anomaly alerts.
Read full definitionWhat are Autonomous Agents?
Autonomous agents are AI systems that take a goal, break it into steps, act through tools, and self-correct from feedback, within security limits and oversight.
Read full definitionWhat are Large Language Models (LLMs)?
Large language models are transformer networks trained on vast text to predict the next token. Learn about parameters, context windows, hallucinations, and RAG.
Read full definitionWhat Are Puffin Files? Apache Iceberg's Format for Statistics and Deletion Vectors
Puffin is the Apache Iceberg file format for data too large for manifests. It stores Theta sketches for distinct counts and, in format v3, deletion vectors.
Read full definitionWhat are Vector Embeddings?
Vector embeddings are numeric arrays that represent the meaning of text, images, or audio so AI models can compare them, and how they fit into a data lakehouse.
Read full definitionWhat is a Business Glossary?
A business glossary is a shared set of agreed definitions for business terms like revenue. Learn how it links to lakehouse tables and to the semantic layer.
Read full definitionWhat is a Chief Data Officer (CDO)?
A chief data officer is the executive responsible for data strategy, governance, and compliance. Learn the role's three pillars and where it reports.
Read full definitionWhat is a Cost-Based Optimizer (CBO)?
A cost-based optimizer is the query planner that uses table statistics to estimate the cost of each execution plan and picks the cheapest, via catalog metadata.
Read full definitionWhat is a Data Architect?
A data architect is the role that designs an organization's overall data platform, models, and governance rules, including choices like an open data lakehouse.
Read full definitionWhat is a Data Catalog?
A data catalog is a searchable inventory of metadata that helps people find, understand, and trust data assets, covering harvesting, glossaries, and lineage.
Read full definitionWhat is a Data Dictionary?
A data dictionary is technical documentation of each table and column in a database, with data types, constraints, and relations, unlike a business glossary.
Read full definitionWhat is a Data Fabric?
A data fabric is an architecture that uses active metadata and automation to connect data across clouds and on-premises systems, often through virtualization.
Read full definitionWhat is a Data Lake?
A data lake is a central repository that stores raw structured and unstructured data in its native format, from Hadoop HDFS to cloud object storage today.
Read full definitionWhat is a Data Lakehouse Platform?
A data lakehouse platform runs warehouse-style SQL and ACID transactions on data lake storage, using open table formats and compute decoupled from storage.
Read full definitionWhat is a Data Mesh?
Data mesh is a decentralized approach where business domains own and serve their data as products. Learn its four principles and how it maps onto a lakehouse.
Read full definitionWhat is a Data Pipeline?
A data pipeline is automated software that extracts, transforms, and loads data from sources into a warehouse or lakehouse. Learn each phase and idempotency.
Read full definitionWhat is a Data Product?
A data product is a dataset built and maintained like software for its users, a core idea in data mesh. Learn the traits that make one discoverable and trusted.
Read full definitionWhat is a Data Warehouse?
A data warehouse is a central, structured store built for analytical SQL queries. See how OLAP, ETL, and cloud warehouses led to the open lakehouse.
Read full definitionWhat is a Database Administrator (DBA)?
A database administrator (DBA) keeps databases available, fast, and secure. Covers HA and disaster recovery, query tuning, RBAC, and the cloud-era shift.
Read full definitionWhat is a Dimension Table?
A dimension table holds the descriptive attributes, like customer, product, and date, that analysts use to filter and group facts in a star schema, plus SCDs.
Read full definitionWhat is a Directed Acyclic Graph (DAG)?
A directed acyclic graph (DAG) is a graph of tasks with one-way dependencies and no cycles. See how Airflow and dbt use DAGs to order and parallelize pipelines.
Read full definitionWhat is a Knowledge Graph?
A knowledge graph models entities as nodes and relationships as edges so queries can follow connections directly. Learn how it supports GraphRAG for AI apps.
Read full definitionWhat is a Materialized View?
A materialized view is a database object that stores precomputed query results. Learn about incremental refresh and query rewriting with Dremio Reflections.
Read full definitionWhat is a Message Queue?
A message queue is a buffer that lets producers and consumers trade messages asynchronously so services stay decoupled. See how it differs from event streams.
Read full definitionWhat is a REST API?
A REST API is a web interface built on constraints like statelessness and standard HTTP verbs. Learn how REST works and where it falls short for bulk data.
Read full definitionWhat is a Semantic Layer?
A semantic layer maps raw tables to shared business metrics and definitions. Learn how headless BI and metric stores work and why AI agents query through it.
Read full definitionWhat is a Snowflake Schema?
A snowflake schema is a dimensional model whose dimension tables are normalized into sub-tables. Compare it with a star schema and when the extra joins pay.
Read full definitionWhat is a Star Schema?
A star schema is a dimensional model with a central fact table joined to denormalized dimension tables. Learn Kimball basics and why it simplifies analytic SQL.
Read full definitionWhat is a Taxonomy?
A data taxonomy is a hierarchical classification scheme for organizing data assets. Learn how data stewards use it to tag catalog tables and drive access rules.
Read full definitionWhat is a Vector Database?
A vector database stores embeddings and finds items by semantic similarity, covering HNSW and IVF indexes, metadata filtering, and hybrid search for AI apps.
Read full definitionWhat is a Webhook?
A webhook is an HTTP callback that pushes an event payload to another app when something happens, and how it feeds real-time lakehouse ingestion with retries.
Read full definitionWhat is ACID Compliance?
ACID compliance is the set of guarantees (atomicity, consistency, isolation, durability) that keep transactions reliable, and how table formats add it to lakes.
Read full definitionWhat is Active Data Governance?
Active data governance enforces access rules, masking, and tagging in software at query time instead of in static policy documents, applied in a data lakehouse.
Read full definitionWhat is Active Metadata?
Active metadata is metadata collected automatically and fed back into systems to trigger actions, unlike passive catalog entries, and its role in a data fabric.
Read full definitionWhat is Agentic AI?
Agentic AI refers to LLM-powered systems that plan, reason, and act through tools, covering memory, function calling, multi-agent setups, and deployment issues.
Read full definitionWhat is Agentic Analytics?
Agentic analytics uses AI agents that turn plain-language questions into SQL on a lakehouse, relying on a semantic layer, a ReAct loop, and read-only access.
Read full definitionWhat is AI Agent Tool Use (Function Calling)?
Tool use, or function calling, lets an LLM ask an application to run a defined function and return the result, which is how agents query data and take action.
Read full definitionWhat is AI Ethics and Bias?
AI ethics and bias covers how models inherit unfair patterns from training data and the practices that address it: alignment, explainability, and red teaming.
Read full definitionWhat is an AI Context Window?
An AI context window is the maximum number of tokens an LLM can process at once, and why that limit led to retrieval-augmented generation and vector search.
Read full definitionWhat is an AI Hallucination?
An AI hallucination is when a language model produces fluent output that is false or unsupported. Learn why it happens and how RAG grounds answers in your data.
Read full definitionWhat is an API Gateway?
An API gateway is a reverse proxy that acts as the single entry point to backend services, handling authentication, rate limiting, load balancing, and routing.
Read full definitionWhat is an Iceberg Manifest File?
An Iceberg manifest file is an Avro metadata file listing data files with their status and column stats, so query engines can prune files without listing S3.
Read full definitionWhat is an Ontology?
An ontology is a formal, machine-readable model of the concepts, properties, and relationships in a domain, used to structure knowledge graphs over a lakehouse.
Read full definitionWhat is Analytics Engineering?
Analytics engineering is the work of turning raw data into tested, documented models for analysis, often with dbt, and curating the lakehouse semantic layer.
Read full definitionWhat is Apache Arrow?
Apache Arrow is an open, language-independent columnar in-memory format for analytics, covering zero-copy reads, SIMD efficiency, and Arrow Flight transport.
Read full definitionWhat is Apache Drill?
Apache Drill is an open-source distributed SQL engine that queries JSON, CSV, and Parquet files without a predefined schema, and its influence on the lakehouse.
Read full definitionWhat is Apache Hadoop?
Apache Hadoop is an open-source framework for storing and processing large datasets on clusters with HDFS, MapReduce, and YARN; a forerunner of the lakehouse.
Read full definitionWhat is Apache Hudi?
Apache Hudi is an open table format and data lake platform for upserts, deletes, and incremental processing, with copy-on-write and merge-on-read tables.
Read full definitionWhat is Apache Iceberg?
Apache Iceberg is an open table format for large analytic tables on object storage, covering its metadata tree, hidden partitioning, time travel, and catalogs.
Read full definitionWhat is Apache Kafka?
Apache Kafka is an open-source distributed event streaming platform built on producers, brokers, consumers, topics, and partitions, often feeding a lakehouse.
Read full definitionWhat is Apache Kudu?
Apache Kudu is an open-source columnar storage engine for fast inserts, updates, and analytic scans using primary key indexes and Raft, and how Iceberg differs.
Read full definitionWhat is Apache Paimon?
Apache Paimon is an open table format built for streaming that uses an LSM-tree to handle real-time updates with primary keys and changelogs, often with Flink.
Read full definitionWhat is Apache Pinot?
Apache Pinot is an open-source real-time distributed OLAP datastore for high-concurrency, user-facing analytics, using star-tree indexes and real-time tables.
Read full definitionWhat is Apache Polaris?
Apache Polaris is an open-source implementation of the Apache Iceberg REST catalog spec, providing centralized role-based access control for many query engines.
Read full definitionWhat is API Integration?
API integration is how systems exchange data through APIs; for data teams it means pulling SaaS data into a lakehouse while handling auth, paging, and limits.
Read full definitionWhat is Attention Mechanism?
The attention mechanism lets a neural network weigh how each token relates to every other token, covering self-attention, parallelism, and multi-head attention.
Read full definitionWhat is Autonomous Resource Optimization?
Autonomous resource optimization is automatic tuning of caching, data layout, and compute by the platform, not by hand, as in Dremio's autonomous features.
Read full definitionWhat is Autonomous Workflows?
Autonomous workflows are processes that run on their own toward set goals, scheduled and monitored by an orchestrator as DAGs with retries and alerts.
Read full definitionWhat is Azure Data Lake Storage (ADLS)?
Azure Data Lake Storage (ADLS) Gen2 is Microsoft's analytics storage that adds a hierarchical namespace to Blob Storage for fast directory operations.
Read full definitionWhat is B-Tree Indexing?
B-tree indexing is a balanced tree structure databases use to find rows without a full table scan. Learn how searches work, B+ trees, and the write penalty.
Read full definitionWhat is Backfilling Data?
Backfilling data means reprocessing historical periods so a new metric covers the past. See how orchestrators and Iceberg partitions handle it.
Read full definitionWhat is Batch Processing vs Stream Processing?
Batch processing handles data in scheduled chunks; stream processing handles each event as it arrives. Compare Spark, Kafka, and Flink, plus Lambda and Kappa.
Read full definitionWhat is Block Storage?
Block storage splits data into fixed-size blocks on a volume attached to one server. Learn why databases use it and why lakehouses use object storage instead.
Read full definitionWhat is Bronze Layer?
The bronze layer is the raw landing zone of a medallion architecture, holding data as ingested from sources. See how it feeds silver and gold lakehouse tables.
Read full definitionWhat is Business Intelligence (BI)?
Business intelligence is the software and practice of turning data into dashboards and reports. See how BI tools sit on a semantic layer over a data lakehouse.
Read full definitionWhat is Caching?
Caching stores the result of an expensive operation in fast memory so repeat requests skip the work. Learn TTL, invalidation, and how Dremio Reflections use it.
Read full definitionWhat is CAP Theorem?
The CAP Theorem is the foundational mental model for data engineering architecture. It proves that there is no such thing as a 'perfect' distributed database.
Read full definitionWhat is CCPA?
The CCPA is a California privacy law giving consumers the right to know, delete, and opt out of the sale of their data. See what it demands of a lakehouse.
Read full definitionWhat is Center of Excellence (CoE)?
A data Center of Excellence is a central team that sets standards, training, and governance for data work. Learn its core functions and its role in a data mesh.
Read full definitionWhat is Chain of Thought?
Chain of thought is a prompting technique where a language model writes out step-by-step reasoning before answering. See how it relates to ReAct and tool use.
Read full definitionWhat is Change Management in Data Initiatives?
Change management in data initiatives is the work of getting people to adopt new data platforms. Covers sponsorship, data champions, training, and rollouts.
Read full definitionWhat is Chroma?
Chroma is an open-source embedding database for building LLM applications. Learn how it stores vectors and runs similarity searches with HNSW.
Read full definitionWhat is Chunking Strategies for RAG?
Chunking for RAG splits documents into smaller pieces before embedding them for retrieval. Compare fixed-size, sentence, structural, and parent-child chunking.
Read full definitionWhat is CI/CD for Data?
CI/CD for data applies automated testing and deployment to data pipelines and SQL. See how dbt tests and Nessie branches validate changes before release.
Read full definitionWhat is ClickHouse?
ClickHouse is an open-source, column-oriented database for real-time analytical queries. Learn how vectorized execution works and how it fits near a lakehouse.
Read full definitionWhat is Column-Level Security?
Column-level security controls which columns a user can see inside a table, often by masking. Covers dynamic masking and central policy control in a lakehouse.
Read full definitionWhat is Columnar Format?
A columnar format stores each column's values together instead of row by row, which speeds up analytical scans. See how it relates to Arrow and Iceberg.
Read full definitionWhat is Compliance Posture?
Compliance posture is an organization's current state of adherence to regulations and security policies. See how access controls and quality checks support it.
Read full definitionWhat is Compute Layer?
The compute layer is the tier of a decoupled architecture that runs queries and transforms data. See how engines like Trino query open lakehouse storage.
Read full definitionWhat is Compute Pushdown?
Compute pushdown sends filters, column selection, and aggregations to the data source instead of the engine. See how Iceberg metadata and Parquet stats use it.
Read full definitionWhat is Computer Vision?
Computer vision is the field of AI that trains models to interpret images and video. Learn how CNNs and vision transformers process pixel data.
Read full definitionWhat is Consistent Hashing?
Consistent hashing maps keys and servers onto a hash ring so adding or removing a node moves only a small share of keys. Virtual nodes balance the load.
Read full definitionWhat is Containerization (Docker)?
Containerization packages an app with its dependencies so it runs the same everywhere; Docker is the standard tool. Compare containers with virtual machines.
Read full definitionWhat is Context Window Management?
Context window management is choosing, chunking, and ranking what goes into a language model's limited input. Learn why it matters for agents that call tools.
Read full definitionWhat is Continuous Aggregates?
Continuous aggregates are materialized views that refresh incrementally as new data arrives, so rollups stay current. See how they relate to streaming and CDC.
Read full definitionWhat is Continuous Deployment (CD)?
Continuous deployment is the practice of releasing every change that passes automated tests straight to production, and how it differs from continuous delivery.
Read full definitionWhat is Continuous Integration (CI)?
Continuous integration is the practice of merging code often and running automated builds and tests on every change, applied here to SQL and data pipeline code.
Read full definitionWhat is Copy-On-Write?
Copy-on-write is a table update strategy that rewrites whole data files when rows change, trading slower writes for faster reads in formats like Apache Iceberg.
Read full definitionWhat is CQRS?
CQRS (Command Query Responsibility Segregation) is a design pattern that separates the operations that write data from those that read it, so each scales alone.
Read full definitionWhat is Customer Data Platform (CDP)?
A customer data platform (CDP) unifies customer records from many channels through identity resolution, and composable CDPs run directly on the data lakehouse.
Read full definitionWhat is Data Anonymization?
Data anonymization alters data so individuals cannot be re-identified, using techniques like generalization, added noise, and k-anonymity rather than masking.
Read full definitionWhat is Data as Code?
Data as Code applies Git-style workflows such as branches, commits, tags, and merges to lakehouse tables, using versioned catalogs like Project Nessie.
Read full definitionWhat is Data Classification?
Data classification is the process of finding and labeling data by sensitivity, from public to restricted, so tag-based security can protect lakehouse tables.
Read full definitionWhat is Data Compaction?
Data compaction is the maintenance job that rewrites many small files into fewer larger ones, keeping open table formats like Apache Iceberg efficient to query.
Read full definitionWhat is Data Consistency?
Data consistency is the guarantee about which version of data a reader sees across replicas. Covers the CAP theorem and strong versus eventual consistency.
Read full definitionWhat is Data Contracts?
A data contract is an agreement between data producers and consumers on schema, quality, and ownership, meant to stop changes breaking downstream analytics.
Read full definitionWhat is Data Democratization?
Data democratization lets every employee access and analyze data without engineering help, through self-service tools, data literacy, and a semantic layer.
Read full definitionWhat is Data Engineering Lifecycle?
The data engineering lifecycle is a framework of five stages data moves through: generation, ingestion, storage, transformation, and serving to BI and apps.
Read full definitionWhat is Data Engineering?
Data engineering is the discipline of building the pipelines and infrastructure that ingest, transform, and store data, and how it differs from data science.
Read full definitionWhat is Data Governance?
Data governance is the set of policies, roles, and technical controls that keep data secure, accurate, and usable, from stewardship to row and column security.
Read full definitionWhat is Data Gravity?
Data gravity is the idea that large datasets attract applications, services, and more data around them, making that data harder and costlier to move over time.
Read full definitionWhat is Data Ingestion?
Data ingestion is the process of extracting data from sources like databases, APIs, and logs and loading it into a lakehouse, by batch or real-time streaming.
Read full definitionWhat is Data Lake vs Data Warehouse?
A data lake stores raw data of any type at low cost, while a data warehouse holds curated data for fast SQL; compare the two and why the lakehouse merged them.
Read full definitionWhat is Data Lakehouse Architecture?
Data lakehouse architecture separates object storage, an open table format metadata layer, and multiple compute engines to give lake files warehouse features.
Read full definitionWhat is Data Lakehouse?
A data lakehouse is an architecture that adds warehouse features like ACID transactions and schema enforcement to data lake files via open formats like Iceberg.
Read full definitionWhat is Data Lineage vs Data Provenance?
Data lineage traces how data moved and changed across pipelines, while data provenance records who created it, when, and under what authority; how they differ.
Read full definitionWhat is Data Lineage?
Data lineage is a record of where data came from, how it was transformed, and where it is used. See how lineage graphs support root cause and impact analysis.
Read full definitionWhat is Data Literacy?
Data literacy is the ability to read, question, and argue with data. Learn its core skills and how organizations build a data-literate culture beyond tools.
Read full definitionWhat is Data Locality?
Data locality is the practice of running computation close to where data is stored to cut network transfer. Learn how columnar formats and caching apply it.
Read full definitionWhat is Data Masking?
Data masking hides or scrambles sensitive values like PII so people can query data without seeing the real values. Compare static and dynamic masking.
Read full definitionWhat is Data Mining?
Data mining is the process of finding patterns, correlations, and anomalies in large datasets. Learn association rules, clustering, classification, and more.
Read full definitionWhat is Data Modeling?
Data modeling defines the entities, attributes, and relationships in a dataset. Learn the conceptual, logical, and physical phases and OLTP vs OLAP design.
Read full definitionWhat is Data Monetization?
Data monetization is the process of turning an organization's data into measurable financial value. Compare internal and external strategies and their risks.
Read full definitionWhat is Data Observability?
Data observability is ongoing monitoring of data and pipeline health to catch silent failures. Its pillars: freshness, volume, distribution, schema, lineage.
Read full definitionWhat is Data Orchestration?
Data orchestration schedules and monitors interdependent pipelines in dependency order. Learn how tools like Apache Airflow use DAGs, retries, and backfills.
Read full definitionWhat is Data Privacy by Design?
Privacy by design is an engineering approach that builds data privacy into systems from the start instead of adding it later. Learn its seven core principles.
Read full definitionWhat is Data Privacy?
Data privacy is handling personal data lawfully, with consent and limited purpose. See how Apache Iceberg row-level deletes help honor erasure requests.
Read full definitionWhat is Data Product Manager?
A data product manager owns a data product the way a PM owns software: finding the real need, setting SLAs, and tracking adoption. Learn the role and skills.
Read full definitionWhat is Data Profiling?
Data profiling is the process of examining a dataset's structure, content, and quality before use. Learn its three phases and how it guides pipeline design.
Read full definitionWhat is Data Provenance?
Data provenance is the documented origin and history of a dataset, used to judge if it can be trusted. See how immutable logs and signatures support it for AI.
Read full definitionWhat is Data Quality?
Data quality is how accurate, complete, consistent, and timely data is for its use. See how teams enforce it with data contracts, tests, and anomaly checks.
Read full definitionWhat is Data Reflections?
Data Reflections, now called Reflections, are Dremio's optimized precomputed copies of data that its planner uses to speed up queries without changes to SQL.
Read full definitionWhat is Data Replication?
Data replication is the continuous copying of data across servers or regions for availability and read scale. Compare synchronous and asynchronous replication.
Read full definitionWhat is Data Science?
Data science combines statistics, computing, and domain knowledge to predict outcomes from data. Learn its lifecycle and why it relies on lakehouse data access.
Read full definitionWhat is Data Security Posture Management (DSPM)?
Data security posture management (DSPM) is a security category that finds, classifies, and scores risk for sensitive data across clouds, unlike CSPM.
Read full definitionWhat is Data Serialization Formats (Protobuf, Thrift)?
Protobuf and Thrift are schema-based binary serialization formats that encode data more compactly than JSON. Learn how Protobuf works and evolves its schemas.
Read full definitionWhat is Data Serialization?
Data serialization converts in-memory data into bytes for storage or transfer. Compare JSON, Protobuf, Avro, and Parquet, and see how Arrow avoids conversion.
Read full definitionWhat is Data Silos?
A data silo is data held in one team's system and isolated from the rest of the business. Learn their costs and how lakehouses and virtualization reduce them.
Read full definitionWhat is Data Stewardship?
Data stewardship assigns people accountability for the quality, meaning, and access rules of specific data domains. See the steward role and its data mesh fit.
Read full definitionWhat is Data Strategy?
A Data Strategy is the comprehensive, executive-level roadmap that aligns an organization's data initiatives directly with its core business objectives.
Read full definitionWhat is Data Tokenization?
Data tokenization replaces sensitive values like card numbers with random tokens, keeping originals in a separate vault, so lakehouse queries still work.
Read full definitionWhat is Data Vault?
Data Vault is a warehouse modeling method built from hubs, links, and satellites. See how its append-only, additive design absorbs change and keeps history.
Read full definitionWhat is Data Virtualization?
Data virtualization lets users query many sources as if they were one database, without copying data. Learn about pushdown and how it pairs with a lakehouse.
Read full definitionWhat is Data Visualization?
Data visualization turns data into charts, graphs, and maps people can read quickly. Learn visual encodings, how charts distort, and data storytelling.
Read full definitionWhat is Data Warehousing?
Their sales are processed in an Oracle database, their website traffic is logged in a MySQL database, and their customer support tickets are managed in Salesforce.
Read full definitionWhat is Data-Driven Decision Making?
Data-driven decision making means basing business choices on measured evidence instead of intuition, using KPIs, A/B tests, and a single source of truth.
Read full definitionWhat is DataOps?
DataOps applies DevOps practices to data pipelines: agile delivery, version control, automated testing, CI/CD, and observability for data teams.
Read full definitionWhat is Deep Learning?
Deep learning is machine learning built on multi-layer neural networks. Covers layers, backpropagation, GPUs, and the large datasets lakehouses often hold.
Read full definitionWhat is Delta Lake?
Delta Lake is an open table format that adds ACID transactions to Parquet data lakes. Covers the Delta log, schema enforcement, time travel, and UniForm.
Read full definitionWhat is Differential Privacy?
Differential privacy is a mathematical privacy guarantee that adds calibrated noise to query results. Learn how epsilon and privacy budgets limit leakage.
Read full definitionWhat is Dimensional Modeling?
Dimensional modeling is Ralph Kimball's method of organizing analytics data into fact and dimension tables, the star schemas common in a lakehouse gold layer.
Read full definitionWhat is Disaster Recovery (DR)?
Disaster recovery (DR) is the plan for restoring data systems after a major outage or loss. Learn how RPO, RTO, immutable backups, and air-gapping shape it.
Read full definitionWhat is Distributed Computing?
Distributed computing splits one workload across many networked machines. Covers master-worker clusters, fault tolerance, and the cost of data shuffling.
Read full definitionWhat is Distributed SQL Engine?
A distributed SQL engine runs one query in parallel across a cluster of nodes. Learn how engines like Trino query lakehouse storage with vectorized execution.
Read full definitionWhat is Distributed Systems?
A distributed system is a set of networked computers that act as one. Covers horizontal scaling, fault tolerance, distributed computing fallacies, and state.
Read full definitionWhat is Dremio Cloud?
Dremio Cloud is Dremio's fully managed lakehouse service. It queries data where it lives, including Apache Iceberg tables, with no clusters to run yourself.
Read full definitionWhat is Dremio Text-to-SQL?
Dremio Text-to-SQL is a Dremio feature that turns natural language questions into SQL, letting business users query lakehouse data without writing code.
Read full definitionWhat is DSPy?
DSPy is a framework from Stanford that optimizes LLM prompts and weights algorithmically, so you program language model pipelines instead of prompts.
Read full definitionWhat is DuckDB?
DuckDB is an in-process analytical SQL database that runs inside your app with no server. See its vectorized engine, Parquet and Arrow support, and MotherDuck.
Read full definitionWhat is ELT?
ELT (Extract, Load, Transform) loads raw data into a warehouse or lakehouse first and transforms it there with SQL tools like dbt, using elastic cloud compute.
Read full definitionWhat is Embeddings?
Embeddings are numeric vectors that capture the meaning of text, images, or audio. Learn how vector search over embeddings supports RAG and hybrid search.
Read full definitionWhat is Encryption at Rest?
Encryption at rest protects stored data, such as S3 objects or disks, by keeping it as ciphertext. Covers AES-256, key management, and server vs client side.
Read full definitionWhat is ETL vs ELT?
ETL transforms data before loading it; ELT loads raw data first and transforms it in the warehouse or lakehouse. Compare the tradeoffs and when to use each.
Read full definitionWhat is ETL?
ETL (Extract, Transform, Load) cleans data on a separate server before loading it into a warehouse. Learn its three phases, schema-on-write, and why ELT rose.
Read full definitionWhat is Event Sourcing?
Event sourcing is a design pattern that records every state change as an event in an append-only log, so current state is rebuilt by replaying those events.
Read full definitionWhat is Event Streaming?
Event streaming moves data as a continuous flow of immutable events through platforms like Kafka, letting many systems react to business events in real time.
Read full definitionWhat is Feature Engineering?
Feature engineering turns raw data into model inputs (features) for machine learning. Covers extraction, windowing, encoding, and avoiding data leakage.
Read full definitionWhat is Federated Data Access?
Federated data access lets one query engine read data across separate sources without copying it first. Learn how engines like Dremio query data where it lives.
Read full definitionWhat is Federated Identity?
Federated identity lets users sign in once through a trusted identity provider and reuse that identity across many systems and their data access policies.
Read full definitionWhat is Federated Learning?
Federated learning trains an ML model across many devices or sites without moving the raw data. Covers local training, gradient aggregation, and tradeoffs.
Read full definitionWhat is Few-Shot Learning?
Few-shot learning is getting a model to perform a task from only a handful of examples, often placed in an LLM prompt, instead of retraining on a large dataset.
Read full definitionWhat is File Storage?
File storage organizes data in a hierarchy of folders shared over NFS or SMB. See where it fits next to object storage and why it struggles at lakehouse scale.
Read full definitionWhat is Fine-Tuning?
Fine-tuning further trains a pre-trained model on task-specific examples to update its weights. Covers training data, LoRA and PEFT, and fine-tuning versus RAG.
Read full definitionWhat is FinOps for Data?
FinOps for data is the practice of making cloud data spend visible and accountable. Covers the inform, optimize, operate cycle and cost in an open lakehouse.
Read full definitionWhat is GDPR?
GDPR is the EU law that governs how organizations collect, store, and process personal data. See how it shapes data residency and row-level deletes on the lake.
Read full definitionWhat is Generative AI?
Generative AI is a class of AI models that create new text, code, images, and audio from prompts. Learn about transformers, diffusion, and AI on lakehouse data.
Read full definitionWhat is Generative Engine Optimization?
Generative engine optimization is the practice of shaping content so AI assistants and answer engines can find, cite, and summarize it accurately in replies.
Read full definitionWhat is Generative Pre-trained Transformer (GPT)?
GPT is a family of autoregressive language models that predict the next token. Learn what generative, pre-trained, and transformer each mean in how GPT works.
Read full definitionWhat is Google Cloud Storage (GCS)?
Google Cloud Storage is Google Cloud's object storage service. Learn about multi-region buckets, storage classes, lifecycle rules, and GCS as lakehouse storage.
Read full definitionWhat is Graph Neural Networks (GNNs)?
Graph neural networks (GNNs) are deep learning models for graph data that learn by passing messages between connected nodes. See their core tasks and uses.
Read full definitionWhat is GraphQL?
GraphQL is an open-source query language for APIs that lets clients request only the fields they need from one endpoint. Learn its tradeoffs for data ingestion.
Read full definitionWhat is GraphRAG?
GraphRAG is a retrieval-augmented generation approach that pulls facts from a knowledge graph, not only vector search, to give an LLM structured, linked facts.
Read full definitionWhat is Hardware Acceleration (GPUs/TPUs) for AI?
Hardware acceleration for AI means running neural network math on GPUs, TPUs, and other parallel chips. Learn how they speed up model training and inference.
Read full definitionWhat is Headless BI?
Headless BI is an approach where metric definitions live in a shared semantic layer apart from dashboards, so every BI tool and AI agent uses the same logic.
Read full definitionWhat is Homomorphic Encryption?
Homomorphic encryption is a form of encryption that lets a system compute on encrypted data without decrypting it. Learn partial vs. full schemes and the cost.
Read full definitionWhat is Hugging Face?
Hugging Face is a company and platform hosting open models and datasets, known for its Transformers library and Model Hub. See how its ecosystem fits together.
Read full definitionWhat is Hybrid Search?
Hybrid search combines semantic vector search with keyword search and merges the rankings, so retrieval matches both meaning and exact terms in AI applications.
Read full definitionWhat is Iceberg Catalog?
An Iceberg catalog is the service that maps each Apache Iceberg table name to its current metadata file and swaps that pointer atomically so commits stay safe.
Read full definitionWhat is Iceberg Snapshot?
An Iceberg snapshot is the recorded state of an Apache Iceberg table at one point in time, listing its data files. Learn how snapshots make time travel work.
Read full definitionWhat is Idempotent Pipelines?
An idempotent pipeline is a data workflow that gives the same result however many times it runs on the same input, so retries and reruns do not duplicate data.
Read full definitionWhat is Infrastructure as Code (IaC)?
Infrastructure as code is the practice of defining servers, networks, and cloud resources in versioned files. Learn how declarative tools like Terraform work.
Read full definitionWhat is JSON?
JSON is a human-readable text format for nested key-value data used by most web APIs. Learn why it is slow for analytics and how lakehouses turn it to Parquet.
Read full definitionWhat is Kubernetes?
Kubernetes is an open-source platform that automates deploying, scaling, and managing containers. Learn how data teams use it to run Spark jobs and ML models.
Read full definitionWhat is LangChain?
LangChain is an open-source framework for building LLM apps with prompts, memory, retrieval, and tool-calling agents. See how agents use it with lakehouse data.
Read full definitionWhat is Large Language Model (LLM) Context Windows?
When interacting with a Large Language Model (LLM), it is easy to anthropomorphize the system and assume it has a continuous, human-like memory.
Read full definitionWhat is Late Materialization?
Late materialization is a query technique where a columnar engine filters on a few columns first and reads the remaining columns only for the rows that match.
Read full definitionWhat is LlamaIndex?
LlamaIndex is an open-source data framework that connects custom data sources to large language models for indexing, retrieval, and RAG and agent applications.
Read full definitionWhat is LLM Routing?
LLM routing is the practice of sending each request to the language model best suited for it, balancing answer quality, latency, and cost across several models.
Read full definitionWhat is Load Balancing?
Load balancing spreads incoming requests across a pool of servers so no one server is overloaded. Learn its algorithms, health checks, and use with SQL engines.
Read full definitionWhat is Log-Structured Merge-Tree (LSM)?
A log-structured merge-tree is a write-optimized data structure that buffers writes in memory, flushes sorted files to disk, and merges them through compaction.
Read full definitionWhat is Looker?
Looker is a Google Cloud BI platform that defines metrics once in LookML code and runs queries in the database, an early model for the headless semantic layer.
Read full definitionWhat is Low-Rank Adaptation (LoRA)?
Low-Rank Adaptation (LoRA) is a fine-tuning method from Microsoft researchers that freezes a model's weights and trains small low-rank matrices added to its layers.
Read full definitionWhat is Machine Learning?
Machine learning is a branch of AI where algorithms learn patterns from data instead of fixed rules. See its main types and how lakehouse tables feed training.
Read full definitionWhat is MapReduce?
MapReduce is a distributed processing model from Google that splits work into map, shuffle, and reduce phases across many servers, and why Spark replaced it.
Read full definitionWhat is Merge-On-Read?
Merge-on-read is a table write strategy that stores updates and deletes in separate files and merges them with data files at query time, as Apache Iceberg does.
Read full definitionWhat is Metadata Catalog?
A metadata catalog is a central registry that tracks where tables live, their schemas, and their current state, so query engines can find lakehouse tables.
Read full definitionWhat is Metadata?
Metadata is data about data: schemas, file locations, statistics, and definitions. Learn how Apache Iceberg metadata drives file pruning and time travel.
Read full definitionWhat is Metric Store?
A metric store is a central layer that defines business metrics and KPI logic once, so every BI tool and AI app reports the same numbers from one calculation.
Read full definitionWhat is Micro-Batches?
Micro-batching is a method that splits a data stream into small time-based batches, and how Spark and Apache Iceberg commit and compact them in a lakehouse.
Read full definitionWhat is Mixture of Experts (MoE)?
Mixture of Experts (MoE) is a neural network design where a router sends each token to a few expert subnetworks, so the model uses only part of its weights.
Read full definitionWhat is MLOps?
MLOps is the practice of applying DevOps methods to machine learning: versioning code and data, automating training and deployment, and watching for drift.
Read full definitionWhat is Model Drift?
Model drift is the drop in a machine learning model's accuracy as real-world data changes. Learn data drift versus concept drift and how to detect and fix each.
Read full definitionWhat is Model Quantization?
Model quantization is a compression method that stores model weights at lower precision, such as 8-bit or 4-bit, so large language models need less memory.
Read full definitionWhat is Monolithic Architecture?
Monolithic architecture is a software design where the whole application is built and deployed as one unit. Learn its strengths and its scaling limits.
Read full definitionWhat is Monte Carlo?
Monte Carlo is a data observability platform that monitors pipelines and tables for anomalies, tracks lineage, and alerts data teams to data downtime.
Read full definitionWhat is MotherDuck?
MotherDuck is a serverless cloud analytics service built on DuckDB that runs queries across your laptop and the cloud and reads Parquet in object storage.
Read full definitionWhat is MPP Architecture?
MPP architecture is a parallel database design where one query is split across many compute nodes that run at once, using vectorized execution and pushdown.
Read full definitionWhat is Multi-Agent Orchestration?
Multi-agent orchestration is the coordination of several AI agents that divide a task, pass data and results between them, and work toward a shared goal.
Read full definitionWhat is Multi-Agent System?
A multi-agent system is a design in which several autonomous software agents interact and collaborate to solve a task too broad for a single agent.
Read full definitionWhat is Multi-Cloud Architecture?
Multi-cloud architecture is the use of more than one cloud provider at once, and how Apache Iceberg tables and federated query engines support it.
Read full definitionWhat is Multi-Modal AI?
Multi-modal AI is a type of model that processes and generates more than one kind of data, such as text, images, and audio, within one shared network.
Read full definitionWhat is Natural Language Processing (NLP)?
Natural language processing (NLP) is the field of AI that lets computers parse, understand, and generate human language, from sentiment to transformers.
Read full definitionWhat is Object Storage?
Object storage keeps data as objects with keys and metadata in a flat namespace, as in Amazon S3. See how Apache Iceberg builds ACID tables on top of it.
Read full definitionWhat is Open Data Architecture?
Open data architecture is an approach that keeps data in open file and table formats so different tools and engines can work on the same data without copies.
Read full definitionWhat is Open Table Format?
An open table format is a metadata spec that lets many engines read and write the same table safely. See how formats like Apache Iceberg add ACID to data lakes.
Read full definitionWhat is Open-Source LLMs (Llama, Mistral)?
Open-source LLMs are language models, such as Meta's Llama and Mistral's models, whose weights are public to run, and why companies host them privately.
Read full definitionWhat is Open-Source Software (OSS)?
Open-source software is code released under a license that lets anyone inspect, modify, and share it. See how OSS licenses and business models shape data tools.
Read full definitionWhat is OpenLineage?
OpenLineage is an open standard for collecting lineage metadata from data pipelines, whatever tool runs them. Learn its role in lakehouse data governance.
Read full definitionWhat is Operational Analytics?
Operational analytics pushes metrics computed in the lakehouse into the SaaS tools frontline teams use. Learn how Reverse ETL and composable CDPs make it work.
Read full definitionWhat is Optimistic Concurrency Control?
Optimistic concurrency control lets writers proceed without locks and checks for conflicts at commit time. Learn how Apache Iceberg uses it for ACID writes.
Read full definitionWhat is Partitioning?
Partitioning splits a large table into groups of files by column values so queries can skip data. Learn how Apache Iceberg's hidden partitioning handles it.
Read full definitionWhat is Paxos and Raft Consensus?
Paxos and Raft are consensus algorithms that let a server cluster agree on shared state despite failures. See how Raft's leader election powers etcd and Consul.
Read full definitionWhat is Pinecone?
Pinecone is a fully managed, serverless vector database for similarity search on embeddings. See how ANN indexing, metadata filters, and hybrid search aid RAG.
Read full definitionWhat is Polaris Catalog?
Polaris is an open-source catalog for Apache Iceberg tables that lets many engines share the same metadata. Learn how it fits into a multi-engine lakehouse.
Read full definitionWhat is Power BI?
Power BI is Microsoft's business intelligence and visualization tool. Learn how Power Query, DAX, DirectQuery, and semantic models connect it to a lakehouse.
Read full definitionWhat is Predicate Pushdown?
Predicate pushdown is a query optimization that applies filters at storage so engines read less data. See how Parquet stats and Iceberg manifests make it work.
Read full definitionWhat is Predictive Analytics?
Predictive analytics uses historical data, statistics, and machine learning to estimate future outcomes. See how models are trained, deployed, and hit by drift.
Read full definitionWhat is Prescriptive Analytics?
Prescriptive analytics recommends actions by combining predictions with optimization and simulation. Learn how it works and why it depends on data quality.
Read full definitionWhat is Presto?
Presto is an open-source distributed SQL engine built at Facebook for interactive data lake queries. Learn how it relates to Trino and how its pipelines run.
Read full definitionWhat is Prompt Engineering?
Prompt engineering is the practice of designing LLM inputs to get accurate, well-formatted output. See how few-shot, chain-of-thought, and system prompts work.
Read full definitionWhat is Pub/Sub Architecture?
Pub/sub is a messaging pattern where publishers send events to topics and subscribers read them on their own. See how brokers like Apache Kafka decouple apps.
Read full definitionWhat is Qdrant?
Qdrant is an open-source vector database and similarity search engine for embeddings. See how HNSW indexing and hybrid dense and sparse search support RAG apps.
Read full definitionWhat is Query Planner?
A query planner is the part of a database or engine that turns SQL into an execution plan. See the plan lifecycle and how lakehouse engines like Trino use metadata.
Read full definitionWhat is Query Planning?
Query planning is how a SQL engine turns a query into an optimized execution plan before running it. See how distributed engines use pushdown and vectorization.
Read full definitionWhat is Raw Reflections?
A Raw Reflection is a Dremio feature that stores an optimized, materialized copy of a dataset's rows and columns to speed up queries on lakehouse tables.
Read full definitionWhat is Re-ranking Models?
A re-ranking model reorders search results by scoring each document against the query, often with a cross-encoder. Learn how two-stage retrieval improves RAG.
Read full definitionWhat is ReAct Framework?
ReAct (Reason and Act) is a prompting pattern where an AI agent alternates reasoning steps with tool calls. See how it lets agents run multi-step data analysis.
Read full definitionWhat is Real-time Analytics?
Real-time analytics means ingesting, processing, and querying data as it is generated. See how Kafka, Flink, real-time OLAP stores, and Apache Iceberg fit in.
Read full definitionWhat is Reasoning Engine?
A reasoning engine is the part of an AI agent that plans steps, picks tools to call, and checks results. See how it drives multi-step analytical workflows.
Read full definitionWhat is Reinforcement Learning from Human Feedback (RLHF)?
RLHF is a training method that aligns a language model with human preferences using a reward model and reinforcement learning. See its three phases and limits.
Read full definitionWhat is REST Catalog Specification?
The Iceberg REST catalog spec is an open API that defines how engines talk to Apache Iceberg catalogs. See how it lets many engines share one lakehouse catalog.
Read full definitionWhat is Retrieval-Augmented Generation (RAG)?
Retrieval-augmented generation (RAG) grounds an LLM's answers in documents retrieved at query time. See how chunking, embeddings, and vector search fit in.
Read full definitionWhat is Return on Investment (ROI) for AI?
ROI for AI measures whether a machine learning or generative AI project returns more business value than it costs, from data engineering and GPUs to revenue.
Read full definitionWhat is Reverse ETL?
Reverse ETL copies modeled data from a warehouse or lakehouse back into operational tools like CRMs and ad platforms, so teams act on metrics analysts built.
Read full definitionWhat is Role-Based Access Control (RBAC)?
Role-based access control (RBAC) grants permissions to roles, not individual users. See how a central catalog like Apache Polaris enforces RBAC across engines.
Read full definitionWhat is Row-Level Deletes?
Row-level deletes remove individual rows from a table without rewriting whole data files. Learn how open table formats like Apache Iceberg support them.
Read full definitionWhat is Run-Length Encoding?
Run-length encoding is a lossless compression method that stores a run of repeated values as one value and a count, which suits sorted columnar data.
Read full definitionWhat is Schema Evolution?
Schema evolution is changing a table's columns without breaking existing data. See how Apache Iceberg column IDs allow adds, drops, renames and type promotion.
Read full definitionWhat is Schema Registry?
A schema registry is a service that stores versioned Avro, Protobuf or JSON schemas for Kafka messages and checks compatibility as streams feed Iceberg tables.
Read full definitionWhat is Self-Service Analytics?
Self-service analytics lets business users explore data without writing code. See how BI tools, a semantic layer and a federated query engine support it safely.
Read full definitionWhat is Semantic Caching?
Semantic caching stores LLM responses and reuses one when a new prompt means the same as an earlier one, matched by embedding similarity, to cut model calls.
Read full definitionWhat is Semantic Search?
Semantic search finds results by meaning instead of exact keywords, using vector embeddings and nearest neighbor similarity, and hybrid search mixes both.
Read full definitionWhat is Semi-Structured Data (JSON, XML)?
Semi-structured data like JSON and XML carries its own tags and nesting instead of a fixed schema. See how lakehouse engines query it and store it in Parquet.
Read full definitionWhat is Serverless Computing?
Serverless computing runs code or queries without managing servers, scaling on demand and billing per use. See how serverless engines query Iceberg data.
Read full definitionWhat is Shuffle Partitioning?
Shuffle partitioning is how a distributed engine redistributes data across workers by key for joins and aggregations, and how salting and AQE address skew.
Read full definitionWhat is Snapshot Isolation?
Snapshot isolation lets each transaction read a consistent snapshot of data, so reads and writes run at once. See how Apache Iceberg snapshots provide it.
Read full definitionWhat is Soda?
Soda is a data quality tool that runs checks written in the YAML-based SodaCL language. Learn how it handles anomaly detection, data contracts and CI/CD checks.
Read full definitionWhat is Speech-to-Text (ASR)?
Speech-to-text, or automatic speech recognition (ASR), converts spoken audio into text. Learn how acoustic and language models and Whisper turn calls into data.
Read full definitionWhat is Storage Abstraction?
Storage abstraction is a layer that hides where and how data is stored from compute engines. See how Apache Iceberg metadata decouples tables from storage.
Read full definitionWhat is Storage Layer?
The storage layer is the tier of a data architecture that holds the physical data files, such as Parquet in object storage, separate from the compute engines.
Read full definitionWhat is Streaming Analytics?
Streaming analytics processes data continuously as events arrive instead of in scheduled batches. Learn how CDC, commit logs and delivery guarantees support it.
Read full definitionWhat is Streaming Ingestion?
Streaming ingestion loads data continuously as it is generated, via a broker like Kafka and a processor like Flink, into sinks such as Apache Iceberg tables.
Read full definitionWhat is Tableau?
Tableau is a business intelligence tool for building visual dashboards. Learn how its live connections and extracts work and how it pairs with a semantic layer.
Read full definitionWhat is Temperature in Generative AI?
Temperature is an LLM sampling setting that controls how random the next-token choice is. Learn how low and high values change output and which tasks suit each.
Read full definitionWhat is Terraform?
Terraform is HashiCorp's infrastructure as code tool that defines cloud resources in declarative files, using state, plans, and providers to deploy lakehouses.
Read full definitionWhat is the Agentic Lakehouse?
The agentic lakehouse is Dremio's term for a lakehouse built for AI agent use, pairing a semantic layer with Apache Arrow access and integrated vector search.
Read full definitionWhat is The Apache Software Foundation?
The Apache Software Foundation is a nonprofit that hosts open source projects like Spark and Iceberg, with the Apache Way, Apache 2.0 license, and incubator.
Read full definitionWhat is The Future of the Data Lakehouse?
The future of the data lakehouse covers where open lakehouse design is heading: agentic AI integration, fewer ETL pipelines, and multi-engine interoperability.
Read full definitionWhat is the Medallion Architecture?
The medallion architecture is a design pattern that refines lakehouse data through Bronze, Silver, and Gold tiers, built on table formats like Apache Iceberg.
Read full definitionWhat is Threat Modeling?
Threat modeling is the practice of finding what can go wrong in a system's design before it is built. Learn the STRIDE method and how teams plan mitigations.
Read full definitionWhat is Time Travel?
Time travel is a table format feature for querying data as it existed in the past, and how Apache Iceberg and Delta Lake use snapshots, timestamps, and cleanup.
Read full definitionWhat is Tokenization in LLMs?
Tokenization in LLMs is the step that splits text into tokens mapped to integer IDs, covering word, character, and subword (BPE) methods and their cost impact.
Read full definitionWhat is Tool Calling?
Tool calling is an LLM capability where the model requests external functions or APIs and uses the results, with notes on agent loops, ReAct, and governance.
Read full definitionWhat is Total Cost of Ownership (TCO) in Data?
Total cost of ownership (TCO) is the full cost of buying, running, and retiring a data system, labor included, and how a decoupled lakehouse changes it.
Read full definitionWhat is Transactional Layer?
A transactional layer adds ACID guarantees to files on a data lake, and this entry covers how open table formats like Apache Iceberg use metadata to provide it.
Read full definitionWhat is Type 3 SCD?
Type 3 SCD is a slowly changing dimension method that keeps limited history by adding a column for the previous value next to the current one in the same row.
Read full definitionWhat is Unity Catalog?
Unity Catalog is Databricks' governance catalog for lakehouse data and AI assets, covering access control, lineage, data sharing, and its open source release.
Read full definitionWhat is Universal Semantic Layer?
A universal semantic layer defines business metrics and logic once so every BI tool reads them the same way, and this entry covers how Dremio applies the idea.
Read full definitionWhat is Unstructured Data?
Unstructured data is information with no predefined schema, like text, PDFs, audio, and images, and how lakehouses and AI make it easier to store and analyze.
Read full definitionWhat is Upserts (Update and Insert)?
An upsert updates a row if it exists and inserts it if it does not. Learn how SQL MERGE INTO and change data capture handle upserts on lakehouse tables.
Read full definitionWhat is Vector Search Indexes?
Vector search indexes are structures that speed up approximate nearest neighbor search over embeddings, with a look at how IVF clustering and HNSW graphs work.
Read full definitionWhat is Vector Similarity Metrics?
Vector similarity metrics measure how close two embeddings are, comparing cosine similarity, Euclidean distance, and dot product and their impact on RAG search.
Read full definitionWhat is Vectorized Execution?
Vectorized execution is a query processing model that works on batches of columnar values instead of rows, using CPU cache and SIMD in engines like DuckDB.
Read full definitionWhat is Vendor Lock-in?
Vendor lock-in is dependence on one provider that makes switching costly, and how open storage plus open file and table formats keep data portable.
Read full definitionWhat is Version Control?
Version control is a system that records every change to code over time, covering commits, branches, pull requests, and reverts for pipelines and dbt models.
Read full definitionWhat is Virtual Data Warehouse?
A virtual data warehouse queries data where it lives through federation instead of copying it into one database, with tradeoffs in freshness, cost, and speed.
Read full definitionWhat is Weaviate?
Weaviate is an open source vector database that stores objects with their embeddings, combining HNSW vector search, keyword filters, and hybrid search for RAG.
Read full definitionWhat is XML?
XML is a text markup format that stores hierarchical data in tags validated by schemas like XSD, and why its verbosity makes ingestion slower than with JSON.
Read full definitionWhat is Z-Ordering?
Z-ordering is a clustering technique that sorts rows along a Z-curve across several columns, so Iceberg and Delta Lake queries can skip more files on filters.
Read full definitionWhat is Zero Trust Architecture?
Zero trust architecture is a security model that verifies every request instead of trusting the network. See how least privilege applies to a data lakehouse.
Read full definitionWhat is Zero-ETL?
Zero-ETL is an approach that connects operational databases to analytics systems without hand-built ETL pipelines, and how it relates to ELT and reverse ETL.
Read full definitionWhat is Zero-Shot Learning?
Zero-shot learning is a model's ability to handle tasks or classes it saw no labeled examples of, relying on general knowledge, as LLMs do with new prompts.
Read full definitionWrite-Audit-Publish (WAP): Staging, Testing, and Publishing Data with Iceberg Branches
Write-Audit-Publish (WAP) is a pattern that stages new data, tests it, and only then exposes it. Here is how Iceberg branches and Nessie make WAP cheap.
Read full definitionZero-Copy Architecture: What It Means with Apache Iceberg and Arrow
Zero-copy architecture means querying or sharing data where it lives instead of duplicating it. See the patterns behind the term in an Iceberg lakehouse.
Read full definitionNo terms match that search — try a broader keyword.