What is a REST API?

What is a REST API?

A REST API (Representational State Transfer Application Programming Interface) is the most common architectural style for how software systems communicate and exchange data over the web. Defined by Roy Fielding in his 2000 doctoral dissertation, REST is not a specific software package or a strict protocol; it is a set of architectural constraints. A system that follows these constraints is considered “RESTful,” which makes it predictable for data engineering pipelines to connect to and extract data from.

Before REST became common, many web service integrations used heavier protocols like SOAP (Simple Object Access Protocol) with verbose XML messages and extensive tooling. REST simplified integration by building directly on the existing language of the web, HTTP (Hypertext Transfer Protocol). REST does not mandate a data format, but most REST APIs today exchange lightweight JSON (JavaScript Object Notation).

The Core Constraints of REST

For an API to be formally classified as RESTful, it must follow several architectural constraints.

1. Statelessness

This is the most important rule for scalability. In a REST API, the server does not keep session state for the client between requests.

If a data pipeline extracts Page 1 of the Customers table from Salesforce, and then immediately requests Page 2, the Salesforce server does not rely on remembering the earlier request. Therefore, every API request generated by the pipeline must contain all the information necessary for the server to process it, including the authentication Bearer Token, the specific endpoint URL, and the pagination cursor. This statelessness helps large SaaS platforms scale horizontally, because any server in the cluster can handle any request.

2. Client-Server Decoupling

The backend data storage (the Server) and the data extraction pipeline (the Client) are decoupled. The Salesforce API does not care if the pipeline extracting the data is written in Python, Java, or executed via a basic command-line curl request. As long as the request matches the API’s interface, the data is delivered.

3. Uniform Interface (HTTP Verbs)

REST APIs conventionally map data actions (CRUD: Create, Read, Update, Delete) directly to standard HTTP methods. When a data engineer builds an extraction pipeline, they normally use the GET method.

• GET https://api.example.com/v1/customers (Retrieves the data). They avoid POST (which creates new data) and DELETE (which removes data), which keeps extraction read-only and predictable.

Limitations for Big Data Extraction

While REST is widely used across the internet, it has drawbacks for Data Engineering and bulk analytical extraction.

REST APIs often suffer from Over-fetching and Under-fetching. If an engineer only needs the email column from the Customers table, they hit the /customers endpoint. However, unless the API supports field selection, the server decides what the response contains. It may return JSON payloads with dozens of fields the pipeline does not need (address, phone number, internal IDs). The pipeline downloads that extra data over the network only to discard most of it. Inefficiencies like this were part of the motivation for query languages like GraphQL, which let the client choose the fields it receives. For large-volume extraction, many SaaS platforms also offer separate bulk export APIs.

Summary of Technical Value

The REST API is one of the most common ways software systems connect. By using stateless requests, lightweight JSON payloads, and standard HTTP verbs, it made software integration far more accessible. It provides the structured, reliable pathways that modern data pipelines use to extract operational data into the central Data Lakehouse.

Learn More

To learn more about the Data Lakehouse, read The Lakehouse Built for Everyone by Alex Merced. You can find this and other books by Alex Merced at books.alexmerced.com.