API Fundamentals

• 1: API Versioning and Deprecation
• 2: Entity Versioning and Conflict Management
• 3: Content-Type Negotiation
• 4: Unique Identifiers

1 - API Versioning and Deprecation

Versioning

We use URI Path versioning, prefixing all endpoints with a simple, incrementing version string (v1). All routes in a resource should have their versions increased at the same time, though this will usually only involve adding additional routes in your gateway.

/v1/myresources/*
/v2/myresources/*

Deprecation

Breaking API changes are inevitable. Should a deprecation become necessary, it is most convenient to use a common pattern for communication and documentation. For this purpose, we will use RFC-8594 and RFC-8288 to communicate the expected deprecation date, as well as any remediation documentation. This is there to help your consumers, as they can update their code at their own pace, and to help you, as it simplifies management and communication around that API.

HTTP/1.1 200 OK
Sunset: Sat, 31 Dec 2018 23:59:59 GMT
Link: </docs/deprecation>; rel="sunset"; title="Deprecation Notice"

Breaking Changes

The following is a table of changes and whether they are considered ‘breaking’. A simple rule of thumb is that the “addition” of something is not considered breaking, while “changing” something is.

Any time a breaking change is introduced, we must follow the appropriate expand/contract change management process.

The Sunset Header

Sunset: Sat, 31 Dec 2018 23:59:59 GMT

As per RFC-8594, every endpoint that is flagged for deprecation must include this header, which specifies the date after which the endpoint will no longer be available. This allows clients to plan for the change and update their code accordingly.

The Link Header

If documentation is available describing the deprecation and the steps to take to update, you should use RFC-8288 to include include a Link header in your API response. This will allow our API clients to notify their authors and/or users that an out-of-date API is still in use, and that they should update their code and/or SDK.

Link: </target/url>; rel="sunset"; title="Human Readable Title"

2 - Entity Versioning and Conflict Management

Entity versioning and conflict management are essential for ensuring data integrity. The headers described in RFC 7232 provide us with the tools we need to communicate the version of an entity, and decide whether it can be modified. Our intent here is to create a client/server contract, providing an opt-in method to manage staleness during API calls. Data can only be returned to the client if it is demonstrably fresh, while modifications should only be allowed if the client can demonstrate awareness of the version they are modifying.

Headers

There are two headers by which an entity’s version or state is communicated by a server to a client: ETag and Last-Modified. You must return both in simple GET requests for a specific resource, though it is not specific on how they should be handled in a query, aggregation, or graph query.

ETag

An ETag is an opaque string that can weakly or strongly assert the content of the entity. An example of a weak ETag is one derived from a second-precision modification date, as multiple changes could have occurred within that time window. An example of a strong ETag is a SHA-256 of the entity body as persisted in the database, as it is the content itself that communicates freshness.

// Weak tag
ETag: W/"123456"

// Strong tag
ETag: "da39a3ee5e6b4b0d3255bfef95601890afd80709"

For the purpose of this contract, only strong ETag values are permitted, even though they may be difficult to generate. All ETags are represented as hexadecimal strings such as those generated by a SHA-256, and must be independently reproducible from the entity.

Last-Modified

Each resource entity should also return the Last-Modified header as per RFC 7231 Section 7.1.1.1, which is a timestamp of the last modification of the entity.

Last-Modified: Sun, 06 Nov 1994 08:49:37 GMT

Simple Responses

If the entity is loaded by itself via a simple GET request, the ETag and Last-Modified headers should both be present as per the RFC.

HTTP 1.1/200 Ok
Last-Modified: Sat, 29 Oct 1994 19:43:31 GMT
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
Content-Type: application/json

{
  ...
}

Requests using ETag

To manage staleness and conflict when accessing a single resource, the client uses an If-None-Match or If-Match header to communicate to the service the conditional parameters under which it wants to operate. This contract is required only on single resources, as calculating ETags for large collections is prohibitively expensive.

If-None-Match

Here, the client wishes to read a resource, but only if there is a new version.

HTTP GET /v1/resource/123456
If-None-Match: ABCDEF

And the server returns that the content has not been modified.

HTTP 1.1/304 Not Modified

If-Match

Here, the client wishes to modify a resource, but only if it has not changed since the client loaded it.

HTTP POST /v1/resource/123456
If-Match: ABCDEF

And the server indicates that the resource has changed.

HTTP 1.1/412 Precondition Failed

Requests using Last-Modified

The other way of managing staleness is to use the If-Modified-Since or If-Unmodified-Since headers. The largest criticism of this method, is that it is dependent on the server’s clock being accurate, and that the date was stored with an appropriate precision. Since it is likely that multiple chances can happen in the same second window, this method is less reliable than ETags.

If-Modified-Since

Here, a client wishes to read a resource, but only if there is a new version.

HTTP GET /v1/resource/123456
If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT

And the server returns that the content has not been modified

HTTP 1.1/304 Not Modified

If-Unmodified-Since

Here, the client wishes to modify a resource, but only if it has not changed since the client loaded it.

HTTP POST /v1/resource/123456
If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT

Server indicates that the resource has changed

HTTP 1.1/412 Precondition Failed

Collections

Both ETag and Last-Modified headers must be included on collection responses, and they should be derived from the returned page of results, rather than the entire result set. As an example, let us presume that a client is querying a collection of resources, but only if it has been modified:

HTTP POST /v1/resource/query
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
If-Modified-Since: Sat, 29 Oct 1994 19:43:31 GMT

{
  ...
  results: [
    ...
  ],
  ...
}

And the server indicates that the list is still fresh.

HTTP 1.1/304 Not Modified

3 - Content-Type Negotiation

Table stakes for this contract is that all APIs must support JSON as a request and response format. Additional formats are left to the discretion of the service. For example, a report service may want to return application/pdf, while some other services may prefer application/yaml.

Content-Type: application/json

When creating a request, there are two components to consider:

1. What format the request is sending.
2. What format the request wants to receive.

In all cases, the API must support requesting and receiving application/json, assuming a body is sent/expected. There are a variety of error cases that need to be outlined.

Requests for HTML

In all cases where the requested response type is Accepts: text/html, the resource server has two choices. Either, is to simply return a 415 Unsupported Media Type. This is the easiest response, but also the least helpful. The better choice is to ensure that your service also offers an API explorer style user interface, such as a Swagger-UI. In this case, the server should return a 303 See Other to this API explorer.

POST /namespace/v1/entity
Accepts: text/html


HTTP/1.1 303 See Other
Location: https://api.example.com

Request with unsupported body formats

If a request sends a body that is not supported, the server must respond with 415 Unsupported Media Type.

POST /v1/entity
Content-Type: text/unsupported


HTTP/1.1 415 Unsupported Media Type

Requests for unsupported media types

If a client asks for a response in a media type that is not supported, we return a 400.

Accepts: text/unsupported


HTTP/1.1 400 Bad Request

Requests for multiple media types

Accepts: text/html, application/xhtml+xml, application/xml;q=0.9, image/webp, */*;q=0.8

It is common for browsers to ask for several HTML-like MIME types in standard HTTP requests, though it’s less common for API requests. If a request such as this is received, the first most relevant media type that is supported by the API must be observed.

Example 1

This resolves to a Location header, as outlined above.

Accepts: text/html, application/json

Example 2

This resolves to application/json.

Accepts: text/plain, application/yaml, application/json;q=0.4

Requests for Files

In cases where a request is asking for a file response (such as a report), two approaches may be taken.

Approach 1: Build the file in JS

A UI can read JSON data and compose the format in memory. This is useful for small, ad-hoc reports and files, scripts that need to be prepopulated with customer data (such as onboarding scripts), and other things that require decoration with the current user context.

Approach 2: Redirect to an authorized GET URL

For larger files, or files that are generated and stored for later download, we must balance authorization requirements against default browser behavior. A browser’s default behavior to download when asked to load a PDF, for example, is too convenient to pass up, though building URI’s that include authorization information in their querystring is problematic.

As such, we require that larger files be downloaded via a two-step process. First, accessing the resource via the regular resource path and the desired Content-Type. The response must be a 302 Found, with a Location header containing such an authorized URL, with the issued authorization method being very short lived.

GET /v1/report/my-report-id
Accepts: application/pdf


HTTP/1.1 302 Found
Location: /v1/report/my-report-id.pdf?<some_short_lived_authorization_method>

We are specifically using 302 Found, because it instructs the browser to convert all requests to GET methods. Thus if an HTTP request asks for a report generation, the POST operation to create that report will - upon completion - be converted into a GET to download this report. For on-demand reports, please also review the standards around Long-Running Operations.

4 - Unique Identifiers

All unique identifiers in the system, that is to say resource ID’s, must be UUID’s or KSUID’s. This is to ensure that the identifiers are globally unique and can be generated without a central authority.

UUID

A UUID is a universally unique identifier, and is defined in RFC 4122. It is a 128-bit number, usually represented as a 32-character hexadecimal string. The UUID is generated using a pseudo-random number generator, and while it is not not guaranteed to be unique, but the probability of a collision is extremely low.

GET /v1/entity/123e4567-e89b-12d3-a456-426614174000

KSUID

A KSUID is a K-Sortable Unique Identifier, and was originally defined by the team at Segment.io. It is notably different from a UUID in that - in addition to being globally unique - it is also sortable by date. This is achieved by encoding the timestamp when the KSUID was generated in the first 4 bytes of the identifier.

These ID’s are most useful when trying to store time-series data with a minimum number of storage indexes, as even a filesystem will be able to sort these ID’s by name.

GET /v1/entity/0ujsszwN8NRY24YaXiTIE2VWDTS