Protocol Version 1.2
1. Overview
Unlike the rest of the Agrest documentation that talks about the server framework, this document should be of interest not only to the server-side developers, but to the client-side devs as well, as it demonstrates how to access Agrest services.
"Agrest Protocol" is based on HTTP and JSON and defines how a client can communicate with Agrest endpoints. It consists of two things:
-
A format of JSON documents used in requests and responses. It can work with any application-specific data model.
-
A set of control parameters for the client to request a specific representation of the model from the server (i.e. the client can control sorting, filtering, offsets and included properties of the response objects). An endpoint may support all or a subset of these parameters, depending on its intention and capabilities.
As we’ll see below, the protocol defines some powerful features that give a lot of flexibility to the client to decide what data is retrieved from the service and how it should look like. The fact that every Agrest service looks the same regardless of a specific data model simplifies the implementation of the client. And the server implementation is assisted by the framework.
1.1. Data Model
The examples in this document are using a simple "bookstore" model. Here it is defined as pseudocode:
Author
- id : long
- name : string
- dateOfBirth : date
- books : list<Book>
Book
- id : long
- title : string
- genre: string
- author : Author
As you see, the model consists of two entities (Author and Book), each with an "id" property, some value properties
(strings, dates, etc.) and related entity properties (list<Book>
, Author
). id
, attribute
and relationship
are
the terms used throughout Agrest to distinguish between these types of properties.
2. Protocol Conventions
As things often are in the REST world, Agrest protocol is quite loose. Many elements are conventions rather than hard requirements, but following them would produce consistent, easy-to-understand APIs. Here are some common conventions for writing Agrest endpoints, some reflected in the default server framework, others left to the developer to follow:
-
Content type of GET, POST, PUT responses serving message documents or collection documents is
application/json
-
Content type of POST and PUT requests consuming update documents is
application/json
-
Control parameters are passed as URL query parameters (e.g.
/my?include=name
) -
Values of control parameters may contain spaces, special chars and small chunks of JSON. So they need to be URL-encoded
-
Individual object ids are passed as URL path components (e.g.
/my/5
) -
Standard HTTP status codes are used to convey either a success or a failure of a request
-
Dates and times in requests and responses are in ISO 8601 format. E.g.
2022-04-19T11:08:53Z
,2023-04-10T11:08
,1979-04-19
3. JSON Documents
3.1. Message Response
This document is used in response to POST, PUT or DELETE when no entity data is returned to the client.
{
"message" : "..."
}
|
string |
An optional human-readable message, usually provided to explain reasons of a failure. |
3.2. Collection Response
This document is used in response to GET, POST or PUT to return data from the server. This is the format used to return any kind of data to the client.
{
"data" : [
{ "id" : 5, "name": "A" },
{ "id" : 8, "name": "B" }
],
"total" : 2
}
|
list or object |
A list (or sometimes, an object) that contains entity objects. The schema of the objects is implicit, and depends on the endpoint model and request control parameters. Each object may contain other objects, with an arbitrary number of nesting levels. |
|
int |
The total number of objects in the collection. This is usually equal to the size of the |
3.3. Update Request
This document is used as a body of a POST or PUT request to pass the data from the client to the server to modify an entity collection. It is either a single object or a collection of objects (so pure custom model, no Agrest-specific properties) :
{ "id" : 5, "name": "X" }
[
{ "id" : 5, "name": "X" },
{ "id" : 8, "name": "Y" }
]
While collection response documents can contain object trees with an unlimited depth (entities with their related entities, with their own related entities and so on), update documents can only have properties of a single entity. It can still have references to the related entities as described below, but no entity nesting. |
4. Control Parameters
Agrest protocol control parameters are exp
, sort
, dir
, start
, limit
, mapBy
, include
, exclude
. They are
passed as URL query parameters (e.g. my?include=name
) with the goal to shape the Collection Response document.
With their help, the client can decide how to sort, filter, paginate the objects in the data
collection, and which
object properties to include.
These parameters are applied to GET
and often to POST
and PUT
requests (when those latter ones return
Collection Response).
Parameter values can be either simple strings/numbers or short JSON structures. JSON structures may require proper URL encoding to be done by the client.
4.1. exp
exp
is used to filter objects in the response collection. It can take one of the three forms, depending on the
parameter binding style:
exp=<exp_string> (1)
exp=["<exp_string>", "value1", "value2"] (2)
exp={"exp"="<exp_string>", "params":{ "param1" : "value1", "param2", "value2"}} (3)
1 | Without parameters, an expression it is just a string (we’ll explain what exp_string is shortly) |
2 | An expression with parameter resolution by position is represented as a JSON list, where the expression string is the first element, and all subsequent elements are positional parameters |
3 | An expression with parameter resolution by name is represented as a JSON object with "exp" and "params" properties |
exp_string
is a conditional expression based on Agrest own "expression DSL", that should be intuitively understood
by most programmers. Here are some examples:
name='Ernest Hemingway'
books.title = 'A Farewell to Arms'
title = not in ('A Farewell to Arms', 'For Whom the Bell Tolls')
title like 'A%' and author.dateOfBirth > $afterDate
TODO: A more comprehensive Agrest exp format specification is forthcoming. Until then, please check Apache Cayenne expressions that use practically the same syntax (except Agrest would not allow "db:" expressions). And see the examples below
A few things to note about the examples above:
-
Expressions use dot-separated paths to refer to the values of object properties
-
There is an implicit "root entity" to which the paths are applied. It is either the main entity of the request or, when an expression is used inside an
include
, a related entity. Soname
andbooks.title
would work withAuthor
, whiletitle
andauthor.dateOfBirth
- withBook
. -
Parameters always start with a "$" sign, regardless of whether they are resolved by name or by position
Here are a few examples using an expression parameter. URL encoding is not shown for clarity:
exp=name='Ernest Hemingway'
exp=name like 'E%'
exp=name likeIgnoreCase '%ab%'
exp=books+ = null
The "+" sign denotes "left join" syntax. Comparing to null returns all authors who have no books in the system.
exp=["author.dateOfBirth > $afterDate","1900-01-01"]
exp={"exp":"author.dateOfBirth > $afterDate", "params":{"afterDate":"1900-01-01" }}
4.2. sort / direction
sort
and direction
are used to order objects in the response collection. The following forms are supported:
sort=<prop_path> (1)
sort=<prop_path>&direction=<direction_spec> (2)
sort=<json_sort_object> (3)
sort=[<json_sort_object>,...] (4)
1 | Ascending sort by a single property |
2 | Sort by a single property with explicit direction |
3 | Sort by a single property with a sort object |
4 | Sort by multiple properties with a list of sort objects |
direction
is added to specify sort direction when sorting is done on a single property. It can have one of these values
(specified in lower or upper case) :
-
asc
: this is also the default, when nodirection
is provided -
desc
-
asc_ci
: for case-insensitive ascending ordering -
desc_ci
: for case-insensitive descending ordering
<json_sort_object>
allows to specify both the property and the direction in one place and has a form of
{"path": .., "direction": .. }
. direction
is optional and takes the same values as direction
. Let’s demonstrate
all the flavors with a few examples:
sort=name
sort=id&direction=desc
sort={"path":"id","direction":"desc"}
sort=[{"path":"name"},{"path":"dateOfBirth","direction":"desc"}]
4.3. start / limit
start
and limit
parameters are used for client-controlled pagination of a response collection. They can be used
together or individually to request a range of objects in a collection.
start
is an offset within the "data" array. All the objects below this offset are discarded from the response
collection. Default start
is 0.
limit
is a maximum number of objects in the collection "data". Default is no limit.
limit
is applied after start
. So for a collection of 10 objects, start=2&limit=5
would result in objects 2
through 6 returned from the server. Returned Collection "total" would still be 10.
4.4. mapBy
mapBy
parameter is used to reshape the data
collection from a list of objects to a map of lists, keyed by some
property. E.g. consider a typical list response:
{
"data" : [
{ "id" : 8, "title" : "One Hundred Years of Solitude", "genre" : "fiction" },
{ "id" : 5, "title" : "Battle Cry of Freedom", "genre" : "history" },
{ "id" : 12, "title" : "For Whom the Bell Tolls", "genre" : "fiction" }
],
"total":3
}
With mapBy=genre
it is transformed to a map. The total here is still the number of objects in all the maps
combined:
{
"data" : {
"fiction" : [
{ "id" : 8, "title" : "One Hundred Years of Solitude", "genre" : "fiction" },
{ "id" : 12, "title" : "For Whom the Bell Tolls", "genre" : "fiction" }
],
"history" : [
{ "id" : 5, "title" : "Battle Cry of Freedom", "genre" : "history" }
]
},
"total" : 3
}
4.5. include / exclude
include
and exclude
are used to recursively shape individual objects in a response collection. These are the
controls that turn your REST endpoints fixed models into graphs that can be dynamically navigated by the clients.
exclude
format:
exclude=<prop_path> (1)
exclude=[<prop_path>,...] (2)
1 | A single property path |
2 | A JSON array of property paths |
include
format:
include=<prop_path> (1)
include=<json_include_object> (2)
include=[<prop_path_or_json_include_object>,...] (3)
1 | A single property path |
2 | A JSON include object |
3 | A JSON array of property paths and include objects |
<json_include_object>
has the following structure:
{
"path": .. , // the only required property
"exp": .. ,
"sort": .. ,
"start": ..,
"limit": ..
"mapBy": ..
"include": ...
}
The only required property is path
that determines which property is included. If the path
points to a relationship,
the object can contain properties corresponding to all the individual controls we’ve seen already (even a nested include
!).
Those controls are applied to the related entity denoted by the path
.
A few more notes before we show the examples:
-
What is included by default? As we’ve discussed above, Agrest model entities consist of id, attribute and relationship properties. If no includes are specified, Collection Response document would contain the id and all the attributes of a given entity, and none of the relationships.
-
Multiple
include
andexclude
parameters can be used in a single request. They will be combined together.
Now let’s see the examples:
In the examples below we will omit the {"data":[..],"total": .. } collection document wrapper, and will only
show the structure of a single individual object within the "data" collection.
|
exclude=genre
{ "id" : 8, "title" : "One Hundred Years of Solitude" }
include=id
{ "id" : 8 }
include=id&include=author.name
{ "id" : 8, "author" : {"name" : "Gabriel García Márquez"} }
include={"path":"books","exp":"title like '%a%'","sort":"title", "include":"title"}
{
"books" : [
{ "title" : "Autumn of the Patriarch" },
{ "title" : "One Hundred Years of Solitude" }
]
}
include={"path":"books","mapBy":"genre", "include":"id"}
{
"books" : {
"fiction" : [
{ "id" : 55 },
{ "id" : 8 }
]
}
}
include=["id","name"]
{ "id" : 45, "name" : "Gabriel García Márquez"}
include=["id","books.title",{"path":"books","exp":"title like %a%'"}]
{
"id" : 45,
"books" : [
{ "title" : "Autumn of the Patriarch" },
{ "title" : "One Hundred Years of Solitude" }
]
}
include=["id",{"books":["id", "title"]}]
{
"id" : 45,
"books" : [
{ "id" : 55, "title" : "Autumn of the Patriarch" },
{ "id" : 8, "title" : "One Hundred Years of Solitude" }
]
}
In this example attributes of both the root entity and the related entity are specified as JSON arrays. Also, there is
a shortcut - instead of {"path":"books","include":[..]}}
, we are using {"books":[..]}
.