Agrest 4.x Stable Protocol Documentation

Protocol Version 1.1

1. Overview

Agrest Protocol is a simple HTTP/JSON-based message protocol. It operates on an object model implicitly shared between a client and a server. It defines the format of JSON documents exchanged between client and server, and a set of control parameters that let the client control representation of the model returned from the server. E.g. the client may request a range of objects, sorted in a specific order, matching a criteria, with each object including a subset of attributes and related entities. This gives the client exactly what it needs, thus simplifying the code, minimizing the number of trips to the server, and optimizing the size of the response.

application/json is used in both requests and responses where applicable. Values of some control parameters below are also represented as JSON.

All examples below use an imaginary CMS data model that is made of 2 entities: Domain and Article, with a 1..N relationship between them:

2. JSON Documents

2.1. Simple Document

This document is used in responses that contain no data, just a boolean status and a message. On success, it might look like this:

HTTP/1.1 200 OK
Content-Type: application/json

{
   "success" : true,
   "message" : "all is good"
}

On failure, it might look like this:

HTTP/1.1 500 Server error
Content-Type: application/json

{
   "success" : false,
   "message" : "Database connection failure"
}

2.2. Collection Document

A document is used in responses that return the data from the server. This is the main representation of data in Agrest.

HTTP/1.1 200 OK
Content-Type: application/json

{
   "data" : [
      { "id" : 5, "name": "A" },
      { "id" : 8, "name": "B" }
   ],
   "total" : 2
}

"data" array contains entity objects. Implicit entity model defines what attributes and relationships (collectively - "properties") each object has. A subset of properties showing in the collection document is a defined by a combination of server-side constraints and client request control parameters. Each object in the data array may contain related objects, those in turn may contain their related objects, with no limit on the depth of nesting.

"total" is a number of objects one would see in the collection if there was no pagination. If pagination is in use (see Pagination with "start" / "limit"), the total may be greater than the number of visible objects in the "data" array. Otherwise it is equal to the size of "data".

2.3. Update Document

Update Document is sent in a request body from the client to the server to modify an entity collection. It is a Collection document stripped down to its "data" section. There are two flavors - a single object and an array of objects:

{ "id" : 5, "name": "X" }

[
   { "id" : 5, "name": "X" },
   { "id" : 8, "name": "Y" }
]

3. Dates and Times

JSON doesn’t have datatypes for either date or time. Both are represented as strings. Server and client must ensure that date/time strings are in ISO 8601 format E.g.:

2015-04-19T11:08:53Z
2015-04-10T11:08
2015-04-19

Developers should not assume that the server is in the same time zone as the browser. All timezone-aware expressions should contain time zone offset or "Z" suffix (for "Zulu" time).

4. Control Parameters

Control parameters, usually passed as URL keys, apply to the Collection Document and let the server to provide a single generic endpoint per entity, while still allowing the client to shape up the response Collection to its liking. These parameters are normally used with GET, however POST/PUT can also return a Collection Document, so many of the parameters are also applicable when modifying the data.

4.1. Filtering Collection with "exp"

"exp" key was first introduced in Agrest 4.1. Previously it was called "cayenneExp". Both are synonymous. "cayenneExp" is still supported, though should be considered deprecated.

A condition expression used to filter the response objects. Expression follow syntax of the Apache Cayenne expressions. An implicit "root" for the property paths is the request entity, (unless "exp" is used within an "include" JSON, in which case the root is that related entity).

Example 1: Filtering on a single property.

exp=vhost='agrest.io'

Example 2: Filtering using outer join (the "+" sign is notation for "outer").

exp=articles+ = null

Example 3: Filtering with parameters using positional bindings.

exp=["articles.body like $b","%Agrest%"]

Example 4: Filtering with parameters using named bindings.

exp={ "exp" : "articles.body like $b", "params":{"b":"%Agrest%"}}

4.2. Ordering Collection with "sort" / "dir"

Example 1: Sort on a single property.

sort=vhost

Example 2: Sort descending on a property.

sort=id&dir=DESC

dir can be one of ASC (default), DESC, ASC_CI (for case-insensitive asending ordering), DESC_CI (for case-insensitive descending ordering)

Example 3: Same as 2, but sort is a JSON object.

sort={"property":"vhost","direction":"DESC"}

"direction" takes the same values as dir above - ASC (implied default), DESC, ASC_CI, DESC_CI

Example 4: Multiple sortings as a single JSON structure.

sort=[{"property":"name"},"property":"vhost","direction":"DESC"}]

These two parameters are used together to request from the server a range of objects for a potentially huge collection. They allow to implement efficient data pagination on the client.

start is an offset within the "data" array. All the objects below this offset are discarded from the collection. Default "start" is 0.

limit is a maximum number of objects in the collection "data". Default is infinity (no limit).

"limit" is applied after "start". So for a collection with a total of 10 objects, ?start=2&limit=5 would result in objects 2..6 returned from the server. Returned Collection "total" would still be 10.

4.4. Structuring Collection with "mapBy"

Agrest presents the response as an array of entities [Response: Collection Document]. E.g. Request of articles returns the following array:

{
"data" : [
    { "title" : "Agrest mapBy",  "body" : "mapBy is used ..", "publishedOn" : "6 July, 2018" },
    { "title" : "Other Tech News",  "body" : "Java community ..", "publishedOn" : "8 October, 2017" },
    { "title" : "Introducing Agrest",  "body" : "Agrest is a ..", "publishedOn" : "6 July, 2018" }
  ],
  "total":3
}

Using mapBy this array can be transformed to a map. One of entity fields is used as the key of required map.

mapBy=publishedOn

{
"data" : {
    "8 October, 2017" : [
        { "title" : "Other Tech News",  "body" : "Java community …", "publishedOn" : "8 October, 2017" }
    ],
    "6 July, 2018" : [
        { "title" : "Agrest mapBy",  "body" : "mapBy is used …", "publishedOn" : "6 July, 2018" },
        { "title" : "Introducing Agrest",  "body" : "Agrest is a …", "publishedOn" : "6 July, 2018" }
    ]
  },
  "total" : 3
}

4.5. Shaping Collection with "include" / "exclude"

Model entities may have "simple" properties (attributes) and properties that point to related entities (relationships). By default Collection Document contains entity representation that includes its "id", all of its attributes, and none of the relationships. "include" and "exclude" parameters allow the client to request a specific subset of entity properties, including related entities. Some examples are given below, showing include/exclude parameters and resulting entity contents.

Example 1: Include default properties (all entity attributes) minus "vhost" attribute.

exclude=vhost

{ "id" : 45, "name" : "Agrest Site" }

Example 2: Exclude all properties, but "id".

include=id

{ "id" : 45 }

Example 3: Multiple includes, one of them points to attributes of related entity.

include=id&include=articles.title

{
   "id" : 45,
   "articles" : [
      { "title" : "Agrest Includes" },
      { "title" : "Other Tech News" },
      { "title" : "Introducing Agrest" }
   ]
}

Example 4: Advanced include. Include specification can itself be a JSON object and contain "exp", "sort", "start" and "limit" keys shaping up a collection of related objects for each root object.

include={"path":"articles","exp":"title like '%Agrest%'","sort":"title"}&include=articles.title

{
   "id" : 45,
   "articles" : [
      { "title" : "Introducing Agrest" },
      { "title" : "Agrest Includes" }
   ]
}

"exp" key was first introduced in Agrest 4.1. Previously it was called "cayenneExp". Both are synonymous. "cayenneExp" is still supported, though should be considered deprecated.

Example 5: Related objects as a map. Here we’ll map article bodies by title.

include={"path":"articles","mapBy":"title"}&include=articles.body

{
   "articles" : {
      "Introducing Agrest" : { "body" : "Agrest is a .." },
      "Agrest Includes" : { "body" : "Includes are .." }
   }
}

Example 6: Include and Exclude parameters have ability to take an array of values:

include=["id","name"]

{ "id" : 45, "name" : "Agrest Site" }

Example 7: The array can contain both the simple include and the advanced include values

include=["id","articles.title",{"path":"articles","exp":"title like '%Agrest%'"}]

{
   "id" : 45,
   "articles" : [
      { "title" : "Introducing Agrest" },
      { "title" : "Agrest Includes" }
   ]
}

Example 8: Attributes of a related entity can be presented as an inner array in JSON format:

include=["id","name",{"articles":["title","body"]}]

{
   "id" : 45,
   "name" : "Agrest Site",
   "articles" : [
      { "title" : "Introducing Agrest", "body" : "Agrest is a .." },
      { "title" : "Agrest Includes", "body" : "Includes are .." }
   ]
}

Example 9: The related entity can be specified as a path value:

include=["id","name",{"articles.categories":["id","name"]}]

Example 10: The advanced include can contain the array of include values:

include={"path":"articles","sort":"title","include":["title",{"categories":["id","name"]}]}

5. Protocol Extensions

Agrest ships with one such optional extension that adapts the framework for the use with Sencha/ExtJS JavaScript client. This extension is described below.

5.1. Sencha Adapter

Provides a few extensions to the Agrest protocol to better handle certain Sencha features:

If a to-one relationship property is included in the Collection, the framework would also generate a companion "synthetic" FK property called "propertyName_id"
"filter" key - an alternative to "exp".
"group" / "groupDir" keys that are functionally equivalent to "sort" / "dir".

Agrest Protocol