Agrest Workflow


1. Create a Simple Agrest App

1.1. Setting up the environment

In this chapter, we install (or check that you already have installed) a minimally needed set of software to build an Agrest-based application.

1.1.1. Install Java

JDK has to be installed. In this tutorial we use JDK 1.8.

1.1.2. Install IntelliJ IDEA

Download and install IntelliJ IDEA Community Edition. This tutorial is based on version 2016.3, still it should work with any recent IntelliJ IDEA version.

1.1.3. The resulting application

The final application could be downloaded from GitHub: Bookstore app

1.2. Starting a project

In this chapter, we create a new Java project in IntelliJ IDEA and introduce a simple Bookstore application that will be used as an example.

1.2.1. Define a Bookstore Domain Model

The application contains two types of entities: Category and Book. The relationship between Category and Book entities is one-to-many.

bookstore er diagram

1.2.2. Create a new Project in IntelliJ IDEA

In IntelliJ IDEA, select File > New > Project... Then select Maven and click Next.

In the dialog shown on the screenshot below, fill in the Group Id and Artifact Id fields and click Next.

tutorial idea project

During the next step, you will be able to customize the directory for your project. Click Finish where you are done. Now you should have a new empty project.

1.2.3. Configure Maven pom.xml

Add the following dependencies:

<dependency>
    <groupId>io.agrest</groupId>
    <artifactId>agrest</artifactId>
    <version>3.4-SNAPSHOT</version>
</dependency>

<dependency>
    <groupId>org.glassfish.jersey.containers</groupId>
    <artifactId>jersey-container-servlet-core</artifactId>
    <version>2.27</version>
</dependency>
<dependency>
    <groupId>org.glassfish.jersey.inject</groupId>
    <artifactId>jersey-hk2</artifactId>
    <version>2.27</version>
</dependency>

<dependency>
    <groupId>org.apache.derby</groupId>
    <artifactId>derby</artifactId>
    <version>10.13.1.1</version>
</dependency>

Configure a jetty Maven plugin to start app using mvn jetty:run command

<plugin>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>9.4.12.v20180830</version>
    <configuration>
        <scanIntervalSeconds>5</scanIntervalSeconds>
        <classesDirectory>${project.basedir}/target/classes</classesDirectory>
        <supportedPackagings><supportedPackaging>jar</supportedPackaging></supportedPackagings>
    </configuration>
</plugin>

1.3. Implementation

In this chapter, we implement a simple application to demonstrate Agrest features. The application uses Cayenne as an ORM framework and for further information regarding a DB mapping please, refer to Apache Cayenne ORM

1.3.1. Configure Cayenne

In the application’s resources folder, create a Cayenne project file:

cayenne-project.xml

<?xml version="1.0" encoding="utf-8"?>
<domain project-version="9">
    <map name="datamap"/>

    <node name="datanode"
          factory="org.apache.cayenne.configuration.server.XMLPoolingDataSourceFactory"
          schema-update-strategy="org.apache.cayenne.access.dbsync.CreateIfNoSchemaStrategy"
    >
        <map-ref name="datamap"/>
        <data-source>
            <driver value="org.apache.derby.jdbc.EmbeddedDriver"/>
            <url value="jdbc:derby:memory:testdb;create=true"/>
            <connectionPool min="1" max="1"/>
            <login/>
        </data-source>
    </node>
</domain>

In the same folder, add a file that contains a basic Cayenne mapping. The mapping is done based on the ER diagram from the Starting a project charter:

datamap.map.xml

<?xml version="1.0" encoding="utf-8"?>
<data-map xmlns="http://cayenne.apache.org/schema/9/modelMap"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://cayenne.apache.org/schema/9/modelMap https://cayenne.apache.org/schema/9/modelMap.xsd"
          project-version="9">
    <property name="defaultPackage" value="org.example.agrest.persistent"/>
    <db-entity name="BOOK">
        <db-attribute name="AUTHOR" type="VARCHAR" length="128"/>
        <db-attribute name="CATEGORY_ID" type="INTEGER"/>
        <db-attribute name="ID" type="INTEGER" isPrimaryKey="true" isMandatory="true"/>
        <db-attribute name="TITLE" type="VARCHAR" isMandatory="true" length="128"/>
    </db-entity>
    <db-entity name="CATEGORY">
        <db-attribute name="DESCRIPTION" type="NCLOB"/>
        <db-attribute name="ID" type="INTEGER" isPrimaryKey="true" isMandatory="true"/>
        <db-attribute name="NAME" type="VARCHAR" isMandatory="true" length="128"/>
    </db-entity>
    <obj-entity name="Book" className="org.example.agrest.persistent.Book" dbEntityName="BOOK">
        <obj-attribute name="author" type="java.lang.String" db-attribute-path="AUTHOR"/>
        <obj-attribute name="title" type="java.lang.String" db-attribute-path="TITLE"/>
    </obj-entity>
    <obj-entity name="Category" className="org.example.agrest.persistent.Category" dbEntityName="CATEGORY">
        <obj-attribute name="description" type="java.lang.String" db-attribute-path="DESCRIPTION"/>
        <obj-attribute name="name" type="java.lang.String" db-attribute-path="NAME"/>
    </obj-entity>
    <db-relationship name="category" source="BOOK" target="CATEGORY" toMany="false">
        <db-attribute-pair source="CATEGORY_ID" target="ID"/>
    </db-relationship>
    <db-relationship name="books" source="CATEGORY" target="BOOK" toMany="true">
        <db-attribute-pair source="ID" target="CATEGORY_ID"/>
    </db-relationship>
    <obj-relationship name="category" source="Book" target="Category" deleteRule="Nullify" db-relationship-path="category"/>
    <obj-relationship name="books" source="Category" target="Book" deleteRule="Deny" db-relationship-path="books"/>
</data-map>

1.3.2. Define domain models

Create two classes to present data objects in package org.example.agrest.persistent:

public class Category extends CayenneDataObject {

    public static final String ID_PK_COLUMN = "ID";

    public static final Property<String> DESCRIPTION = Property.create("description", String.class);
    public static final Property<String> NAME = Property.create("name", String.class);
    public static final Property<List<Book>> BOOKS = Property.create("books", List.class);
}
public class Book extends CayenneDataObject {

    public static final String ID_PK_COLUMN = "ID";

    public static final Property<String> AUTHOR = Property.create("author", String.class);
    public static final Property<String> TITLE = Property.create("title", String.class);
    public static final Property<Category> CATEGORY = Property.create("category", Category.class);
}

1.3.3. Implement Agrest application classes

Create an application and a resource class in package org.example.agrest:

@ApplicationPath("/api/*")
public class BookstoreApplication extends ResourceConfig {

    public BookstoreApplication() {

        ServerRuntime cayenneRuntime
                = ServerRuntime.builder()
                               .addConfig("cayenne-project.xml")
                               .build();

        AgRuntime agRuntime = AgBuilder.build(cayenneRuntime);
        super.register(agRuntime);

        packages("org.example.agrest");
    }
}
@Path("category")
@Produces(MediaType.APPLICATION_JSON)
public class CategoryResource {

    @Context
    private Configuration config;

    @POST
    public SimpleResponse create(String data) {
        return Ag.create(Category.class, config).sync(data);
    }

    @GET
    public DataResponse<Category> getAll(@Context UriInfo uriInfo) {
        return Ag.select(Category.class, config).uri(uriInfo).get();
    }
}

1.3.4. Configure web.xml

Provide a servlet configuration and a mapping based on the application class that you already created.

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         metadata-complete="false"
         version="3.1">

    <servlet>
        <servlet-name>BookstoreApp</servlet-name>
        <servlet-class>org.glassfish.jersey.servlet.ServletContainer</servlet-class>
        <init-param>
            <param-name>javax.ws.rs.Application</param-name>
            <param-value>org.example.agrest.BookstoreApplication</param-value>
        </init-param>
    </servlet>

    <servlet-mapping>
        <servlet-name>BookstoreApp</servlet-name>
        <url-pattern>/api/*</url-pattern>
    </servlet-mapping>

</web-app>

1.4. Running the Application

In this chapter, we run and test our application. After you have completed the above steps, the structure of your project will look like this:

bookstore ide prjct

1.4.1. Building and running

To build the application, use the mvn clean install command.

To run a jetty server with our application, use the mvn jetty:run command.

1.4.2. Configure IntelliJ IDEA Maven plugin

To run the application using IDE, add a new Maven configuration on a menu path Run → Edit Configurations…​. Then set a Name: (e.g. Bookstore) and a Command line: in jetty:run

1.4.3. Testing

After running the application, you can call this endpoint to get a list of categories:

curl -i -X GET 'http://localhost:8080/api/category'

And get the following response:

HTTP/1.1 200 OK
Date: Wed, 03 Oct 2018 10:14:51 GMT
Content-Type: application/json
Content-Length: 21
Server: Jetty(9.3.14.v20161028)

{"data":[],"total":0}

As you may see, the list is empty. So, use the 'POST' command to add some categories:

curl -i -X POST 'http://localhost:8080/api/category'  -d '{"id":"1","name":"Science Fiction"}'

Repeat the command, if necessary:

curl -i -X POST 'http://localhost:8080/api/category' -d '{"id":"2","name":"Horror"}'

The response will be:

HTTP/1.1 201 Created
Date: Wed, 03 Oct 2018 10:42:17 GMT
Content-Type: application/json
Content-Length: 16
Server: Jetty(9.3.14.v20161028)

{"success":true}

Now make the 'GET' request again and you will receive the following:

HTTP/1.1 200 OK
Date: Wed, 03 Oct 2018 10:44:44 GMT
Content-Type: application/json
Content-Length: 117
Server: Jetty(9.3.14.v20161028)

{"data":[{"id":1,"description":null,"name":"Science Fiction"},{"id":2,"description":null,"name":"Horror"}],"total":2}

2. Design-First Approach

2.1. Introduction

The API Design-First (or API-First) approach prescribes writing your API definition first as a contract before writing any code. This approach is more modern than the traditional Code-First approach. If main consumers of your API are third parties, partners or customers, the Design-First approach is the best choice. It allows you to provide good design for mission-critical APIs.

  • The contract that represents API specification is the best point for discussion. It can be visualized by using such tools as Swagger UI.

  • Based on API specification, you can even run a mock service. This way, developers and stakeholders will be able to preview and discuss the suggested design.

  • You can fix most high-level design issues before writing any code.

  • The final contract that is approved by all players tends to lead do better API.

  • API documentation and appropriated tests could be generated from the contract. This means you can have the application ready sooner.

In the beginning of developing an API, it is very important to understand the difference between Design-First, Code-First and DB-First approaches. And to help with it there are listed three principles you have to follow to reach all benefits of API Design-First approach:

  1. The API is the first user interface of the application

  2. The API comes first, than the implementation

  3. The API is self-descriptive

2.2. Setting up the environment

In this chapter, we check that you already have installed an Agrest-based application that can be used to demonstrate the Design-First approach.

2.2.1. Prerequisites

Set up and build an application example from previous chapter Create a Simple Agrest App. You can either follow a step-by-step process to create an application from scratch or get a ready-made application from GitHub Bookstore app

2.2.2. Prepare the Bookstore application

As we described above, the Design-First approach means the API specification comes first, and the code comes second.

So, we have to remove from the application classes that defined API resources. In our case it is CategoryResource.java we have created manually. Later all our API resources will be generated from .yaml definition automatically according to the Design-First approach.

Then update the pom.xml.

Add the openapi-generator-maven-plugin plugin and appropriate settings. For more details, please refer tothe next section Configure and run API generation

<plugin>
    <groupId>org.openapitools</groupId>
    <artifactId>openapi-generator-maven-plugin</artifactId>
    <version>3.0.2</version>
    <executions>
        <execution>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <inputSpec>${project.basedir}/src/main/resources/bookstore-api.yaml</inputSpec>
                <generatorName>io.swagger.codegen.languages.AgServerCodegen</generatorName>
                <output>${project.basedir}</output>
                <apiPackage>org.example.agrest</apiPackage>
                <modelPackage>org.example.agrest.persistent</modelPackage>
                <invokerPackage>org.example.agrest</invokerPackage>
                <generateModels>false</generateModels>
                <skipOverwrite>false</skipOverwrite>
            </configuration>
        </execution>
    </executions>

    <dependencies>
        <dependency>
            <groupId>io.agrest.openapi</groupId>
            <artifactId>agrest-openapi-designfirst</artifactId>
            <version>3.4-SNAPSHOT</version>
        </dependency>
    </dependencies>
</plugin>

2.2.3. The resulting application

The final application that implements Design-First approach is available on GitHub at: Bookstore Design-first app

2.3. Defining the API

Agrest provides the AgServerCodegen class and a set of custom Mustache templates to generate an API implementation based on OpenAPI 3.0 specification.

The top-down workflow for creating the API is as follows:

2.3.1. 1. Start with domain models

Create a bookstore-api.yaml file to define your API and put it in the src/main/java/resources folder. Add general information regarding your API:

openapi: 3.0.0
servers:
  - url: 'http://127.0.0.1/v1'
info:
  title: Agrest-based API of Bookstore
  description: An API for interacting with the Bookstore backend server
  version: v1

Then add definition of your models. If you want to create an updated API (e.g. POST, PUT) of your model, you have to define a 'requestBodies' element in addition to a 'schemas' element.

Please make sure that you can either specify existing Java-DB mapping classes (based on CayenneDataObject e.g. our Category and Book) or generate simple POJO models by Maven plugin.

For further information, please refer to Configure and run API generation section.

But in either case you have to define models in the .yaml file:

tags:
  - name: Category
    description: |
      This model represents a Category type and is used to retrieve, create and update a book Category information.

components:
  schemas:
    Category:
      type: object
      properties:
       id:
         type: string
         description: Unique ID of Category
         example: 1
       name:
         type: string
         description: Book Category name
         example: Science Fiction
       description:
         type: string
         description: Description of Category
         example: Science fiction (often shortened to Sci-Fi or SF) is a genre of speculative fiction.

  requestBodies:
    Category:
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Category'
      description: Category object that needs to be created or updated
      required: true

2.3.2. 2. Attach Agrest protocol definition

The Agrest protocol file protocol.yaml contains definition of all Control Parameters. Just place this protocol.yaml in the catalog were your main bookstore-api.yaml file is located (e.g. 'src/main/java/resources').

2.3.3. 3. Define resources

Add REST API resource definition to your bookstore-api.yaml file. Make sure that Agrest protocol parameters are defined as references.

paths:
  /category:
    get:
      summary: Get list of all Book Categories
      operationId: getAll
      tags:
        - Category
      parameters:
        - $ref: '../resources/protocol.yaml#/components/queryParams/Limit'
        - $ref: '../resources/protocol.yaml#/components/queryParams/Start'
      responses:
        '200':
          description: Success response.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Category'
        default:
          description: Unexpected error
    post:
      tags:
        - Category
      summary: Create a new Book Category
      operationId: create
      requestBody:
        $ref: "#/components/requestBodies/Category"
      responses:
        default:
          description: successful operation

  /category/{id}:
    get:
      description: Returns a particular Book Category
      operationId: getOne
      tags:
        - Category
      parameters:
        - name: id
          in: path
          description: ID of Category to fetch
          required: true
          schema:
            type: integer
            format: int32
      responses:
        '200':
          description: Success responce
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Category'
          description: Unexpected error

2.3.4. 4. Run build-time API generation

To generate an Agrest-based API, a special Maven plugin is used. This plugin should be configured in accordance with your .yaml files location [inputSpec], packages [apiPackage], output catalog [output], etc.

mvn clean install runs generation of the API.

For more details, please refer to the Configure and run API generation section

2.4. Validate the API implementation

After it has been successfully generated, CategoryResource.java could be found in the output catalog. This class has a ready-to-use implementation (not a stub) of all methods defined in the .yaml file.

@Path("/")
public class CategoryResource {

    @Context
    private Configuration config;

    @POST
    @Path("/v1/category")
    @Consumes({ "application/json" })
    public DataResponse<Category> create(String category) {
        AgRequest agRequest = Ag.request(config)
                .build();

        return Ag.create(Category.class, config)
                 .request(agRequest)
                 .syncAndSelect(category);
    }

    @GET
    @Path("/v1/category")
    @Produces({ "application/json" })
    public DataResponse<Category> getAll(@QueryParam("limit") Integer limit, @QueryParam("start") Integer start) {
        AgRequest agRequest = Ag.request(config)
                .limit(limit)
                .start(start)
                .build();

        return Ag.select(Category.class, config)
                 .request(agRequest)
                 .get();
    }

    @GET
    @Path("/v1/category/{id}")
    @Produces({ "application/json" })
        public DataResponse<Category> getOne(@PathParam("id") Integer id) {
        AgRequest agRequest = Ag.request(config)
                .build();

        return Ag.select(Category.class, config)
                 .byId(id)
                 .request(agRequest)
                 .get();
    }
}

If you configure Maven plugin to generate models [generateModels], the POJO Category.java will be generated.

public class Category   {

    private Integer id = null;
    private String name = null;
    private String description = null;

...
    /**
     * Unique ID of Category
     * @return id
     **/
    @AgAttribute
    @ApiModelProperty(example = "1", value = "Unique ID of Category")
    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

...
    /**
     * Book Category name
     * @return name
     **/
    @AgAttribute
    @ApiModelProperty(example = "Science Fiction", value = "Book Category name")
    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

2.5. Configure and run API generation

There is an example of the Maven plugin configuration:

<plugin>
    <groupId>org.openapitools</groupId>
    <artifactId>openapi-generator-maven-plugin</artifactId>
    <version>3.0.2</version>
    <executions>
        <execution>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <inputSpec>${project.basedir}/src/main/resources/bookstore-api.yaml</inputSpec>
                <generatorName>io.swagger.codegen.languages.AgServerCodegen</generatorName>
                <output>${project.basedir}</output>
                <apiPackage>org.example.agrest</apiPackage>
                <modelPackage>org.example.agrest.persistent</modelPackage>
                <invokerPackage>org.example.agrest</invokerPackage>
                <generateModels>false</generateModels>
                <skipOverwrite>false</skipOverwrite>
            </configuration>
        </execution>
    </executions>

    <dependencies>
        <dependency>
            <groupId>io.agrest.openapi</groupId>
            <artifactId>agrest-openapi-designfirst</artifactId>
            <version>3.4-SNAPSHOT</version>
        </dependency>
    </dependencies>
</plugin>

<inputSpec>

Points to your API definition in .yaml or .json formats.

<generatorName>

Sets the Agrest custom code generator.

<output>

Sets the output catalog for all generated items.

<apiPackage>

Contains full package name of resource implementation classes to be generated.

<modelPackage>

Contains full package name of model classes. If [generateModels] is set to true, the POJO stubs of models will be generated. Otherwise, existing model classes from this package will be used.

<generateModels>

Generates POJO of models or uses existing ones.

<skipOverwrite>

If it is set to false, all generated files will be overwritten each time during mvn clean install. So, if you are planning to customize the generated API implementation, this parameter should be set to true.

2.5.1. Run the Application

As we mentioned in the chapter Building and running the Application is run by command mvn jetty:run. After the Jetty server starts, the following curl commands can be used for the API testing:

curl -i -X GET 'http://localhost:8080/api/v1/category'
curl -i -X POST -H 'Content-Type: application/json' 'http://localhost:8080/api/v1/category'  -d '{"id":"1","name":"Science Fiction"}'

Please, pay attention that the POST command has to contain the Content-Type parameter according to the annotation of the 'create' method of the CategoryResource class.

2.6. More examples with integration tests

Agrest has more Examples of Design-First approach implementation that are provided as separate module with set of tests. To run it use mvn clean install command. The API description (contract) is defined in file src/test/java/resources/api.yaml.

All generated APIs will be located on src/test/java together with corresponding integration tests.

During the build time the API implementation is generated by Maven plugin and then this implementation is checked by integration tests. We use testing fixtures from agrest module as models.

2.7. protocol.yaml

components:
  queryParams:

    CayenneExp:
      name: cayenneExp
      in: query
      style: form
      explode: false
      schema:
        type: object
        properties:
          exp:
            type: string
            description: A conditional expression that is used to filter the response objects
            example: articles.body like $b
          params:
            type: object
            additionalProperties:
              type: string
      description: cayenneExp query
      required: false

    Dir:
      name: dir
      in: query
      style: form
      explode: false
      schema:
        type: string
        enum:
          - ASC
          - DESC
      description: sorting direction
      required: false

    Excludes:
      name: exclude
      in: query
      style: form
      explode: false
      schema:
        type: array
        items:
          $ref: '#/components/queryParams/Exclude'
      description: list of excludes
      required: false

    Exclude:
      name: exclude
      in: query
      schema:
        type: object
        properties:
          path:
            type: string
          excludes:
            type: array
            items:
              $ref: '#/components/queryParams/Exclude'
      description: An exclude parameter
      required: false

    Includes:
      name: include
      in: query
      style: form
      explode: false
      schema:
        type: array
        items:
          $ref: '#/components/queryParams/Include'
      description: list of includes
      required: false

    Include:
      name: include
      in: query
      schema:
        type: object
        properties:
          value:
            type: string
          cayenneExp:
            $ref: '#/components/queryParams/CayenneExp'
          sort:
            $ref: '#/components/queryParams/Sort'
          mapBy:
            $ref: '#/components/queryParams/MapBy'
          path:
            type: string
          start:
            $ref: '#/components/queryParams/Start'
          limit:
            $ref: '#/components/queryParams/Limit'
          includes:
            type: array
            items:
              $ref: '#/components/queryParams/Include'
      description: An include parameter
      required: false

    Limit:
      name: limit
      in: query
      style: form
      explode: false
      schema:
        type: object
        properties:
          value:
            type: integer
            format: int32
            description:
      description: limit query param. Used for pagination.
      required: false

    Start:
      name: start
      in: query
      style: form
      explode: false
      schema:
        type: object
        properties:
          value:
            type: integer
            format: int32
            description:
      description: start query param. Used for pagination.
      required: false

    MapBy:
      name: mapBy
      in: query
      style: form
      explode: false
      schema:
        type: object
        properties:
          path:
            type: string
            description:
      description:
      required: false

    Sort:
      name: sort
      in: query
      style: form
      explode: false
      schema:
        type: object
        properties:
          property:
            type: string
            description:
          direction:
            type: object
            $ref: '#/components/queryParams/Dir'
          sorts:
            type: array
            items:
              $ref: '#/components/queryParams/Sort'
      description: sort
      required: false

3. Code-First Approach

3.1. Introduction

The Code-First approach is useful mainly in Domain Driven Design. In the Code-First approach, you focus on the domain of your application and start creating classes for your domain entity rather than creating your database (DB-First) or creating your API (Design-First) first. And only after you can create domain classes, a DB structure and an appropriate API specification will be created.

With regards to creating API specification, it means that the specification can be automatically generated from the class sources. This generation has to use a classes meta information that can be provided using annotations, for example.

Agrest provides the following annotations:

@AgAttribute

@AgId

@AgRelationship

@AgResource

If you specify your models and resources using these annotations, the Agrest Maven plugin will generate an API specification in the openapi v.3.0 format.