Beginning Serverless Journey

Serverless is a paradigm which lays its foundations on the fact that "We don't have to provision and manage servers".

This article series explores various aspects involved in a serverless application lifecycle including - development, testing and deployment.

Our serverless journey which starts from building to deploying an application will be using multiple serverless components including AWS Lambda, AWS API Gateway, AWS DynamoDB, LocalStack and AWS CDK.

Let's deep dive step by step into what it takes to build a Serverless application.

Building a Serverless application

Let's assume a hypothetical "Order Service" that allows creation of an order and its retrieval by id.

As a part of this article we will be building just one part of this service which will expose a REST API to allow users to "find an order" by "orderId". Below diagram highlights different AWS components involved in finding an order by its id.

We will be using TypeScript for writing our Serverless application. Why Typescript? For a few reasons -

  • Small cold start time
  • Supports static typing and type inference
  • Existing Javascript libraries can be used with Typescript

So, let's start building our application.

Step 1: Setting up the project

Let's quickly setup our project -

  • Install typescript globally by executing npm install typescript -g
  • Create a directory named serverless-order-service representing our project
  • Execute npm init -y inside our project directory (serverless-order-service)
  • Add typescript as a dependency by executing npm install typescript --save inside our project directory

As a final step, add tsconfig.json with a very simple configuration as mentioned below -

{
  "compilerOptions": {
    "noEmitOnError": true,
    "moduleResolution": "node",
    "module": "commonjs",
    "target": "es6",
    "outDir": "dist",
    "inlineSourceMap": true
  }
}

That should be it. Our project set up is done and we are ready to make our first commit.

Before we start coding

Let's take a moment to think about the overall design of the project.

This application is a classic web application which involves a REST API, a database and an object representing the persistent state of order. With this very small context, I feel it would not be unfair to organise the project in Model-View-Controller fashion which means execution of a user request will involve the following components -

Step 2: Let's start with lambda handler

Lambda handler is a function which will be invoked by AWS Lambda Service in response to an event. An event could be - an object uploaded on an S3 bucket, an event on SQS or an https request via API gateway and many more. In our example a request to an API Gateway will be an event.

Before we start our lambda function let's install type definition for aws-lambda by executing - npm install @types/aws-lambda --save-dev and create a commit.

After the dependency is installed we are ready to code our handler. Let's put this in a file named handler.ts under src directory.

import {APIGatewayEvent} from "aws-lambda";
export const ordersHandler = async (event: APIGatewayEvent): Promise<any> => {
    //your code goes here
};

This is the simplest handler function that could be created at this stage. One thing to note is we are using the type APIGatewayEvent imported from "aws-lambda" to get type-safe events as parameter to handler function.

We want to keep our handler function as thin as possible so we will delegate the request to a controller class which instead of taking APIGatewayEvent will take a domain object that wraps APIGatewayEvent.

In this example, OrderRequest is that domain object, effectively a wrapper over APIGatewayEvent.

import {APIGatewayEvent} from "aws-lambda";

export const ordersHandler = async (event: APIGatewayEvent): Promise<any> => {
    return new OrderController().handle(new OrderRequest(event)); //handler invokes controller
};

export class OrderController {
    handle(orderRequest: OrderRequest) { //accepts OrderRequest
    }
}

//domain object which wraps APIGatewayEvent
export class OrderRequest {
    constructor(private readonly event: APIGatewayEvent) {
    }
}

Let's move OrderController and OrderRequest classes to controller and model packages (or directories) respectively and invoke OrderController from handler. This is how the handler function will look like after the classes have been moved.

import {APIGatewayEvent} from "aws-lambda";
import {OrderRequest}    from "./model/OrderRequest";
import {OrderController} from "./controller/OrderController";

export const ordersHandler = async (event: APIGatewayEvent): Promise<any> => {
    return new OrderController().handle(new OrderRequest(event));
};

I think we are ready to make our next commit. I know we haven't written any unit tests but as far as this article is concerned, we will write unit tests before we start the next one.

Step 3: Let's jump into controller

Controller is a class which will handle the incoming request and return an appropriate response. By this definition, controller will be expected to accept an OrderRequest and find an order by its id.

import {OrderRequest} from "../model/OrderRequest";

export class OrderController {

    handle(orderRequest: OrderRequest) {
       if (orderRequest.isAGetOrder()) { //if the request is for finding an order
         return this.findAnOrderBy(orderRequest.orderId()); //find an order by its id
       }
       return null;
    }

    private findAnOrderBy(id: string): Order {
        return null; //fake implementation
    }
}

export class Order {
}

Few quick observations -

  • OrderRequest is the domain object which encapsulates APIGatewayEvent and provides domain methods like orderId(), isAGetOrder() without exposing APIGatewayEvent
  • Currently orderId() and isAGetOrder() methods of OrderRequest return fixed (or fake) values

Let's make a few quick changes in OrderController -

  • Move Order class into model package
  • Invoke Service method to find an order by its id

This is how different classes look at this stage.

//OrderController.ts
import {OrderRequest} from "../model/OrderRequest";
import {Order}        from "../model/Order";
import {OrderService} from "../service/OrderService";

export class OrderController {
    private orderService: OrderService;

    constructor() {
        this.orderService = new OrderService();
    }

    handle(orderRequest: OrderRequest) {
       if (orderRequest.isAGetOrder()) {
         return this.findAnOrderBy(orderRequest.orderId())
       }
       return null;
    }

    private findAnOrderBy(id: string): Order {
        return this.orderService.findAnOrderBy(id); //controller invokes service to find an order by its id
    }
}

//OrderRequest.ts
import {APIGatewayEvent} from "aws-lambda";

export class OrderRequest {
    constructor(private readonly event: APIGatewayEvent) {
    }

    isAGetOrder(): boolean {
       return true; //fake implementation
    }
    orderId(): string {
        return ""; //fake implementation
    }
}

//OrderService.ts
export class OrderService {

    findAnOrderBy(id: string) {
        return null; //fake implementation
    }
}

Step 4: Let's jump into service

Service layer will interact with repository to find an order by id. It doesn't look like service layer is really needed for this example, but let's proceed with it -

export class OrderService {
    private repository: OrderRepository;

    constructor() {
        this.repository = new OrderRepository();
    }

    findAnOrderBy(id: string) {
        return this.repository.findAnOrderBy(id); //service invokes repository to find an order by its id
    }
}

export class OrderRepository {
    findAnOrderBy(id: string) {
        return null; //fake implementation
    }
}

Let's move OrderRepository into repository package.

Step 5: Let's jump into repository

Repository will interact with our database which in this example is a DynamoDB and fetch an order by its id. Let's assume a table named "orders" with "orderId" as the HASH key and an attribute named "amount".

We will be using "aws-sdk" for querying DynamoDB. "aws-sdk" is a dependency which is available in the runtime environment of lambda which means this dependency can be added as a "devDependency".

So, let's add it as a "devDependency" by executing npm install aws-sdk --save-dev. Let's also add type definitions for aws-sdk by executing npm install @types/aws-sdk --save-dev.

Now we are ready to query "orders" table.

import {DynamoDB} from "aws-sdk"
import {GetItemInput} from "aws-sdk/clients/dynamodb";
import {Order} from "../model/Order";

const dynamoDb = new DynamoDB({
    "region": "ap-south-1"
});

export class OrderRepository {

    async findAnOrderBy(id: string) {
        const getItemInputOptions: GetItemInput = {
            TableName: "orders", //table name
            Key: {
                "orderId": {S: id} //query against orderId attribute of order item
            }
        };
        const response = await dynamoDb.getItem(getItemInputOptions).promise(); //get a dynamo item by passing its id
        return response.Item;
    }
}

Few quick observations -

  • We have hard-coded the region and table name, which we might want to fetch from configuration / properties for below mentioned reasons -
    • if region is different for actual deployment and integration testing (using localstack)
    • if there are multiple deployment environments and dynamo table name is different for each environment
      • With DynamoDB lowest level of abstraction is a table, hence, we might need different table name for each environment
  • Method name and return type of the method do not go hand-in-hand. We expect this method to return an "order" but this method seems to be returning some type specified by "response.Item"

Let's quickly make a change to return Order instead of response.Item.

import {DynamoDB} from "aws-sdk"
import {GetItemInput} from "aws-sdk/clients/dynamodb";
import {Order} from "../model/Order";

const dynamoDb = new DynamoDB({
    "region": "ap-south-1"
});

export class OrderRepository {

    async findAnOrderBy(id: string) {
        const getItemInputOptions: GetItemInput = {
            TableName: "orders", //table name
            Key: {
                "orderId": {S: id} //query against orderId attribute of order item
            }
        };
        const response = await dynamoDb.getItem(getItemInputOptions).promise(); //get a dynamo item by passing its id
        return response.Item ? Order.from(response.Item) : null;  //map dynamo item to Order
    }
}

and this is how Order.ts looks like -

import {DocumentClient} from "aws-sdk/clients/dynamodb";

export class Order {
    static from(item: DocumentClient.AttributeMap): Order {
        return null; //fake implementation
    }
}

This completes our repository. We still have some open items. Let's take them one by one -

  • Pending implementation of from() in Order
  • Pending implementation of orderId() in OrderRequest
  • Pending changes relating to async/await in controller, service and handler

Step 6: Finishing Order class

Order class provides a static method which accepts and instance of DocumentClient.AttributeMap and returns an instance of Order consisting of orderId and amount.

import {DocumentClient} from "aws-sdk/clients/dynamodb";

export class Order {  

    static from(item: DocumentClient.AttributeMap): Order {
        return new Order(item.orderId.S, Number(item.amount.N)); //create an instance of Order from an instance of AttributeMap
    }
    private constructor(private readonly orderId: string,
                        private readonly amount: number) {
    }
}

AttributeMap within aws-sdk is defined as -

export type AttributeMap = {[key: string]: AttributeValue};

and AttributeValue is an interface which is defined as -

export interface AttributeValue {
    S?: StringAttributeValue;
    N?: NumberAttributeValue;
  .....
}

Hence, item.orderId gives us an instance of AttributeValue and then we use .S or .N to get the corresponding value

Step 7: Finishing OrderRequest class

  • isAGetOrder() should return TRUE given a GET request beginning with /orders as the path part
  • orderId() should return the value of pathParameter "orderId"

import {APIGatewayEvent} from "aws-lambda";

export class OrderRequest {
    constructor(private readonly event: APIGatewayEvent) {
    }

    //return true if the request is a GET request, with path starting from /orders and containing a path parameter
    isAGetOrder(): boolean {
        return this.event.httpMethod === "GET" &&
               this.event.path.startsWith("/orders") &&
               this.event.pathParameters.orderId != null;
    }
    orderId(): string {
        return this.event.pathParameters.orderId; //order id is passed as path parameter
    }
}

Step 8: Introducing async/await in service and controller

Let's introduce async and await in handler, controller and service.

//handler.ts
export const ordersHandler = async (event: APIGatewayEvent): Promise<any> => {
    return await new OrderController().handle(new OrderRequest(event));
};

//OrderController.ts
async handle(orderRequest: OrderRequest) {
    if (orderRequest.isAGetOrder()) {
        return await this.findAnOrderBy(orderRequest.orderId())
    }
    return null;
}

private async findAnOrderBy(id: string) {
    return await this.orderService.findAnOrderBy(id);
}

//OrderService.ts
async findAnOrderBy(id: string) {
    return await this.repository.findAnOrderBy(id);
}

Step 9: Lambda response with API gateway

When AWS lambda works behind an API gateway, it is expected to return a response in a specific format. This looks like -

{
   "statusCode": Http Status Code,
   "body": Response body,
   "headers": Response headers
}

It would be great if controller knows the least about this structure. All it should do is return a response with Order object. Let's create an abstraction which takes an object T and knows about the final HTTP response. Let's name this abstraction as Response.

export class Response<T> {
    private constructor(readonly status: HttpStatus, readonly body?: T) {
    }

    //signifies 200 response
    static ok<T>(body: T) {
        return new Response(HttpStatus.OK, body);
    }
    //signifies 404 response
    static notFound() {
        return new Response(HttpStatus.NOT_FOUND);
    }
}

export class HttpStatus {
    static readonly OK = "200";
    static readonly NOT_FOUND = "404";
}

With the introduction of Response, following will be the view of some classes -

import {OrderRequest} from "../model/OrderRequest";
import {OrderService} from "../service/OrderService";

export class OrderController {
    private orderService: OrderService;

    constructor() {
        this.orderService = new OrderService();
    }

    async handle(orderRequest: OrderRequest): Promise<Response<Order | unknown>> {
        if (orderRequest.isAGetOrder()) {
            const order = await this.findAnOrderBy(orderRequest.orderId());
            //return an Ok response if order is found else NotFound
            return order === null ? Response.notFound() : Response.ok<Order>(order);
        }
        return Response.notFound(); //return NotFound response
    }

    private async findAnOrderBy(id: string) {
        return await this.orderService.findAnOrderBy(id);
    }
}

export class Response<T> {
    private constructor(readonly status: HttpStatus, readonly body?: T) {
    }

    static ok<T>(body: T) {
        return new Response(HttpStatus.OK, body);
    }
    static notFound() {
        return new Response(HttpStatus.NOT_FOUND);
    }
    get() {
        //return a well formed JSON response
        return this.body === null ? {"statusCode": this.status} : {
            "statusCode": this.status,
            "body": JSON.stringify(this.body)
        }
    }
}

export class HttpStatus {
    static readonly OK = "200";
    static readonly NOT_FOUND = "404";
}

If controller returns an instance of Response, handler code can be changed to invoke get() on the returned instance.

import {APIGatewayEvent} from "aws-lambda";
import {OrderRequest}    from "./model/OrderRequest";
import {OrderController} from "./controller/OrderController";

export const ordersHandler = async (event: APIGatewayEvent): Promise<any> => {
    const response = await new OrderController().handle(new OrderRequest(event));
    return response.get(); //handler invokes get() on the response returned from controller
};

That's it. We have connected all the pieces, yes without any form of tests.

Step 10: Adding unit tests

Let's add a couple of unit tests before we close the article. I will add all the necessary unit tests offline and commit the code. Before we can start with tests let's add following dependencies -

  • npm install jest --save-dev
  • npm install @types/jest --save-dev
  • npm install ts-jest --save-dev
  • npm install sinon --save-dev

jest configuration in jest.config.js -

module.exports = {
    "testMatch": [
        "**/__tests__/**/*.+(ts|tsx|js)",
        "**/?(*.)+(spec|test).+(ts|tsx|js)"
    ],
    "transform": {
        "^.+\\.(ts|tsx)$": "ts-jest"
    },
};

test script in package.json -

"scripts": {
  "test": "jest test/**"
}

Controller unit tests

Let's add our first test which attempts to validate the status for finding an order by its id.

import {OrderController} from "../../src/controller/OrderController";
import {OrderRequest} from "../../src/model/OrderRequest";
import {APIGatewayEvent} from "aws-lambda";
import {HttpStatus} from "../../src/model/ModelAndResponseStatus";
import {OrderService} from "../../src/service/OrderService";
import {Order} from "../../src/model/Order";

import * as sinon from "sinon";

test("should return Ok as the response status given a request to find an order by id", async () =>{
    sinon.stub(OrderService.prototype, "findAnOrderBy")
         .callsFake(() => sinon.mock(Order));

    const response = await new OrderController().handle(orderRequest("id-100"));

    expect(response.status).toEqual(HttpStatus.OK);
});

afterEach(() => {
    sinon.restore();
});

const orderRequest = (id: string) => {
    const apiGatewayEvent: APIGatewayEvent = {
        httpMethod: "GET",
        path: `/orders/${id}`,
        pathParameters: {
            "orderId": id
        },
        body: null,
        isBase64Encoded: false,
        headers: {},
        multiValueHeaders: {},
        queryStringParameters: {},
        multiValueQueryStringParameters: {},
        stageVariables: {},
        requestContext: null,
        resource: ""
    };
    return new OrderRequest(apiGatewayEvent);
};

One quick observation -

  • apiGatewayEvent had to be constructed with all the attributes even though we needed only pathParameters because APIGatewayEvent type mandates all the attributes
  • visibility of status and model (in the next test) had to be changed from private to public to assert on these fields

Another test could be to check the order returned from controller given an id.

test("should return an order given a request to find an order by id", async () => {
    sinon.stub(OrderService.prototype, "findAnOrderBy")
         .callsFake(() => new Order("id-100", 1445));

    const response = await new OrderController().handle(orderRequest("id-100"));

    expect(response.body!!).toEqual(new Order("id-100", 1445));
});

I guess we are ready to do TDD as well for Serverless.

Summary

Finally we have come to an end of our first article where we made an attempt to design a small part of a serverless application which uses AWS Lambda, API Gateway and DynamoDB.

As a part of this application we have tried to draw some parallels with MVC design pattern and bring the same to the serverless world.

Items that we have left -

  • Exception handling is missing
  • Controller checks if the request is for getting an order. This if/else ladder will grow given the same lambda handles creation and deletion of orders.
  • Every component is unit testable in itself, except Repository layer which needs dynamo db

I am sure you will be able to fill these gaps and at this stage, I will move forward.

There is a lot of work still left before we can deploy the code -

  • We need to have integration test(s) which can give us confidence if this entire application is actually working or not
  • We need to integrate CDK (Cloud Development Kit) for deploying our infrastructure
  • We need to have unit tests and snapshot tests for our CDK based infra code

Code is available here.

Let's move on to our next article which explores integration testing using Localstack for our serverless application.