Automating container environments for testing microservices

Testing microservices is a fundamental task in a microservice oriented system and containerization offers a great opportunity for automating it. Containers can be created and connected on demand, thus they provide the perfect environment where performing tests because it can be created and then destroyed when the tests are terminated. Moreover, we could create a testing environment system which is the exact copy of the production one.

In my previous post I showed Jocker, a Jolie component able to interact with Docker by offering a subset of its functionalities as Jolie operations instead of REST ones. In this post I am going to exploit Jocker for automatizing a test on a simple Jolie microservice by orchestrating it from another jolie orchestrator. In order to do that I will use a simple example you can find in the jocker git repo under the folder: ExampleOrchestrator/TestingDBSystem

The system under test
The system under test is very simple and it is just represented by a microservice connected with a PostgreSQL database.

You can find the code of this simple Jolie microservice here. As you can see, this microservice has only one RequestResponse operation called getContent which is in charge to select the field of a row (field2) from table testTable of the DB depending on the value of column field1. Very simple.

Orchestrating the Test
Now, I'll show you how to test the microservice by checking if it properly returns the correct responses when some request messages are sent. In order to do so I use this simple orchestrator for interacting with Jocker which executes all the actions I need. In particular, the orchestrator performs three main activities:

• Preparation of the testing environment
• Test execution
• Removal of the testing environment

Preparation of the testing environment
The main idea is to prepare the testing environment by creating a container for each basic component of the system. Here we have two basic components: the PostgreSQL database and the Jolie microservice.

The PostgreSQL Database container is obtained in the following way:

1. pulling down the postgresql image from docker hub
2. creating the corresponding container
3. starting the container
4. initializing the database we need by creating it from scratch
5. initializing the required known data into the database

Steps 4 and 5 could be skipped if we consider to already have a postgresql test image with a pre-installed database initialized with the required data.

On the other hand, the Jolie microservice image can be built by following the same steps explained here. In particular:

1. a temporary directory is created and all the content of the ServiceToTest folder is copied in. 
2. a Dockerfile is dynamically created and added to the temporary folder
3. a tar file of the temporary folder is created 
4. a docker image of the microservice is created invoking Jocker
5. a container is created starting from that image
6. the container is started

The final environment is a system like the following one where the two basic components are now encapsulated within two different containers:

Test execution
The test execution is a very simple phase because the orchestrator just sends all the requests I want to test to the microservice inputPort and checks if the results are like expected. For the sake of this example there is only one request to test but, clearly, they can be hundreds depending on the operations to test and the variety of data to be considered.

Removal of the testing environment
When the test is done, I don't need the test environment any more and I can destroy it. In particular:

1. I stop the two containers
2. I remove the two containers 
3. I remove the two images (in reality, as you can see in the code, I do not remove the postgresql image just because it takes time for pulling it down from the docker hub, but it is up to you).

Some interesting notes

• In this example we do not exploit the possibility to create links among different docker containers but we directly call the container on the ports they provide. In order to do this, once a container is created we also query Jocker in order to get their info details for extracting the local IP assigned by Docker to them. We will use these IPs as reference hosts for connecting the microservice with the PostgreSQL database and the tests to the microservice.
• In order to run the example it is sufficient to run the jocker container as described here and the run the orchestrator placed in the folder  ExampleOrchestrator/TestingDBSystem by using the following command
  
  jolie orchestrator.ol

Conclusions
I hope this post could be of inspirations for all those software engineers who are addressing testing issues with microservices and containers. Let me know if you have questions or doubts.

Jocker: orchestrating Docker containers with Jolie

recently we spent time in integrating Jolie and Docker. Our idea was very simple: since Jolie is a very good language for orchestrating microservices in general, why not use it also for orchestrating docker containers??

Thanks to Andrea Junior Berselli who started to work on this topic during his University degree at the University of Bologna, we can now say that a first component able to integrate Docker with Jolie exists! Its name is Jocker [github project]!

How does Jocker work?
Jocker is a jolie microservice which is able to call the REST API of Docker (we implemented only a subset so far) and it offers them as simple jolie operations thus avoiding to deal with all the details related to rest json calls. Here you can see the jolie interface of Jocker. The architecture is very simple:

Jocker must be executed in the same machine where docker server is running. It is communicating by exploiting localsocket //var/run/docker.sock and it will supply jolie operations in the default location localhost:8008 with protocol sodep.

Jocker container
The easy way for running jocker is to pulling down its container and then starting it. The Jocker image is available at jolielang section on Docker Hub and it can easily pulled down by using the followingcommand:

docker pull jolielang/jocker

When pulled down run the following command for executing it:

docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 8008:8008 jolielang/jocker

Jocker from sources
If you want to run Jocker from sources, you need some extra steps before continuining:

• you need to install Jolie in your machine 
• you need to install libmatthew Java libraries in order to enable localsockets.

Running Jocker is very simple, just go into the jocker folder and then type the followin command:

jolie dockerAPI.ol

Jocker listening location can be changed by editing file config.ini.

Jocker Clients
It is very easy to interact with Jocker, just create the following outputPort in your Jolie microservice and use it as usual:

outputPort DockerIn {
    Location: "socket://localhost:8008"
    Protocol: sodep
    Interfaces: InterfaceAPI
}

where InterfaceAPI can be downloaded from here. As an example you can request for the list of all the containers with the following client:

include "console.iol"
include "string_utils.iol"
include "InterfaceAPI.iol"

outputPort DockerIn {
    Location: "socket://localhost:8008"
    Protocol: sodep
    Interfaces: InterfaceAPI
}

main {
    rq.all = true;
    containers@DockerIn( rq )( response );
    valueToPrettyString@StringUtils( response )( s );
    println@Console( s )()
}

In the github repository of the project there are some sample clients you can use for testing Jocker.

Enjoy Jocker and, please, let us know comments and suggestions for improving it.

What is a microservice? (from a linguistic point of view)

As we described here, with Jolie we are pioneering the linguistic approach for dealing with microservices. Our idea is that microservices are introducing a new programming paradigm which can be crystallized within a programming language. The focus of this post is about the definition of microservice starting from our point of view: a linguistic point of view.

A service is the single unit of programmable software
In the last years the concept of service has been investigated in the area of Service Oriented Computing and several definitions have been provided for defining service contracts, service providers, service discovery and so on. All these definitions are quite abstract because services have been conceived to be technology agnostic both in the case of SOA and microservices. Such a fact means that it is possible to develop a service in any given technology. They say that services are technology agnostic.

Technology agnosticism is a very important feature which allows us to engineer a software system independently from any technology lock-in. But our purpose here is to give the definition of a service as a single unit of programmable software which cannot be fragmented in sub-parts. For this reason, you will find that all the definitions I am going to provide here are strongly related to a specific technology: Jolie. If you like to see how we chose to model the service oriented programming paradigm in a single language you can continue to read, otherwise you can skip this post. If we are wrong, or we are missing some points, or you know other technologies which match the definitions please write us your feedbacks. We are very exited to share ideas on this open topic.

As a starting point, let me explain the first assumption we made in the linguistic paradigm: the service is the single unit of programmable software. Usually, a service is always obtained by programming a server (it is not important if it is simple or not) joint with some business logic which represent the functionalities to serve:

                             SERVER + BUSINESS LOGIC = SERVICE

In a linguistic paradigm such an equation is not more valid just because servers do not exist. Only services exist. It is not possible to program a server because you can program only services. So, forget servers (do not confuse with serverless, it is a different approach). So, if there are no servers but only services, what is a service? As it happens for Object Orientation where classes are logical definitions and objects are the instances of classes in a running environment, let me call the logical definition of services with the term service and its running instance with the term microservice.

                                     SERVICE --> MICROSERVICE

For each service there could be more than one microservices, but each microservice is just the running instance of a service. The service is the single unit of programmable software. In the following I am going to build the definition of a service by giving some qualities it has to provide. At the end of this post I'll give the definition of service.

Services exchange messages
The only way for exchanging data among services are messages. There are no other way. A message is just a limited portion of data transmitted in a limited portion of time. A service can both receive messages and send messages. In a SOA a service which receives and sends messages is usually called orchestrator. Such a difference in the linguistic paradigm does not exist, an orchestrator is just a service. In particular, in Jolie message exchange is enabled by means of ports. Messages are received by means of inputPorts and they are sent by means of outputPorts. Similar constructs are used in WS-BPEL, they are called partnerLinkTypes.

Services can have a behaviour
The behaviour defines the business logic to be executed when a message is received.  In the behaviour it is possible to compute the received data and/or send messages to other services. In Jolie the behaviour is expressed in scope main. A behaviour can define different operations. Different business logics can be joint to different operations. In Jolie multiple operations can be expressed in the same behaviour by using the non deterministic operator:

main {
[ op1( req1 )( res1 ) {
    businessLogic1
}]  

[ op2( req2 )( res2 ) {
    businessLogic2
}]  

...

[ opn( reqn )( resn ) {
    businessLogicn
}]  
}

An operation must always express a finite computation in time. In other words, when triggered an operation must always reach an end state. I say that an operation is divergent if its behaviour defines an infinite computation. Jolie allows for the definition of divergent operations by defining infinite loops:

divergentOperation( request )( response ) {
    while( true ) {
           nullProcess 
    }
}

Divergent operations are deprecated in Jolie.

Services declare interfaces
The operations of the behaviour must be declared in a machine readable interface. The interface defines all the available operations of a given service. Interfaces are used as a concrete contract for interacting with a service. In Jolie interfaces are also equipped with message type declarations.

type MyRequestType: void {
    .msg: string
}

type MyResponseType: void {
    .resp_msg: string
}

interface myInterface {
RequestResponse:
     myOperation( MyRequestType )( MyResponseType )
}

Services execute sessions
Services execute sessions for serving requests. A session is a running instance of a service operation. Sessions are independently executed and their data are independently scoped. If we suppose to send three messages to a microservice which implements the following service we will receive three different messages for each request message.

main {
   test( request )( response ) {
      response = request
   }
}

if we concurrently send three request messages with content "1", "2", "3", we will receive three related reply messages with the same content.

A definition of service

Here I try to summarize a definition of service starting from the basic concepts highlighted above. Jolie provides more linguistic elements w.r.t. those highlighted here. Maybe more basic concepts could be added to the definition and other should be removed. This is just a starting point for trying to investigate microservices from a linguistic point of view. All the contributions are encouraged!

A service is a unit of programmable software able to exchange messages with other services whose behaviour is triggered by incoming messages and it is defined by a set of finite computation logics called operations which are declared within a machine readable interface. A running instance of a service is called microservice whose inner instances of triggered operations are called sessions. Sessions are executed independently and their variables are independently scoped.