Wednesday, October 12, 2016

Fast Jolie microservices deployment with Docker

Jolie is now available on Docker and now it is possible to develop and run a microservice inside a container.

But what about the deployment of a microservice in Docker? How can we build a deployable docker container which includes the microservice we are working on?

Actually, it is a very easy task. It is sufficient to develop the microservice by following some simple rules and your docker image will be ready in few seconds!

Rule 1 : you need a Dockerfile for building an image of your microservice
First of all, create a file named Dockerfile in your working directory and write the following lines:

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME


where SURNAME, NAME and EMAIL must be replaced with the maintainer's surname, name and email respectively. The Dockerfile will be used by Docker for building the image related to your microservice. As you can see, the image you are creating is layered upon a previously created image called jolielang/jolie-docker-deployer.
You can find this image in the docker hub of jolielang here. Such a docker image, is prepared for facilitating the deployment of a jolie microservice as a docker image. In order to use it in the right way, just follow the next rules. As an example, let me suppose to deploy the following microservice saved in file helloservice.ol:

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}

execution{ concurrent }

inputPort Hello {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: HelloInterface
}

main {
  hello( request )( response ) {
        response = request
  }
}


Rule 2 : EXPOSE inputPorts ports
Remember that all the inputPorts of your microservice must be reached from outside the container, thus you need to expose them in the Dockerfile.



In the example the inputPort is located at localhost:8000, thus we need to add EXPOSE 8000 in the Dockerfile. So, your Dockerfile now becomes like this:

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME

EXPOSE 8000

Rule 3 : COPY the files of your project and define the main.ol
Now, everything is quite done for preparing the image, we just need to copy the files of the project in the docker image. When doing this, pay attention to rename the file which must be run with the name main.ol.

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME

EXPOSE 8000
COPY helloservice.ol main.ol

Building your image

When the Dockerfile is ready we can build the docker image of the microservice. In order to do this you just need to run the following command within your working directory which also contains the Dockerfile.

docker build -t hello .

where hello is the name we give to the image. Once it is finished, you can easily check the presence of the image in the local registry by running the following command:

docker images

Running a container
Now, starting from the image, you can run all the containers you want. A container can be run by launching the following command:

docker run --name hello-cnt -p 8000:8000 hello

where hello-cnt is the name we give to the container. Note that the parameter -p allows you to map the microservice port (8000) to the port 8000 of your localhost. You can check that the container is running just launching the following command which lists all the running containers:

docker ps

Your microservice is now deployed and it is listening for requests at port 8000. You can just try to invoke it with a client like the following one. Remember to launch the client in a separate shell of your localhost!



include "console.iol"

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}


outputPort Hello {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: HelloInterface
}

main {
  hello@Hello( "hello" )( response );
  println@Console( response )() 
}



Advanced settings
So far, we have deployed a very simple service but, usually we deal with microservices that are more complicated than the hello service presented before. In particular, it is very common the case where some constants or outputPort locations must be defined at deploying time. In order to show this point, let me now consider the following service:

interface HelloPlusInterface {
RequestResponse:
     helloPlus( string )( string )
}

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}

execution{ concurrent } 

constants {
   CUSTOM_MESSAGE = " :plus!"


outputPort Hello {
Location: "socket://localhost:8000"
Protocol: sodep
Interfaces: HelloInterface
}

inputPort HelloPlus {
Location: "socket://localhost:8001"
Protocol: sodep
Interfaces: HelloPlusInterface 
}

main {
  helloPlus( request )( response ) {
        hello@Hello( request )( response );
        response = response + CUSTOM_MESSAGE
  }


This is a very simple microservice which has a dependency on the previous one. Indeed, in order to implement its operation helloPlus, it requires to invoke the operation hello of the previously deployed microservice. Moreover, it uses a constants CUSTOM_MESSAGE for defining a string to be added to the response string.




Usually, we would like that some of these parameters can be defined at deploying time because they directly deal with the architectural context where the microservice will run. Thus, we would like to create an image which is configurable when it is run as a container. How can we achieve this?

Rule 4 : Prepare constants to be defined at deploying time
The image jolielang/jolie-docker-deployer we prepared for deploying microservice has been built with some specific scripts which are executed before running the main.ol. These scripts just read the environment variables passed to the docker container and transform them in a file of constants which must be read by your microservice. The most important facts we need to know here are:
  1. Only the environment variables prefixed with JDEP_ will be processed
  2. The processed environment variables will be collected in a file of constants  named dependencies.iol. If it exists it will be overwritten.
From these two points we change the microservice as it follows:
 

interface HelloPlusInterface {
RequestResponse:
     helloPlus( string )( string )
}

interface HelloInterface {
RequestResponse:
     hello( string )( string )
}

include "dependencies.iol"

execution{ concurrent } 

outputPort Hello {
Location: JDEP_HELLO_LOCATION
Protocol: sodep
Interfaces: HelloInterface
}

inputPort HelloPlus {
Location: "socket://localhost:8001"
Protocol: sodep
Interfaces: HelloPlusInterface 
}

main {
  helloPlus( request )( response ) {
        hello@Hello( request )( response );
        response = response + JDEP_CUSTOM_MESSAGE
  }
}

As you can notice here there are two constants JDEP_HELLO_LOCATION and JDEP_CUSTOM_MESSAGE which require to be defined at the start of the microservice. They must be defined in the file dependencies.iol which MUST BE included in your microservice. This file just contains the declaration of the two constants.

constants {
JDEP_HELLO_LOCATION = "socket://localhost:8000",
JDEP_CUSTOM_MESSAGE = " :plus!"
}

During the development keep this file in your project and collect here all the constants you want to define at deploying time. When the service will be run in the container this file will be overwritten thus, you don't need to copy it into the docker image.

The Dockerfile of the helloPlus service is very similar to the previous one:

FROM jolielang/jolie-docker-deployer
MAINTAINER SURNAME NAME

EXPOSE 8001
COPY helloservicePlus.ol main.ol
We can create the image with the same command used before, but where the name is hello-plus.

 docker build -t hello_plus .

Configuring the container
Now, we just need to know how to pass the constants to the running container and everything is done. Docker allows to pass environment variables with the parameter -e available for command run. Thus the command is:

docker run --name hello-plus-cnt -p 8001:8001 -e JDEP_HELLO_LOCATION="socket://172.17.0.4:8000" -e JDEP_CUSTOM_MESSAGE=" :plus!" hello_plus

where hello-plus-cnt is the name we give to the container. Note that the constant JDEP_HELLO_LOCATION is set to  "socket://172.17.0.4:8000" where the IP is set to 172.17.0.4. It is just an example, here you need to specify the IP that Docker assigned to the container hello-cnt which is executing the service hello.ol. You can retrieve it just launching the following command:

docker inspect --format '{{ .NetworkSettings.IPAddress }}' hello-cnt

Once the hello-plus-cnt container is running, you can simply invoke it with the following client:

include "console.iol"

interface HelloPlusInterface {
RequestResponse:
     helloPlus( string )( string )
}

outputPort HelloPlus {
Location: "socket://localhost:8001"
Protocol: sodep
Interfaces: HelloPlusInterface
}

main {
  helloPlus@HelloPlus( "hello" )( response );
  println@Console( response )()
}


Conclusion
In this post I show how it is possible to deploy a microservice developed with Jolie as a docker container. The procedure is very easy, just pay attention to inputPorts and the constants you want to configure at deploying time. For all the other things you can just rely on the Jolie language. Don't forget that you can also exploit embedding for packaging more microservices into one, thus deploying all of them inside the same container if necessary.

Enjoy!

Monday, September 5, 2016

An easy way to build microRESTservices with Jolie

This post is about a couple of tools [https://github.com/jolie/jester] we developed for facilitating the programming of REST microservices with Jolie. Personally I am not a big supporter of REST services, but I think that a technology which aims at being a reference in the area of microservices like Jolie must have some tools for supporting REST services programming. Why? Because REST services are widely adopted and we cannot ignore such a big evidence.

Ideally, Jolie as a language is already well equipped for supporting API programming also using http, but REST approach is so deep coupled with the usage of the HTTP protocol that it introduces some strong limitations in the service programming paradigm. Which ones? The most evident one is that a REST service only exploits four basic operations: GET, POST, PUT and DELETE. The consequence of such a strong limitation on the possible actions is that the resulting programming style must provide expressiveness on data. This is why the idea of resources has been introduced in REST! Since we cannot programming actions we can only program resources.

Ok,  let's go with REST services!

...But,
...here we have a language, Jolie, that is more expressive than REST because the programmer is free to develop all the operations she wants. From a theoretical point of view, in Jolie she can program infinite actions instead of only four!



- Houston we have a problem! We need to put infinite operations inside just four!!!

- Why only four when we can have infinite?

- This does not matter now Houston, we have to do it!



Ok no problem! Follow our instructions!
First of all, take note that here we want to preserve all the benefits of using Jolie, thus the possibility to develop all the operations we want, but finally publish the microservice as a REST service. We achieve such a result by exploiting a specific microservice architetcure which is a composition of a router and the target microservice we want to publish as a REST one. The router, as described in Fabrizio's paper [https://arxiv.org/abs/1410.3712], is in charge to transform REST calls into standard Jolie calls.



Ok, in order to exaplain how to proceed, let us consider as a target microservice the demo one reported in the jester project [https://github.com/jolie/jester/tree/master/src/jolie/tools/demo]. It is a very simple service which emulates a manager of orders which supplies four operations: getOrders, getOrdersByItem, putOrder, deleteOrders.

Ok, there are four operations but it is just an example, we could have more than four operations :-). The main question now is:

What we have to do for transforming these operations in REST operations?
We need to use the jolie2rest.ol tool you can find the jester project. Such a tool analyzes the interface of the demo service and it extracts a descriptor for the router which enables it to publish the demo service as a REST service.

Very simple! But before running the tool we need to know something more. We need to define how to convert the single operations of the demo microservice will be tranformed into the REST one. In particular, fro each target operations we need to specify if we want a GET operation, a POST operation, a PUT operation or a DELETE operation. It is possible to provide these instructions, directly into the interface of the demo service by exploting the annotation @Rest into the documentation comments. As an example let us consider the operation getOrders:

/**! @Rest: method=get, template=/orders/{userId}?maxItems={maxItems}; */
getOrders( GetOrdersRequest )( GetOrdersResponse )


The annotation defines that the operation getOrders it must be transformed into a GET http method and the URL template that must be adopted is /orders/{userId}?maxItems={maxItems}. What is the template URL?
Since the REST services deal with resources, the URL is used as a means for expressing the resource we want to access. Here we use the template as the means for transforming a call to an operation of the target service into a resource. In particular, the parameters between curly brackets will be directly filled by using the corresponding values of the request message node which is defined in the GetOrdersRequest type:

type GetOrdersRequest: void {
    .userId: string
    .maxItems: int
}


Now, we can proceed by running the following command:

jolie jolie2rest.ol localhost:8080 swagger_enable=true

where localhost:8080 is the location where the router is deployed and the parameter swagger_enable specifies if we want to enable the creation of the swagger json descriptor file.
Note: the file service_list.txt contains the list of the target microservices to be transformed and the related inputPorts to be converted (more instructions can be found here: ref).

As a result the tool will generate two files:
  • router_import.ol
  • swagger_DEMO.json
The former file must be copied in the router folder and it contains the script which enables the router to transform the REST calls into the operations call of the target service. The latter is just the swagger descriptor to be provided in input into a SwaggerUI. You don't have a local SwaggerUI available? Follow these instructions to have it locally, otherwise go to point 6:



  1. Prepare the web server of the SwaggerUI application by downloading Leonardo from here [https://github.com/jolie/leonardo]
  2. Go to this SwaggerUI URL [https://github.com/swagger-api/swagger-ui/archive/v2.2.3.zip] and download the related web application project. 
  3. Copy the content of the folder dist of the SwaggerUI project inside the folder www of Leonardo.
  4. Open a shell and run jolie leonardo.ol
  5. Open a browser at the url http://localhost:8000, the SwaggerUI web application should appear.
  6. Copy the swagger_DEMO.json file into the folder where it is reachable from the SwaggerUI, in the Leonardo scenario put it inside the www folder.
  7. Write in the explorer bar of the SwaggerUI the URL for reaching the swagger descriptor, in the Leonardo case write: http://localhost:8000/swagger_DEMO.json
After this step, you should see the Swagger definition of your target demo service transformed into a REST one. In particular, you should see something like this:







As you can see the four operations of the target service have been transformed in the four different types of REST operations. Have a look to the interface annotations of the demo service for finding the matches with the Swagger interface!

Nice, but what happen if I want to transform more than one Jolie microservices instead of a single one?
This question is dealing with an architecture like the following one where the router is connected with several microservices:



There are no particular problems in achieving such a result. It is sufficient to list all the target microservices with the related input ports in the file service_list.txt and re-launch the jolie2rest tool!

Ok Houston! Here we are! But is it possible to avoid the usage of the router?
Yes, it is possible. But only if you accept to not completely adhere to the REST approach. A jolie microservice can be directly published by using a http inputPort with message format set to json [http://docs.jolie-lang.org/#!documentation/protocols/http.html#http-parameters]. In this case the microservice will be able to serve the http requests without requiring any router or proxy in the middle. If you want this, change the inputPort protocol of the demo service to a http one and then use the jolie2rest tool with the parameter easy_interface set to true.

jolie jolie2rest.ol localhost:8080 swagger_enable=true easy_interface=true

In such a case the router_import.ol file is not generated but only the swagger_DEMO.json one. The operations are all converted in POST methods and the URLs do not follow the templates but the request json messages must be entirely defined in the body of the message. Try to replace the file swagger_DEMO.json in the SwaggerUI and perform some calls.

Generating client stubs from an existing Swagger definition:
A last tool which can be very useful when integrating existing REST services in a Jolie architecture, is jolie_stubs_from_swagger.ol. Such a tool takes in input an existing Swagger definition and generates a Jolie client for each published api.

As an example you could try it by generating the clients for the petstore example supplied by the Swagger project [http://petstore.swagger.io/v2/swagger.json]. In order to do it, create a target folder where you want to store all the generated clients (for example petstoreFolder) and then run the following command:

jolie jolie_stubs_from_swagger.ol http://petstore.swagger.io/v2/swagger.json petstoreService petstoreFolder

where petstoreService is the token that will be used for generating the Jolie outputPort name of the petstore service inside the clients. As a result, in folder petstoreFolder you will find a list of Jolie clients. In particular, you will have a client for each api defined in the petstore swagger definition.

If you want to try to send a request just open one of them and create the request message. For example open the getOrderById.ol and prepare the request message by adding the following jolie code:

with( request ) {
.orderId = 8
};

then run the client as a usual jolie script:

jolie getOrderById.ol


the result should be printed out on the console!

You can exploit these clients inside your existing jolie microservices. Just note that the generated file outputPort.iol defines all the information necessary to declare the outputPort to be used. Thus just include it in your microservice project and then make the calls when is more useful for you!

Conclusion
Houston, everything is clear now! :-) With the REST tools described in this article we want to improve the Jolie language providing the possibility to publish Jolie microservices as REST services and giving an easy way for generating clients from existing REST services. We hope this could be useful for your  projects and, please, do not forget to send us your feedbacks and improvements!