How to deploy onesait Platform in CaaS Rancher?

Introduction

The platform is ready to be deployed completely containerized on Docker, as you can see in this other tutorial: (Deployment) How to execute the platform with Docker

However, in production environments, you may want to have a CaaS platform to help you manage all the Docker containers in execution.

When we offer the platform under a PaaS approach (Platform as a Service), we use RedHat OpenShift as our support CaaS platform.

But the platform also works on other CaaS platforms. If we have to deploy the platform on an On-Premise infrastructure provided by the customer, we can adapt to the Kubernetes platform used by the client. In other scenarios, the client has no platform. In that case, we suggest Rancher, a lightweight, open-source container manager platform, easy to build, as we will see...

How can you install Rancher in an environment with one Master and one Worker?

To install Rancher, you need at least two machines (virtual or physical), with a 64-bit Linux. The Rancher server will be installed in one of them (Master); and the platform's VM in the other (Worker).

NOTE: If needed, all of it can be installed in a single, more powerful VM.

IN MASTER NODE:

1.- Install Docker CE in Master Node

You need a Docker version compatible with the Rancher version that will be installed; or, if you are going to use a pre-existing Master node, the Docker version compatible with the Rancher version in the Master node (Rancher compatibility matrix)

          -CentOS: https://docs.docker.com/install/linux/docker-ce/centos/

          -Ubuntu: https://docs.docker.com/install/linux/docker-ce/ubuntu/

(Don't start Docker yet)

2.- Configure Docker 

Before starting Docker, you must change the path where Docker stores all its information: images, containers, etc. If you don't change it, it will use the default directory, /var/lib/docker. In the commonly used Azure machines, this directory is in a 35-Gb ephemeral disc, so it can be spent during execution, causing the machine to fail. We recommend you use a disc with more space to avoid this problem and, in the worst of cases, if the disc is full, it does not cause the whole machine to fail. To configure this, you must edit the file /lib/systemd/system/docker.service (sudo vi /lib/systemd/system/docker.service) and add option -g to the ExecStart configuration, as seen in the following example:

Finally, with the Docker service stopped, if there are files or directories in /var/lib/docker, you must move them to the new directory.

Add your repository as insecure-registry in Docker's daemon.

sudo vi /etc/docker/daemon.json 

and add:

3.- Start Docker CE 

Start it with >sudo systemctl start docker

and check that it started correctly by running the hello-world image:

>sudo docker run hello-world

Configure that thte Docker starts in the VM's boot:

>sudo systemctl enable docker.service

4.- Install Docker Compose

As Rancher is deployed with Docker Compose, you must install a docker-compose version that is compatible with Docker, Rancher and the versions of the yml in the modules in the Master node.

To do this, download Docker Compose last version (You can see it here: https://github.com/docker/compose/releases)

>sudo curl -L https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m) -o /bin/docker-compose

and give permissions >sudo chmod +x /bin/docker-compose

And finally check the version with >sudo docker-compose -v

(Steps in https://docs.docker.com/compose/install/#install-compose)

5. - Install and execute the Rancher Server

Go to the user's directory (In this case, /home/administrador/)

Do >sudo mkdir -p /datadrive/onesaitplatform/rancher

Create a .env file with this content:

REPOSITORY=registry.onesaitplatform.com

SERVERNAME=sofia2-dockerstack.westeurope.cloudapp.azure.com

HOST_VOLUME_PERSIST=/datadrive/onesaitplatform/rancher

and create a docker-compose.yml file (to execute Rancher on Docker Compose) like this one:

version: "2.1"

services:

 rancherinstance:

 image: rancher/server:latest

 container_name: rancherserver

 ports:

 - "8080:8080"

 volumes:

 - ${HOST_VOLUME_PERSIST}:/var/lib/mysql:rw

 privileged: true

 networks:

 - datanetworknetworks:

 datanetwork:

 ipam:

 driver: default

 config:

 - subnet: 172.28.0.0/16

You must open UDP ports 500 and 4500 for Rancher to work correctly, and you must also check that the firewall is not blocking those ports.

Once you have created the docker-compose.yml file, go to the directory and launch >sudo docker-compose up -d

With this, you have launched the Rancher console in the address you have configured, and you can access it in http://<IP_pública_vm>:8080

(NOTE: Bear in mind that the TCP port 8080 must also be open).

6.- Create environment in Rancher

Once Rancher is installed in the Master node, you must create an Environment from the Rancher console, to deploy the platform in the Worker node. To do this, from Manage Environments --> Add Environment, create an environment (name and description).

Select Cattle as your template.

then select

After this, you will add a new host to the Environment. You will later deploy, containerized, the Open Platform on it.

7.- Install Rancher agent in the Open Platform's node

In the just-created environment:

select Infraestructure --> Hosts --> Add Host

It will ask you if you want to use that registration URL. Validate if it is correct, and go on:

Leave the Custom value

In the Label section, assign the key "NODE" and a name. Commonly the latter is "Worker-N"

where N is the Rancher worker's sequential, numeric identifier in the cluster.

You must also give the private IP of the machine you want to add to the cluster, in this case:

Once you've filled in those two fields, select the Docker command you have to copy and launch in the VM you are adding to the Rancher cluster.

Start by installing Docker in the Worker node, following the same steps as before.

          -CentOS: https://docs.docker.com/install/linux/docker-ce/centos/

          -Ubuntu: https://docs.docker.com/install/linux/docker-ce/ubuntu/

and remembering the configurations:

You will change the path where Docker stores all its information:

sudo vi /lib/systemd/system/docker.service

and add option -g to the ExecStart configuration, as seen in the following example:

ExecStart=/usr/bin/dockerd -g /datadrive/docker

sudo vi /etc/docker/daemon.json

and add:

{

"insecure-registries" : [ "registry.onesaitplatform.com" ]

}

Start Docker: >sudo systemctl start docker

Copy the text specified in the Rancher console, paste it and execute it in the Worker node's shell:

When running this command, Docker will download the needed images to activate the Rancher agent. This will take some minutes.

Once downloaded, in the Rancher environment's infrastructure (Infraestructure --> Hosts) you can see the Cattle agent's different services starting up (scheduler, healthcheck, ip-sec, etc...):

Once these steps are completed, you have deployed Rancher and you also have a node installed as Worker in Rancher to install the platform.

Deploying the Rancher-managed onesait Cloud Platform

1.- Generate the deployment files in Rancher

In the Platform's Git project (https://github.com/onesaitplatform/onesaitplatform-cloud/tree/1.1.0-ce/devops/build-deploy/rancher/onesaitplatform) directory, you will find the needed scripts and templates to generate the files that you will load in Rancher to deploy the Open Platform.

The config.properties file, in the onesaitplatform-cloud/devops/build-deploy/rancher/onesaitplatform/scripts directory, has customizable values depending on the installation:

• PROJECT_NAME → project name.
• WORKER2DEPLOY → it is the node or host where you will deploy the Onesait-Platform. You must provide the Rancher tag that identifies it with NODE
• DOMAIN_NAME → it is the dns through which you will access this OnesaitPlatform instance.
• IMAGE_TAG → a tag identifying the version of the Docker images that will be deployed.

In the example, having the IP 139.59.133.69 for our worker:

2. Launch the deployment files in the Worker node

Once the previous configuration file is edited, copy the four files to the VM serving as Worker. In this case, create a deploy folder in the user's home and upload the four files:

Give execution permission to the file and launch the generate-templates.sh script:

This script generates, in the folder where you are, a directory with the project's name (onesaitplatform_cloudlabs-Worker1) with the Rancher files, from the docker-compose-ce.yml and rancher-compose.yml templates.

There are two .yml-extension files in this directory:

• docker-compose.ym → This file is compatible with Docker Compose and defines sservices, images, tags, networks, volumes, etc...

• rancher-compose.ymll → Rancher specific configuration (number of replicas for each service, start-up order, etc...).

NOTE: These two files can be manually modified if needed.

3.- Create persistent volume directories

Before launching the deployment in the Worker node, you need to create directories in the host or hosts (your Worker) where the deployment will be made.

These directories are those that the different containers will need to dump data in their volumes, and you can see them in the generated docker-compose.yml file:

To create the volumes directory we'll execute the script create-directories.sh, located in the scripts directory we previously uploaded. 

In this example, they are located in the host's external disc, in /datadrive, and you need the following ones:

• /datadrive/onesaitplatform/platform-logs → directory with each container's logs.
• /datadrive/onesaitplatform/nginx/ → directory to store nginx configuration.
• /datadrive/onesaitplatform/nginx/certs → directory to store auto-signed certificates, typically "/etc/nginx/ssl", in the nginx container.
• /datadrive/onesaitplatform/configdb → directory with ConfigDB data.
• /datadrive/onesaitplatform/realtimedb → directory with RealTimeDB data on Mongo.
• /datadrive/onesaitplatform/schedulerdb → directory with scheduler database data.
• /datadrive/onesaitplatform/elasticdb → directory with ElasticSearch indexer data.
• /datadrive/onesaitplatform/flowengine → directory where the FlowEngine flows are stored (Node RED).
• /datadrive/onesaitplatform/webprojects → directory containing the web projects.
• /datadrive/onesaitplatform/export → directory for RTDBMaintainer.
• /datadrive/onesaitplatform/zeppelin/notebook→ directory to store Notebooks.
• /datadrive/onesaitplatform/zeppelin/conf → directory for Notebooks configuration.
• /datadrive/onesaitplatform/kafka-logs → directory with Kafka persistence.
• /datadrive/onesaitplatform/streamsets/data→ directory for DataFlow data.

4.- Generate auto-signed certificates

To generate auto-signed certificates, go to the /datadrive/onesaitplatform/nginx/certs directory and execute:

openssl genrsa 2048 > ca-key.pem
openssl req -new -x509 -nodes -days 3600 -key ca-key.pem -out ca.pem
[Rellenamos los campos que nos indica]

openssl req -newkey rsa:2048 -days 3600 -nodes -keyout server-key.pem -out server-req.pem
[Rellenamos los campos que nos indica]
openssl rsa -in server-key.pem -out server.key
openssl x509 -req -in server-req.pem -days 3600 -CA ca.pem -CAkey ca-key.pem -set_serial 01 -out platform.cer

When you run it, it will ask you a number of parameters such as city, country, company, workgroup, etc. Insert all the parameters you are asked, paying special attention to "common name". There, you must insert the DNS or IP from where you are going to access to the platform. We will validate the correct generation of the certificate executing

>openssl verify -CAfile ca.pem platform.cer

5.- Create configuration file for the nginx proxy

In order to expose the platform publicly, we need a balancer to do so. We will use nginx as reverse proxy. We will copy the configuration file located at <upload_directory>/deploy/op_config/nginx.conf to the directory /datadrive/onesaitplatform/nginx/. Give the file 777 permissions. Also copy the configuration files of the rest of onesait services from <upload_directory>/deploy/op_config/conf.d to datadrive/onesaitplatform/nginx/conf.d

In case of needing more redirections for custom services, we must include the configuration of the new service into the /datadrive/onesaitplatform/nginx/conf.d directory. This directory is mapped to /usr/local/conf.d within the container. To finish we must specify nginx to load such file using the include order.

The configuration would resemble this