Integration of the MinIO console into the platform

Owned by Onesait Platform
Last updated: 22/06/23
4 min read

Since release 3.2.0, the MinIO console has been integrated into the platform.

MinIO is an object storage engine that is compatible with the Amazon S3 API. Its main component is the MinIO (MinIO object storage) server, which offers the S3 API and manages physical storage in a distributed manner, forming a cluster of MinIO object storage servers.

Besides, to facilitate the management of the MinIO cluster (buckets, policies, users...), that a user can browse the structure of directories and files of a bucket through a web UI, and that other applications can be integrated via REST API, MinIO has the MinIO Console module. This module connects to the MinIO Object Storage server, and offers all those services without storing anything itself: all management concepts reside in the MinIO Object Storage.

 

To integrate the MinIO console within the platform's control panel, we had to made a set of technical decisions, mainly derived from the following problems:

  1. The MinIO console web UI is exposed using an http address without a contextpath (http://minioconsole/). This poses a problem when exposing it to the Internet (within a controlpanel iframe), since it can not use a proxy_pass on the loadbalancer that redirects to the internal MinIO console endpoint.

  2. MinIO exposes the web console with a set of security headers that prevent the console from being easily integrated into an iFrame, so certain headers have to be filtered through an intermediary (NGINX)

  3. The MinIO session cookie is created with the SameSite attribute set to LAX, which prevents it from being displayed in an iframe – so we will have to replace the MinIO cookie with one generated by us from the control panel.

  4. The MinIO console has its own menu, meaning that, if it is integrated directly into the platform, it can be quite a confusing screen. To avoid this, we have touched the MinIO console code to remove the menu and some other options.

 

For problem #4, we have chosen to have two MinIO consoles:

 

  • Minio-browser: Modified MinIO console to remove the menu and other options that allow the users to leave the administration of their bucket. This is the console displayed on the MyFiles page of the control panel:

Having modified this console, we now manage its source code, its built, and its release as a container ourselves. Its location in the repository is:

https://gitlab.devops.onesait.com/onesait/platform/engine/onesait-platform/onesait-cloud-platform/-/tree/develop/tools/MinioBrowserConsole

where you can find a README.MD with the instructions of what has been touched in the source code, how the Front is compiled, and how the Docker image is generated to publish it later with a push in the registry: : registry.onesaitplatform.com/minio/console

Note: Before modifying the console and testing to generate the image, you must:

  1. Install Go.

  2. Install node:

    1. sudo apt install nodejs

  3. Install yarn:

    1. sudo apt install npm

    2. sudo npm install -g yarn

With this, now you can follow the steps described in the README.MD:

  1. Compile the Front: From /console-0.10.1/portal-ui

    1. yarn install

    2. make default or yarn default

To test it locally, once the front has been compiled, you have to build the project and run it. This is the way it is done:

  1. Compile the project: From /console-0.10.1

    1. make install

  2. Test it by starting the console from $HOME/go/bin

    1. export CONSOLE_PBKDF_PASSPHRASE=SECRET

    2. export CONSOLE_PBKDF_SALT=SECRET

    3. export CONSOLE_MINIO_SERVER=http://<minio-server>:<minio-port>

    4. ./console server --port=9001

 

  • MInio-console: Generic MinIO console. We use the containerized image from the official release of the MinIO console. This is the console that is displayed to administrators when they select the Minio Console button.

 

 

Problem #1 is solved by exposing the minio consoles as subdomains instead of mediating a context-path. To do this, you have to add both subdomains in the DNS of the installation or in the hosts file of the PCs that are going to access the console. E.g:

  • minioadmin-development.onesaitplatform.com

  • miniobrowser-developmentonesaitplatform.com

These subdomains are solved in the loadbalancer by adding a “server” type entry for each of them. This can be done by adding both files to nginx/conf.d/

Each of them already does the proxy_pass to the corresponding MinIO console service:

For minioadmin-development.onesaitplatform.com

server {        listen       80;        server_name  minioadmin-development.onesaitplatform.com;        location / {           proxy_set_header X-Real-IP $remote_addr;           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;           proxy_set_header X-Forwarded-Proto $scheme;           proxy_set_header Host $http_host;           proxy_hide_header X-XSS-Protection;           proxy_hide_header X-Frame-Options;           proxy_hide_header 'Access-Control-Allow-Origin';           add_header 'Access-Control-Allow-Origin' '*';           add_header 'Access-Control-Allow-Credentials' 'true';          proxy_connect_timeout 500;           proxy_http_version 1.1;           proxy_set_header Connection "";           chunked_transfer_encoding off;           proxy_pass http://minio-console:9090/;        } }

For miniobrowser-development.onesaitplatform.com:

server {        listen       80;        server_name  miniobrowser-development.onesaitplatform.com;        location / {           proxy_set_header X-Real-IP $remote_addr;           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;           proxy_set_header X-Forwarded-Proto $scheme;           proxy_set_header Host $http_host;            proxy_hide_header X-XSS-Protection;           proxy_hide_header X-Frame-Options;           proxy_hide_header 'Access-Control-Allow-Origin';            add_header 'Access-Control-Allow-Origin' '*';           add_header 'Access-Control-Allow-Credentials' 'true';          proxy_connect_timeout 500;           proxy_http_version 1.1;           proxy_set_header Connection "";           chunked_transfer_encoding off;           proxy_pass http://minio-browser:9090/;        } }

 

If needed, add the corresponding include in nginx.conf

include /usr/local/conf.d/minio-browser-nginx.conf; include /usr/local/conf.d/minio-console-nginx.conf;

 

Problem #2 is solved with the header configuration added (add some and remove others) in the previous step when yu create the console server in NGINX.

 

Problem #3 is solved with code in the ObjectStorageController.java class, where you log in to the MinIO console via REST API and create the session cookie in the browser programmatically.

response.setHeader("Set-Cookie", "token=" + userTokenForCookie + "; Domain=" + this.cookieDomain + "; Path=/; Secure; SameSite=None");

 

It is very important to configure the cookieDomain correctly (it is done through the configuration in Endpoint Modules in the control panel), which must be the superior domain to which the subdomains created for the MinIO consoles belong:

E.g.: for miniobrowser-development.onesaitplatform.com –> It has to be set to “onesaitplatform.com”.

Likewise, the consoles must be exposed securely via HTTPs.