Data Models in Onesait Platform

User Interface: 6.1.0-Warcraft

Introduction

In the Platform, the entity of the data model is simply called Entity, although initially it was called “Ontology”. These Entities will allow modeling from simple concepts such as a measure or a record, to complex concepts such as an organization. These Entities are defined in JSON + JSON-Schema format.

Origin of Ontologies

The concept of Ontology - now Entities, we recall - comes from the European R&D project SOFIA, from which the Sofia2 platform originated, which used RDF/OWL Ontologies as data models, in accordance with the principles of the semantic web.

image-20240718-092101.png

When in 2013 Indra considered evolving SOFIA to create an enterprise platform (Sofia2), which could be used in productive and complex projects, an analysis and empirical tests were performed on Sofia2, and it was concluded that the technology underlying the traditional ontologies modeled in OWL did not scale according to the needs of IoT and Big Data projects.

After considering various options, JSON + JSON Schema was considered to be the best present and future proposition.

A bit of history: Sofia2 data model

To learn more about how data models work in Onesait Platform, it is interesting to take a look back and analyze the data model that previously existed in the Sofia2 platform, where although the key concept of the data model was the Ontology, there were other important concepts to consider, such as the “Template” and the “Ontology Instance”.

Let's see what they consist of:

 

Data Model or Template

Ontology

Ontology Instance

 

Data Model or Template

Ontology

Ontology Instance

Represents

Template, either created by an Administrator, or created according to a particular standard (such as FIWARE Data Model), that allows Ontologies to be created.

Entity that represents a concept on which the Platform works.

It is a concrete record of the entity that defines the Ontology.

Example

Template defining the environmental quality attributes, according to the FIWARE Data Model.

  • Environmental quality (obtained from a device).

  • Weather forecast (obtained by an algorithm).

  • Environmental quality obtained at a specific time at a specific point.

  • Forecast for a specific region and month.

Formats

JSON-Schema

JSON-Schema

JSON

Where are?

It is not stored, it is a definition.

Independent of the chosen persistence engine: in a relational model they represent a table, in a document-type NoSQL DB they represent a collection of documents, etc.

Independent of the chosen persistence engine: in a relational model they represent a record, in a document-type NoSQL DB they represent a specific document, etc.

Data models in the Onesait Platform Control Panel

Our current data model is a natural evolution of the Sofia2 model, in which we retain concepts such as template or instances. In this case, the data model concept represents a template on which the Entities can then be created.

This is why the creation of templates is restricted to those users who have an “Administrator” role.

To view the different data models available, from the Platform Control Panel navigate to the Administration > Data Models Management menu:

image-20240718-122705.png

 

We will then be shown a list with the existing data templates, where we can see their name, owner, template type, categorization tags and creation and edition dates.

To analyze one of the data model templates, simply click on the “Show” button (eye icon) located on the right side of each template.

The general information of the data model template will then be displayed, as well as the JSON schema:

Entities and data models

The Entities are the key concept of the data model, as well as of the complete operation of the Platform, since the rest of the processes are triggered on them, such as, for example:

  • Rules: which are applied upon the arrival of an Entity instance (or planned), and allow access to the attributes of the Entities to act based on this.

  • Dashboards: dashboards are built, either representing in real time the instances arriving to the Platform, or through a query made on these (DataSources, API REST, etc.).

  • Analytics: Machine Learning models are typically performed on the Entities stored in the Platform's Big Data infrastructure (BDH).

Entities can be created by users with “Developer” or “Analyst” roles, as well as “Administrator” roles. When creating these Entities, we have several mechanisms that will facilitate our work and that we explain below:

https://onesaitplatform.atlassian.net/wiki/spaces/DOCT/pages/4142661637

Entity Instances

As already mentioned, an Entity instance represents a specific time and position within an Entity.

The Platform offers several tools to access the instances, being the Query Tool the most used by developers, since it allows through a wizard to quickly generate queries on the Entities, both in SQL and native, obtaining the result of the query in JSON format, which we can download in our local both in JSON and CSV format.

 

Another option we have to view the content of an Entity is through the Entity CRUD tools, where we will visualize the existing records:

From the Entity CRUD we will be able to edit the records in a simple way. This is not practical when you have thousands of records, but for a quick correction it is very useful.