FIWARE NGSIv2 support on Onesait Platform
What is FIWARE?
FIWARE is a platform, driven by the European Union, for the development and global deployment of Future Internet applications. It is based on an open, public and free architecture and a set of specifications that enable developers, service providers, enterprises and other organizations to develop products that meet their needs.
FIWARE provides a set of Open Source components that can be used in conjunction with other developments to build solutions quickly and cost-effectively.
It provides an API for component integration (FIWARE NGSI), which provides interoperability, extension and replication capabilities.
Soporte FIWARE NGSI en Onesait Platform
Onesait Platform can provide native support for applications developed on the FIWARE platform, allowing applications that use the NGSI standard to connect to interoperate with each other, and also to use the rest of the platform's capabilities (dashboards, analytical capabilities, KPI calculation, data classification, etc.).
Native support for the NGSI protocol is achieved by incorporating three components of the open FIWARE architecture into the Onesait Platform architecture:
Orion Context Broker
Cygnus
STH-Comet (Short Time History)
This results in the following diagram of components in Onesait Platform:
The functionalities covered by each FIWARE component incorporated into Onesait Platform are as follows:
Orion Context Broker:
The main component of any FIWARE solution, it is responsible for managing the context information of the systems and applications connected to the platform, allowing them to send, modify, consult and subscribe to notifications of any data related to the current state of the system.
Information management is done through entities, which are representations of objects modeled ontologically as Data Models.
It implements the OMA NGSI specification by exposing a RESTful API of the NGSI v2 standard. Through this API, different applications (context providers) connected to the broker can integrate with each other by exchanging information.
The context information is persisted in the platform's Semantic Datahub so that it always reflects the updated state of the system.
Cygnus:
The context information managed by the Orion Context Broker is useful for knowing the state of the system at all times, with any change being immediately reflected in the system. In a "data-centric" platform such as Onesait Platform, it is also essential to store the entire history of changes in any entity managed by the Orion Context Broker, as this allows analytical processes, machine learning, building dashboards with historical data series, calculating KPIs, offering open data to citizens...
Within the FIWARE project Cygnus is the connector in charge of providing historical persistence to the context data managed by the Orion Context Broker.
Based on Apache Flume, Cygnus provides a distributed process that allows to get all the changes in the context information from the Orion Context Broker, and communicate them to different destinations:
Short Time Historic: Historical database for NGSI applications and systems
Digital Broker: Onesait Platform's native Broker, stores the information in the semantic datahub, ready to be exploited by the different platform modules (Api Manager, notebooks, dataflows, flow engine, BPM, dashboards, reports...) and trigger all the processes associated in the platform to the corresponding ontology.
STH-Comet (Short Time Historic):
It provides applications and systems with REST NGSI interface with historical context data in the form of time series. It allows to calculate and return data, both "raw" and aggregated by attributes, as well as to temporally narrow the query range.
The historical information is persisted in the platform's Semantic Datahub by the corresponding Cygnus agent, so that STH-Comet is exclusively dedicated to answer requests via REST and resolve queries on the database.
NGSIv2 compatibility
As mentioned above, Onesait Platform can incorporate the fundamental elements of the FIWARE scheme into its architecture to enable applications and systems developed under the NGSIv2 standard to connect to the platform to interoperate with each other, as well as for the information they produce to be managed for use with Onesait Platform's own capabilities.
In more detail, the Onesait Platform integration diagram of these FIWARE components is described below:
For applications and systems using NGSI v2 REST protocol either through direct connection (context providers) or through an IoT Agent type FIWARE intermediary using REST, MQTT, AMQP... protocols in Ultralight 2.0, JSON... formats, the REST interface remains intact.
This REST interface, exposed from the platform acquisition layer, is the brokers' own:
Orion Context Broker: Platform context manager.
Short Time Historic Comet: Manager of historical queries on the context.
For security purposes these two brokers are not exposed directly, but through an NGINX reverse proxy, configured to intercept the requests and delegate to the platform's Identity Manager to check the OAuth2 token of the REST requests addressed to the Orion Context Broker and the STH Comet and verify if the client to which the token corresponds is valid and has permission to access the entity (ontology) on which the request is made.
Orion Context Broker, as the platform's context manager, handles NGSI v2 requests to create, query, modify and delete entities, as well as to manage subscriptions and notifications to changes in the context. It keeps all the information updated in the platform's Semantic Datahub, where a database for the context is created in the MongoDB instance.
Cygnus, as a bus of persistence of changes in the context has a double task:
Maintain for consultation from the STH Comet broker, the Short Time Historic with the context history in the Semantic Datahub of the platform, where a database has been created for the STH in the MongoDB instance
Notify the main broker of the platform (Semantic Broker) of the changes in the context so that it can transfer them to the ontologies corresponding to the entities. In this way, all the information will be available in the platform, ready to be exploited by the different capabilities of the platform: Api manager, KPI engine, reporting engine, Dashboards engine, Analytical Notebooks, Machine Learning, Dataflows...
For this purpose Cygnus, based on Apache Flume, subscribes to Orion Context Broker, to receive notifications of changes in the context and generate events that, after being processed and prepared, are finally sent to the corresponding destination (STH in MongoDB and Digital Broker).
Short Time Historic Comet, as a historical query engine on the context, handles requests to query historical time series on the STH database previously fed by Cygnus.
Data compatibility between FIWARE and Onesait Platform
NGSI is an open standard published by OMA (Open Mobile Alliance) to provide an interoperability protocol through the centralized management and exchange of what is known as context information. The first version dates back to 2013 and in 2016 evolved to its second version NGSI v2, converting the standard to Restful, simplifying operations, using native JSON format data types, supporting GeoJSON and Datetime data types and enriching the operations supported by the first version.
The protocol is based on the management of entities with their corresponding attributes and on the exchange between applications of the information of these entities.
An entity is any object of interest to an application and is defined by a set of attributes or properties:
The set of all the entities existing at any given time, with their corresponding attributes updated to the real value, is known as context.
Therefore, the fundamental element in an NGSI v2 solution is the broker or context manager, to which all applications (context providers) are connected to keep the entities they work with in the real world updated in this context. The applications interoperate with each other by sharing the same context updated in the broker through the operations it provides to access the context.
The operations contemplated by the NGSI v2 standard allow:
Create new entities in the context.
Modify attributes of an entity to update the context.
Query and filter existing entities in the context.
Subscribe to changes in the context to be notified
Consult all types of existing entities in the context and the detail of their attributes.
Batch insertions, updates and deletions.
Onesait Platform provides support for the NGSI v2 standard by including in its architecture the set of pieces of the FIWARE ecosystem mentioned above. This facilitates the seamless integration of systems and applications developed according to the NGSI v2 standard with others that connect to the platform through any other module of the acquisition layer or the interoperability layer.
Transparent integration both between NGSI v2 applications and with the rest of the platform's utilities is possible thanks to the semantics of the entities used, which provides a univocal interpretation of the data independent of the applications.
For this purpose, entities belonging to the same entity type are modeled as ontologies through their corresponding Data Model. The platform provides an extensible set of templates with different Data Models following standards in areas such as IoT, Smart Cities, Social Media, GSMA...
Lhe Data Models provided in the platform to create ontologies that model entities are configurable and extensible from the control panel administration, which allows that with an adequate data governance all entities of the same type comply with the requirements and restrictions previously established when designing the Data Model from which the developers will create the ontologies.
The ontologies in the platform have physical storage for the data, both in real time and historical, in the chosen database.
All changes in the context that affect entities belonging to the type of data modeled by an ontology are immediately reflected in the ontology database, which provides transparent integration with other applications connected to the platform that use that ontology, regardless of the broker or platform module they use to connect. Pej: Digital Broker, Dataflow, Digital Twin Broker, Api Manager, Portal Open Data (CKAN)...
NGSIv2 security in Onesait Platform
The FIWARE parts integrated in the platform (Orion Context Broker and STH Comet) that expose interfaces to the applications are secured using the same model as the rest of the Onesait Platform components.
As these are REST interfaces, security is provided by means of the OAuth 2 protocol using the platform's own Identity Manager.
Lhe security architecture is the one proposed by FIWARE. Brokers are not directly exposed to applications, but a reverse proxy such as NGINX is used as an intermediary. This proxy intercepts the REST requests and redirects them first to a PEP-Proxy that extracts the OAuth 2 token from the request, checks its validity in the configured OAuth 2 server (Identity Manager) and verifies if the token is valid, the user it belongs to has privileges on the accessed resource (entity, present in the request path) and the access mode (http verb of the request). In case of a valid request, the PEP-Proxy returns a response with code 204 (No Content) and the NGINX reverse proxy lets the request pass to the broker. Otherwise, a code 401 (Unauthorized) is returned if the OAuth 2 token is invalid or 403 (Forbidden) if the user does not have access to the resource, so the NGINX reverse proxy does not let the request pass to the broker.
The platform's Identity Manager is extended with a plugin that acts as a FIWARE PEP-Proxy, analyzing the token received in each request to the brokers and verifying if the token is valid and if the operation on the requested entity is allowed for the user to whom the token belongs.
The security management of the applications that connect to the Orion Context Broker and STH Comet is carried out from the platform's own control panel, following Onesait Platform's standard guidelines:
If the application manages application roles, a Realm is created for the application, where the application roles, users and role or roles associated to each user are registered. For example, let's assume bicycle rental management with three types of sites connected to the platform: Rental Stations, Stores and Workshops, so we can consider each one as a role, and each specific site will belong to one or more roles.
The application is registered as a project of the platform, to which it is associated:
Users: They can be associated one by one or in block by associating a Realm with its corresponding users to the project.
Resources: Elements registered in the platform (Ontologies, Notebooks, Dataflows, Apis in Api Manager...).
For each resource, the permission that each user in the project or each application role has over it is indicated if a Realm has been associated to the project and you want to set permissions by role.
Once the application is registered as a project in the control panel of the platform, any user associated to it can identify himself/herself using the REST OAuth 2 API of the Identity Manager, to request an access token. This token will be included in the NSGI v2 REST requests through the X-Auth-Token header.
Onesait Platform's Identity Manager PEP-Proxy is prepared to provide three levels of security:
Authentication:
Verifying that the OAuth 2 token included in the X-Auth-Token header is valid in the Identity Manager and belongs to an authenticated user on the platform.
Authorization based on user permissions:
In this case the PEP-Proxy parses the NGSIv2 REST request extracting:
X-Auth-Token header: OAuth 2 token. It allows validating that it is an authenticated user, as well as knowing the user's identity.
Resource accessed in the REST request path: Allows to know the entity (ontology) on which the request is made.
HTTP verb: Access mode to the resource. Allows to know the type of operation requested: Add, Query, Modify, Delete.
With this information, the PEP-Proxy within the Identity Manager determines if the user is associated to the application (project registered in the control panel) and if within this project the user has the access requested in the request to the entity (ontology).
Advanced authorization based on application roles:
It is identical to the authorization based on user permissions, except that in this case, the project registered in the control panel for the application has a Realm associated with the application's own roles, and the permissions on each entity (ontology) have been established at the role level and not at the nominal user level, since the users have been profiled in the Realm with these roles from the control panel.
Thus, by installing a simple FIWARE-compliant plugin, the platform's native security is used to provide native NGSI v2 applications with access control to information in the Context Broker and the Short Time Historic.