...
...
| Table of Contents |
|---|
Introduction
In this example you are going to load data into the Platform through the FlowEngine, connecting with a dataset from an Open Data Portal.
How to develop it
- As a data origin, we you will use the "London DataStore" (https://data.london.gov.uk/ ). We have searched Search for the total population of the cities with more than 300,000 habitants, "Global City Population Estimates."
To Download download the historical population dataset: https://data.london.gov.uk/download/global-city-population-estimates/604a6a6f-2162-4d6b-bcd0-bee051703de1/global-city-population-estimates.xls
...
This is the structure of the downloaded dataset:
An interesting tab is titled named "CITIES-OVER-300K". For every city, we you have a row with the population per year:
...
Now you can use a template. In our this case, we will use an empty template called "EMPTYBASE". We You are going to represent the information per year and population instead of using a register per city. This will facilitate the temporal series management and the data insertion in the future. As a result, every register in the Excel file will be transformed into different registers (one per year):
...
The last step will be selecting the cell set to process and the output format. In the example, we will use the entire document from A0 (first cell) to the X1693. We will also Also, indicate that we you want the output with raw values, to keep the data type from the Excel file in the JSON.
The output of the "sheet to json" will be a JSON with the registries records of the datasheet.
- Information transformation: Here you will transform the JSON into the registries records to insert in the ontology. This will be made with the "Split to cities" node by creating a message per every combination of registry record and year. The code is:
At the end, we you will have in msg.payload a JSON with the same structure of the ontology instance.
...
5. To check the data into the ontology,select "TOOLS → Query Tool" in the menu.
Then, select the "CityPopulation" ontology:
To search any registerrecord, you can write the query "select * from CityPopulation limit 1":
Each of the inserted values into the defined fields can be seen.
...
This tutorial continues with the representation of the data at (Dashboards) Visualization of Data loaded into the Platform How to visualize data on Dashboards loaded from an Open Data Portal.?