...
The Platform's FlowEngine component offers a large number of connectors to work with social networks. Specifically, it includes 2 two components, : one to "listen" to what happens on Twitter (for example, receiving tweets about a hashtag, or doing a search) and another one to publish tweets:
...
In this post we , you will see how to use these components to:
Listen to what is being said on Twitter about a topic.
Store the information of that a tweet in Platform.
Post a Tweet tweet with information received from an external service (OpenWeatherMaps).
Step 1: Create APP in Portal Developer on Twitter
The first thing we you must do is create an account on Twitter. Once the account is created, we will go to the Twitter Developer Portal at https://developer.twitter.com/
...
Once there, we will go to the Apps section and once there we will , select Create an App:
...
It will ask us you to confirm that we you want to create a developer account, then to complete a set of data. After completing itthose, we you will receive an email to confirm that we you want to create a developer account:
...
After this, we you can create our your App and get the Keys:
...
As indicated on the web, we you must save these Keys, since we you will have to configure them in the FlowEngine components:
...
After this, we you can test the Twitter REST API by pressing the Test an endpoint button.
As they explainindicated, we you can test it with the command:
curl -X GET -H "Authorization: Bearer <BEARER TOKEN>" "https://api.twitter.com/2/tweets/440322224407314432"
...
After testing this, we will go to the Dashboard. There, we you can configure our your App.
...
We'll go Go to the Keys and tokens section and select Generate Access Token and Secret:
...
Which we will You must also save this, together with the API Key and the API Key Secret:
...
IMPORTANT: Until a few months ago, this was enough to finish the necessary configuration in Twitter; but, with the change in the access policy to the Twitter APIs, when registering we will , you must do it with the Essential level that does not allow access to API 1.1, which is the one that uses the Node-Red component of the FlowEngine.
...
Meaning This means that, for the communication with Twitter to work, we you will have to go to the Portal and request the Apply for Elevated which will give us you permission to access the API v1.1:
...
It will ask us you for a series of some information to enable access to this level, such as the use we you want to make. We You can indicate that we you want to try to collect tweets with Node-red.
...
The first thing will be to have a user in the Platform instance that you want to use. For the example, we you will use the Platform CloudLab environment (free environment for experimentation purposes). So we'll , go to https://lab.onesaitplatform.com/ and if we you don't have a user, we'll create it with the SIGN UP button. It is important that we Bear in mind that you must create a DEVELOPER type user:
...
With the user created (We You will receive an email to validate the creation), we you can now log in to the Platform Control Panel:
...
Once there, we will go to the DEVELOPMENT > My Flows menu optionDEVELOPMENT> My Flows:
...
If this is the first time that we you enter, we you will see that we you have not created a FlowEngine domain, so we will press the + button to create it:
...
We'll give Give it a name and click Create:
...
Now we you can see our your domain, which is stopped by default - so, firstly, we will start it:
...
Once started, we you can access it with the View It button:
...
Finally we will , access the FlowEngine Palette. We will start Start by selecting that we you do not want to create a project:
...
We You can maximize the work area with the button in the right corner:
...
Now we will , search for the Twitter components with the browser, and drag the Twitter In component to the canvas:
...
The triangle means that we you must configure it. To do this, we click twice on it so that the configuration window appears.
...
In it, we you must configure the Twitter credentials that we you had obtained in step 1:
...
Then we , you can configure what we you want to listen to. For example, we you can choose a trend. In this example, we you are going to choose #TheBatman on the day of its premiere in Spain:
...
Then, we will configure the debug node to debug the input through the console:
...
And to test it, we simply do a DEPLOY:
...
And we are you will already be seeing, in streaming, the tweets on the topic:
...
To stop it, we you can remove the search criteria.
We You can also publish a tweet with the Twitter Out component:
...
In the FlowEngine, it is very easy to build a business logic so that, when certain events happen (the arrival of an event to an Entity/Ontology for example), a message is composed and published.
In this simple example, we you will simply post a text. To do that, we will use the Inject component configured to inject a text:
...
After this, we will make the deployment and press the button of in the Inject component.
After this, we you can see the message:
...
Step 3: Store Tweet Information in a Platform Entity
If we you go back to the example where we was listening to what was being said about The Batman:
...
and put a debug component configured to output the entire msg object:
...
then we you will see, in the debug tab, a structure like this:
...
To insert this data in the Platform, we you can create an Entity with these attributes: topic, payload, lang, tweet; or you can use the feature of the Platform to create an Entity from a JSON. This later second way is much easier when we you start from a JSON, as in is this case.
From the Debug tab, we can select one of the objects with the Copy value button:
...
If we you paste the content into an editor, we you will see something like this:
...
Since we you want to use this JSON as an example to create our Entity, we you can remove the attributes inside the tweet, so we you will leave have a JSON structure like this:
| Code Block |
|---|
{
"topic": "tweets/playlsit",
"payload": "robert pattinson as batman >>>>>>>",
"lang": "en",
"tweet": {},
"_msgid": "31a45aac.7fc556"
} |
Now we go to the DEVELOPMENT/My Entities option in the Control Panel and select the + button to create a new Entity:
...
There, we you will find several options. In this case, we you are interested in the Create From File option to use the JSON as a basis to create our your Entity:
...
When selecting it, we you can copy the JSON that we you already have left as a template and , then press the Generate JSON-Schema button:
...
Then we Thenm click Create and complete the data of our your Entity:
...
Once created, we you can go back to the FlowEngine to continue with our your flow.
We will look Look for the onesaitplatform-insert component and drag it in, then configure it with our your Entity:
...
If we you connect it directly to the Twitter output and debug the output, we you will see that we you get an error:
...
This is because the output of the components in the FlowEngine is the msg.payload, which in our your case is simply a text:
...
while the Insert node expects a complete JSON structure as we you have defined, since we you are interested in storing all the tweet info, including the tweeting user.
...
where the Function node converts the msg to a payload on output:
...
If we you now do the DEPLOY, we you will see that the insertion is correct:
...
To verify that this information is really being stored in an Entity, we access the Query Tool in TOOLS>Query Tool.
Inside we can consult From there, you can query the stored data, just by selecting the Entity and configuring a query, or by using the default query:
...
Once we have the data stored in the Platform, we you can, for example:
Exploit the data with the Dashboard Engine, for example, showing tweets by zone, users who tweet the most about a topic, …
Create a Model with the Notebooks doing a sentiment analysis on the tweets, or a geographical distribution, …
Publish a REST API with the API Manager, with added information on a specific topic.