Introduction

The Platform's FlowEngine component offers a large number of connectors to work with social networks. Specifically, it includes 2 components, one to "listen" to what happens on Twitter (for example, receiving tweets about a hashtag, or doing a search) and another one to publish tweets:

In this post we will see how to use these components to:

• Listen to what is being said on Twitter about a topic.

• Store the information of that tweet in Platform.

• Post a Tweet with information received from an external service (OpenWeatherMaps).

Step 1: Create APP in Portal Developer on Twitter

The first thing we must do is create an account on Twitter. Once the account is created, we will go to the Twitter Developer Portal at https://developer.twitter.com/

Once there, we will go to the Apps section and there we will select Create an App:

It will ask us to confirm that we want to create a developer account, then to complete a set of data. After completing it, we will receive an email to confirm that we want to create a developer account:

After this, we can create our App and get the Keys:

As indicated on the web, we must save these Keys, since we will have to configure them in the FlowEngine components:

After this, we can test the Twitter REST API by pressing the Test an endpoint button.

As they explain, we can test it with the command:

curl -X GET -H "Authorization: Bearer <BEARER TOKEN>" "https://api.twitter.com/2/tweets/440322224407314432"

After testing this, we will go to the Dashboard. There, we can configure our App.

We'll go to the Keys and tokens section and select Generate Access Token and Secret:

Which we will also save, together with the API Key and the API Key Secret:

IMPORTANT: Until a few months ago, this was enough to finish the necessary configuration in Twitter; but, with the change in the access policy to the Twitter APIs, when registering we will do it with the Essential level that does not allow access to API 1.1, which is the one that uses the Node-Red component of the FlowEngine.

Meaning that, for the communication with Twitter to work, we will have to go to the Portal and request the Apply for Elevated which will give us permission to access the API v1.1:

It will ask us for a series of information to enable access to this level, such as the use we want to make. We can indicate that we want to try to collect tweets with Node-red.

Step 2: Configure Twitter Component in Flowengine and retrieve and post Tweets

The first thing will be to have a user in the Platform instance that you want to use. For the example, we will use the Platform CloudLab environment (free environment for experimentation purposes). So we'll go to https://lab.onesaitplatform.com/ and if we don't have a user, we'll create it with the SIGN UP button. It is important that we must create a DEVELOPER type user:

With the user created (We will receive an email to validate the creation), we can now log in to the Platform Control Panel:

Once there, we will go to the menu option DEVELOPMENT> My Flows:

If this is the first time that we enter, we will see that we have not created a FlowEngine domain, so we will press the + button to create it:

We'll give it a name and click Create:

Now we can see our domain, which is stopped by default - so, firstly, we will start it:

Once started, we can access it with the View It button:

Finally we will access the FlowEngine Palette. We will start by selecting that we do not want to create a project: 

We can maximize the work area with the button in the right corner:

Now we will search for the Twitter components with the browser, and drag the Twitter In component to the canvas: 

The triangle means that we must configure it. To do this, we click twice on it so that the configuration window appears.

In it, we must configure the Twitter credentials that we had obtained in step 1:

Then we can configure what we want to listen to. For example, we can choose a trend. In this example, we are going to choose #TheBatman on the day of its premiere in Spain:

Then, we will configure the debug node to debug the input through the console:

And to test it, we simply do a DEPLOY:

And we are already seeing in streaming the tweets on the topic:

To stop it, we can remove the search criteria.

We can also publish a tweet with the Twitter Out component:

In the FlowEngine, it is very easy to build a business logic so that, when certain events happen (the arrival of an event to an Entity/Ontology for example), a message is composed and published.

In this simple example, we will simply post a text. To do that, we will use the Inject component configured to inject a text:

After this, we will make the deployment and press the button of the Inject component.

After this, we can see the message:

Step 3: Store Tweet Information in a Platform Entity

If we go back to the example where we was listening to what was being said about The Batman:

and put a debug component configured to output the entire msg object:

then we will see, in the debug tab, a structure like this:

To insert this data in the Platform, we can create an Entity with these attributes: topic, payload, lang, tweet; or use the feature of the Platform to create an Entity from a JSON. This later way is much easier when we start from a JSON, as in this case.

From the Debug tab, we can select one of the objects with the Copy value button:

If we paste the content into an editor, we will see something like this:

Since we want to use this JSON as an example to create our Entity, we can remove the attributes inside the tweet, so we will leave a JSON structure like this:

{
    "topic": "tweets/playlsit",
    "payload": "robert pattinson as batman >>>>>>>",
    "lang": "en",
    "tweet": {},
    "_msgid": "31a45aac.7fc556"
}

Now we go to the DEVELOPMENT/My Entities option in the Control Panel and select the + button to create a new Entity:

There, we will find several options. In this case, we are interested in the Create From File option to use the JSON as a basis to create our Entity:

When selecting it, we can copy the JSON that we have left as a template and press the Generate JSON-Schema button:

Then we click Create and complete the data of our Entity:

Once created, we can go back to the FlowEngine to continue with our flow.

We will look for the onesaitplatform-insert component and drag it in, then configure it with our Entity:

If we connect it directly to the Twitter output and debug the output, we will see that we get an error:

This is because the output of the components in the FlowEngine is the msg.payload, which in our case is simply a text:

while the Insert node expects a complete JSON structure as we have defined, since we are interested in storing all the tweet info, including the tweeting user.

The way to solve this is to include a Function node between the two, like this:

where the Function node converts the msg to a payload on output:

If we now do the DEPLOY, we will see that the insertion is correct:

To verify that this information is really being stored in an Entity, we access the Query Tool in TOOLS>Query Tool.

Inside we can consult the stored data, just by selecting the Entity and configuring a query, or by using the default query:

Next steps

Once we have the data stored in the Platform, we can, for example:

• Exploit the data with the Dashboard Engine, for example, showing tweets by zone, users who tweet the most about a topic, …

• Create a Model with the Notebooks doing a sentiment analysis on the tweets, or a geographical distribution, …

• Publish a REST API with the API Manager, with added information on a specific topic.