| Table of Contents | ||||
|---|---|---|---|---|
|
Introducción
...
Introduction
The Platform's FlowEngine component offers a large number of connectors to work with social networks. Specifically, it includes 2 components, one to "listen" to what happens on Twitter (for example, receiving tweets about a hashtag, or doing a search) and another one to publish tweets:
...
En este post vamos a ver cómo usar estos componentes para ver en streaming lo que se dice en Twitter alrededor de un tema y posteriormente publicar un tuit con información recibida desde un servicio externo.
Paso 1: Crear APP en Portal Developer en Twitter
Lo primero que debemos hacer es crearnos una cuenta en Twitter. Una vez creada la cuenta iremos al Portal del Developer de Twitter en In this post we will see how to use these components to:
Listen to what is being said on Twitter about a topic.
Store the information of that tweet in Platform.
Post a Tweet with information received from an external service (OpenWeatherMaps).
Step 1: Create APP in Portal Developer on Twitter
The first thing we must do is create an account on Twitter. Once the account is created, we will go to the Twitter Developer Portal at https://developer.twitter.com/
...
Una vez dentro iremos a la sección Apps y dentro seleccionaremos Once there, we will go to the Apps section and there we will select Create an App:
...
Nos pedirá que confirmemos que queremos crear una cuenta de desarrollador y completar un conjunto de datos y tras completar nos llegará un mail para confirmar que queremos crear una cuenta de desarrollador:
...
Tras esto ya puedo crear mi App y conseguir las It will ask us to confirm that we want to create a developer account, then to complete a set of data. After completing it, we will receive an email to confirm that we want to create a developer account:
...
After this, we can create our App and get the Keys:
...
Como indican en la web debo guardar estas Keys ya que tendré que configurarlas en los componentes del FlowEngineAs indicated on the web, we must save these Keys, since we will have to configure them in the FlowEngine components:
...
Después de esto puedo probar el API REST de Twitter pulsando el botón After this, we can test the Twitter REST API by pressing the Test an endpoint .Como indican puedo probarlo con el comandobutton.
As they explain, we can test it with the command:
curl -X GET -H "Authorization: Bearer <BEARER TOKEN>" "https://api.twitter.com/2/tweets/440322224407314432"
...
Tras probar esto iré al Dashboard, dentro puedo configurar mi App
...
Iré a la sección Keys and tokens y seleccionaré After testing this, we will go to the Dashboard. There, we can configure our App.
...
We'll go to the Keys and tokens section and select Generate Access Token and Secret:
...
Que guardaré también junto con el API Key y el Which we will also save, together with the API Key and the API Key Secret:
...
Hasta hace unos meses con esto he concluido la configuración necesaria en Twitter, pero con el cambio en la política de acceso a las APIs de Twitter al registrarnos lo haremos con el nivel Essential que no permite el acceso al API 1.1 que es la que usa el componente Node-Red del FlowEngine.
...
Tendremos que ir al Portal y solicitar el Apply for Elevated que ya nos da permiso para acceder al IMPORTANT: Until a few months ago, this was enough to finish the necessary configuration in Twitter; but, with the change in the access policy to the Twitter APIs, when registering we will do it with the Essential level that does not allow access to API 1.1, which is the one that uses the Node-Red component of the FlowEngine.
...
Meaning that, for the communication with Twitter to work, we will have to go to the Portal and request the Apply for Elevated which will give us permission to access the API v1.1:
...
Os
pedirá una serie de información para habilitaros el acceso a este nivel, como el uso que queremos hacer, podemos indicar que queremos probar a recoger tuits con It will ask us for a series of information to enable access to this level, such as the use we want to make. We can indicate that we want to try to collect tweets with Node-red.
...
Step 2:
...
Configure Twitter
...
Component in Flowengine and retrieve and post Tweets
The first thing will be to have a user in the Platform instance that you want to use. For the example, we will use the Platform CloudLab environment (free environment for experimentation purposes). So we'll go to https://lab.onesaitplatform.com/ y si no tengo un usuario lo crearé con el botón SIGN UP. Es importante que cree un usuario tipo DEVELOPER and if we don't have a user, we'll create it with the SIGN UP button. It is important that we must create a DEVELOPER type user:
...
Con el usuario creado (recibiré un mail para validar la creación) ya puedo logarme en el Control Panel de PlataformaWith the user created (We will receive an email to validate the creation), we can now log in to the Platform Control Panel:
...
Una vez dentro iré a la opción de menú DEVELOPMENT>My Flows
...
Si es la primera vez que entro veré que no tengo creado un dominio del FlowEngine, así que pulsaré el botón + para crearlo:
...
Le daré un nombre y pulsaré Create:
...
Ahora ya veré mi dominio, que por defecto está parado, así que lo arrancaré primero:
...
Una vez arrancado puedo acceder a él con el botón de View It:
...
Finalmente accederé a la Paleta del FlowEngine, empezaré por seleccionar que no quiero crear un proyecto:
...
Puedo maximizar el área de trabajo con el botón de la esquina derecha:
...
Ahora buscaré los componentes de Twitter con el buscador y arrastraré el de Twitter In a la paleta:
...
El triángulo me indica que debo configurarlo, para esto pincho 2 veces sobre él para que aparezca la ventana de configuración.
...
En esta debo configurar los credenciales de Twitter que obtuvimos en el paso 1:
...
Después ya puedo configurar lo que quiero escuchar, por ejemplo puedo elegir una tendencia, en el ejemplo vamos a elegir #TheBatman en el día de su estreno en España:
...
Después configuraré el nodo debug para depurar por consola lo que llega:
...
Y para probarlo simplemente hago un DEPLOY:
...
Y ya veo en streaming los tuits sobre el tópico:
...
Para pararlo puedo eliminar el criterio de búsqueda.
Igualmente puedo publicar un tuit con el componente Twitter Out:
...
En el FlowEngine es muy fácil componer una lógica de negocio de modo que ante ciertos eventos (la llegada de un evento a una Entidad/Ontología por ejemplo) se componga un mensaje que se publique.
En este sencillo ejemplo simplemente publicaremos un texto, para eso usaremos el componente Inject configurado para inyectar un texto:
...
Tras esto haremos el despliegue y pulsaremos el botón del componente Inject.
Tras esto podremos ver el mensaje
...
Próximos pasos
En otro post posterior ampliaremos el uso de Twitter para crear un sencillo Bot que sea capaz de darnos la información meteorológica de la ciudad por la que preguntemos.
Once there, we will go to the menu option DEVELOPMENT> My Flows:
...
If this is the first time that we enter, we will see that we have not created a FlowEngine domain, so we will press the + button to create it:
...
We'll give it a name and click Create:
...
Now we can see our domain, which is stopped by default - so, firstly, we will start it:
...
Once started, we can access it with the View It button:
...
Finally we will access the FlowEngine Palette. We will start by selecting that we do not want to create a project:
...
We can maximize the work area with the button in the right corner:
...
Now we will search for the Twitter components with the browser, and drag the Twitter In component to the canvas:
...
The triangle means that we must configure it. To do this, we click twice on it so that the configuration window appears.
...
In it, we must configure the Twitter credentials that we had obtained in step 1:
...
Then we can configure what we want to listen to. For example, we can choose a trend. In this example, we are going to choose #TheBatman on the day of its premiere in Spain:
...
Then, we will configure the debug node to debug the input through the console:
...
And to test it, we simply do a DEPLOY:
...
And we are already seeing in streaming the tweets on the topic:
...
To stop it, we can remove the search criteria.
We can also publish a tweet with the Twitter Out component:
...
In the FlowEngine, it is very easy to build a business logic so that, when certain events happen (the arrival of an event to an Entity/Ontology for example), a message is composed and published.
In this simple example, we will simply post a text. To do that, we will use the Inject component configured to inject a text:
...
After this, we will make the deployment and press the button of the Inject component.
After this, we can see the message:
...
Step 3: Store Tweet Information in a Platform Entity
If we go back to the example where we was listening to what was being said about The Batman:
...
and put a debug component configured to output the entire msg object:
...
then we will see, in the debug tab, a structure like this:
...
To insert this data in the Platform, we can create an Entity with these attributes: topic, payload, lang, tweet; or use the feature of the Platform to create an Entity from a JSON. This later way is much easier when we start from a JSON, as in this case.
From the Debug tab, we can select one of the objects with the Copy value button:
...
If we paste the content into an editor, we will see something like this:
...
Since we want to use this JSON as an example to create our Entity, we can remove the attributes inside the tweet, so we will leave a JSON structure like this:
| Code Block |
|---|
{
"topic": "tweets/playlsit",
"payload": "robert pattinson as batman >>>>>>>",
"lang": "en",
"tweet": {},
"_msgid": "31a45aac.7fc556"
} |
Now we go to the DEVELOPMENT/My Entities option in the Control Panel and select the + button to create a new Entity:
...
There, we will find several options. In this case, we are interested in the Create From File option to use the JSON as a basis to create our Entity:
...
When selecting it, we can copy the JSON that we have left as a template and press the Generate JSON-Schema button:
...
Then we click Create and complete the data of our Entity:
...
Once created, we can go back to the FlowEngine to continue with our flow.
We will look for the onesaitplatform-insert component and drag it in, then configure it with our Entity:
...
If we connect it directly to the Twitter output and debug the output, we will see that we get an error:
...
This is because the output of the components in the FlowEngine is the msg.payload, which in our case is simply a text:
...
while the Insert node expects a complete JSON structure as we have defined, since we are interested in storing all the tweet info, including the tweeting user.
The way to solve this is to include a Function node between the two, like this:
...
where the Function node converts the msg to a payload on output:
...
If we now do the DEPLOY, we will see that the insertion is correct:
...
To verify that this information is really being stored in an Entity, we access the Query Tool in TOOLS>Query Tool.
Inside we can consult the stored data, just by selecting the Entity and configuring a query, or by using the default query:
...
Next steps
Once we have the data stored in the Platform, we can, for example:
Exploit the data with the Dashboard Engine, for example, showing tweets by zone, users who tweet the most about a topic, …
Create a Model with the Notebooks doing a sentiment analysis on the tweets, or a geographical distribution, …
Publish a REST API with the API Manager, with added information on a specific topic.