/
New internal connection stages in DataFlow

New internal connection stages in DataFlow

Available since version 6.3.0-Yoshi

Introduction

Until version 6.3.0, all the connection stages with the Platform to deal with Entity operations (origination, insertion, update, lookup, etc.) have been done through the Digital Broker.

https://onesaitplatform.atlassian.net/wiki/spaces/DOCT/pages/2220786158

All stages have had a ‘Connection’ tab with the following fields, indicating the Digital Client and the connection to the Digital Broker:

image-20250124-092154.png
View of the stage connectivity to the Digital Broker.

In this mode, every operation is validated by the permissions and accesses defined in the Digital Client used in the stage configuration.

Target

Following the implementation of SSO for DataFlow in version 6.1.0-Warcraft, all operations are marked at user level, which means that there is no need to depend on the security of the Digital Client.

Therefore, a set of new stages has been created, with the same functionality as the existing ones, for those operations based on Entities. In this way, the configuration is simplified, as it is only necessary to fill in the Entity affected and the timeout of the requests.

image-20250124-095625.png

The following stages have been created:

  • Origins:

    • Internal Origin OnesaitPlatform

  • Destinations:

    • Internal Destination OnesaitPlatform

    • Internal Update OnesaitPlatform

    • Internal Delete OnesaitPlatform

    • Internal CRUD OnesaitPlatform

  • Processors:

    • Internal Lookup OnesaitPlatform

Each of these stages is equivalent to the existing ones, but with internal connectivity to the Platform via Semantic DataHub, which will report simplicity in the configuration and more optimal processes, since it is not necessary to go through the Digital Broker.

In terms of security, as always, it is the one that would apply to any assignment of permissions of a resource to the user who executes the pipeline. This assignment of resources can be any of those available on the Platform (public resources, shared resources, use of applications, etc.). In the event that a user can execute a pipeline, but does not have permissions on the Entity, the corresponding message of lack of permissions will be displayed in the operation:

Finally, it should be noted that the internal operations will always be applied to the user, vertical and tenant that is launching the execution, and only the environment in which the process is launched will be accessible. If, for example, you want to pass data from one Entity from one environment to another, you will have to continue using the (non-internal) stages with connection to the Digital Broker.