Transformations

Learn how to create and manage transformations for your data

What Is a Transformation

A Transformation is an entity that enables data to be modified. It can be thought of as a mathematical function that takes in input data (e.g. produced by a connector) and outputs results (i.e. output data). Multiple transformations can be applied either globally to a canopy, and/or locally to specific connectors in a canopy. Transformations are processed sequentially, one after another, according to the order defined by you. A transformation consists of one or more actions.

What Is an Action

An action (or transformation action) is a subunit of transformations. It represents a single type of transformation operation to be applied to input data. Actions are applied sequentially in the order defined by you. For further information on specific action types, refer to ActionTypes.

Action Types

Shift

Shift

A shift action moves a specified field and its value to the destination field and its value, and then deletes the original field.

Configurable Fields

• key: Field you want to be set to
• value Field you have

Shift can be set at the top level or over which can then be each inside of an array. Note: most transformations will be using shift and make sure to use the over field for objects in an array.

Concat

Concat

Allows the combination of multiple fields into a single field.

Configurable Fields

• sources: an array of values or json paths to group together
• targetPath: the destination JSON path
• delim: the separator between all the fields

Coalesce

Coalesce

Will look through multiple possible keys in order to find the value and set to a path on the json.

Configurable Fields

• key the JSON path to search followed by an array of json paths
• ignore an array of values to ignore on the JSON input (Example: null, "")

Example:

{
    "pathToSetTo": ["uuid", "uid", "id"],
    "ignore": ["null", "undefined", ""]
}

on the input

{
    "id": "value",
    "uuid": "",
    "uid": "value2"
}

Will Result in the follwing:

{
    "id": "value",
    "uuid": "value2"
    "uid": "value2",
    "pathToSetTo": "value2"
}

Delete

Delete

A delete action removes a specified field and its value.

Configurable Fields

• paths: the JSON path to delete for

Extract

Extract

Get the value of a field.

Configurable Fields

• path: the JSON path to search for the value

Default

Default

A default action adds a new field and value to a designated path.

Configurable Fields

• ’key’: the path to set the value to
• value: the JSON value to set

Base64 Encode

Base64 Encode

Take a value and encode it to base64.

Configurable Fields

• input path: the path to grab the value from
• output path: the JSON path to set value to

Using Transformations

Creating Transformations

New transformations can be created to be attached to your canopies and/or connectors. Transformations are shared within your organization. Under your organizations projects in the left side bar, click on the ”+ Create New” button and select “Transformation” from the dropdown menu.

1. Transformation Builder

   Once you create a transformation, you will be be redirected to the Transformation Builder page. The transformation builder will give you ability to insert your custom JSON and apply actions from the center column. See guide below:

   

   1 - Input - This is the input JSON, this should represent the base data payload that you want to transform.

   2 - Transformation Info - This is the overview of the transformation info, here you can update the name and description.

   3 - Transformation Output - This it the transformation output preview, this is the final result after the actions have been applied.

   4 - Actions Tab - Link to the actions tab pane. Here you can view, edit, and set the order of the actions.

   5 - New Action Tab - this tab is where you can configure new actions as well as use the AI action generator to speed up the proccess.

   6 - Add Custom Json - this is where you can view, edit, and set the order of the actions.

   7 - Save Input - save any custom JSON inputs so that way if you need to edit the transformation you can load a previous input.

   8 - Load Input - here you can load an input that you have already configured and saved.

   9 - Preview Input - Anytime you want to see the result of the changes you have made to the transformation actions you can preview the transformed result.

   10 - Updated Transformation - Apply all the changes you have made to the transformation.

2. Add Actions

   Starting by choosing the action Type. Then, configure each action by defining one or more Search criteria (By Name, Wildcard, Double Wildcard). Depending on the action type, configure additional criteria listed.

   By Name

   identify a field (to apply action) by literal field name

   Wildcard

   identify all fields in the current depth level of JSON input

   Double Wildcard

   identify all fields in the current and all sub depth levels of JSON input

3. Use Preview to Confirm Desired Action Behavior

   The transformation builder provides an Input and Preview window. Enter a valid JSON snippet into the Input window and review the Preview window to confirm successful transformation results. Each action produces its own Preview window. You can scroll down the Preview window to confirm each action being applied to the Input JSON snippet.

4. Create Transformation

   Once all configurations are set, be sure to click on the green Save Transformation button bottom right corner.

Modifying Transformations

The process to modify a transformation is identical to creating a new transformation. The one difference is selecting an existing transformation. Please note that modifying a transformation that is already attached to canopies or connectors will have immediate effect on the canopies and/or connectors.

Attaching Transformations

You can attach transformations you or your organization members created to canopies and connectors through the canopy page. You can also attach identical transformations to different connectors or canopies.

1. Attach Transformations To A Canopy

   To attach a transformation to a Canopy, first navigate to your desired Canopy. If there are no transformations you will see an empty transformation state indicated by dotted lines around the Transformation slots.

   

   You should now see a modal to add a transformation to your Canopy.

   

   If you have already attached transformations to your Canopy you can click the + Transformation in the top right of the Canopy Transformations section.

   
2. Attach transformations to a Connector Instance

   To attach a transformation to a Connector Instance, select the Connector Instance you want to add a transformation to. Navigate to the transformation tab, and similar to the Canopy Transformations section you’ll see dotted lines around the Transformation slots if there are no Transformations attached to the Connector Instance.

   
3. Preview your transformations by calling your canopy from the top right corner.

   Note You can expand your connector transformations to preview each individual action.

Detaching transformations

You can detach transformations from a canopy or connector. Detaching a transformation does not delete the transformation. This is also done through the Canopy page.

1. Open the canopy you want to detach transformations from.

2. First Select the transformation you want to detach.

   

   Then inside the transformations modal, click “Remove Transformation From Canopy”.

   

Deleting Transformations

You can delete transformations you or your organization members have created. This will remove the transformation from your organization’s transformation library. This is done through the transformation tab.

1. Open the transformation tab

2. Click on the option button (three dots) of the transformation you want to delete and click delete.