DataSense Manual

DataSense is a tool, which allows using messages metadata to simplify application design. Messages in Mule are data, which go through components of flow application. Within the context of Anypoint Studio, metadata represent information about data structure and types, used in the message elements (payload, flow variables, etc).

Thus, a connector user does not have to use the third-party documentation once again to find out which data are received by the connector or what this connector will return after doing any given operation.
DataSense is used for reviewing information about the message in DataSense Explorer and during message transformation using Transform Message component.

DataSense Explorer is a window, which contains information about a structure of passing through Mule flow messages, and about data types of message parts. Here we can see payload, inbound and outbound properties, flow variables and session variables. Press the button as it is shown in the picture to open DataSense Explorer:

Clicking on variable elements of flow, we can get information about messages in the different points of flow. For example, due to DataSense we know, that after performing operation Get Posts, the connector returns Json with a specific structure and data types:

Also, information about metadata of the connector can be specified using the tab Metadata:

DataWeave is a language to describe message transformation in Mule flow. To perform message transformation, it is necessary to know the structure of the message. DataSense helps us to perform this.

DataWeave can be used as a part of MEL. For example:

#[dw("{'payload': payload, 'interface':'CompInvoice'}","application/json")]
#[dw("{'payload': payload, 'interface':'CompInvoice'}","application/json")]

Also DataWeave can be used via Transform Message component:

This component allows changing message structure by means of the graphic interface:

Connecting different message elements with each other, we get a code in Dataweave language:

Press Preview button to see message structure after transformation.

DataWeave supports  transformation of the following types:

Thus, the transformation from json to xml, from csv to json etc. can be performed with the help of the Transform Message component.

The following order of flow build is the most preferable:

1. To set the configuration for connectors. The configuration usually contains a password/login, an address to the external service, various data affecting the work of the connector in general. It is impossible to connect the external service without this information. Press “+” button and fill in all necessary fields to add configuration.

This allows establishing connection with an external service and using DataSense for extracting metadata from this service. Also, metadata can be set by the tab Metadata of the connector, as it has been described before.

2. To add components for messages processing, filtration etc to flow.

3. To add elements of messages transformation DataWeave (Transform Message component or components, which use DataWeave Language).

This Mule flow building strategy can be characterized as a flow structure configuration using different components and connecting them with the help of DataWeave.

As it was mentioned before, DataSense gives a lot of advantages during Mule flow design. Adding DataSense is not an obligatory step during connector building, nevertheless it is strongly recommended to add it. As noted before, DataSense can initially simplify the usage of connector at the stage of application design.

DataSense implementation consists of two stages:

For example:

At the first stage we create the class, where statically (when Email and Prescriptions values are set manually) or dynamically (values are set on the base of information from the external service) we specify the list of metadata keys (Email and Prescriptions) for this operation and assign the structure, which is displayed in DataSense Explorer. This structure can also be assigned dynamically or statically. In this example, object attributes (Location and Subject) were got dynamically from the external service. All other fields were set statically.

At the second stage, we specify the class, which is used for any given operation. In this case, for “Create Interactions” operation we specified the class, which had extracted Email and Prescriptions types and created the specific structure of metadata for input and output of the connector. Also, we can set which metadata will be shown in the connector (input or output; or both).

The class should be marked with @MetaDataCategory annotation, to inject a connector and contain 2 methods marked with @MetaDataKeyRetriever and @MetaDataRetriever annotations. In this case metadata in output and input of the connector are similar (with the structure, defined in the method, which is marked with @MetaDataRetriever annotation). In the case, when one metadata structure should be in input of the connector and the other one should be in output, we add to class one more method with @MetaDataOutputRetriever annotation. Then, in the method marked with @MetaDataRetriever annotation the metadata structure will be assigned in input of connector and in the method marked with @MetaDataOutputRetriever annotation the metadata structure will be assigned in output of connector.

As noted previously, a list of keys with metadata can be specified statically or dynamically in the method, marked with @MetaDataKeyRetriever annotation.

A metadata structure can be set statically or dynamically in the method, marked with @MetaDataRetriever (or @MetaDataOutputRetriever) annotation.

Let’s consider some of such cases.

This way, in the method marked with @MetaDataKeyRetriever annotation, we defined the following types of keys: Book with Book_id identifier , Author with Author_id identifier. For each key, we set the metadata structure using POJO classes in the method marked with @MetaDataRetriever annotation.

This example demonstrates the following situations:

At this stage, the class needed for getting information about metadata should be set @MetaDataScope annotation is used for this purpose. This annotation can be used on separate processor level or on class level.
In the case of using annotation on class level for every processor in the connector, there will be used the same DataSenseResolver.

The simultaneous using of the annotation on both classes and processors levels is possible. In this case the processor which is marked with @MetaDataScope annotation, will use the class, set in the annotation. And other processors will use DataSenseResolver, which is set in the annotation on the class level. If it is required, that the separate processor doesn’t show the metadata information, @NoMetaData annotation can be used here.

An example of the processor with @MetaDataScope annotation:

One of the method parameters is marked with @MetaDataKeyParam annotation. The parameter marked with this annotation, will accept one of the metadata keys list values, which we have got in DataSenseResolver method marked with @MetaDataKeyRetriever annotation.

Using this annotation, you can also arrange the metadata information displayed only in input or output of the connector:

@MetaDataKeyParam(affects = MetaDataKeyParamAffectsType.INPUT)

or

@MetaDataKeyParam(affects = MetaDataKeyParamAffectsType.OUTPUT)

In case if the keys list is statically set only with one value, the processor can be marked with @MetaDataStaticKey annotation (type = “Book”), where type is the statically set key. It is possible when the metadata structure of the connector doesn’t depend on the chosen key.

An example:

In this example the removal operation result always has the same structure, which doesn’t depend on the chosen key, that’s why we can assign the key statically.