(v24) Source Creation Tab

Source Creation Tab

This applies to: Visual Data Discovery

Use the Source Creation tab to define new sources, edit sources, and define the data entity or entities that make up your source.

Note: You must be logged in as user with the Administer Sources or Create New Data Sources privilege to see the Source Creation tab, or have Read and Write permissions on the source.

Note: When you upload a flat file, Select Schema and Select Entity fields are not present, and the option to create Custom SQL is not available. You can instead Select File, Edit File, and configure API Endpoints.

The general structure of this tab includes:

• In the upper portion of the tab, you can edit and update the Name and Description of this source in the Source Definition work area.

• In the lower portion of the tab, you can add and manage data entities in the Source Definition work area.

• In the bottom portion of the tab, you can manage Joins and Join Settings. The Join Definition work area is visible after you've added multiple data entities to create a Fusion source.

Source Creation Tab

Define basic information about this source.

Source Definition

Add and edit the unique Name and optional Description of your source.

Data Entity Definition

Add and edit data entities for this source. After adding a data entity or making changes, select Save Source to save your changes, or Preview Source to preview your data.

Entities

Select Add to add a new data entity from a connection or an uploaded file.

• Select From Connection to add a data entity from an existing source.

• Select From File to add a data entity from a file.

After you've entered a data entity, you can add a filter values entity to provide an alternative source of metadata values. Use these entities in a dynamic override on Filter Values tab in Fields to improve the filter experience of heavy data entities.

Entity Details - From Connection

Define a unique Data Entity Name, then define the details for how the information is accessed and presented.

Note: Depending on the your source, different options are available to define it.

When you select From Connection, details you can define can include:

• Entity Name - The unique name for this data entity.

• Select Connection: Select a connection for this data entity. You will only see connections you have access to.

• Select Schema: If a Schema is available for your connection, you can select a schema to filter a list of entities for this connection. See Connector Feature Support. Visualize your schema by selecting the view option. See View Relationships of a Schema in a Source.

• Select Indices: ElasticSearch connections support Indices: select one or more to include.

  • Indice Selection: Select Manually to create a merged list of fields from selected indices, or Automatically to select a pattern that automatically selects indices.

    

• Existing Entity and Custom SQL:

  • Select Existing Entity to use an available entity from your source.

  • Custom SQL: Select to define a custom SQL to retrieve the data you want from your source. Select Entity is not available if you select Custom SQL.

• Select Entity: Select an available entity. A list of all native fields from this entity populates Available Fields and the Preview table.

  • Available Fields: All fields available from this source are included by default. Disable (uncheck) specific fields to exclude them from the source. The fields, when disabled, are not included in any work areas of the source. If you attempt to remove a field in use by a visual or other object, Symphony will prevent you from saving your changes to the source configuration.

  • Data Entity Preview table: A preview of the data from the source for all fields, even if deselected in Available Fields.

  • Source Preview table: A preview of data from the source as defined in Available Fields. Select Preview Source to preview your selected data.

    In the example shown here, the date field is deselected in Available Fields. It is visible in the Data Entity Preview table, but not the Source Preview table.

    

Select Apply to apply your changes, or add another data entity to create joins for a Fusion source.

Custom SQL

When you select this option, a Custom SQL editing pane opens that you can use to write and run your SQL query. After you have run a successful query, the results populates Available Fields and the Preview table, and you can Apply your changes. You can't save invalid SQL.

Table visuals and Details dialogs display fields in the order they are retrieved from the source. When you create a source using custom SQL, your fields are shown in the order you specify.

If your Symphony environment makes use of Logi AI, you can download or create your own chatflows to generate custom SQL statements for use in your environment. See Generate SQL Statements.

Important: Custom SQL queries are a powerful tool for performing complex data queries. However, be careful when creating custom SQL queries because it is easy to define a heavy query or a query that may overwhelm your database. Use this feature carefully.

Variables (specified as custom user attributes) can be inserted in custom SQL. See Specify Custom User Attributes. In addition, you can use a vertical bar (|) in the SQL to separate the custom attribute name from a default value used for user definitions that do not have the custom user attribute defined. For example, the following custom SQL uses the value of the state customer user attribute to filter source data for records from whatever state the user's state custom user attribute is set to. If a state custom user attribute is not defined for a user, a default of Alabama is used.

SELECT * FROM Orders WHERE state = '${User.state|Alabama}'

Data Entity Details - From File

Define a unique Data Entity Name, then define the details for how the information is accessed and presented.

When you select From File, details you can define can include:

• Data Entity Name - The unique name for this data entity.

• Select File: Select an available uploaded file for this data entity. A list of all native fields from this entity populates Available Fields and the Preview table. Use Upload New File to add files to this source. See Manage File Uploads.

• API Endpoints: Select for more information about appending data, replacing data, or deleting data from your file upload.

• Edit File: Select to edit the File Details of your uploaded file.

Select Apply to apply your changes, or add another data entity to create joins for a Fusion source.

Join Definition

Create joins between pairs of data entities to create a Fusion source. You must have at least two data entities in source to create a join. If you have more than one data entity in your source, all entities must be used in a join.

To create a new join, select Add, then define the entities, join type, and fields. Select Apply to finish creating the join.

When you create a join, settings you can define can include:

• Entity Left: Select an available entity from the data entities you defined.

• Join Type: Select Left, Inner, or Full Outer.

• Entity Right: Select an available entity from the data entities you defined.

• Enable Dimension Entity: Select the gear () icon to enable dimension for one or both entities. This improves the performance of queries execution by removing unused data entities from the join.

• Field Left: Select at least one field from this entity. You can add multiple fields by selecting the add field button.

• Field Right: Select at least one field from this entity. You can add multiple fields by selecting the add field button.

You can also view the relationships of your joins and add more joins in a visualization. See Create a Fusion Source.