Dynamic File Processing

Situation: This data integration example takes (folder path + source schema entity id + mapping entity id + target schema entity id + target path) from a table and dynamically uses this information in the process flow.

Logic:

1) Iterate through the database table and pickup a folder path from each record.

2) Go to the actual folder location and check if any file is there or not by setting up a count (loop)  (it will not consider sub-folders as files)

• If the count is > 0 then it will go into the folder and process each file one by one (loop) by using the entity ids and assigning them to the activities
• If the count is < 0 then we go and get the next record from the source result set.

Important components:

1. Understanding how the record queue producer can queue and read one record at a time from db source.

2. Understanding how the custom plugin is used to count files in a folder and how it can be used in gateways

3. Using count in a gateway rule.

4. Understanding how the entity ids are assigned through put-context-var.

5. Learning how to use a front end Web Form to feed the source database table and manage the records in that table.

How to deploy the process at your end:

Use the Migration Utility to deploy the zip file “DynamicFileProcessing.zip” by going to Enviroment Promotion > Deploy > Upload this zip file

and follow the steps to complete the migration.