I have a large data set (20MB-50MB) in xml; generated through other data transformations. I need to group this data set up by a single element and output the results of that single element.
I used the xpath expression from the article, http://support.adeptia.com/entries/22255876-how-to-use-duplicate-rule-in-mapping, which works great on small data sets, however, on a large data set, the xpath expression is very slow.
I then applied the splitting feature to the data set. This was fast, very nice, however, the results were a merged data set of the individual chunks grouped instead of the total of all chunks grouped.
Any ideas of a better way to group up an element of a large data set?
1. Create a key. Data Mapper->Global->Key Functions
2. Apply record filter: Data Mapper->(Select Destination Record)->Node->Properties->ForEach
3. Apply Mapping Rules: Data Mapper->(Select Destination Element)->Node->Textual Rules