Grouping Large Data


I have a large data set (20MB-50MB) in xml; generated through other data transformations. I need to group this data set up by a single element and output the results of that single element.

I used the xpath expression from the article,, which works great on small data sets, however, on a large data set, the xpath expression is very slow.

I then applied the splitting feature to the data set. This was fast, very nice, however, the results were a merged data set of the individual chunks grouped instead of the total of all chunks grouped.

Any ideas of a better way to group up an element of a large data set?



1. Create a key. Data Mapper->Global->Key Functions

2. Apply record filter: Data Mapper->(Select Destination Record)->Node->Properties->ForEach


3. Apply Mapping Rules: Data Mapper->(Select Destination Element)->Node->Textual Rules



Generated XSL


Have more questions? Submit a request


Article is closed for comments.