In this case we cannot go for remove duplicate stage as u cannot capture the duplicate records,
Hence we should go for a sort stage.
In sort stage, set the option "create keychange column" to true.
This will create one new column Key change at the ouput of the sort stage which will be 1 for the unique record and 0 for duplicate record.
Now by using a transformer u can separate it.
No comments:
Post a Comment