Thursday, April 3, 2014

Extracting every nth row

Extracting every nth row

Scenario: How to load every nth row from a Flat file/ relational DB to the target? Suppose n=3, then in above condition the row numbered 3,6,9,12,....so on, This example takes every 3 row to target table.
Solution:
  1. Connect an expression transformation after source qualifier.
    Add the next value port of sequence generator to expression transformation.
    Q14_src_seq_exp_1.PNG
  2. In expression create a new port (validate) and write the expression like in the picture below.
    exp mapping
  3. Connect a filter transformation to expression and write the condition in property like in the picture below.
    filter mapping
  4. Finally connect to target.
    final
 

No comments:

Post a Comment

 BEST PYSPARK LEARNING SITES https://www.youtube.com/watch?v=s3B8HXLlLTM&list=PL2IsFZBGM_IHCl9zhRVC1EXTomkEp_1zm&index=5 https://www...