Data Partitioning & Collecting Methods in DataStage | +91 937 936 5515
Data Partitioning & Collecting Methods in DataStage
The following partitioning methods are available:
Auto:-. InfoSphere DataStage attempts to work out the best partitioning method depending on execution modes of current and preceding stages and how many nodes are specified in the configuration file. This is the default partitioning method for the Difference stage. If the incoming data is already key-partitioned and sorted, InfoSphere DataStage will use the Same method. Otherwise it will key partition and sort for you.
Entire:- Each file written to receives the entire data set.
Hash:- The records are hashed into partitions based on the value of a key column or columns selected from the Available list.
Modulus:-
The records are partitioned using a modulus function on the key column selected from the Available list. This is commonly used to partition on tag fields.
Random:-
The records are partitioned randomly, based on the output of a random number generator.
Round Robin:-
The records are partitioned on a round-robin basis as they enter the stage.
Same:-
Preserves the partitioning already in place.
Db2:-
Replicates the Db2 partitioning method of a specific Db2 table. Requires extra properties to be set. Access these properties by clicking the properties button.
Range:-
Divides a data set into approximately equal size partitions based on one or more partitioning keys. Range partitioning is often a preprocessing step to performing a total sort on a data set. Requires extra properties to be set. Access these properties by clicking the properties button.
The following Collection methods are available:
Auto:-
This is the default collection method for Difference stages. Normally, when you are using Auto mode, InfoSphere DataStage will eagerly read any row from any input partition as it becomes available. For the Difference stage, InfoSphere DataStage will ensure that the data is sorted as it is collected.
Ordered:-
Reads all records from the first partition, then all records from the second partition, and so on.
Round Robin:-
Reads a record from the first input partition, then from the second partition, and so on. After reaching the last partition, the operator starts over.
Sort Merge:-
Reads records in an order based on one or more columns of the record. This requires you to select a collecting key column from the Available list.