CqlInputFormat (apache-cassandra API)
Date: 2014-10-21

org.apache.cassandra.hadoop.cql3

## Class CqlInputFormat

- java.lang.Object
- - org.apache.hadoop.mapreduce.InputFormat<K,Y>
  - - org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat<java.lang.Long,com.datastax.driver.core.Row>
    - - org.apache.cassandra.hadoop.cql3.CqlInputFormat

- All Implemented Interfaces:
    org.apache.hadoop.mapred.InputFormat<java.lang.Long,com.datastax.driver.core.Row>

      public class CqlInputFormat
      extends AbstractColumnFamilyInputFormat<java.lang.Long,com.datastax.driver.core.Row>

  Hadoop InputFormat allowing map/reduce against Cassandra rows within one ColumnFamily. At minimum, you need to set the KS and CF in your Hadoop job Configuration. The ConfigHelper class is provided to make this simple: ConfigHelper.setInputColumnFamily You can also configure the number of rows per InputSplit with ConfigHelper.setInputSplitSize. The default split size is 64k rows. the number of CQL rows per page CQLConfigHelper.setInputCQLPageRowSize. The default page row size is 1000. You should set it to "as big as possible, but no bigger." It set the LIMIT for the CQL query, so you need set it big enough to minimize the network overhead, and also not too big to avoid out of memory issue. other native protocol connection parameters in CqlConfigHelper

- - ### Field Summary

    - ### Fields inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat

      `CASSANDRA_HADOOP_MAX_KEY_SIZE, CASSANDRA_HADOOP_MAX_KEY_SIZE_DEFAULT, MAPRED_TASK_ID`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `CqlInputFormat()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,com.datastax.driver.core.Row>` | `createRecordReader(org.apache.hadoop.mapreduce.InputSplit arg0, org.apache.hadoop.mapreduce.TaskAttemptContext arg1)`
    `org.apache.hadoop.mapred.RecordReader<java.lang.Long,com.datastax.driver.core.Row>` | `getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.mapred.Reporter reporter)`

    - ### Methods inherited from class org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat

      `createAuthenticatedClient, getSplits, getSplits, validateConfiguration`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### CqlInputFormat

          public CqlInputFormat()

  - ### Method Detail

    - #### getRecordReader

          public org.apache.hadoop.mapred.RecordReader<java.lang.Long,com.datastax.driver.core.Row> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                                                                           org.apache.hadoop.mapred.JobConf jobConf,
                                                                                                           org.apache.hadoop.mapred.Reporter reporter)
                                                                                                             throws java.io.IOException

      Throws:
        `java.io.IOException`

    - #### createRecordReader

          public org.apache.hadoop.mapreduce.RecordReader<java.lang.Long,com.datastax.driver.core.Row> createRecordReader(org.apache.hadoop.mapreduce.InputSplit arg0,
                                                                                                                 org.apache.hadoop.mapreduce.TaskAttemptContext arg1)
                                                                                                                   throws java.io.IOException,
                                                                                                                          java.lang.InterruptedException

      Specified by:
        `createRecordReader` in class `org.apache.hadoop.mapreduce.InputFormat<java.lang.Long,com.datastax.driver.core.Row>`
      Throws:
        `java.io.IOException`
        `java.lang.InterruptedException`

Copyright © 2014 The Apache Software Foundation