LocalJobRunner (Hadoop 1.2.1 API)
Date: 2013-07-22

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.hadoop.mapred
Class LocalJobRunner

    java.lang.Object
      org.apache.hadoop.mapred.LocalJobRunner

    public class LocalJobRunner

extends Object

Implements MapReduce locally, in-process, for debugging.

Field Summary
`static String` | `LOCAL_MAX_MAPS`
`static org.apache.commons.logging.Log` | `LOG`
`static long` | `versionID`

Constructor Summary
`LocalJobRunner(JobConf conf)`

Method Summary
`void` | `cancelDelegationToken(Token<DelegationTokenIdentifier> token)` Cancel a delegation token.
`JobStatus[]` | `getAllJobs()` Get all the jobs submitted.
`TaskReport[]` | `getCleanupTaskReports(JobID id)` Grab a bunch of info on the cleanup tasks that make up the job
`ClusterStatus` | `getClusterStatus(boolean detailed)` Get the current status of the cluster
`Token<DelegationTokenIdentifier>` | `getDelegationToken(Text renewer)` Get a new delegation token.
`String` | `getFilesystemName()` A MapReduce system always operates on a single filesystem.
`Counters` | `getJobCounters(JobID id)` Grab the current job counters
`JobProfile` | `getJobProfile(JobID id)` Grab a handle to a job that is already known to the JobTracker.
`JobStatus[]` | `getJobsFromQueue(String queue)` Gets all the jobs submitted to the particular Queue
`JobStatus` | `getJobStatus(JobID id)` Grab a handle to a job that is already known to the JobTracker.
`static int` | `getLocalMaxRunningMaps(JobContext job)`
`TaskReport[]` | `getMapTaskReports(JobID id)` Grab a bunch of info on the map tasks that make up the job
`JobID` | `getNewJobId()` Allocate a name for the job.
`long` | `getProtocolVersion(String protocol, long clientVersion)` Return protocol version corresponding to protocol interface.
`QueueAclsInfo[]` | `getQueueAclsForCurrentUser()` Gets the Queue ACLs for current user
`AccessControlList` | `getQueueAdmins(String queueName)` Get the administrators of the given job-queue.
`JobQueueInfo` | `getQueueInfo(String queue)` Gets scheduling information associated with the particular Job queue
`JobQueueInfo[]` | `getQueues()` Gets set of Job Queues associated with the Job Tracker
`TaskReport[]` | `getReduceTaskReports(JobID id)` Grab a bunch of info on the reduce tasks that make up the job
`TaskReport[]` | `getSetupTaskReports(JobID id)` Grab a bunch of info on the setup tasks that make up the job
`String` | `getStagingAreaDir()` Get a hint from the JobTracker where job-specific files are to be placed.
`String` | `getSystemDir()` Grab the jobtracker system directory path where job-specific files are to be placed.
`TaskCompletionEvent[]` | `getTaskCompletionEvents(JobID jobid, int fromEventId, int maxEvents)` Get task completion events for the jobid, starting from fromEventId.
`String[]` | `getTaskDiagnostics(TaskAttemptID taskid)` Returns the diagnostic information for a particular task in the given job.
`JobStatus[]` | `jobsToComplete()` Get the jobs that are not completed and not failed
`void` | `killJob(JobID id)` Kill the indicated job
`boolean` | `killTask(TaskAttemptID taskId, boolean shouldFail)` Throws `UnsupportedOperationException`
`long` | `renewDelegationToken(Token<DelegationTokenIdentifier> token)` Renew an existing delegation token
`void` | `setJobPriority(JobID id, String jp)` Set the priority of the specified job
`static void` | `setLocalMaxRunningMaps(JobContext job, int maxMaps)` Set the max number of map tasks to run concurrently in the LocalJobRunner.
`JobStatus` | `submitJob(JobID jobid, String jobSubmitDir, Credentials credentials)` Submit a Job for execution.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### LOG

    public static final org.apache.commons.logging.Log LOG

### LOCAL_MAX_MAPS

    public static final String LOCAL_MAX_MAPS

See Also:
  Constant Field Values

### versionID

    public static final long versionID

See Also:
  Constant Field Values

Constructor Detail

### LocalJobRunner

    public LocalJobRunner(JobConf conf)
                   throws IOException

Throws:
  `IOException`

Method Detail

### getProtocolVersion

    public long getProtocolVersion(String protocol,
                                   long clientVersion)

  Description copied from interface: `VersionedProtocol`
  Return protocol version corresponding to protocol interface.

  Parameters:
    `protocol` - The classname of the protocol interface
    `clientVersion` - The version of the protocol that the client speaks
  Returns:
    the version that the server will speak

### getNewJobId

    public JobID getNewJobId()

  Allocate a name for the job.

  Returns:
    a unique job name for submitting jobs.

### submitJob

    public JobStatus submitJob(JobID jobid,
                               String jobSubmitDir,
                               Credentials credentials)
                        throws IOException

  Submit a Job for execution. Returns the latest profile for that job. The job files should be submitted in jobSubmitDir.

  Throws:
    `IOException`

### killJob

    public void killJob(JobID id)

  Kill the indicated job

### setJobPriority

    public void setJobPriority(JobID id,
                               String jp)
                        throws IOException

  Set the priority of the specified job

  Parameters:
    `id` - ID of the job
    `jp` - Priority to be set for the job
  Throws:
    `IOException`

### killTask

    public boolean killTask(TaskAttemptID taskId,
                            boolean shouldFail)
                     throws IOException

  Throws `UnsupportedOperationException`

  Parameters:
    `taskId` - the id of the task to kill.
    `shouldFail` - if true the task is failed and added to failed tasks list, otherwise it is just killed, w/o affecting job failure status.
  Throws:
    `IOException`

### getJobProfile

    public JobProfile getJobProfile(JobID id)

  Grab a handle to a job that is already known to the JobTracker.

  Returns:
    Profile of the job, or null if not found.

### getMapTaskReports

    public TaskReport[] getMapTaskReports(JobID id)

  Grab a bunch of info on the map tasks that make up the job

### getReduceTaskReports

    public TaskReport[] getReduceTaskReports(JobID id)

  Grab a bunch of info on the reduce tasks that make up the job

### getCleanupTaskReports

    public TaskReport[] getCleanupTaskReports(JobID id)

  Grab a bunch of info on the cleanup tasks that make up the job

### getSetupTaskReports

    public TaskReport[] getSetupTaskReports(JobID id)

  Grab a bunch of info on the setup tasks that make up the job

### getJobStatus

    public JobStatus getJobStatus(JobID id)

  Grab a handle to a job that is already known to the JobTracker.

  Returns:
    Status of the job, or null if not found.

### getJobCounters

    public Counters getJobCounters(JobID id)

  Grab the current job counters

### getFilesystemName

    public String getFilesystemName()
                             throws IOException

  A MapReduce system always operates on a single filesystem. This function returns the fs name. ('local' if the localfs; 'addr:port' if dfs). The client can then copy files into the right locations prior to submitting the job.

  Throws:
    `IOException`

### getClusterStatus

    public ClusterStatus getClusterStatus(boolean detailed)

  Get the current status of the cluster

  Parameters:
    `detailed` - if true then report tracker names and memory usage
  Returns:
    summary of the state of the cluster

### jobsToComplete

    public JobStatus[] jobsToComplete()

  Get the jobs that are not completed and not failed

  Returns:
    array of JobStatus for the running/to-be-run jobs.

### getTaskCompletionEvents

    public TaskCompletionEvent[] getTaskCompletionEvents(JobID jobid,
                                                         int fromEventId,
                                                         int maxEvents)
                                                  throws IOException

  Get task completion events for the jobid, starting from fromEventId. Returns empty aray if no events are available.

  Parameters:
    `jobid` - job id
    `fromEventId` - event id to start from.
    `maxEvents` - the max number of events we want to look at
  Returns:
    array of task completion events.
  Throws:
    `IOException`

### getAllJobs

    public JobStatus[] getAllJobs()

  Get all the jobs submitted.

  Returns:
    array of JobStatus for the submitted jobs

### getTaskDiagnostics

    public String[] getTaskDiagnostics(TaskAttemptID taskid)
                                throws IOException

  Returns the diagnostic information for a particular task in the given job. To be implemented

  Parameters:
    `taskid` - the id of the task
  Returns:
    an array of the diagnostic messages
  Throws:
    `IOException`

### getSystemDir

    public String getSystemDir()

  Grab the jobtracker system directory path where job-specific files are to be placed.

  Returns:
    the system directory where job-specific files are to be placed.
  See Also:
    `JobSubmissionProtocol.getSystemDir()`

### getQueueAdmins

    public AccessControlList getQueueAdmins(String queueName)
                                     throws IOException

  Get the administrators of the given job-queue. This method is for hadoop internal use only.

  Returns:
    Queue administrators ACL for the queue to which job is submitted to
  Throws:
    `IOException`
  See Also:
    `org.apache.hadoop.mapred.JobSubmissionProtocol#getQueueAdmins()`

### getStagingAreaDir

    public String getStagingAreaDir()
                             throws IOException

  Get a hint from the JobTracker where job-specific files are to be placed.

  Returns:
    the directory where job-specific files are to be placed.
  Throws:
    `IOException`
  See Also:
    `JobSubmissionProtocol.getStagingAreaDir()`

### getJobsFromQueue

    public JobStatus[] getJobsFromQueue(String queue)
                                 throws IOException

  Gets all the jobs submitted to the particular Queue

  Parameters:
    `queue` - Queue name
  Returns:
    array of JobStatus for the submitted jobs
  Throws:
    `IOException`

### getQueues

    public JobQueueInfo[] getQueues()
                             throws IOException

  Gets set of Job Queues associated with the Job Tracker

  Returns:
    Array of the Job Queue Information Object
  Throws:
    `IOException`

### getQueueInfo

    public JobQueueInfo getQueueInfo(String queue)
                              throws IOException

  Gets scheduling information associated with the particular Job queue

  Parameters:
    `queue` - Queue Name
  Returns:
    Scheduling Information of the Queue
  Throws:
    `IOException`

### getQueueAclsForCurrentUser

    public QueueAclsInfo[] getQueueAclsForCurrentUser()
                                               throws IOException

  Gets the Queue ACLs for current user

  Returns:
    array of QueueAclsInfo object for current user.
  Throws:
    `IOException`

### setLocalMaxRunningMaps

    public static void setLocalMaxRunningMaps(JobContext job,
                                              int maxMaps)

  Set the max number of map tasks to run concurrently in the LocalJobRunner.

  Parameters:
    `job` - the job to configure
    `maxMaps` - the maximum number of map tasks to allow.

### getLocalMaxRunningMaps

    public static int getLocalMaxRunningMaps(JobContext job)

  Returns:
    the max number of map tasks to run concurrently in the LocalJobRunner.

### cancelDelegationToken

    public void cancelDelegationToken(Token<DelegationTokenIdentifier> token)
                               throws IOException,
                                      InterruptedException

  Cancel a delegation token.

  Parameters:
    `token` - the token to cancel
  Throws:
    `IOException`
    `InterruptedException`

### getDelegationToken

    public Token<DelegationTokenIdentifier> getDelegationToken(Text renewer)
                                                        throws IOException,
                                                               InterruptedException

  Get a new delegation token.

  Parameters:
    `renewer` - the user other than the creator (if any) that can renew the token
  Returns:
    the new delegation token
  Throws:
    `IOException`
    `InterruptedException`

### renewDelegationToken

    public long renewDelegationToken(Token<DelegationTokenIdentifier> token)
                              throws IOException,
                                     InterruptedException

  Renew an existing delegation token

  Parameters:
    `token` - the token to renew
  Returns:
    the new expiration time
  Throws:
    `IOException`
    `InterruptedException`

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2009 The Apache Software Foundation