Slurm Workload Manager -

    .SH "DESCRIPTION"
    \fBsqueue\fR is used to view job and job step information for jobs managed by
    Slurm.

    .SH "OPTIONS"

    .TP
    \fB\-A <account_list>\fR, \fB\-\-account=<account_list>\fR
    Specify the accounts of the jobs to view. Accepts a comma separated
    list of account names. This has no effect when listing job steps.

    .TP
    \fB\-a\fR, \fB\-\-all\fR
    Display information about jobs and job steps in all partitions.
    This causes information to be displayed about partitions that are configured as
    hidden and partitions that are unavailable to user's group.

    .TP
    \fB\-r\fR, \fB\-\-array\fR
    Display one job array element per line.
    Without this option, the display will be optimized for use with job arrays
    (pending job array elements will be combined on one line of output with the
    array index values printed using a regular expression).

    .TP
    \fB\-\-array-unique\fR
    Display one unique pending job array element per line. Without this option,
    the pending job array elements will be grouped into the master array job to
    optimize the display.  This can also be set with the environment variable
    SQUEUE_ARRAY_UNIQUE.

    .TP
    \fB\-\-federation\fR
    Show jobs from the federation if a member of one.

    .TP
    \fB\-h\fR, \fB\-\-noheader\fR
    Do not print a header on the output.

    .TP
    \fB\-\-help\fR
    Print a help message describing all options \fBsqueue\fR.

    .TP
    \fB\-\-hide\fR
    Do not display information about jobs and job steps in all partitions. By default,
    information about partitions that are configured as hidden or are not available
    to the user's group will not be displayed (i.e. this is the default behavior).

    .TP
    \fB\-i <seconds>\fR, \fB\-\-iterate=<seconds>\fR
    The job ID format is "job_id[_array_id]".
    Performance of the command can be measurably improved for systems with large
    numbers of jobs when a single job ID is specified.
    By default, this field size will be limited to 64 bytes.
    Use the environment variable SLURM_BITSTR_LEN to specify larger field sizes.

    .TP
    \fB\-\-local\fR
    Show only jobs local to this cluster. Ignore other clusters in this federation
    (if any). Overrides \-\-federation.

    .TP
    \fB\-l\fR, \fB\-\-long\fR
    Report more of the available information for the selected jobs or job steps,
    subject to any constraints specified.

    .TP
    \fB\-L\fR, \fB\-\-licenses=<license_list>\fR
    Request jobs requesting or using one or more of the named licenses.
    The license list consists of a comma separated list of license names.

    .TP
    \fB\-\-me\fR
    Equivalent to \fB\-\-user=<my username>\fR.

    .TP
    \fB\-M\fR, \fB\-\-clusters=<cluster_name>\fR
    Clusters to issue commands to.  Multiple cluster names may be comma separated.
    A value of '\fIall\fR' will query to run on all clusters.
    This option implicitly sets the \fB\-\-local\fR option.

    .TP
    \fB\-n\fR, \fB\-\-name=<name_list>\fR
    Request jobs or job steps having one of the specified names.  The
    list consists of a comma separated list of job names.

    .TP
    \fB\-\-noconvert\fR
    Don't convert units from their original type (e.g. 2048M won't be converted to
    2G).

    .TP
    \fB\-o <output_format>\fR, \fB\-\-format=<output_format>\fR
    Specify the information to be displayed, its size and position
    (right or left justified).
    Also see the \fB\-O <output_format>\fR, \fB\-\-Format=<output_format>\fR
    option described below (which supports less flexibility in formatting, but
    supports access to all fields).
    If the command is executed in a federated cluster environment and information
    about more than one cluster is to be displayed and the \fB\-h, \-\-noheader\fR
    option is used, then the cluster name will be displayed before the default

    .IP
    The format of each field is "%[[.]size]type[suffix]"
    .RS 10
    .TP
    \fIsize\fR
    Minimum field size. If no size is specified, whatever is needed to print the
    information will be used.
    .TP
    \fI.\fR
    Indicates the output should be right justified and size must be specified.
    By default output is left justified.
    .TP
    \fIsuffix\fR
    Arbitrary string to append to the end of the field.
    .RE

    .IP
    Note that many of these \fItype\fR specifications are valid
    only for jobs while others are valid only for job steps.
    Valid \fItype\fR specifications include:

    .RS
    .TP 6
    \fB%all\fR
    Print all fields available for this data type with a vertical bar separating
    each field.
    .TP
    \fB%a\fR
    Account associated with the job.
    (Valid for jobs only)
    .TP
    \fB%A\fR
    Number of tasks created by a job step.
    This reports the value of the \fBsrun \-\-ntasks\fR option.
    (Valid for job steps only)
    .TP
    \fB%A\fR
    Job id.
    This will have a unique value for each element of job arrays.
    (Valid for jobs only)
    .TP
    \fB%B\fR
    Executing (batch) host. For an allocated session, this is the host on which
    the session is executing (i.e. the node from which the \fBsrun\fR or the
    \fBsalloc\fR command was executed). For a batch job, this is the node executing
    the batch script. In the case of a typical Linux cluster, this would be the
    compute node zero of the allocation. In the case of a Cray ALPS
    system, this would be the front\-end host whose slurmd daemon executes the job
    script.
    .TP
    \fB%c\fR
    .TP
    \fB%D\fR
    Number of nodes allocated to the job or the minimum number of nodes
    required by a pending job. The actual number of nodes allocated to a pending
    job may exceed this number if the job specified a node range count (e.g.
    minimum and maximum node counts) or the job specifies a processor
    count instead of a node count. As a job is completing this number will reflect
    the current number of nodes allocated.
    (Valid for jobs only)
    .TP
    \fB%e\fR
    Time at which the job ended or is expected to end (based upon its time limit).
    (Valid for jobs only)
    .TP
    \fB%E\fR
    Job dependencies remaining. This job will not begin execution until these
    dependent jobs complete. In the case of a job that can not run due to job
    dependencies never being satisfied, the full original job dependency
    specification will be reported. A value of NULL implies this job has no
    dependencies.
    (Valid for jobs only)
    .TP
    \fB%f\fR
    Features required by the job.
    (Valid for jobs only)
    .TP
    \fB%F\fR
    Job array's job ID. This is the base job ID.
    For non\-array jobs, this is the job ID.
    (Valid for jobs only)
    .TP
    \fB%g\fR
    Group name of the job.
    (Valid for jobs only)
    .TP
    \fB%G\fR
    Group ID of the job.
    (Valid for jobs only)
    .TP
    \fB%h\fR
    Can the compute resources allocated to the job be over subscribed by other jobs.
    The resources to be over subscribed can be nodes, sockets, cores, or
    hyperthreads depending upon configuration.
    The value will be "YES" if the job was submitted with the oversubscribe option
    or the partition is configured with OverSubscribe=Force,
    "NO" if the job requires exclusive node access,
    "USER" if the allocated compute nodes are dedicated to a single user,
    "MCS" if the allocated compute nodes are dedicated to a single security class
    (See MCSPlugin and MCSParameters configuration parameters for more information),
    "OK" otherwise (typically allocated dedicated CPUs),
    (Valid for jobs only)
    .TP
    form "#+#" where the first number is the "heterogeneous job leader" and the
    second number the zero origin offset for each component of the job.
    .TP
    \fB%I\fR
    Number of cores per socket requested by the job.
    This reports the value of the \fBsrun \-\-cores\-per\-socket\fR option.
    When \-\-cores\-per\-socket has not been set, "*" is displayed.
    (Valid for jobs only)
    .TP
    \fB%j\fR
    Job or job step name.
    (Valid for jobs and job steps)
    .TP
    \fB%J\fR
    Number of threads per core requested by the job.
    This reports the value of the \fBsrun \-\-threads\-per\-core\fR option.
    When \-\-threads\-per\-core has not been set, "*" is displayed.
    (Valid for jobs only)
    .TP
    \fB%k\fR
    Comment associated with the job.
    (Valid for jobs only)
    .TP
    \fB%K\fR
    Job array index.
    By default, this field size will be limited to 64 bytes.
    Use the environment variable SLURM_BITSTR_LEN to specify larger field sizes.
    (Valid for jobs only)
    .TP
    \fB%l\fR
    Time limit of the job or job step in days\-hours:minutes:seconds.
    The value may be "NOT_SET" if not yet established or "UNLIMITED" for no limit.
    (Valid for jobs and job steps)
    .TP
    \fB%L\fR
    Time left for the job to execute in days\-hours:minutes:seconds.
    This value is calculated by subtracting the job's time used from its time
    limit.
    The value may be "NOT_SET" if not yet established or "UNLIMITED" for no limit.
    (Valid for jobs only)
    .TP
    \fB%m\fR
    Minimum size of memory (in MB) requested by the job.
    (Valid for jobs only)
    .TP
    \fB%M\fR
    Time used by the job or job step in days\-hours:minutes:seconds.
    The days and hours are printed only as needed.
    For job steps this field shows the elapsed time since execution began
    and thus will be inaccurate for job steps which have been suspended.
    Clock skew between nodes in the cluster will cause the time to be inaccurate.
    If the time is obviously wrong (e.g. negative), it displays as "INVALID".
    .TP
    \fB%O\fR
    Are contiguous nodes requested by the job.
    (Valid for jobs only)
    .TP
    \fB%p\fR
    Priority of the job (converted to a floating point number between 0.0 and 1.0).
    Also see \fB%Q\fR.
    (Valid for jobs only)
    .TP
    \fB%P\fR
    Partition of the job or job step.
    (Valid for jobs and job steps)
    .TP
    \fB%q\fR
    Quality of service associated with the job.
    (Valid for jobs only)
    .TP
    \fB%Q\fR
    Priority of the job (generally a very large unsigned integer).
    Also see \fB%p\fR.
    (Valid for jobs only)
    .TP
    \fB%r\fR
    The reason a job is in its current state.
    See the \fBJOB REASON CODES\fR section below for more information.
    (Valid for jobs only)
    .TP
    \fB%R\fR
    For pending jobs: the reason a job is waiting for execution
    is printed within parenthesis.
    For terminated jobs with failure: an explanation as to why the
    job failed is printed within parenthesis.
    For all other job states: the list of allocate nodes.
    See the \fBJOB REASON CODES\fR section below for more information.
    (Valid for jobs only)
    .TP
    \fB%s\fR
    Node selection plugin specific data for a job. Possible data includes:
    Geometry requirement of resource allocation (X,Y,Z dimensions),
    Connection type (TORUS, MESH, or NAV == torus else mesh),
    Permit rotation of geometry (yes or no),
    Node use (VIRTUAL or COPROCESSOR), etc.
    (Valid for jobs only)
    .TP
    \fB%S\fR
    Actual or expected start time of the job or job step.
    (Valid for jobs and job steps)
    .TP
    \fB%t\fR
    Job state in compact form.
    See the \fBJOB STATE CODES\fR section below for a list of possible states.
    .TP
    \fB%v\fR
    Reservation for the job.
    (Valid for jobs only)
    .TP
    \fB%V\fR
    The job's submission time.
    .TP
    \fB%w\fR
    Workload Characterization Key (wckey).
    (Valid for jobs only)
    .TP
    \fB%W\fR
    Licenses reserved for the job.
    (Valid for jobs only)
    .TP
    \fB%x\fR
    List of node names explicitly excluded by the job.
    (Valid for jobs only)
    .TP
    \fB%X\fR
    Count of cores reserved on each node for system use (core specialization).
    (Valid for jobs only)
    .TP
    \fB%y\fR
    Nice value (adjustment to a job's scheduling priority).
    (Valid for jobs only)
    .TP
    \fB%Y\fR
    For pending jobs, a list of the nodes expected to be used when the job is
    started.
    .TP
    \fB%z\fR
    Number of requested sockets, cores, and threads (S:C:T) per node for the job.
    When (S:C:T) has not been set, "*" is displayed.
    (Valid for jobs only)
    .TP
    \fB%Z\fR
    The job's working directory.
    .RE

    .TP
    \fB\-O <output_format>\fR, \fB\-\-Format=<output_format>\fR
    Specify the information to be displayed.
    Also see the \fB\-o <output_format>\fR, \fB\-\-format=<output_format>\fR
    option described below (which supports greater flexibility in formatting, but
    does not support access to all fields because we ran out of letters).
    Requests a comma separated list of job information to be displayed.

    .IP
    The format of each field is "type[:[.][size][sufix]]"
    .IP
    Note that many of these \fItype\fR specifications are valid
    only for jobs while others are valid only for job steps.
    Valid \fItype\fR specifications include:

    .RS
    .TP 7
    \fBAccount\fR
    Print the account associated with the job.
    (Valid for jobs only)
    .TP
    \fBAccrueTime\fR
    Print the accrue time associated with the job.
    (Valid for jobs only)
    .TP
    \fBadmin_comment\fR
    Administrator comment associated with the job.
    (Valid for jobs only)
    .TP
    \fBAllocNodes\fR
    Print the nodes allocated to the job.
    (Valid for jobs only)
    .TP
    \fBAllocSID\fR
    Print the session ID used to submit the job.
    (Valid for jobs only)
    .TP
    \fBArrayJobID\fR
    Prints the job ID of the job array.
    (Valid for jobs and job steps)
    .TP
    \fBArrayTaskID\fR
    Prints the task ID of the job array.
    (Valid for jobs and job steps)
    .TP
    \fBAssocID\fR
    Prints the ID of the job association.
    (Valid for jobs only)
    .TP
    \fBBatchFlag\fR
    Prints whether the batch flag has been set.
    (Valid for jobs only)
    .TP
    \fBBatchHost\fR
    Executing (batch) host. For an allocated session, this is the host on which
    the session is executing (i.e. the node from which the \fBsrun\fR or the
    \fBsalloc\fR command was executed). For a batch job, this is the node executing
    the batch script. In the case of a typical Linux cluster, this would be the
    compute node zero of the allocation. In the case of a Cray ALPS
    system, this would be the front\-end host whose slurmd daemon executes the job
    script.
    (Valid for jobs only)
    Name of the cluster that is running the job or job step.
    .TP
    \fBClusterFeature\fR
    Cluster features required by the job.
    (Valid for jobs only)
    .TP
    \fBCommand\fR
    The command to be executed.
    (Valid for jobs only)
    .TP
    \fBComment\fR
    Comment associated with the job.
    (Valid for jobs only)
    .TP
    \fBContiguous\fR
    Are contiguous nodes requested by the job.
    (Valid for jobs only)
    .TP
    \fBCores\fR
    .RS
    Number of cores per socket requested by the job.
    This reports the value of the \fBsrun \-\-cores\-per\-socket\fR option.
    When \fB\-\-cores\-per\-socket\fR has not been set, "*" is displayed.
    (Valid for jobs only)
    .RE
    .TP
    \fBCoreSpec\fR
    Count of cores reserved on each node for system use (core specialization).
    (Valid for jobs only)
    .TP
    \fBCPUFreq\fR
    Prints the frequency of the allocated CPUs.
    (Valid for job steps only)
    .TP
    \fBcpus\-per\-task\fR
    Prints the number of CPUs per tasks allocated to the job.
    (Valid for jobs only)
    .TP
    \fBcpus\-per\-tres\fR
    Print the memory required per trackable resources allocated to the job or job step.
    .TP
    \fBDeadline\fR
    Prints the deadline affected to the job
    (Valid for jobs only)
    .TP
    \fBDelayBoot\fR
    Delay boot time.
    (Valid for jobs only)
    .TP
    \fBDependency\fR
    Job dependencies remaining. This job will not begin execution until these
    dependent jobs complete. In the case of a job that can not run due to job
    The time of job termination, actual or expected.
    (Valid for jobs only)
    .TP
    \fBexit_code\fR
    The exit code for the job.
    (Valid for jobs only)
    .TP
    \fBFeature\fR
    Features required by the job.
    (Valid for jobs only)
    .TP
    \fBGroupID\fR
    Group ID of the job.
    (Valid for jobs only)
    .TP
    \fBGroupName\fR
    Group name of the job.
    (Valid for jobs only)
    .TP
    \fBHetJobID\fR
    Job ID of the heterogeneous job leader.
    .TP
    \fBHetJobIDSet\fR
    Expression identifying all components job IDs within a heterogeneous job.
    .TP
    \fBHetJobOffset\fR
    Zero origin offset within a collection of heterogeneous job components.
    .TP
    \fBJobArrayID\fR
    Job array's job ID. This is the base job ID.
    For non\-array jobs, this is the job ID.
    (Valid for jobs only)
    .TP
    \fBJobID\fR
    .RS
    Job ID.
    This will have a unique value for each element of job arrays and each
    component of heterogeneous jobs.
    (Valid for jobs only)
    .RE
    .TP
    \fBLastSchedEval\fR
    Prints the last time the job was evaluated for scheduling.
    (Valid for jobs only)
    .TP
    \fBLicenses\fR
    Licenses reserved for the job.
    (Valid for jobs only)
    .TP
    \fBMaxCPUs\fR
    Prints the max number of CPUs allocated to the job.
    (Valid for jobs only)
    Minimum number of CPUs (processors) per node requested by the job.
    This reports the value of the \fBsrun \-\-mincpus\fR option with a
    default value of zero.
    (Valid for jobs only)
    .TP
    \fBMinMemory\fR
    Minimum size of memory (in MB) requested by the job.
    (Valid for jobs only)
    .TP
    \fBMinTime\fR
    Minimum time limit of the job
    (Valid for jobs only)
    .TP
    \fBMinTmpDisk\fR
    Minimum size of temporary disk space (in MB) requested by the job.
    (Valid for jobs only)
    .TP
    \fBName\fR
    .RS
    Job or job step name.
    (Valid for jobs and job steps)
    .RE
    .TP
    \fBNetwork\fR
    The network that the job is running on.
    (Valid for jobs and job steps)
    .TP
    \fBNice\fR
    .RS
    Nice value (adjustment to a job's scheduling priority).
    (Valid for jobs only)
    .RE
    .TP
    \fBNodeList\fR
    List of nodes allocated to the job or job step. In the case of a
    \fICOMPLETING\fR job, the list of nodes will comprise only those
    nodes that have not yet been returned to service.
    (Valid for jobs only)
    .TP
    \fBNodes\fR
    .RS
    List of nodes allocated to the job or job step. In the case of a
    \fICOMPLETING\fR job, the list of nodes will comprise only those
    nodes that have not yet been returned to service.
    (Valid job steps only)
    .RE
    .TP
    \fBNTPerBoard\fR
    The number of tasks per board allocated to the job.
    (Valid for jobs only)
    .TP
    \fBNTPerCore\fR
    reflect the current number of CPUs allocated.
    (Valid for jobs and job steps)
    .TP
    \fBNumNodes\fR
    Number of nodes allocated to the job or the minimum number of nodes
    required by a pending job. The actual number of nodes allocated to a pending
    job may exceed this number if the job specified a node range count (e.g.
    minimum and maximum node counts) or the job specifies a processor
    count instead of a node count. As a job is completing this number will reflect
    the current number of nodes allocated.
    (Valid for jobs only)
    .TP
    \fBNumTasks\fR
    Number of tasks requested by a job or job step.
    This reports the value of the \fB\-\-ntasks\fR option.
    (Valid for jobs and job steps)
    .TP
    \fBOrigin\fR
    .RS
    Cluster name where federated job originated from.
    (Valid for federated jobs only)
    .RE
    .TP
    \fBOriginRaw\fR
    Cluster ID where federated job originated from.
    (Valid for federated jobs only)
    .TP
    \fBOverSubscribe\fR
    Can the compute resources allocated to the job be over subscribed by other jobs.
    The resources to be over subscribed can be nodes, sockets, cores, or
    hyperthreads depending upon configuration.
    The value will be "YES" if the job was submitted with the oversubscribe option
    or the partition is configured with OverSubscribe=Force,
    "NO" if the job requires exclusive node access,
    "USER" if the allocated compute nodes are dedicated to a single user,
    "MCS" if the allocated compute nodes are dedicated to a single security class
    (See MCSPlugin and MCSParameters configuration parameters for more information),
    "OK" otherwise (typically allocated dedicated CPUs),
    (Valid for jobs only)
    .TP
    \fBPartition\fR
    Partition of the job or job step.
    (Valid for jobs and job steps)
    .TP
    \fBPreemptTime\fR
    The preempt time for the job.
    (Valid for jobs only)
    .TP
    \fBPriority\fR
    Priority of the job (converted to a floating point number between 0.0 and 1.0).
    Also see \fBprioritylong\fR.
    (Valid for jobs only)
    .RE
    .TP
    \fBReason\fR
    .RS
    The reason a job is in its current state.
    See the \fBJOB REASON CODES\fR section below for more information.
    (Valid for jobs only)
    .RE
    .TP
    \fBReasonList\fR
    For pending jobs: the reason a job is waiting for execution
    is printed within parenthesis.
    For terminated jobs with failure: an explanation as to why the
    job failed is printed within parenthesis.
    For all other job states: the list of allocate nodes.
    See the \fBJOB REASON CODES\fR section below for more information.
    (Valid for jobs only)
    .TP
    \fBReboot\fR
    .RS
    Indicates if the allocated nodes should be rebooted before starting the job.
    (Valid on jobs only)
    .RE
    .TP
    \fBReqNodes\fR
    List of node names explicitly requested by the job.
    (Valid for jobs only)
    .TP
    \fBReqSwitch\fR
    The max number of requested switches by for the job.
    (Valid for jobs only)
    .TP
    \fBRequeue\fR
    Prints whether the job will be requeued on failure.
    (Valid for jobs only)
    .TP
    \fBReservation\fR
    Reservation for the job.
    (Valid for jobs only)
    .TP
    \fBResizeTime\fR
    The amount of time changed for the job to run.
    (Valid for jobs only)
    .TP
    \fBRestartCnt\fR
    The number of restarts for the job.
    (Valid for jobs only)
    .TP
    \fBResvPort\fR
    Reserved ports of the job.
    (Valid for job steps only)
    .TP
    Geometry requirement of resource allocation (X,Y,Z dimensions),
    Connection type (TORUS, MESH, or NAV == torus else mesh),
    Permit rotation of geometry (yes or no),
    Node use (VIRTUAL or COPROCESSOR), etc.
    (Valid for jobs only)
    .TP
    \fBSiblingsActive\fR
    Cluster names of where federated sibling jobs exist.
    (Valid for federated jobs only)
    .TP
    \fBSiblingsActiveRaw\fR
    Cluster IDs of where federated sibling jobs exist.
    (Valid for federated jobs only)
    .TP
    \fBSiblingsViable\fR
    Cluster names of where federated sibling jobs are viable to run.
    (Valid for federated jobs only)
    .TP
    \fBSiblingsViableRaw\fR
    Cluster IDs of where federated sibling jobs viable to run.
    (Valid for federated jobs only)
    .TP
    \fBSockets\fR
    Number of sockets per node requested by the job.
    This reports the value of the \fBsrun \-\-sockets\-per\-node\fR option.
    When \fB\-\-sockets\-per\-node\fR has not been set, "*" is displayed.
    (Valid for jobs only)
    .TP
    \fBSPerBoard\fR
    Number of sockets per board allocated to the job.
    (Valid for jobs only)
    .TP
    \fBStartTime\fR
    Actual or expected start time of the job or job step.
    (Valid for jobs and job steps)
    .TP
    \fBState\fR
    .RS
    Job state in extended form.
    See the \fBJOB STATE CODES\fR section below for a list of possible states.
    (Valid for jobs only)
    .RE
    .TP
    \fBStateCompact\fR
    Job state in compact form.
    See the \fBJOB STATE CODES\fR section below for a list of possible states.
    (Valid for jobs only)
    .TP
    \fBSTDERR\fR
    .RS
    The directory for standard error to output to.
    (Valid for jobs only)
    \fBStepID\fR
    .RS
    Job or job step ID.
    In the case of job arrays, the job ID format will be of the form
    "<base_job_id>_<index>".
    (Valid forjob steps only)
    .RE
    .TP
    \fBStepName\fR
    Job step name.
    (Valid for job steps only)
    .TP
    \fBStepState\fR
    The state of the job step.
    (Valid for job steps only)
    .TP
    \fBSubmitTime\fR
    The time that the job was submitted at.
    (Valid for jobs only)
    .TP
    \fBsystem_comment\fR
    System comment associated with the job.
    (Valid for jobs only)
    .TP
    \fBThreads\fR
    Number of threads per core requested by the job.
    This reports the value of the \fBsrun \-\-threads\-per\-core\fR option.
    When \fB\-\-threads\-per\-core\fR has not been set, "*" is displayed.
    (Valid for jobs only)
    .TP
    \fBTimeLeft\fR
    Time left for the job to execute in days\-hours:minutes:seconds.
    This value is calculated by subtracting the job's time used from its time
    limit.
    The value may be "NOT_SET" if not yet established or "UNLIMITED" for no limit.
    (Valid for jobs only)
    .TP
    \fBTimeLimit\fR
    Timelimit for the job or job step.
    (Valid for jobs and job steps)
    .TP
    \fBTimeUsed\fR
    Time used by the job or job step in days\-hours:minutes:seconds.
    The days and hours are printed only as needed.
    For job steps this field shows the elapsed time since execution began
    and thus will be inaccurate for job steps which have been suspended.
    Clock skew between nodes in the cluster will cause the time to be inaccurate.
    If the time is obviously wrong (e.g. negative), it displays as "INVALID".
    (Valid for jobs and job steps)
    .TP
    \fBtres\-alloc\fR
    Print the trackable resources allocated to the job if running.
    \fBtres\-per\-socket\fR
    Print the trackable resources per socket requested by the job or job step.
    .TP
    \fBtres\-per\-step\fR
    Print the trackable resources requested by the job step.
    .TP
    \fBtres\-per\-task\fR
    Print the trackable resources per task requested by the job or job step.
    .TP
    \fBUserID\fR
    .RS
    User ID for a job or job step.
    (Valid for jobs and job steps)
    .RE
    .TP
    \fBUserName\fR
    User name for a job or job step.
    (Valid for jobs and job steps)
    .TP
    \fBWait4Switch\fR
    The amount of time to wait for the desired number of switches.
    (Valid for jobs only)
    .TP
    \fBWCKey\fR
    .RS
    Workload Characterization Key (wckey).
    (Valid for jobs only)
    .RE
    .TP
    \fBWorkDir\fR
    The job's working directory.
    (Valid for jobs only)
    .RE

    .TP
    \fB\-p <part_list>\fR, \fB\-\-partition=<part_list>\fR
    Specify the partitions of the jobs or steps to view. Accepts a comma separated
    list of partition names.

    .TP
    \fB\-P\fR, \fB\-\-priority\fR
    For pending jobs submitted to multiple partitions, list the job once per
    partition. In addition, if jobs are sorted by priority, consider both the
    partition and job priority. This option can be used to produce a list of
    pending jobs in the same order considered for scheduling by Slurm with
    appropriate additional options (e.g. "\-\-sort=\-p,i \-\-states=PD").

    .TP
    \fB\-q <qos_list>\fR, \fB\-\-qos=<qos_list>\fR
    Specify the qos(s) of the jobs or steps to view. Accepts a comma
    separated list of qos's.

    .TP
    \fB\-\-sibling\fR
    Show all sibling jobs on a federated cluster. Implies \-\-federation.

    .TP
    \fB\-S <sort_list>\fR, \fB\-\-sort=<sort_list>\fR
    Specification of the order in which records should be reported.
    This uses the same field specification as the <output_format>.
    The long format option "cluster" can also be used to sort jobs or job steps by
    cluster name (e.g. federated jobs).
    Multiple sorts may be performed by listing multiple sort fields
    separated by commas.
    The field specifications may be preceded by "+" or "\-" for
    ascending (default) and descending order respectively.
    For example, a sort value of "P,U" will sort the
    records by partition name then by user id.
    The default value of sort for jobs is "P,t,\-p" (increasing partition
    name then within a given partition by increasing job state and then
    decreasing priority).
    The default value of sort for job steps is "P,i" (increasing partition
    name then within a given partition by increasing step id).

    .TP
    \fB\-\-start\fR
    Report the expected start time and resources to be allocated for pending jobs
    in order of increasing start time.
    This is equivalent to the following options:
    \fB\-\-format="%.18i %.9P %.8j %.8u %.2t  %.19S %.6D %20Y %R"\fR,
    \fB\-\-sort=S\fR and \fB\-\-states=PENDING\fR.
    Any of these options may be explicitly changed as desired by
    combining the \fB\-\-start\fR option with other option values
    (e.g. to use a different output format).
    The expected start time of pending jobs is only available if the
    Slurm is configured to use the backfill scheduling plugin.

    .TP
    \fB\-t <state_list>\fR, \fB\-\-states=<state_list>\fR
    Specify the states of jobs to view.  Accepts a comma separated list of
    state names or "all". If "all" is specified then jobs of all states will be
    reported. If no state is specified then pending, running, and completing
    jobs are reported. See the \fBJOB STATE CODES\fR section below for a list of
    valid states. Both extended and compact forms are valid.
    Note the \fB<state_list>\fR supplied is case insensitive ("pd" and "PD" are
    equivalent).

    .TP
    \fB\-u <user_list>\fR, \fB\-\-user=<user_list>\fR
    Request jobs or job steps from a comma separated list of users.
    The list can consist of user names or user id numbers.
    Performance of the command can be measurably improved for systems with large
    numbers of jobs when a single user is specified.

    Report only on jobs allocated to the specified node or list of nodes.
    This may either be the \fBNodeName\fR or \fBNodeHostname\fR
    as defined in \fBslurm.conf(5)\fR in the event that they differ.
    A node_name of \fBlocalhost\fR is mapped to the current host name.

    .SH "JOB REASON CODES"
    These codes identify the reason that a job is waiting for execution.
    A job may be waiting for more than one reason, in which case only
    one of those reasons is displayed.
    .TP 22
    \fBAssociationJobLimit\fR
    The job's association has reached its maximum job count.
    .TP
    \fBAssociationResourceLimit\fR
    The job's association has reached some resource limit.
    .TP
    \fBAssociationTimeLimit\fR
    The job's association has reached its time limit.
    .TP
    \fBBadConstraints\fR
    The job's constraints can not be satisfied.
    .TP
    \fBBeginTime\fR
    The job's earliest start time has not yet been reached.
    .TP
    \fBCleaning\fR
    The job is being requeued and still cleaning up from its previous execution.
    .TP
    \fBDependency\fR
    This job is waiting for a dependent job to complete.
    .TP
    \fBFrontEndDown\fR
    No front end node is available to execute this job.
    .TP
    \fBInactiveLimit\fR
    The job reached the system InactiveLimit.
    .TP
    \fBInvalidAccount\fR
    The job's account is invalid.
    .TP
    \fBInvalidQOS\fR
    The job's QOS is invalid.
    .TP
    \fBJobHeldAdmin\fR
    The job is held by a system administrator.
    .TP
    \fBJobHeldUser\fR
    The job is held by the user.
    .TP
    \fBJobLaunchFailure\fR
    The job could not be launched.
    This may be due to a file system problem, invalid program name, etc.
    The partition required by this job is in an Inactive state and not able to
    start jobs.
    .TP
    \fBPartitionNodeLimit\fR
    The number of nodes required by this job is outside of its partition's current
    limits.
    Can also indicate that required nodes are DOWN or DRAINED.
    .TP
    \fBPartitionTimeLimit\fR
    The job's time limit exceeds its partition's current time limit.
    .TP
    \fBPriority\fR
    One or more higher priority jobs exist for this partition or advanced reservation.
    .TP
    \fBProlog\fR
    Its PrologSlurmctld program is still running.
    .TP
    \fBQOSJobLimit\fR
    The job's QOS has reached its maximum job count.
    .TP
    \fBQOSResourceLimit\fR
    The job's QOS has reached some resource limit.
    .TP
    \fBQOSTimeLimit\fR
    The job's QOS has reached its time limit.
    .TP
    \fBReqNodeNotAvail\fR
    Some node specifically required by the job is not currently available.
    The node may currently be in use, reserved for another job, in an advanced
    reservation, DOWN, DRAINED, or not responding.
    Nodes which are DOWN, DRAINED, or not responding will be identified as part
    of the job's "reason" field as "UnavailableNodes". Such nodes will typically
    require the intervention of a system administrator to make available.
    .TP
    \fBReservation\fR
    The job is waiting its advanced reservation to become available.
    .TP
    \fBResources\fR
    The job is waiting for resources to become available.
    .TP
    \fBSystemFailure\fR
    Failure of the Slurm system, a file system, the network, etc.
    .TP
    \fBTimeLimit\fR
    The job exhausted its time limit.
    .TP
    \fBQOSUsageThreshold\fR
    Required QOS threshold has been breached.
    .TP
    \fBWaitingForScheduling\fR
    No reason has been set for this job yet.
    Waiting for the scheduler to determine the appropriate reason.
    .TP
    \fBCD  COMPLETED\fR
    Job has terminated all processes on all nodes with an exit code of zero.
    .TP
    \fBCF  CONFIGURING\fR
    Job has been allocated resources, but are waiting for them to become ready for use
    (e.g. booting).
    .TP
    \fBCG  COMPLETING\fR
    Job is in the process of completing. Some processes on some nodes may still be active.
    .TP
    \fBDL  DEADLINE\fR
    Job terminated on deadline.
    .TP
    \fBF   FAILED\fR
    Job terminated with non\-zero exit code or other failure condition.
    .TP
    \fBNF  NODE_FAIL\fR
    Job terminated due to failure of one or more allocated nodes.
    .TP
    \fBOOM OUT_OF_MEMORY\fR
    Job experienced out of memory error.
    .TP
    \fBPD  PENDING\fR
    Job is awaiting resource allocation.
    .TP
    \fBPR  PREEMPTED\fR
    Job terminated due to preemption.
    .TP
    \fBR   RUNNING\fR
    Job currently has an allocation.
    .TP
    \fBRD  RESV_DEL_HOLD\fR
    Job is held.
    .TP
    \fBRF  REQUEUE_FED\fR
    Job is being requeued by a federation.
    .TP
    \fBRH  REQUEUE_HOLD\fR
    Held job is being requeued.
    .TP
    \fBRQ  REQUEUED\fR
    Completing job is being requeued.
    .TP
    \fBRS  RESIZING\fR
    Job is about to change size.
    .TP
    \fBRV  REVOKED\fR
    Sibling was removed from cluster due to other cluster starting the job.
    .TP
    \fBSI  SIGNALING\fR
    Job is being signaled.
    Job has an allocation, but execution has been suspended and CPUs have been
    released for other jobs.
    .TP
    \fBTO  TIMEOUT\fR
    Job terminated upon reaching its time limit.

    .SH "PERFORMANCE"
    .PP
    Executing \fBsqueue\fR sends a remote procedure call to \fBslurmctld\fR. If
    enough calls from \fBsqueue\fR or other Slurm client commands that send remote
    procedure calls to the \fBslurmctld\fR daemon come in at once, it can result in
    a degradation of performance of the \fBslurmctld\fR daemon, possibly resulting
    in a denial of service.
    .PP
    Do not run \fBsqueue\fR or other Slurm client commands that send remote
    procedure calls to \fBslurmctld\fR from loops in shell scripts or other
    programs. Ensure that programs limit calls to \fBsqueue\fR to the minimum
    necessary for the information you are trying to gather.

    .SH "ENVIRONMENT VARIABLES"
    .PP
    Some \fBsqueue\fR options may be set via environment variables. These
    environment variables, along with their corresponding options, are listed
    below. (Note: Commandline options will always override these settings.)
    .TP 20
    \fBSLURM_BITSTR_LEN\fR
    Specifies the string length to be used for holding a job array's task ID
    expression.
    The default value is 64 bytes.
    A value of 0 will print the full expression with any length required.
    Larger values may adversely impact the application performance.
    .TP
    \fBSLURM_CLUSTERS\fR
    Same as \fB\-\-clusters\fR
    .TP
    \fBSLURM_CONF\fR
    The location of the Slurm configuration file.
    .TP
    \fBSLURM_TIME_FORMAT\fR
    Specify the format used to report time stamps. A value of \fIstandard\fR, the
    default value, generates output in the form "year\-month\-dateThour:minute:second".
    A value of \fIrelative\fR returns only "hour:minute:second" if the current day.
    For other dates in the current year it prints the "hour:minute" preceded by
    "Tomorr" (tomorrow), "Ystday" (yesterday), the name of the day for the coming
    week (e.g. "Mon", "Tue", etc.), otherwise the date (e.g. "25 Apr").
    For other years it returns a date month and year without a time (e.g.
    "6 Jun 2012"). All of the time stamps use a 24 hour format.

    A valid strftime() format can also be specified. For example, a value of
    "%a %T" will report the day of the week and a time stamp (e.g. "Mon 12:34:56").
    .TP
    \fBSQUEUE_ACCOUNT\fR
    \fBSQUEUE_FORMAT\fR
    \fB\-o <output_format>, \-\-format=<output_format>\fR
    .TP
    \fBSQUEUE_FORMAT2\fR
    \fB\-O <output_format>, \-\-Format=<output_format>\fR
    .TP
    \fBSQUEUE_LICENSES\fR
    \fB\-p-l <license_list>, \-\-license=<license_list>\fR
    .TP
    \fBSQUEUE_LOCAL\fR
    \fB\-\-local\fR
    .TP
    \fBSQUEUE_PARTITION\fR
    \fB\-p <part_list>, \-\-partition=<part_list>\fR
    .TP
    \fBSQUEUE_PRIORITY\fR
    \fB\-P\fR, \fB\-\-priority\fR
    .TP
    \fBSQUEUE_QOS\fR
    \fB\-p <qos_list>, \-\-qos=<qos_list>\fR
    .TP
    \fBSQUEUE_SIBLING\fR
    \fB\-\-sibling\fR
    .TP
    \fBSQUEUE_SORT\fR
    \fB\-S <sort_list>, \-\-sort=<sort_list>\fR
    .TP
    \fBSQUEUE_STATES\fR
    \fB\-t <state_list>, \-\-states=<state_list>\fR
    .TP
    \fBSQUEUE_USERS\fR
    \fB\-u <user_list>, \-\-users=<user_list>\fR

    .SH "EXAMPLES"
    .eo
    Print the jobs scheduled in the debug partition and in the
    COMPLETED state in the format with six right justified digits for
    the job id followed by the priority with an arbitrary fields size:
    .br
    # squeue -p debug -t COMPLETED -o "%.6i %p"
    .br
     JOBID PRIORITY
    .br
     65543 99993
    .br
     65544 99992
    .br
     65545 99991
    .ec

    .eo
    Print the job steps in the debug partition sorted by user:
    .br
    # squeue --jobs 12345,12346,12348
    .br
     JOBID PARTITION NAME USER ST  TIME  NODES NODELIST(REASON)
    .br
     12345     debug job1 dave  R   0:21     4 dev[9-12]
    .br
     12346     debug job2 dave PD   0:00     8 (Resources)
    .br
     12348     debug job3 ed   PD   0:00     4 (Priority)
    .ec

    .eo
    Print information only about job step 65552.1:
    .br
    # squeue --steps 65552.1
    .br
      STEPID     NAME PARTITION    USER    TIME  NODELIST
    .br
     65552.1    test2     debug   alice   12:49  dev[1-4]
    .ec

    .SH "COPYING"
    Copyright (C) 2002\-2007 The Regents of the University of California.
    Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
    .br
    Copyright (C) 2008\-2010 Lawrence Livermore National Security.
    .br
    Copyright (C) 2010\-2016 SchedMD LLC.
    .LP
    This file is part of Slurm, a resource management program.
    For details, see <https://slurm.schedmd.com/>.
    .LP
    Slurm is free software; you can redistribute it and/or modify it under
    the terms of the GNU General Public License as published by the Free
    Software Foundation; either version 2 of the License, or (at your option)
    any later version.
    .LP
    Slurm is distributed in the hope that it will be useful, but WITHOUT ANY
    WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
    FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
    details.
    .SH "SEE ALSO"
    \fBscancel\fR(1), \fBscontrol\fR(1), \fBsinfo\fR(1), \fBsrun\fR(1),
    \fBslurm_load_ctl_conf\fR (3), \fBslurm_load_jobs\fR (3),
    \fBslurm_load_node\fR (3),
    \fBslurm_load_partitions\fR (3)