FilterFileSystem (Hadoop 1.2.1 API)
Date: 2013-07-22

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.hadoop.fs
Class FilterFileSystem

    java.lang.Object
      org.apache.hadoop.conf.Configured
          org.apache.hadoop.fs.FileSystem
              org.apache.hadoop.fs.FilterFileSystem

All Implemented Interfaces:
  Closeable, Configurable

Direct Known Subclasses:
  ChecksumFileSystem, HarFileSystem

    public class FilterFileSystem

extends FileSystem

A `FilterFileSystem` contains some other file system, which it uses as its basic file system, possibly transforming the data along the way or providing additional functionality. The class `FilterFileSystem` itself simply overrides all methods of `FileSystem` with versions that pass all requests to the contained file system. Subclasses of `FilterFileSystem` may further override some of these methods and may also provide additional methods and fields.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem
`FileSystem.Statistics`

Field Summary
`protected FileSystem` | `fs`

Fields inherited from class org.apache.hadoop.fs.FileSystem
`FS_DEFAULT_NAME_KEY, LOG, statistics`

Constructor Summary
`FilterFileSystem()`
`FilterFileSystem(FileSystem fs)`

Method Summary
`FSDataOutputStream` | `append(Path f, int bufferSize, Progressable progress)` Append to an existing file (optional operation).
`protected void` | `checkPath(Path path)` Check that a Path belongs to this FileSystem.
`void` | `close()` No more filesystem operations are needed.
`void` | `completeLocalOutput(Path fsOutputFile, Path tmpLocalFile)` Called when we're all done writing to the target.
`void` | `concat(Path f, Path[] psrcs)` Concat existing files together.
`void` | `copyFromLocalFile(boolean delSrc, Path src, Path dst)` The src file is on the local disk.
`void` | `copyToLocalFile(boolean delSrc, Path src, Path dst)` The src file is under FS, and the dst is on the local disk.
`FSDataOutputStream` | `create(Path f, FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress)` Opens an FSDataOutputStream at the indicated Path with write-progress reporting.
`boolean` | `delete(Path f)` Deprecated.
`boolean` | `delete(Path f, boolean recursive)` Delete a file
`String` | `getCanonicalServiceName()` Get a canonical service name for this file system.
`Configuration` | `getConf()` Return the configuration used by this object.
`long` | `getDefaultBlockSize()` Return the number of bytes that large input files should be optimally be split into to minimize i/o time.
`short` | `getDefaultReplication()` Get the default replication.
`BlockLocation[]` | `getFileBlockLocations(FileStatus file, long start, long len)` Return an array containing hostnames, offset and size of portions of the given file.
`FileChecksum` | `getFileChecksum(Path f)` Get the checksum of a file.
`FileStatus` | `getFileStatus(Path f)` Get file status.
`Path` | `getHomeDirectory()` Return the current user's home directory in this filesystem.
`String` | `getName()` Deprecated. call #getUri() instead.
`URI` | `getUri()` Returns a URI whose scheme and authority identify this FileSystem.
`Path` | `getWorkingDirectory()` Get the current working directory for the given file system
`void` | `initialize(URI name, Configuration conf)` Called after a new FileSystem instance is constructed.
`FileStatus[]` | `listStatus(Path f)` List files in a directory.
`Path` | `makeQualified(Path path)` Make sure that a path specifies a FileSystem.
`boolean` | `mkdirs(Path f, FsPermission permission)` Make the given file and all non-existent parents into directories.
`FSDataInputStream` | `open(Path f, int bufferSize)` Opens an FSDataInputStream at the indicated Path.
`boolean` | `rename(Path src, Path dst)` Renames Path src to Path dst.
`void` | `setOwner(Path p, String username, String groupname)` Set owner of a path (i.e.
`void` | `setPermission(Path p, FsPermission permission)` Set permission of a path.
`boolean` | `setReplication(Path src, short replication)` Set replication for an existing file.
`void` | `setVerifyChecksum(boolean verifyChecksum)` Set the verify checksum flag.
`void` | `setWorkingDirectory(Path newDir)` Set the current working directory for the given file system.
`Path` | `startLocalOutput(Path fsOutputFile, Path tmpLocalFile)` Returns a local File that the user can write output to.

Methods inherited from class org.apache.hadoop.fs.FileSystem
`addFileSystemForTesting, append, append, clearStatistics, closeAll, closeAllForUGI, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, deleteOnExit, exists, get, get, get, getAllStatistics, getBlockSize, getCacheSize, getCanonicalUri, getContentSummary, getDefaultBlockSize, getDefaultPort, getDefaultReplication, getDefaultUri, getDelegationToken, getLength, getLocal, getNamed, getReplication, getStatistics, getStatistics, getUsed, globStatus, globStatus, isDirectory, isFile, listStatus, listStatus, listStatus, mkdirs, mkdirs, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, open, printStatistics, processDeleteOnExit, setDefaultUri, setDefaultUri, setTimes`

Methods inherited from class org.apache.hadoop.conf.Configured
`setConf`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### fs

    protected FileSystem fs

Constructor Detail

### FilterFileSystem

    public FilterFileSystem()

### FilterFileSystem

    public FilterFileSystem(FileSystem fs)

Method Detail

### initialize

    public void initialize(URI name,
                           Configuration conf)
                    throws IOException

  Called after a new FileSystem instance is constructed.

  Overrides:
    `initialize` in class `FileSystem`
  Parameters:
    `name` - a uri whose authority section names the host, port, etc. for this FileSystem
    `conf` - the configuration
  Throws:
    `IOException`

### getUri

    public URI getUri()

  Returns a URI whose scheme and authority identify this FileSystem.

  Specified by:
    `getUri` in class `FileSystem`

### getCanonicalServiceName

    public String getCanonicalServiceName()

  Description copied from class: `FileSystem`
  Get a canonical service name for this file system. The token cache is the only user of this value, and uses it to lookup this filesystem's service tokens. The token cache will not attempt to acquire tokens if the service is null.

  Overrides:
    `getCanonicalServiceName` in class `FileSystem`
  Returns:
    a service string that uniquely identifies this file system, null if the filesystem does not implement tokens
  See Also:
    `SecurityUtil.buildDTServiceName(URI, int)`

### getName

    public String getName()

  Deprecated. call #getUri() instead.

  Overrides:
    `getName` in class `FileSystem`

### makeQualified

    public Path makeQualified(Path path)

  Make sure that a path specifies a FileSystem.

  Overrides:
    `makeQualified` in class `FileSystem`

### checkPath

    protected void checkPath(Path path)

  Check that a Path belongs to this FileSystem.

  Overrides:
    `checkPath` in class `FileSystem`

### getFileBlockLocations

    public BlockLocation[] getFileBlockLocations(FileStatus file,
                                                 long start,
                                                 long len)
                                          throws IOException

  Description copied from class: `FileSystem`
  Return an array containing hostnames, offset and size of portions of the given file. For a nonexistent file or regions, null will be returned. This call is most helpful with DFS, where it returns hostnames of machines that contain the given file. The FileSystem will simply return an elt containing 'localhost'.

  Overrides:
    `getFileBlockLocations` in class `FileSystem`
  Throws:
    `IOException`

### open

    public FSDataInputStream open(Path f,
                                  int bufferSize)
                           throws IOException

  Opens an FSDataInputStream at the indicated Path.

  Specified by:
    `open` in class `FileSystem`
  Parameters:
    `f` - the file name to open
    `bufferSize` - the size of the buffer to be used.
  Throws:
    `IOException`

### append

    public FSDataOutputStream append(Path f,
                                     int bufferSize,
                                     Progressable progress)
                              throws IOException

  Append to an existing file (optional operation).

  Specified by:
    `append` in class `FileSystem`
  Parameters:
    `f` - the existing file to be appended.
    `bufferSize` - the size of the buffer to be used.
    `progress` - for reporting progress if it is not null.
  Throws:
    `IOException`

### concat

    public void concat(Path f,
                       Path[] psrcs)
                throws IOException

  Description copied from class: `FileSystem`
  Concat existing files together.

  Overrides:
    `concat` in class `FileSystem`
  Parameters:
    `f` - the path to the target destination.
  Throws:
    `IOException`

### create

    public FSDataOutputStream create(Path f,
                                     FsPermission permission,
                                     boolean overwrite,
                                     int bufferSize,
                                     short replication,
                                     long blockSize,
                                     Progressable progress)
                              throws IOException

  Description copied from class: `FileSystem`
  Opens an FSDataOutputStream at the indicated Path with write-progress reporting.

  Specified by:
    `create` in class `FileSystem`
  Parameters:
    `f` - the file name to open
    `overwrite` - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.
    `bufferSize` - the size of the buffer to be used.
    `replication` - required block replication for the file.
  Throws:
    `IOException`
  See Also:
    `FileSystem.setPermission(Path, FsPermission)`

### setReplication

    public boolean setReplication(Path src,
                                  short replication)
                           throws IOException

  Set replication for an existing file.

  Overrides:
    `setReplication` in class `FileSystem`
  Parameters:
    `src` - file name
    `replication` - new replication
  Returns:
    true if successful; false if file does not exist or is a directory
  Throws:
    `IOException`

### rename

    public boolean rename(Path src,
                          Path dst)
                   throws IOException

  Renames Path src to Path dst. Can take place on local fs or remote DFS.

  Specified by:
    `rename` in class `FileSystem`
  Throws:
    `IOException`

### delete

    @Deprecated
    public boolean delete(Path f)
                   throws IOException

  Deprecated.

  Delete a file

  Specified by:
    `delete` in class `FileSystem`
  Throws:
    `IOException`

### delete

    public boolean delete(Path f,
                          boolean recursive)
                   throws IOException

  Delete a file

  Specified by:
    `delete` in class `FileSystem`
  Parameters:
    `f` - the path to delete.
    `recursive` - if path is a directory and set to true, the directory is deleted else throws an exception. In case of a file the recursive can be set to either true or false.
  Returns:
    true if delete is successful else false.
  Throws:
    `IOException`

### listStatus

    public FileStatus[] listStatus(Path f)
                            throws IOException

  List files in a directory.

  Specified by:
    `listStatus` in class `FileSystem`
  Parameters:
    `f` - given path
  Returns:
    the statuses of the files/directories in the given patch returns null, if Path f does not exist in the FileSystem
  Throws:
    `IOException`

### getHomeDirectory

    public Path getHomeDirectory()

  Description copied from class: `FileSystem`
  Return the current user's home directory in this filesystem. The default implementation returns "/user/$USER/".

  Overrides:
    `getHomeDirectory` in class `FileSystem`

### setWorkingDirectory

    public void setWorkingDirectory(Path newDir)

  Set the current working directory for the given file system. All relative paths will be resolved relative to it.

  Specified by:
    `setWorkingDirectory` in class `FileSystem`
  Parameters:
    `newDir` -

### getWorkingDirectory

    public Path getWorkingDirectory()

  Get the current working directory for the given file system

  Specified by:
    `getWorkingDirectory` in class `FileSystem`
  Returns:
    the directory pathname

### mkdirs

    public boolean mkdirs(Path f,
                          FsPermission permission)
                   throws IOException

  Make the given file and all non-existent parents into directories. Has the semantics of Unix 'mkdir -p'. Existence of the directory hierarchy is not an error.

  Specified by:
    `mkdirs` in class `FileSystem`
  Throws:
    `IOException`

### copyFromLocalFile

    public void copyFromLocalFile(boolean delSrc,
                                  Path src,
                                  Path dst)
                           throws IOException

  The src file is on the local disk. Add it to FS at the given dst name. delSrc indicates if the source should be removed

  Overrides:
    `copyFromLocalFile` in class `FileSystem`
  Throws:
    `IOException`

### copyToLocalFile

    public void copyToLocalFile(boolean delSrc,
                                Path src,
                                Path dst)
                         throws IOException

  The src file is under FS, and the dst is on the local disk. Copy it from FS control to the local dst name. delSrc indicates if the src will be removed or not.

  Overrides:
    `copyToLocalFile` in class `FileSystem`
  Throws:
    `IOException`

### startLocalOutput

    public Path startLocalOutput(Path fsOutputFile,
                                 Path tmpLocalFile)
                          throws IOException

  Returns a local File that the user can write output to. The caller provides both the eventual FS target name and the local working file. If the FS is local, we write directly into the target. If the FS is remote, we write into the tmp local area.

  Overrides:
    `startLocalOutput` in class `FileSystem`
  Throws:
    `IOException`

### completeLocalOutput

    public void completeLocalOutput(Path fsOutputFile,
                                    Path tmpLocalFile)
                             throws IOException

  Called when we're all done writing to the target. A local FS will do nothing, because we've written to exactly the right place. A remote FS will copy the contents of tmpLocalFile to the correct target at fsOutputFile.

  Overrides:
    `completeLocalOutput` in class `FileSystem`
  Throws:
    `IOException`

### getDefaultBlockSize

    public long getDefaultBlockSize()

  Return the number of bytes that large input files should be optimally be split into to minimize i/o time.

  Overrides:
    `getDefaultBlockSize` in class `FileSystem`

### getDefaultReplication

    public short getDefaultReplication()

  Get the default replication.

  Overrides:
    `getDefaultReplication` in class `FileSystem`

### getFileStatus

    public FileStatus getFileStatus(Path f)
                             throws IOException

  Get file status.

  Specified by:
    `getFileStatus` in class `FileSystem`
  Parameters:
    `f` - The path we want information from
  Returns:
    a FileStatus object
  Throws:
    `FileNotFoundException` - when the path does not exist; IOException see specific implementation
    `IOException`

### getFileChecksum

    public FileChecksum getFileChecksum(Path f)
                                 throws IOException

  Get the checksum of a file.

  Overrides:
    `getFileChecksum` in class `FileSystem`
  Parameters:
    `f` - The file path
  Returns:
    The file checksum. The default return value is null, which indicates that no checksum algorithm is implemented in the corresponding FileSystem.
  Throws:
    `IOException`

### setVerifyChecksum

    public void setVerifyChecksum(boolean verifyChecksum)

  Set the verify checksum flag. This is only applicable if the corresponding FileSystem supports checksum. By default doesn't do anything.

  Overrides:
    `setVerifyChecksum` in class `FileSystem`

### getConf

    public Configuration getConf()

  Description copied from interface: `Configurable`
  Return the configuration used by this object.

  Specified by:
    `getConf` in interface `Configurable`
  Overrides:
    `getConf` in class `Configured`

### close

    public void close()
               throws IOException

  Description copied from class: `FileSystem`
  No more filesystem operations are needed. Will release any held locks.

  Specified by:
    `close` in interface `Closeable`
  Overrides:
    `close` in class `FileSystem`
  Throws:
    `IOException`

### setOwner

    public void setOwner(Path p,
                         String username,
                         String groupname)
                  throws IOException

  Set owner of a path (i.e. a file or a directory). The parameters username and groupname cannot both be null.

  Overrides:
    `setOwner` in class `FileSystem`
  Parameters:
    `p` - The path
    `username` - If it is null, the original username remains unchanged.
    `groupname` - If it is null, the original groupname remains unchanged.
  Throws:
    `IOException`

### setPermission

    public void setPermission(Path p,
                              FsPermission permission)
                       throws IOException

  Set permission of a path.

  Overrides:
    `setPermission` in class `FileSystem`
  Throws:
    `IOException`

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2009 The Apache Software Foundation