org.apache.hadoop.io

## Class IOUtils

- java.lang.Object
- - org.apache.hadoop.io.IOUtils

-     @InterfaceAudience.Public
       @InterfaceStability.Evolving
      public class IOUtils
      extends Object

  An utility class for I/O related functionality.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `IOUtils.NullOutputStream` The /dev/null of OutputStreams.

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static org.slf4j.Logger` | `LOG`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `IOUtils()`

  - ### Method Summary

    All Methods Static Methods Concrete Methods Deprecated Methods
    Modifier and Type | Method and Description
    --- | ---
    `static void` | `cleanup(org.apache.commons.logging.Log log, Closeable... closeables)` Deprecated. use `cleanupWithLogger(Logger, java.io.Closeable...)` instead
    `static void` | `cleanupWithLogger(org.slf4j.Logger logger, Closeable... closeables)` Close the Closeable objects and ignore any `Throwable` or null pointers.
    `static void` | `closeSocket(Socket sock)` Closes the socket ignoring `IOException`
    `static void` | `closeStream(Closeable stream)` Closes the stream ignoring `Throwable`.
    `static void` | `closeStreams(Closeable... streams)` Closes the streams ignoring `Throwable`.
    `static void` | `copyBytes(InputStream in, OutputStream out, Configuration conf)` Copies from one stream to another.
    `static void` | `copyBytes(InputStream in, OutputStream out, Configuration conf, boolean close)` Copies from one stream to another.
    `static void` | `copyBytes(InputStream in, OutputStream out, int buffSize)` Copies from one stream to another.
    `static void` | `copyBytes(InputStream in, OutputStream out, int buffSize, boolean close)` Copies from one stream to another.
    `static void` | `copyBytes(InputStream in, OutputStream out, long count, boolean close)` Copies count bytes from one stream to another.
    `static void` | `fsync(File fileToSync)` Ensure that any writes to the given file is written to the storage device that contains it.
    `static void` | `fsync(FileChannel channel, boolean isDir)` Ensure that any writes to the given file is written to the storage device that contains it.
    `static List<String>` | `listDirectory(File dir, FilenameFilter filter)` Return the complete list of files in a directory as strings. This is better than File#listDir because it does not ignore IOExceptions.
    `static void` | `readFully(InputStream in, byte[] buf, int off, int len)` Reads len bytes in a loop.
    `static byte[]` | `readFullyToByteArray(DataInput in)` Reads a DataInput until EOF and returns a byte array.
    `static void` | `skipFully(InputStream in, long len)` Similar to readFully().
    `static IOException` | `wrapException(String path, String methodName, IOException exception)` Takes an IOException, file/directory path, and method name and returns an IOException with the input exception as the cause and also include the file,method details.
    `static int` | `wrappedReadForCompressedData(InputStream is, byte[] buf, int off, int len)` Utility wrapper for reading from `InputStream`.
    `static void` | `writeFully(FileChannel fc, ByteBuffer buf, long offset)` Write a ByteBuffer to a FileChannel at a given offset, handling short writes.
    `static void` | `writeFully(WritableByteChannel bc, ByteBuffer buf)` Write a ByteBuffer to a WritableByteChannel, handling short writes.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### LOG

          public static final org.slf4j.Logger LOG

  - ### Constructor Detail

    - #### IOUtils

          public IOUtils()

  - ### Method Detail

    - #### copyBytes

          public static void copyBytes(InputStream in,
                                       OutputStream out,
                                       int buffSize,
                                       boolean close)
                                throws IOException

      Copies from one stream to another.
      Parameters:
        `in` - InputStrem to read from
        `out` - OutputStream to write to
        `buffSize` - the size of the buffer
        `close` - whether or not close the InputStream and OutputStream at the end. The streams are closed in the finally clause.
      Throws:
        `IOException`

    - #### copyBytes

          public static void copyBytes(InputStream in,
                                       OutputStream out,
                                       int buffSize)
                                throws IOException

      Copies from one stream to another.
      Parameters:
        `in` - InputStrem to read from
        `out` - OutputStream to write to
        `buffSize` - the size of the buffer
      Throws:
        `IOException`

    - #### copyBytes

          public static void copyBytes(InputStream in,
                                       OutputStream out,
                                       Configuration conf)
                                throws IOException

      Copies from one stream to another. closes the input and output streams at the end.
      Parameters:
        `in` - InputStrem to read from
        `out` - OutputStream to write to
        `conf` - the Configuration object
      Throws:
        `IOException`

    - #### copyBytes

          public static void copyBytes(InputStream in,
                                       OutputStream out,
                                       Configuration conf,
                                       boolean close)
                                throws IOException

      Copies from one stream to another.
      Parameters:
        `in` - InputStream to read from
        `out` - OutputStream to write to
        `conf` - the Configuration object
        `close` - whether or not close the InputStream and OutputStream at the end. The streams are closed in the finally clause.
      Throws:
        `IOException`

    - #### copyBytes

          public static void copyBytes(InputStream in,
                                       OutputStream out,
                                       long count,
                                       boolean close)
                                throws IOException

      Copies count bytes from one stream to another.
      Parameters:
        `in` - InputStream to read from
        `out` - OutputStream to write to
        `count` - number of bytes to copy
        `close` - whether to close the streams
      Throws:
        `IOException` - if bytes can not be read or written

    - #### wrappedReadForCompressedData

          public static int wrappedReadForCompressedData(InputStream is,
                                                         byte[] buf,
                                                         int off,
                                                         int len)
                                                  throws IOException

      Utility wrapper for reading from `InputStream`. It catches any errors thrown by the underlying stream (either IO or decompression-related), and re-throws as an IOException.
      Parameters:
        `is` - - InputStream to be read from
        `buf` - - buffer the data is read into
        `off` - - offset within buf
        `len` - - amount of data to be read
      Returns:
        number of bytes read
      Throws:
        `IOException`

    - #### readFully

          public static void readFully(InputStream in,
                                       byte[] buf,
                                       int off,
                                       int len)
                                throws IOException

      Reads len bytes in a loop.
      Parameters:
        `in` - InputStream to read from
        `buf` - The buffer to fill
        `off` - offset from the buffer
        `len` - the length of bytes to read
      Throws:
        `IOException` - if it could not read requested number of bytes for any reason (including EOF)

    - #### skipFully

          public static void skipFully(InputStream in,
                                       long len)
                                throws IOException

      Similar to readFully(). Skips bytes in a loop.
      Parameters:
        `in` - The InputStream to skip bytes from
        `len` - number of bytes to skip.
      Throws:
        `IOException` - if it could not skip requested number of bytes for any reason (including EOF)

    - #### cleanup

          @Deprecated
          public static void cleanup(org.apache.commons.logging.Log log,
                                                 Closeable... closeables)

      Deprecated. use `cleanupWithLogger(Logger, java.io.Closeable...)` instead
      Close the Closeable objects and ignore any `Throwable` or null pointers. Must only be used for cleanup in exception handlers.
      Parameters:
        `log` - the log to record problems to at debug level. Can be null.
        `closeables` - the objects to close

    - #### cleanupWithLogger

          public static void cleanupWithLogger(org.slf4j.Logger logger,
                                               Closeable... closeables)

      Close the Closeable objects and ignore any `Throwable` or null pointers. Must only be used for cleanup in exception handlers.
      Parameters:
        `logger` - the log to record problems to at debug level. Can be null.
        `closeables` - the objects to close

    - #### closeStream

          public static void closeStream(Closeable stream)

      Closes the stream ignoring `Throwable`. Must only be called in cleaning up from exception handlers.
      Parameters:
        `stream` - the Stream to close

    - #### closeStreams

          public static void closeStreams(Closeable... streams)

      Closes the streams ignoring `Throwable`. Must only be called in cleaning up from exception handlers.
      Parameters:
        `streams` - the Streams to close

    - #### closeSocket

          public static void closeSocket(Socket sock)

      Closes the socket ignoring `IOException`
      Parameters:
        `sock` - the Socket to close

    - #### writeFully

          public static void writeFully(WritableByteChannel bc,
                                        ByteBuffer buf)
                                 throws IOException

      Write a ByteBuffer to a WritableByteChannel, handling short writes.
      Parameters:
        `bc` - The WritableByteChannel to write to
        `buf` - The input buffer
      Throws:
        `IOException` - On I/O error

    - #### writeFully

          public static void writeFully(FileChannel fc,
                                        ByteBuffer buf,
                                        long offset)
                                 throws IOException

      Write a ByteBuffer to a FileChannel at a given offset, handling short writes.
      Parameters:
        `fc` - The FileChannel to write to
        `buf` - The input buffer
        `offset` - The offset in the file to start writing at
      Throws:
        `IOException` - On I/O error

    - #### listDirectory

          public static List<String> listDirectory(File dir,
                                                   FilenameFilter filter)
                                            throws IOException

      Return the complete list of files in a directory as strings.

      This is better than File#listDir because it does not ignore IOExceptions.
      Parameters:
        `dir` - The directory to list.
        `filter` - If non-null, the filter to use when listing this directory.
      Returns:
        The list of files in the directory.
      Throws:
        `IOException` - On I/O error

    - #### fsync

          public static void fsync(File fileToSync)
                            throws IOException

      Ensure that any writes to the given file is written to the storage device that contains it. This method opens channel on given File and closes it once the sync is done.
      Borrowed from Uwe Schindler in LUCENE-5588
      Parameters:
        `fileToSync` - the file to fsync
      Throws:
        `IOException`

    - #### fsync

          public static void fsync(FileChannel channel,
                                   boolean isDir)
                            throws IOException

      Ensure that any writes to the given file is written to the storage device that contains it. This method opens channel on given File and closes it once the sync is done. Borrowed from Uwe Schindler in LUCENE-5588
      Parameters:
        `channel` - Channel to sync
        `isDir` - if true, the given file is a directory (Channel should be opened for read and ignore IOExceptions, because not all file systems and operating systems allow to fsync on a directory)
      Throws:
        `IOException`

    - #### wrapException

          public static IOException wrapException(String path,
                                                  String methodName,
                                                  IOException exception)

      Takes an IOException, file/directory path, and method name and returns an IOException with the input exception as the cause and also include the file,method details. The new exception provides the stack trace of the place where the exception is thrown and some extra diagnostics information. Return instance of same exception if exception class has a public string constructor; Otherwise return an PathIOException. InterruptedIOException and PathIOException are returned unwrapped.
      Parameters:
        `path` - file/directory path
        `methodName` - method name
        `exception` - the caught exception.
      Returns:
        an exception to throw

    - #### readFullyToByteArray

          public static byte[] readFullyToByteArray(DataInput in)
                                             throws IOException

      Reads a DataInput until EOF and returns a byte array. Make sure not to pass in an infinite DataInput or this will never return.
      Parameters:
        `in` - A DataInput
      Returns:
        a byte array containing the data from the DataInput
      Throws:
        `IOException` - on I/O error, other than EOF

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.