org.apache.hadoop.hdfs.server.datanode

## Class LocalReplica

- java.lang.Object
- - org.apache.hadoop.hdfs.protocol.Block
  - - org.apache.hadoop.hdfs.server.datanode.ReplicaInfo
    - - org.apache.hadoop.hdfs.server.datanode.LocalReplica

- All Implemented Interfaces:
    Comparable<org.apache.hadoop.hdfs.protocol.Block>, org.apache.hadoop.hdfs.server.datanode.Replica, org.apache.hadoop.io.Writable, org.apache.hadoop.util.LightWeightGSet.LinkedElement
  Direct Known Subclasses:
    FinalizedReplica, LocalReplicaInPipeline, ReplicaUnderRecovery, ReplicaWaitingToBeRecovered

      public abstract class LocalReplica
      extends org.apache.hadoop.hdfs.server.datanode.ReplicaInfo

  This class is used for all replicas which are on local storage media and hence, are backed by files.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `LocalReplica.ReplicaDirInfo`

  - ### Field Summary

    - ### Fields inherited from class org.apache.hadoop.hdfs.protocol.Block

      `BLOCK_FILE_PREFIX, blockFilePattern, METADATA_EXTENSION, metaFilePattern, metaOrBlockFilePattern`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `blockDataExists()`
    `boolean` | `breakHardLinksIfNeeded()` This function "breaks hardlinks" to the current replica file.
    `void` | `bumpReplicaGS(long newGS)` Bump a replica's generation stamp to a new one.
    `int` | `compareWith(FsVolumeSpi.ScanInfo info)`
    `void` | `copyBlockdata(URI destination)`
    `void` | `copyMetadata(URI destination)`
    `boolean` | `deleteBlockData()` Used to deletes the replica's block data.
    `boolean` | `deleteMetadata()` Used to deletes the replica's metadata.
    `void` | `fsyncDirectory()` Sync the parent directory changes to durable device.
    `long` | `getBlockDataLength()`
    `File` | `getBlockFile()` Get the full path of this replica's data file.
    `URI` | `getBlockURI()` Get the `URI` for where the data of this replica is stored.
    `InputStream` | `getDataInputStream(long seekOffset)` Returns an `InputStream` to the replica's data.
    `OutputStream` | `getDataOutputStream(boolean append)` Returns an `OutputStream` to the replica's data.
    `protected File` | `getDir()` Return the parent directory path where this replica is located.
    `LengthInputStream` | `getMetadataInputStream(long offset)` Returns an `InputStream` to the replica's metadata.
    `long` | `getMetadataLength()`
    `OutputStream` | `getMetadataOutputStream(boolean append)` Returns an `OutputStream` to the replica's metadata.
    `URI` | `getMetadataURI()` Get the `URI` for where the metadata of this replica is stored.
    `File` | `getMetaFile()` Get the full path of this replica's meta file.
    `boolean` | `getPinning(org.apache.hadoop.fs.LocalFileSystem localFS)` Check whether the block was pinned.
    `boolean` | `getPinning(org.apache.hadoop.fs.LocalFileSystem localFS, org.apache.hadoop.fs.Path path)` Get pin status of a file by checking the sticky bit.
    `boolean` | `metadataExists()`
    `static LocalReplica.ReplicaDirInfo` | `parseBaseDir(File dir, long blockId)`
    `boolean` | `renameData(URI destURI)` Rename the data `URI` to that referenced by `destURI`.
    `boolean` | `renameMeta(URI destURI)` Rename the metadata `URI` to that referenced by `destURI`.
    `void` | `setPinning(org.apache.hadoop.fs.LocalFileSystem localFS)` Set a block to be pinned on this datanode so that it cannot be moved by Balancer/Mover.
    `void` | `setPinning(org.apache.hadoop.fs.LocalFileSystem localFS, org.apache.hadoop.fs.Path path)` Set sticky bit on path to pin file.
    `static void` | `truncateBlock(FsVolumeSpi volume, File blockFile, File metaFile, long oldlen, long newlen, org.apache.hadoop.hdfs.server.datanode.FileIoProvider fileIoProvider)`
    `void` | `truncateBlock(long newLength)`
    `void` | `updateWithReplica(org.apache.hadoop.hdfs.server.datanode.StorageLocation replicaLocation)` Update this replica with the `StorageLocation` found.

    - ### Methods inherited from class org.apache.hadoop.hdfs.server.datanode.ReplicaInfo

      `createInfo, getBytesReserved, getFileIoProvider, getNext, getOriginalBytesReserved, getOriginalReplica, getRecoveryID, getStorageUuid, getVolume, isOnTransientStorage, setNext, setRecoveryID, toString`

    - ### Methods inherited from class org.apache.hadoop.hdfs.protocol.Block

      `appendStringTo, compareTo, equals, filename2id, getBlockId, getBlockId, getBlockName, getGenerationStamp, getGenerationStamp, getNumBytes, hashCode, isBlockFilename, isMetaFilename, matchingIdAndGenStamp, metaToBlockFile, readFields, readId, set, setBlockId, setGenerationStamp, setNumBytes, toString, write, writeId`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

    - ### Methods inherited from interface org.apache.hadoop.hdfs.server.datanode.Replica

      `getBlockId, getBytesOnDisk, getGenerationStamp, getNumBytes, getState, getVisibleLength`

- - ### Method Detail

    - #### getBlockFile

          public File getBlockFile()

      Get the full path of this replica's data file.
      Returns:
        the full path of this replica's data file

    - #### getMetaFile

          public File getMetaFile()

      Get the full path of this replica's meta file.
      Returns:
        the full path of this replica's meta file

    - #### getDir

          protected File getDir()

      Return the parent directory path where this replica is located.
      Returns:
        the parent directory path where this replica is located

    - #### parseBaseDir

          public static LocalReplica.ReplicaDirInfo parseBaseDir(File dir,
                                                                 long blockId)

    - #### breakHardLinksIfNeeded

          public boolean breakHardLinksIfNeeded()
                                         throws IOException

      This function "breaks hardlinks" to the current replica file. When doing a DataNode upgrade, we create a bunch of hardlinks to each block file. This cleverly ensures that both the old and the new storage directories can contain the same block file, without using additional space for the data. However, when we want to append to the replica file, we need to "break" the hardlink to ensure that the old snapshot continues to contain the old data length. If we failed to do that, we could roll back to the previous/ directory during a downgrade, and find that the block contents were longer than they were at the time of upgrade.
      Specified by:
        `breakHardLinksIfNeeded` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        true only if data was copied.
      Throws:
        `IOException`

    - #### getBlockURI

          public URI getBlockURI()

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Get the `URI` for where the data of this replica is stored.
      Specified by:
        `getBlockURI` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        `URI` for the location of replica data.

    - #### getDataInputStream

          public InputStream getDataInputStream(long seekOffset)
                                         throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns an `InputStream` to the replica's data.
      Specified by:
        `getDataInputStream` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `seekOffset` - the offset at which the read is started from.
      Returns:
        the `InputStream` to read the replica data.
      Throws:
        `IOException` - if an error occurs in opening a stream to the data.

    - #### getDataOutputStream

          public OutputStream getDataOutputStream(boolean append)
                                           throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns an `OutputStream` to the replica's data.
      Specified by:
        `getDataOutputStream` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `append` - indicates if the block should be opened for append.
      Returns:
        the `OutputStream` to write to the replica.
      Throws:
        `IOException` - if an error occurs in creating an `OutputStream`.

    - #### blockDataExists

          public boolean blockDataExists()

      Specified by:
        `blockDataExists` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        true if the replica's data exists.

    - #### deleteBlockData

          public boolean deleteBlockData()

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Used to deletes the replica's block data.
      Specified by:
        `deleteBlockData` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        true if the replica's data is successfully deleted.

    - #### getBlockDataLength

          public long getBlockDataLength()

      Specified by:
        `getBlockDataLength` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        the length of the block on storage.

    - #### getMetadataURI

          public URI getMetadataURI()

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Get the `URI` for where the metadata of this replica is stored.
      Specified by:
        `getMetadataURI` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        `URI` for the location of replica metadata.

    - #### getMetadataInputStream

          public LengthInputStream getMetadataInputStream(long offset)
                                                   throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns an `InputStream` to the replica's metadata.
      Specified by:
        `getMetadataInputStream` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `offset` - the offset at which the read is started from.
      Returns:
        the `LengthInputStream` to read the replica metadata.
      Throws:
        `IOException`

    - #### getMetadataOutputStream

          public OutputStream getMetadataOutputStream(boolean append)
                                               throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns an `OutputStream` to the replica's metadata.
      Specified by:
        `getMetadataOutputStream` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `append` - indicates if the block metadata should be opened for append.
      Returns:
        the `OutputStream` to write to the replica's metadata.
      Throws:
        `IOException` - if an error occurs in creating an `OutputStream`.

    - #### metadataExists

          public boolean metadataExists()

      Specified by:
        `metadataExists` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        true if the replica's metadata exists.

    - #### deleteMetadata

          public boolean deleteMetadata()

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Used to deletes the replica's metadata.
      Specified by:
        `deleteMetadata` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        true if the replica's metadata is successfully deleted.

    - #### getMetadataLength

          public long getMetadataLength()

      Specified by:
        `getMetadataLength` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Returns:
        the length of the metadata on storage.

    - #### renameMeta

          public boolean renameMeta(URI destURI)
                             throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Rename the metadata `URI` to that referenced by `destURI`.
      Specified by:
        `renameMeta` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `destURI` - the target `URI`.
      Returns:
        true if the rename is successful.
      Throws:
        `IOException` - if an exception occurs in the rename.

    - #### renameData

          public boolean renameData(URI destURI)
                             throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Rename the data `URI` to that referenced by `destURI`.
      Specified by:
        `renameData` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `destURI` - the target `URI`.
      Returns:
        true if the rename is successful.
      Throws:
        `IOException` - if an exception occurs in the rename.

    - #### updateWithReplica

          public void updateWithReplica(org.apache.hadoop.hdfs.server.datanode.StorageLocation replicaLocation)

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Update this replica with the `StorageLocation` found.
      Specified by:
        `updateWithReplica` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `replicaLocation` - the `StorageLocation` found for this replica.

    - #### getPinning

          public boolean getPinning(org.apache.hadoop.fs.LocalFileSystem localFS)
                             throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Check whether the block was pinned.
      Specified by:
        `getPinning` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `localFS` - the local filesystem to use.
      Returns:
        true if the block is pinned.
      Throws:
        `IOException`

    - #### setPinning

          public void setPinning(org.apache.hadoop.fs.LocalFileSystem localFS)
                          throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Set a block to be pinned on this datanode so that it cannot be moved by Balancer/Mover.
      Specified by:
        `setPinning` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `localFS` - the local filesystem to use.
      Throws:
        `IOException` - if there is an exception in the pinning.

    - #### bumpReplicaGS

          public void bumpReplicaGS(long newGS)
                             throws IOException

      Description copied from class: `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Bump a replica's generation stamp to a new one. Its on-disk meta file name is renamed to be the new one too.
      Specified by:
        `bumpReplicaGS` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Parameters:
        `newGS` - new generation stamp
      Throws:
        `IOException` - if the change fails

    - #### truncateBlock

          public void truncateBlock(long newLength)
                             throws IOException

      Specified by:
        `truncateBlock` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Throws:
        `IOException`

    - #### compareWith

          public int compareWith(FsVolumeSpi.ScanInfo info)

      Specified by:
        `compareWith` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`

    - #### copyMetadata

          public void copyMetadata(URI destination)
                            throws IOException

      Specified by:
        `copyMetadata` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Throws:
        `IOException`

    - #### copyBlockdata

          public void copyBlockdata(URI destination)
                             throws IOException

      Specified by:
        `copyBlockdata` in class `org.apache.hadoop.hdfs.server.datanode.ReplicaInfo`
      Throws:
        `IOException`

    - #### getPinning

          public boolean getPinning(org.apache.hadoop.fs.LocalFileSystem localFS,
                                    org.apache.hadoop.fs.Path path)
                             throws IOException

      Get pin status of a file by checking the sticky bit.
      Parameters:
        `localFS` - local file system
        `path` - path to be checked
      Returns:
        true if the file is pinned with sticky bit
      Throws:
        `IOException`

    - #### setPinning

          public void setPinning(org.apache.hadoop.fs.LocalFileSystem localFS,
                                 org.apache.hadoop.fs.Path path)
                          throws IOException

      Set sticky bit on path to pin file.
      Parameters:
        `localFS` - local file system
        `path` - path to be pinned with sticky bit
      Throws:
        `IOException`

    - #### truncateBlock

          public static void truncateBlock(FsVolumeSpi volume,
                                           File blockFile,
                                           File metaFile,
                                           long oldlen,
                                           long newlen,
                                           org.apache.hadoop.hdfs.server.datanode.FileIoProvider fileIoProvider)
                                    throws IOException

      Throws:
        `IOException`

    - #### fsyncDirectory

          public void fsyncDirectory()
                              throws IOException

      Sync the parent directory changes to durable device.
      Throws:
        `IOException`

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.