org.apache.hadoop.yarn.logaggregation.filecontroller.ifile

## Class LogAggregationIndexedFileController

- java.lang.Object
- - org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController
  - - org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController

-     @InterfaceAudience.Private
       @InterfaceStability.Unstable
      public class LogAggregationIndexedFileController
      extends LogAggregationFileController

  The Indexed Log Aggregation File Format implementation.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `LogAggregationIndexedFileController.IndexedFileLogMeta` This IndexedFileLogMeta includes the meta information for a single file which would be aggregated in one Log aggregation cycle.
    `static class` | `LogAggregationIndexedFileController.IndexedLogsMeta` This IndexedLogsMeta includes all the meta information for the aggregated log file.
    `static class` | `LogAggregationIndexedFileController.IndexedPerAggregationLogMeta` This IndexedPerAggregationLogMeta includes the meta information for all files which would be aggregated in one Log aggregation cycle.

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `CHECK_SUM_FILE_SUFFIX`

    - ### Fields inherited from class org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController

      `APP_DIR_PERMISSIONS, APP_LOG_FILE_UMASK, conf, fileControllerName, fsSupportsChmod, LOG_AGGREGATION_FS_SUPPORT_APPEND, remoteRootLogDir, remoteRootLogDirSuffix, retentionSize, TLDIR_PERMISSIONS`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `LogAggregationIndexedFileController()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `closeWriter()` Close the writer.
    `org.apache.hadoop.fs.FileStatus` | `getAllChecksumFiles(Map<String,org.apache.hadoop.fs.FileStatus> fileMap, String fileName)`
    `Map<org.apache.hadoop.yarn.api.records.ApplicationAccessType,String>` | `getApplicationAcls(org.apache.hadoop.fs.Path aggregatedLogPath, org.apache.hadoop.yarn.api.records.ApplicationId appId)` Returns ACLs for the application.
    `String` | `getApplicationOwner(org.apache.hadoop.fs.Path aggregatedLogPath, org.apache.hadoop.yarn.api.records.ApplicationId appId)` Returns the owner of the application.
    `static int` | `getFSInputBufferSize(org.apache.hadoop.conf.Configuration conf)`
    `static int` | `getFSOutputBufferSize(org.apache.hadoop.conf.Configuration conf)`
    `List<org.apache.hadoop.fs.FileStatus>` | `getNodeLogFileToRead(List<org.apache.hadoop.fs.FileStatus> nodeFiles, String nodeId, org.apache.hadoop.yarn.api.records.ApplicationId appId)`
    `org.apache.hadoop.fs.Path` | `getRemoteAppLogDir(org.apache.hadoop.yarn.api.records.ApplicationId appId, String user)` Get the remote application directory for log aggregation.
    `long` | `getRollOverLogMaxSize(org.apache.hadoop.conf.Configuration conf)`
    `Clock` | `getSystemClock()`
    `void` | `initializeWriter(LogAggregationFileControllerContext context)` Initialize the writer.
    `void` | `initInternal(org.apache.hadoop.conf.Configuration conf)` Derived classes initialize themselves using this method.
    `boolean` | `isRollover(org.apache.hadoop.fs.FileContext fc, org.apache.hadoop.fs.Path candidate)`
    `LogAggregationIndexedFileController.IndexedLogsMeta` | `loadIndexedLogsMeta(org.apache.hadoop.fs.Path remoteLogPath, long end, org.apache.hadoop.yarn.api.records.ApplicationId appId)`
    `Map<String,Long>` | `parseCheckSumFiles(List<org.apache.hadoop.fs.FileStatus> fileList)`
    `void` | `postWrite(LogAggregationFileControllerContext record)` Operations needed after write the log content.
    `boolean` | `readAggregatedLogs(ContainerLogsRequest logRequest, OutputStream os)` Output container log.
    `List<ContainerLogMeta>` | `readAggregatedLogsMeta(ContainerLogsRequest logRequest)` Return a list of `ContainerLogMeta` for an application from Remote FileSystem.
    `void` | `renderAggregatedLogsBlock(HtmlBlock.Block html, View.ViewContext context)` Render Aggregated Logs block.
    `void` | `write(AggregatedLogFormat.LogKey logKey, AggregatedLogFormat.LogValue logValue)` Write the log content.

    - ### Methods inherited from class org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController

      `aggregatedLogSuffix, belongsToAppAttempt, checkExists, cleanOldLogs, closePrintStream, createAppDir, createDir, getFileSystem, getRemoteNodeLogFileForApp, getRemoteRootLogDir, getRemoteRootLogDirSuffix, initialize, isFsSupportsChmod, verifyAndCreateRemoteLogDir`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### CHECK_SUM_FILE_SUFFIX

          public static final String CHECK_SUM_FILE_SUFFIX

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### LogAggregationIndexedFileController

          public LogAggregationIndexedFileController()

  - ### Method Detail

    - #### initInternal

          public void initInternal(org.apache.hadoop.conf.Configuration conf)

      Description copied from class: `LogAggregationFileController`
      Derived classes initialize themselves using this method.
      Specified by:
        `initInternal` in class `LogAggregationFileController`
      Parameters:
        `conf` - the Configuration

    - #### initializeWriter

          public void initializeWriter(LogAggregationFileControllerContext context)
                                throws IOException

      Description copied from class: `LogAggregationFileController`
      Initialize the writer.
      Specified by:
        `initializeWriter` in class `LogAggregationFileController`
      Parameters:
        `context` - the `LogAggregationFileControllerContext`
      Throws:
        `IOException` - if fails to initialize the writer

    - #### closeWriter

          public void closeWriter()

      Description copied from class: `LogAggregationFileController`
      Close the writer.
      Specified by:
        `closeWriter` in class `LogAggregationFileController`

    - #### write

          public void write(AggregatedLogFormat.LogKey logKey,
                            AggregatedLogFormat.LogValue logValue)
                     throws IOException

      Description copied from class: `LogAggregationFileController`
      Write the log content.
      Specified by:
        `write` in class `LogAggregationFileController`
      Parameters:
        `logKey` - the log key
        `logValue` - the log content
      Throws:
        `IOException` - if fails to write the logs

    - #### postWrite

          public void postWrite(LogAggregationFileControllerContext record)
                         throws Exception

      Description copied from class: `LogAggregationFileController`
      Operations needed after write the log content.
      Specified by:
        `postWrite` in class `LogAggregationFileController`
      Parameters:
        `record` - the `LogAggregationFileControllerContext`
      Throws:
        `Exception` - if anything fails

    - #### readAggregatedLogs

          public boolean readAggregatedLogs(ContainerLogsRequest logRequest,
                                            OutputStream os)
                                     throws IOException

      Description copied from class: `LogAggregationFileController`
      Output container log.
      Specified by:
        `readAggregatedLogs` in class `LogAggregationFileController`
      Parameters:
        `logRequest` - `ContainerLogsRequest`
        `os` - the output stream
      Returns:
        true if we can read the aggregated logs successfully
      Throws:
        `IOException` - if we can not access the log file.

    - #### readAggregatedLogsMeta

          public List<ContainerLogMeta> readAggregatedLogsMeta(ContainerLogsRequest logRequest)
                                                        throws IOException

      Description copied from class: `LogAggregationFileController`
      Return a list of `ContainerLogMeta` for an application from Remote FileSystem.
      Specified by:
        `readAggregatedLogsMeta` in class `LogAggregationFileController`
      Parameters:
        `logRequest` - `ContainerLogsRequest`
      Returns:
        a list of `ContainerLogMeta`
      Throws:
        `IOException` - if there is no available log file

    - #### parseCheckSumFiles

          @InterfaceAudience.Private
          public Map<String,Long> parseCheckSumFiles(List<org.apache.hadoop.fs.FileStatus> fileList)
                                                                         throws IOException

      Throws:
        `IOException`

    - #### getNodeLogFileToRead

          @InterfaceAudience.Private
          public List<org.apache.hadoop.fs.FileStatus> getNodeLogFileToRead(List<org.apache.hadoop.fs.FileStatus> nodeFiles,
                                                                                                       String nodeId,
                                                                                                       org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                                                                throws IOException

      Throws:
        `IOException`

    - #### getAllChecksumFiles

          @InterfaceAudience.Private
          public org.apache.hadoop.fs.FileStatus getAllChecksumFiles(Map<String,org.apache.hadoop.fs.FileStatus> fileMap,
                                                                                                String fileName)

    - #### renderAggregatedLogsBlock

          public void renderAggregatedLogsBlock(HtmlBlock.Block html,
                                                View.ViewContext context)

      Description copied from class: `LogAggregationFileController`
      Render Aggregated Logs block.
      Specified by:
        `renderAggregatedLogsBlock` in class `LogAggregationFileController`
      Parameters:
        `html` - the html
        `context` - the ViewContext

    - #### getApplicationOwner

          public String getApplicationOwner(org.apache.hadoop.fs.Path aggregatedLogPath,
                                            org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                     throws IOException

      Description copied from class: `LogAggregationFileController`
      Returns the owner of the application.
      Specified by:
        `getApplicationOwner` in class `LogAggregationFileController`
      Parameters:
        `aggregatedLogPath` - the aggregatedLog path
        `appId` - the ApplicationId
      Returns:
        the application owner
      Throws:
        `IOException` - if we can not get the application owner

    - #### getApplicationAcls

          public Map<org.apache.hadoop.yarn.api.records.ApplicationAccessType,String> getApplicationAcls(org.apache.hadoop.fs.Path aggregatedLogPath,
                                                                                                         org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                                                                  throws IOException

      Description copied from class: `LogAggregationFileController`
      Returns ACLs for the application. An empty map is returned if no ACLs are found.
      Specified by:
        `getApplicationAcls` in class `LogAggregationFileController`
      Parameters:
        `aggregatedLogPath` - the aggregatedLog path.
        `appId` - the ApplicationId
      Returns:
        a map of the Application ACLs.
      Throws:
        `IOException` - if we can not get the application acls

    - #### getRemoteAppLogDir

          public org.apache.hadoop.fs.Path getRemoteAppLogDir(org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                              String user)
                                                       throws IOException

      Description copied from class: `LogAggregationFileController`
      Get the remote application directory for log aggregation.
      Overrides:
        `getRemoteAppLogDir` in class `LogAggregationFileController`
      Parameters:
        `appId` - the Application ID
        `user` - the Application Owner
      Returns:
        the remote application directory
      Throws:
        `IOException` - if can not find the remote application directory

    - #### loadIndexedLogsMeta

          @InterfaceAudience.Private
          public LogAggregationIndexedFileController.IndexedLogsMeta loadIndexedLogsMeta(org.apache.hadoop.fs.Path remoteLogPath,
                                                                                                                    long end,
                                                                                                                    org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                                                                             throws IOException

      Throws:
        `IOException`

    - #### getFSOutputBufferSize

          @InterfaceAudience.Private
          public static int getFSOutputBufferSize(org.apache.hadoop.conf.Configuration conf)

    - #### getFSInputBufferSize

          @InterfaceAudience.Private
          public static int getFSInputBufferSize(org.apache.hadoop.conf.Configuration conf)

    - #### getRollOverLogMaxSize

          @InterfaceAudience.Private
          public long getRollOverLogMaxSize(org.apache.hadoop.conf.Configuration conf)

    - #### isRollover

          @InterfaceAudience.Private
          public boolean isRollover(org.apache.hadoop.fs.FileContext fc,
                                                               org.apache.hadoop.fs.Path candidate)
                                                        throws IOException

      Throws:
        `IOException`

    - #### getSystemClock

          @InterfaceAudience.Private
          public Clock getSystemClock()

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.