org.apache.hadoop.yarn.server.nodemanager

## Class WindowsSecureContainerExecutor

- java.lang.Object
- - org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor
  - - org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor
    - - org.apache.hadoop.yarn.server.nodemanager.WindowsSecureContainerExecutor

- All Implemented Interfaces:
    org.apache.hadoop.conf.Configurable

      public class WindowsSecureContainerExecutor
      extends DefaultContainerExecutor

  Windows secure container executor (WSCE). This class offers a secure container executor on Windows, similar to the LinuxContainerExecutor. As the NM does not run on a high privileged context, this class delegates elevated operations to the helper hadoopwintuilsvc, implemented by the winutils.exe running as a service. JNI and LRPC is used to communicate with the privileged service.

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

      `DefaultContainerExecutor.LocalWrapperScriptBuilder`

    - ### Nested classes/interfaces inherited from class org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

      `ContainerExecutor.DelayedProcessKiller, ContainerExecutor.ExitCode, ContainerExecutor.Signal`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static String` | `LOCALIZER_PID_FORMAT`

    - ### Fields inherited from class org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

      `lfs`

    - ### Fields inherited from class org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

      `DIRECTORY_CONTENTS, TASK_LAUNCH_SCRIPT_PERMISSION, TOKEN_FILE_NAME_FMT, WILDCARD`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `WindowsSecureContainerExecutor()`

  - ### Method Summary

    All Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected org.apache.hadoop.util.Shell.CommandExecutor` | `buildCommandExecutor(String wrapperScriptPath, String containerIdStr, String userName, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.yarn.api.records.Resource resource, File wordDir, Map<String,String> environment)` Create a new `Shell.ShellCommandExecutor` using the parameters.
    `protected void` | `copyFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, String owner)` Copy a file using the `DefaultContainerExecutor.lfs` `FileContext`.
    `protected void` | `createDir(org.apache.hadoop.fs.Path dirPath, org.apache.hadoop.fs.permission.FsPermission perms, boolean createParent, String owner)` Use the `DefaultContainerExecutor.lfs` `FileContext` to create the target directory.
    `protected DefaultContainerExecutor.LocalWrapperScriptBuilder` | `getLocalWrapperScriptBuilder(String containerIdStr, org.apache.hadoop.fs.Path containerWorkDir)` Create a `DefaultContainerExecutor.LocalWrapperScriptBuilder` for the given container ID and path that is appropriate to the current platform.
    `protected String[]` | `getRunCommand(String command, String groupId, String userName, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.conf.Configuration conf)` Return a command line to execute the given command in the OS shell.
    `protected void` | `killContainer(String pid, ContainerExecutor.Signal signal)` Send a specified signal to the specified pid
    `org.apache.hadoop.fs.Path` | `localizeClasspathJar(org.apache.hadoop.fs.Path jarPath, org.apache.hadoop.fs.Path target, String owner)` This function localizes the JAR file on-demand.
    `void` | `setConf(org.apache.hadoop.conf.Configuration conf)`
    `protected void` | `setScriptExecutable(org.apache.hadoop.fs.Path script, String owner)` Make a file executable using the `DefaultContainerExecutor.lfs` `FileContext`.
    `void` | `startLocalizer(LocalizerStartContext ctx)` Prepare the environment for containers in this application to execute.

    - ### Methods inherited from class org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

      `clearLogDirPermissions, containerIsAlive, createContainerLocalizer, deleteAsUser, getLogDirPermissions, getWorkingDir, init, isContainerAlive, launchContainer, reapContainer, relaunchContainer, signalContainer, symLink`

    - ### Methods inherited from class org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

      `activateContainer, cleanupBeforeRelaunch, deactivateContainer, getConf, getIpAndHost, getLocalIpAndHost, getNMEnvVar, getPidFilePath, getProcessId, getRunCommand, getRunCommandForOther, getRunCommandForWindows, isContainerActive, logOutput, pauseContainer, prepareContainer, reacquireContainer, readDirAsUser, resumeContainer, writeLaunchEnv, writeLaunchEnv`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### LOCALIZER_PID_FORMAT

          public static final String LOCALIZER_PID_FORMAT

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### WindowsSecureContainerExecutor

          public WindowsSecureContainerExecutor()
                                         throws IOException,
                                                URISyntaxException

      Throws:
        `IOException`
        `URISyntaxException`

  - ### Method Detail

    - #### setConf

          public void setConf(org.apache.hadoop.conf.Configuration conf)

      Specified by:
        `setConf` in interface `org.apache.hadoop.conf.Configurable`
      Overrides:
        `setConf` in class `ContainerExecutor`

    - #### getRunCommand

          protected String[] getRunCommand(String command,
                                           String groupId,
                                           String userName,
                                           org.apache.hadoop.fs.Path pidFile,
                                           org.apache.hadoop.conf.Configuration conf)

      Description copied from class: `ContainerExecutor`
      Return a command line to execute the given command in the OS shell. On Windows, the {code}groupId{code} parameter can be used to launch and associate the given GID with a process group. On non-Windows hosts, the {code}groupId{code} parameter is ignored.
      Overrides:
        `getRunCommand` in class `ContainerExecutor`
      Parameters:
        `command` - the command to execute
        `groupId` - the job owner's GID
        `userName` - the job owner's username
        `pidFile` - the path to the container's PID file
        `conf` - the configuration
      Returns:
        the command line to execute

    - #### getLocalWrapperScriptBuilder

          protected DefaultContainerExecutor.LocalWrapperScriptBuilder getLocalWrapperScriptBuilder(String containerIdStr,
                                                                                                    org.apache.hadoop.fs.Path containerWorkDir)

      Description copied from class: `DefaultContainerExecutor`
      Create a `DefaultContainerExecutor.LocalWrapperScriptBuilder` for the given container ID and path that is appropriate to the current platform.
      Overrides:
        `getLocalWrapperScriptBuilder` in class `DefaultContainerExecutor`
      Parameters:
        `containerIdStr` - the container ID
        `containerWorkDir` - the container's working directory
      Returns:
        a new `DefaultContainerExecutor.LocalWrapperScriptBuilder`

    - #### copyFile

          protected void copyFile(org.apache.hadoop.fs.Path src,
                                  org.apache.hadoop.fs.Path dst,
                                  String owner)
                           throws IOException

      Description copied from class: `DefaultContainerExecutor`
      Copy a file using the `DefaultContainerExecutor.lfs` `FileContext`.
      Overrides:
        `copyFile` in class `DefaultContainerExecutor`
      Parameters:
        `src` - the file to copy
        `dst` - where to copy the file
        `owner` - the owner of the new copy. Used only in secure Windows clusters
      Throws:
        `IOException` - when the copy fails
      See Also:
        `WindowsSecureContainerExecutor`

    - #### createDir

          protected void createDir(org.apache.hadoop.fs.Path dirPath,
                                   org.apache.hadoop.fs.permission.FsPermission perms,
                                   boolean createParent,
                                   String owner)
                            throws IOException

      Description copied from class: `DefaultContainerExecutor`
      Use the `DefaultContainerExecutor.lfs` `FileContext` to create the target directory.
      Overrides:
        `createDir` in class `DefaultContainerExecutor`
      Parameters:
        `dirPath` - the target directory
        `perms` - the target permissions for the target directory
        `createParent` - whether the parent directories should also be created
        `owner` - the user as whom the target directory should be created. Used only on secure Windows hosts.
      Throws:
        `IOException` - if there's a failure performing a file operation
      See Also:
        `WindowsSecureContainerExecutor`

    - #### setScriptExecutable

          protected void setScriptExecutable(org.apache.hadoop.fs.Path script,
                                             String owner)
                                      throws IOException

      Description copied from class: `DefaultContainerExecutor`
      Make a file executable using the `DefaultContainerExecutor.lfs` `FileContext`.
      Overrides:
        `setScriptExecutable` in class `DefaultContainerExecutor`
      Parameters:
        `script` - the path to make executable
        `owner` - the new owner for the file. Used only in secure Windows clusters
      Throws:
        `IOException` - when the change mode operation fails
      See Also:
        `WindowsSecureContainerExecutor`

    - #### localizeClasspathJar

          public org.apache.hadoop.fs.Path localizeClasspathJar(org.apache.hadoop.fs.Path jarPath,
                                                                org.apache.hadoop.fs.Path target,
                                                                String owner)
                                                         throws IOException

      Description copied from class: `ContainerExecutor`
      This function localizes the JAR file on-demand. On Windows the ContainerLaunch creates a temporary special JAR manifest of other JARs to workaround the CLASSPATH length. In a secure cluster this JAR must be localized so that the container has access to it. The default implementation returns the classpath passed to it, which is expected to have been created in the node manager's fprivate folder, which will not work with secure Windows clusters.
      Overrides:
        `localizeClasspathJar` in class `ContainerExecutor`
      Parameters:
        `jarPath` - the path to the JAR to localize
        `target` - the directory where the JAR file should be localized
        `owner` - the name of the user who should own the localized file
      Returns:
        the path to the localized JAR file
      Throws:
        `IOException` - if localization fails

    - #### startLocalizer

          public void startLocalizer(LocalizerStartContext ctx)
                              throws IOException,
                                     InterruptedException

      Description copied from class: `ContainerExecutor`
      Prepare the environment for containers in this application to execute.

           For $x in local.dirs
             create $x/$user/$appId
           Copy $nmLocal/appTokens  -> $N/$user/$appId
           For $rsrc in private resources
             Copy $rsrc  -> $N/$user/filecache/[idef]
           For $rsrc in job resources
             Copy $rsrc  -> $N/$user/$appId/filecache/idef

      Overrides:
        `startLocalizer` in class `DefaultContainerExecutor`
      Parameters:
        `ctx` - LocalizerStartContext that encapsulates necessary information for starting a localizer.
      Throws:
        `IOException` - for most application init failures
        `InterruptedException` - if application init thread is halted by NM

    - #### buildCommandExecutor

          protected org.apache.hadoop.util.Shell.CommandExecutor buildCommandExecutor(String wrapperScriptPath,
                                                                                      String containerIdStr,
                                                                                      String userName,
                                                                                      org.apache.hadoop.fs.Path pidFile,
                                                                                      org.apache.hadoop.yarn.api.records.Resource resource,
                                                                                      File wordDir,
                                                                                      Map<String,String> environment)

      Description copied from class: `DefaultContainerExecutor`
      Create a new `Shell.ShellCommandExecutor` using the parameters.
      Overrides:
        `buildCommandExecutor` in class `DefaultContainerExecutor`
      Parameters:
        `wrapperScriptPath` - the path to the script to execute
        `containerIdStr` - the container ID
        `userName` - the application owner's username
        `pidFile` - the path to the container's PID file
        `resource` - this parameter controls memory and CPU limits.
        `wordDir` - If not-null, specifies the directory which should be set as the current working directory for the command. If null, the current working directory is not modified.
        `environment` - the container environment
      Returns:
        the new `Shell.ShellCommandExecutor`
      See Also:
        `Shell.ShellCommandExecutor`

    - #### killContainer

          protected void killContainer(String pid,
                                       ContainerExecutor.Signal signal)
                                throws IOException

      Description copied from class: `DefaultContainerExecutor`
      Send a specified signal to the specified pid
      Overrides:
        `killContainer` in class `DefaultContainerExecutor`
      Parameters:
        `pid` - the pid of the process [group] to signal.
        `signal` - signal to send
      Throws:
        `IOException` - if the command to kill the process fails

Copyright © 2008–2022 Apache Software Foundation. All rights reserved.