##### General

- Overview
- Single Node Setup
- Cluster Setup
- Commands Reference
- FileSystem Shell
- Compatibility Specification
- Downstream Developer's Guide
- Admin Compatibility Guide
- Interface Classification
- FileSystem Specification

##### Common

- CLI Mini Cluster
- Native Libraries
- Proxy User
- Rack Awareness
- Secure Mode
- Service Level Authorization
- HTTP Authentication
- Credential Provider API
- Hadoop KMS
- Tracing
- Unix Shell Guide

##### HDFS

- Architecture
- User Guide
- Commands Reference
- NameNode HA With QJM
- NameNode HA With NFS
- Observer NameNode
- Federation
- ViewFs
- ViewFsOverloadScheme
- Snapshots
- Edits Viewer
- Image Viewer
- Permissions and HDFS
- Quotas and HDFS
- libhdfs (C API)
- WebHDFS (REST API)
- HttpFS
- Short Circuit Local Reads
- Centralized Cache Management
- NFS Gateway
- Rolling Upgrade
- Extended Attributes
- Transparent Encryption
- Multihoming
- Storage Policies
- Memory Storage Support
- Synthetic Load Generator
- Erasure Coding
- Disk Balancer
- Upgrade Domain
- DataNode Admin
- Router Federation
- Provided Storage

##### MapReduce

- Tutorial
- Commands Reference
- Compatibility with 1.x
- Encrypted Shuffle
- Pluggable Shuffle/Sort
- Distributed Cache Deploy
- Support for YARN Shared Cache

##### MapReduce REST APIs

- MR Application Master
- MR History Server

##### YARN

- Architecture
- Commands Reference
- Capacity Scheduler
- Fair Scheduler
- ResourceManager Restart
- ResourceManager HA
- Resource Model
- Node Labels
- Node Attributes
- Web Application Proxy
- Timeline Server
- Timeline Service V.2
- Writing YARN Applications
- YARN Application Security
- NodeManager
- Running Applications in Docker Containers
- Using CGroups
- Secure Containers
- Registry
- Reservation System
- Graceful Decommission
- Opportunistic Containers
- YARN Federation
- Shared Cache
- Using GPU
- Using FPGA
- Placement Constraints
- YARN UI2

##### YARN REST APIs

- Introduction
- Resource Manager
- Node Manager
- Timeline Server
- Timeline Service V.2

##### YARN Service

- Overview
- QuickStart
- Concepts
- Yarn Service API
- Service Discovery
- System Services

##### Submarine

- Index
- QuickStart
- Examples

##### Hadoop Compatible File Systems

- Aliyun OSS
- Amazon S3
- Azure Blob Storage
- Azure Data Lake Storage
- OpenStack Swift

##### Auth

- Overview
- Examples
- Configuration
- Building

##### Tools

- Hadoop Streaming
- Hadoop Archives
- Hadoop Archive Logs
- DistCp
- GridMix
- Rumen
- Resource Estimator Service
- Scheduler Load Simulator
- Hadoop Benchmarking

##### Reference

- Changelog and Release Notes
- Java API docs
- Unix Shell API
- Metrics

##### Configuration

- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml
- kms-default.xml
- httpfs-default.xml
- Deprecated Properties

# Apache Hadoop Changelog

## Release 2.10.0 - Unreleased (as of 2018-09-02)

### INCOMPATIBLE CHANGES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-12883 | RBF: Document Router and State Store metrics | Major | documentation | Yiqun Lin | Yiqun Lin
HDFS-12895 | RBF: Add ACL support for mount table | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13099 | RBF: Use the ZooKeeper as the default State Store | Minor | documentation | Yiqun Lin | Yiqun Lin

### IMPORTANT ISSUES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-13083 | RBF: Fix doc error setting up client | Major | federation | tartarus | tartarus

### NEW FEATURES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-13283 | Percentage based Reserved Space Calculation for DataNode | Major | datanode, hdfs | Lukas Majercak | Lukas Majercak
HDFS-13553 | RBF: Support global quota | Major | . | Íñigo Goiri | Yiqun Lin

### IMPROVEMENTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-14987 | Improve KMSClientProvider log around delegation token checking | Major | . | Xiaoyu Yao | Xiaoyu Yao
HADOOP-14872 | CryptoInputStream should implement unbuffer | Major | fs, security | John Zhuge | John Zhuge
HADOOP-14960 | Add GC time percentage monitor/alerter | Major | . | Misha Dmitriev | Misha Dmitriev
HADOOP-15023 | ValueQueue should also validate (lowWatermark * numValues) > 0 on construction | Minor | . | Xiao Chen | Xiao Chen
YARN-6851 | Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat | Minor | . | Wei Yan | Wei Yan
YARN-7495 | Improve robustness of the AggregatedLogDeletionService | Major | log-aggregation | Jonathan Eagles | Jonathan Eagles
HADOOP-15056 | Fix TestUnbuffer#testUnbufferException failure | Minor | test | Jack Bearden | Jack Bearden
HADOOP-15012 | Add readahead, dropbehind, and unbuffer to StreamCapabilities | Major | fs | John Zhuge | John Zhuge
HADOOP-15104 | AliyunOSS: change the default value of max error retry | Major | fs/oss | wujinhu | wujinhu
YARN-7642 | Add test case to verify context update after container promotion or demotion with or without auto update | Minor | nodemanager | Weiwei Yang | Weiwei Yang
HADOOP-15111 | AliyunOSS: backport HADOOP-14993 to branch-2 | Major | fs/oss | Genmao Yu | Genmao Yu
HDFS-12818 | Support multiple storages in DataNodeCluster / SimulatedFSDataset | Minor | datanode, test | Erik Krogen | Erik Krogen
HDFS-9023 | When NN is not able to identify DN for replication, reason behind it can be logged | Critical | hdfs-client, namenode | Surendra Singh Lilhore | Xiao Chen
YARN-7678 | Ability to enable logging of container memory stats | Major | nodemanager | Jim Brennan | Jim Brennan
HDFS-12945 | Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods | Minor | . | Wei Yan | Wei Yan
YARN-7622 | Allow fair-scheduler configuration on HDFS | Minor | fairscheduler, resourcemanager | Greg Phillips | Greg Phillips
YARN-7590 | Improve container-executor validation check | Major | security, yarn | Eric Yang | Eric Yang
MAPREDUCE-7029 | FileOutputCommitter is slow on filesystems lacking recursive delete | Minor | . | Karthik Palaniappan | Karthik Palaniappan
MAPREDUCE-6984 | MR AM to clean up temporary files from previous attempt in case of no recovery | Major | applicationmaster | Gergo Repas | Gergo Repas
HADOOP-15189 | backport HADOOP-15039 to branch-2 and branch-3 | Blocker | . | Genmao Yu | Genmao Yu
HADOOP-15212 | Add independent secret manager method for logging expired tokens | Major | security | Daryn Sharp | Daryn Sharp
YARN-7728 | Expose container preemptions related information in Capacity Scheduler queue metrics | Major | . | Eric Payne | Eric Payne
MAPREDUCE-7048 | Uber AM can crash due to unknown task in statusUpdate | Major | mr-am | Peter Bacsko | Peter Bacsko
HADOOP-13972 | ADLS to support per-store configuration | Major | fs/adl | John Zhuge | Sharad Sonker
YARN-7813 | Capacity Scheduler Intra-queue Preemption should be configurable for each queue | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne
HADOOP-15235 | Authentication Tokens should use HMAC instead of MAC | Major | security | Robert Kanter | Robert Kanter
HDFS-11187 | Optimize disk access for last partial chunk checksum of Finalized replica | Major | datanode | Wei-Chiu Chuang | Gabor Bota
HADOOP-15266 | [branch-2] Upper/Lower case conversion support for group names in LdapGroupsMapping | Major | . | Nanda kumar | Nanda kumar
HADOOP-15279 | increase maven heap size recommendations | Minor | build, documentation, test | Allen Wittenauer | Allen Wittenauer
HDFS-12884 | BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo | Major | namenode | Konstantin Shvachko | chencan
HADOOP-15334 | Upgrade Maven surefire plugin | Major | build | Arpit Agarwal | Arpit Agarwal
HADOOP-15312 | Undocumented KeyProvider configuration keys | Major | . | Wei-Chiu Chuang | LiXin Ge
YARN-7623 | Fix the CapacityScheduler Queue configuration documentation | Major | . | Arun Suresh | Jonathan Hung
HDFS-13314 | NameNode should optionally exit if it detects FsImage corruption | Major | namenode | Arpit Agarwal | Arpit Agarwal
HDFS-13418 | NetworkTopology should be configurable when enable DFSNetworkTopology | Major | . | Tao Jie | Tao Jie
HADOOP-15394 | Backport PowerShell NodeFencer HADOOP-14309 to branch-2 | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13462 | Add BIND_HOST configuration for JournalNode’s HTTP and RPC Servers | Major | hdfs, journal-node | Lukas Majercak | Lukas Majercak
HDFS-13492 | Limit httpfs binds to certain IP addresses in branch-2 | Major | httpfs | Wei-Chiu Chuang | Wei-Chiu Chuang
HDFS-12981 | renameSnapshot a Non-Existent snapshot to itself should throw error | Minor | hdfs | Sailesh Patel | Kitti Nanasi
HADOOP-15441 | Log kms url and token service at debug level. | Minor | . | Wei-Chiu Chuang | Gabor Bota
YARN-8249 | Few REST api’s in RMWebServices are missing static user check | Critical | webapp, yarn | Sunil Govindan | Sunil Govindan
HADOOP-15486 | Make NetworkTopology#netLock fair | Major | net | Nanda kumar | Nanda kumar
HADOOP-15449 | Increase default timeout of ZK session to avoid frequent NameNode failover | Critical | common | Karthik Palanisamy | Karthik Palanisamy
HDFS-13602 | Add checkOperation(WRITE) checks in FSNamesystem | Major | ha, namenode | Erik Krogen | Chao Sun
HDFS-13644 | Backport HDFS-10376 to branch-2 | Major | . | Yiqun Lin | Zsolt Venczel
HDFS-13653 | Make dfs.client.failover.random.order a per nameservice configuration | Major | federation | Ekanth Sethuramalingam | Ekanth Sethuramalingam
HDFS-13714 | Fix TestNameNodePrunesMissingStorages test failures on Windows | Major | hdfs, namenode, test | Lukas Majercak | Lukas Majercak
HDFS-13719 | Docs around dfs.image.transfer.timeout are misleading | Major | . | Kitti Nanasi | Kitti Nanasi
HDFS-11060 | make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable | Minor | hdfs | Lantao Jin | Lantao Jin
YARN-8155 | Improve ATSv2 client logging in RM and NM publisher | Major | . | Rohith Sharma K S | Abhishek Modi
HDFS-13814 | Remove super user privilege requirement for NameNode.getServiceStatus | Minor | namenode | Chao Sun | Chao Sun
YARN-8559 | Expose mutable-conf scheduler’s configuration in RM /scheduler-conf endpoint | Major | resourcemanager | Anna Savarin | Weiwei Yang
HDFS-13813 | Exit NameNode if dangling child inode is detected when saving FsImage | Major | hdfs, namenode | Siyao Meng | Siyao Meng
HDFS-13821 | RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache | Major | hdfs | Fei Hui | Fei Hui
HADOOP-15689 | Add “*.patch” into .gitignore file of branch-2 | Major | . | Rui Gao | Rui Gao
HDFS-13831 | Make block increment deletion number configurable | Major | . | Yiqun Lin | Ryan Wu
YARN-8051 | TestRMEmbeddedElector#testCallbackSynchronization is flakey | Major | test | Robert Kanter | Robert Kanter

### BUG FIXES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-12052 | Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS | Major | httpfs, webhdfs | Zoran Dimitrijevic | Zoran Dimitrijevic
HDFS-12318 | Fix IOException condition for openInfo in DFSInputStream | Major | . | legend | legend
HDFS-12614 | FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured | Major | . | Manoj Govindassamy | Manoj Govindassamy
YARN-7370 | Preemption properties should be refreshable | Major | capacity scheduler, scheduler preemption | Eric Payne | Gergely Novák
YARN-7428 | Add containerId to Localizer failed logs | Minor | nodemanager | Prabhu Joseph | Prabhu Joseph
HDFS-12783 | [branch-2] “dfsrouter” should use hdfsScript | Major | . | Brahma Reddy Battula | Brahma Reddy Battula
HDFS-12788 | Reset the upload button when file upload fails | Critical | ui, webhdfs | Brahma Reddy Battula | Brahma Reddy Battula
YARN-7469 | Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit | Major | capacity scheduler, yarn | Eric Payne | Eric Payne
HADOOP-14982 | Clients using FailoverOnNetworkExceptionRetry can go into a loop if they’re used without authenticating with kerberos in HA env | Major | common | Peter Bacsko | Peter Bacsko
YARN-7489 | ConcurrentModificationException in RMAppImpl#getRMAppMetrics | Major | capacityscheduler | Tao Yang | Tao Yang
YARN-7525 | Incorrect query parameters in cluster nodes REST API document | Minor | documentation | Tao Yang | Tao Yang
HDFS-12813 | RequestHedgingProxyProvider can hide Exception thrown from the Namenode for proxy size of 1 | Major | ha | Mukul Kumar Singh | Mukul Kumar Singh
HADOOP-15045 | ISA-L build options are documented in branch-2 | Major | build, documentation | Akira Ajisaka | Akira Ajisaka
HADOOP-15067 | GC time percentage reported in JvmMetrics should be a gauge, not counter | Major | . | Misha Dmitriev | Misha Dmitriev
YARN-7363 | ContainerLocalizer doesn’t have a valid log4j config when using LinuxContainerExecutor | Major | nodemanager | Yufei Gu | Yufei Gu
HDFS-12754 | Lease renewal can hit a deadlock | Major | . | Kuhu Shukla | Kuhu Shukla
HDFS-12832 | INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit | Critical | namenode | DENG FEI | Konstantin Shvachko
HDFS-11754 | Make FsServerDefaults cache configurable. | Minor | . | Rushabh S Shah | Mikhail Erofeev
HADOOP-15042 | Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0 | Minor | fs/azure | Rajesh Balamohan | Rajesh Balamohan
HDFS-12638 | Delete copy-on-truncate block along with the original block, when deleting a file being truncated | Blocker | hdfs | Jiandan Yang | Konstantin Shvachko
YARN-4813 | TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently | Major | resourcemanager | Daniel Templeton | Gergo Repas
MAPREDUCE-5124 | AM lacks flow control for task events | Major | mr-am | Jason Lowe | Peter Bacsko
YARN-7455 | quote_and_append_arg can overflow buffer | Major | nodemanager | Jason Lowe | Jim Brennan
YARN-5594 | Handle old RMDelegationToken format when recovering RM | Major | resourcemanager | Tatyana But | Robert Kanter
HADOOP-14985 | Remove subversion related code from VersionInfoMojo.java | Minor | build | Akira Ajisaka | Ajay Kumar
HDFS-11751 | DFSZKFailoverController daemon exits with wrong status code | Major | auto-failover | Doris Gu | Bharat Viswanadham
HDFS-12889 | Router UI is missing robots.txt file | Major | . | Bharat Viswanadham | Bharat Viswanadham
YARN-7607 | Remove the trailing duplicated timestamp in container diagnostics message | Minor | nodemanager | Weiwei Yang | Weiwei Yang
HADOOP-15080 | Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x “json-lib” | Blocker | fs/oss | Chris Douglas | Sammi Chen
YARN-7608 | Incorrect sTarget column causing DataTable warning on RM application and scheduler web page | Major | resourcemanager, webapp | Weiwei Yang | Gergely Novák
HDFS-12833 | Distcp : Update the usage of delete option for dependency with update and overwrite option | Minor | distcp, hdfs | Harshakiran Reddy | usharani
YARN-7647 | NM print inappropriate error log when node-labels is enabled | Minor | . | Yang Wang | Yang Wang
HDFS-12907 | Allow read-only access to reserved raw for non-superusers | Major | namenode | Daryn Sharp | Rushabh S Shah
HDFS-12881 | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Lowe | Ajay Kumar
YARN-7595 | Container launching code suppresses close exceptions after writes | Major | nodemanager | Jason Lowe | Jim Brennan
HADOOP-15085 | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Lowe | Jim Brennan
HADOOP-15123 | KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5_CONFIG | Minor | security | Vipin Rathor | Vipin Rathor
YARN-7661 | NodeManager metrics return wrong value after update node resource | Major | . | Yang Wang | Yang Wang
HDFS-12347 | TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently | Critical | test | Xiao Chen | Bharat Viswanadham
YARN-7662 | [Atsv2] Define new set of configurations for reader and collectors to bind. | Major | . | Rohith Sharma K S | Rohith Sharma K S
YARN-7674 | Update Timeline Reader web app address in UI2 | Major | . | Rohith Sharma K S | Sunil Govindan
YARN-7542 | Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED | Major | . | Arun Suresh | Sampada Dehankar
HADOOP-15143 | NPE due to Invalid KerberosTicket in UGI | Major | . | Jitendra Nath Pandey | Mukul Kumar Singh
YARN-7692 | Skip validating priority acls while recovering applications | Blocker | resourcemanager | Charan Hebri | Sunil Govindan
MAPREDUCE-7028 | Concurrent task progress updates causing NPE in Application Master | Blocker | mr-am | Gergo Repas | Gergo Repas
YARN-7619 | Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user | Major | capacity scheduler, yarn | Eric Payne | Eric Payne
YARN-7699 | queueUsagePercentage is coming as INF for getApp REST api call | Major | webapp | Sunil Govindan | Sunil Govindan
HDFS-12985 | NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted | Major | hdfs | Manoj Govindassamy | Manoj Govindassamy
YARN-4227 | Ignore expired containers from removed nodes in FairScheduler | Critical | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg
YARN-7508 | NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode | Major | capacityscheduler | Tao Yang | Tao Yang
YARN-7663 | RMAppImpl:Invalid event: START at KILLED | Major | resourcemanager | lujie | lujie
YARN-6948 | Invalid event: ATTEMPT_ADDED at FINAL_SAVING | Major | yarn | lujie | lujie
HADOOP-15060 | TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky | Major | . | Miklos Szegedi | Miklos Szegedi
YARN-7735 | Fix typo in YARN documentation | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma
YARN-7727 | Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector | Minor | yarn | Prabhu Joseph | Prabhu Joseph
HDFS-11915 | Sync rbw dir on the first hsync() to avoid file lost on power failure | Critical | . | Kanaka Kumar Avvaru | Vinayakumar B
YARN-7705 | Create the container log directory with correct sticky bit in C code | Major | nodemanager | Yufei Gu | Yufei Gu
HDFS-9049 | Make Datanode Netty reverse proxy port to be configurable | Major | datanode | Vinayakumar B | Vinayakumar B
YARN-7758 | Add an additional check to the validity of container and application ids passed to container-executor | Major | nodemanager | Miklos Szegedi | Yufei Gu
HADOOP-15150 | in FsShell, UGI params should be overidden through env vars(-D arg) | Major | . | Brahma Reddy Battula | Brahma Reddy Battula
HADOOP-15181 | Typo in SecureMode.md | Trivial | documentation | Masahiro Tanaka | Masahiro Tanaka
YARN-7806 | Distributed Shell should use timeline async api’s | Major | distributed-shell | Sumana Sathish | Rohith Sharma K S
HADOOP-15121 | Encounter NullPointerException when using DecayRpcScheduler | Major | . | Tao Jie | Tao Jie
MAPREDUCE-7015 | Possible race condition in JHS if the job is not loaded | Major | jobhistoryserver | Peter Bacsko | Peter Bacsko
YARN-7737 | prelaunch.err file not found exception on container failure | Major | . | Jonathan Hung | Keqiu Hu
HDFS-13063 | Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md | Trivial | documentation | Jianfei Jiang | Jianfei Jiang
YARN-7102 | NM heartbeat stuck when responseId overflows MAX_INT | Critical | . | Botong Huang | Botong Huang
MAPREDUCE-7041 | MR should not try to clean up at first job attempt | Major | . | Takanobu Asanuma | Gergo Repas
MAPREDUCE-7020 | Task timeout in uber mode can crash AM | Major | mr-am | Akira Ajisaka | Peter Bacsko
YARN-7765 | [Atsv2] GSSException: No valid credentials provided - Failed to find any Kerberos tgt thrown by Timelinev2Client & HBaseClient in NM | Blocker | . | Sumana Sathish | Rohith Sharma K S
HDFS-12974 | Exception message is not printed when creating an encryption zone fails with AuthorizationException | Minor | encryption | fang zhenyi | fang zhenyi
YARN-7698 | A misleading variable’s name in ApplicationAttemptEventDispatcher | Minor | resourcemanager | Jinjiang Ling | Jinjiang Ling
HDFS-12528 | Add an option to not disable short-circuit reads on failures | Major | hdfs-client, performance | Andre Araujo | Xiao Chen
HDFS-13100 | Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. | Critical | hdfs, webhdfs | Yongjun Zhang | Yongjun Zhang
YARN-7849 | TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error | Major | test | Jason Lowe | Botong Huang
YARN-7801 | AmFilterInitializer should addFilter after fill all parameters | Critical | . | Sumana Sathish | Wangda Tan
YARN-7890 | NPE during container relaunch | Major | . | Billie Rinaldi | Jason Lowe
HDFS-13115 | In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted | Major | . | Yongjun Zhang | Yongjun Zhang
HDFS-12935 | Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up | Major | tools | Jianfei Jiang | Jianfei Jiang
HDFS-13120 | Snapshot diff could be corrupted after concat | Major | namenode, snapshots | Xiaoyu Yao | Xiaoyu Yao
HDFS-10453 | ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster. | Major | namenode | He Xiaoqiao | He Xiaoqiao
HDFS-8693 | refreshNamenodes does not support adding a new standby to a running DN | Critical | datanode, ha | Jian Fang | Ajith S
MAPREDUCE-7052 | TestFixedLengthInputFormat#testFormatCompressedIn is flaky | Major | client, test | Peter Bacsko | Peter Bacsko
HDFS-13112 | Token expiration edits may cause log corruption or deadlock | Critical | namenode | Daryn Sharp | Daryn Sharp
MAPREDUCE-7053 | Timed out tasks can fail to produce thread dump | Major | . | Jason Lowe | Jason Lowe
HADOOP-15206 | BZip2 drops and duplicates records when input split size is small | Major | . | Aki Tanaka | Aki Tanaka
YARN-7937 | Fix http method name in Cluster Application Timeout Update API example request | Minor | docs, documentation | Charan Hebri | Charan Hebri
YARN-7947 | Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne
YARN-7945 | Java Doc error in UnmanagedAMPoolManager for branch-2 | Major | . | Rohith Sharma K S | Botong Huang
HADOOP-14903 | Add json-smart explicitly to pom.xml | Major | common | Ray Chiang | Ray Chiang
HADOOP-15236 | Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider | Trivial | documentation | Akira Ajisaka | Gabor Bota
MAPREDUCE-7027 | HadoopArchiveLogs shouldn’t delete the original logs if the HAR creation fails | Critical | mrv2 | Gergely Novák | Gergely Novák
HDFS-12781 | After Datanode down, In Namenode UI Datanode tab is throwing warning message. | Major | datanode | Harshakiran Reddy | Brahma Reddy Battula
HDFS-12070 | Failed block recovery leaves files open indefinitely and at risk for data loss | Major | . | Daryn Sharp | Kihwal Lee
HADOOP-15251 | Backport HADOOP-13514 (surefire upgrade) to branch-2 | Major | test | Chris Douglas | Chris Douglas
HDFS-13194 | CachePool permissions incorrectly checked | Major | . | Yiqun Lin | Jianfei Jiang
HADOOP-15276 | branch-2 site not building after ADL troubleshooting doc added | Major | documentation | Steve Loughran | Steve Loughran
YARN-7835 | [Atsv2] Race condition in NM while publishing events if second attempt is launched on the same node | Critical | . | Rohith Sharma K S | Rohith Sharma K S
HADOOP-15275 | Incorrect javadoc for return type of RetryPolicy#shouldRetry | Minor | documentation | Nanda kumar | Nanda kumar
YARN-7511 | NPE in ContainerLocalizer when localization failed for running container | Major | nodemanager | Tao Yang | Tao Yang
MAPREDUCE-7023 | TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun | Minor | test | Gergely Novák | Gergely Novák
HADOOP-15283 | Upgrade from findbugs 3.0.1 to spotbugs 3.1.2 in branch-2 to fix docker image build | Major | . | Xiao Chen | Akira Ajisaka
HADOOP-15286 | Remove unused imports from TestKMSWithZK.java | Minor | test | Akira Ajisaka | Ajay Kumar
HDFS-13040 | Kerberized inotify client fails despite kinit properly | Major | namenode | Wei-Chiu Chuang | Xiao Chen
YARN-7736 | Fix itemization in YARN federation document | Minor | documentation | Akira Ajisaka | Sen Zhao
HDFS-13164 | File not closed if streamer fail with DSQuotaExceededException | Major | hdfs-client | Xiao Chen | Xiao Chen
HDFS-13109 | Support fully qualified hdfs path in EZ commands | Major | hdfs | Hanisha Koneru | Hanisha Koneru
MAPREDUCE-6930 | mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml | Major | mrv2 | Daniel Templeton | Sen Zhao
HDFS-10618 | TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition | Major | . | Eric Badger | Eric Badger
HDFS-10803 | TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available | Major | . | Yiqun Lin | Yiqun Lin
HDFS-12156 | TestFSImage fails without -Pnative | Major | test | Akira Ajisaka | Akira Ajisaka
HDFS-13261 | Fix incorrect null value check | Minor | hdfs | Jianfei Jiang | Jianfei Jiang
HDFS-12886 | Ignore minReplication for block recovery | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak
YARN-8039 | Clean up log dir configuration in TestLinuxContainerExecutorWithMocks.testStartLocalizer | Minor | . | Miklos Szegedi | Miklos Szegedi
HDFS-13296 | GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases | Major | . | Xiao Liang | Xiao Liang
HDFS-13268 | TestWebHdfsFileContextMainOperations fails on Windows | Major | . | Íñigo Goiri | Xiao Liang
YARN-8054 | Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread | Major | . | Jonathan Eagles | Jonathan Eagles
YARN-7873 | Revert YARN-6078 | Blocker | . | Billie Rinaldi | Billie Rinaldi
HDFS-13195 | DataNode conf page cannot display the current value after reconfig | Minor | datanode | maobaolong | maobaolong
YARN-8063 | DistributedShellTimelinePlugin wrongly check for entityId instead of entityType | Major | . | Rohith Sharma K S | Rohith Sharma K S
YARN-8068 | Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ | Blocker | yarn | Sunil Govindan | Sunil Govindan
HADOOP-12862 | LDAP Group Mapping over SSL can not specify trust store | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang
HADOOP-15317 | Improve NetworkTopology chooseRandom’s loop | Major | . | Xiao Chen | Xiao Chen
HADOOP-15355 | TestCommonConfigurationFields is broken by HADOOP-15312 | Major | test | Konstantin Shvachko | LiXin Ge
HDFS-13176 | WebHdfs file path gets truncated when having semicolon (;) inside | Major | webhdfs | Zsolt Venczel | Zsolt Venczel
HADOOP-15375 | Branch-2 pre-commit failed to build docker image | Major | . | Xiao Chen | Xiao Chen
HADOOP-15357 | Configuration.getPropsWithPrefix no longer does variable substitution | Major | . | Jim Brennan | Jim Brennan
MAPREDUCE-7062 | Update mapreduce.job.tags description for making use for ATSv2 purpose. | Major | . | Charan Hebri | Charan Hebri
YARN-8073 | TimelineClientImpl doesn’t honor yarn.timeline-service.versions configuration | Major | . | Rohith Sharma K S | Rohith Sharma K S
YARN-6629 | NPE occurred when container allocation proposal is applied but its resource requests are removed before | Critical | . | Tao Yang | Tao Yang
HDFS-13427 | Fix the section titles of transparent encryption document | Minor | documentation | Akira Ajisaka | Akira Ajisaka
YARN-7527 | Over-allocate node resource in async-scheduling mode of CapacityScheduler | Major | capacityscheduler | Tao Yang | Tao Yang
HDFS-7101 | Potential null dereference in DFSck#doWork() | Minor | . | Ted Yu | skrho
YARN-8120 | JVM can crash with SIGSEGV when exiting due to custom leveldb logger | Major | nodemanager, resourcemanager | Jason Lowe | Jason Lowe
YARN-8147 | TestClientRMService#testGetApplications sporadically fails | Major | test | Jason Lowe | Jason Lowe
HADOOP-14970 | MiniHadoopClusterManager doesn’t respect lack of format option | Minor | . | Erik Krogen | Erik Krogen
YARN-8156 | Increase the default value of yarn.timeline-service.app-collector.linger-period.ms | Major | . | Rohith Sharma K S | Charan Hebri
YARN-8165 | Incorrect queue name logging in AbstractContainerAllocator | Trivial | capacityscheduler | Weiwei Yang | Weiwei Yang
YARN-8164 | Fix a potential NPE in AbstractSchedulerPlanFollower | Major | . | lujie | lujie
HDFS-12828 | OIV ReverseXML Processor fails with escaped characters | Critical | hdfs | Erik Krogen | Erik Krogen
HADOOP-15180 | branch-2 : daemon processes’ sysout overwrites ‘ulimit -a’ in daemon’s out file | Minor | scripts | Ranith Sardar | Ranith Sardar
HADOOP-15396 | Some java source files are executable | Minor | . | Akira Ajisaka | Shashikant Banerjee
YARN-6827 | [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S
YARN-7786 | NullPointerException while launching ApplicationMaster | Major | . | lujie | lujie
HDFS-13408 | MiniDFSCluster to support being built on randomized base directory | Major | test | Xiao Liang | Xiao Liang
HADOOP-15390 | Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens | Critical | . | Xiao Chen | Xiao Chen
HDFS-13336 | Test cases of TestWriteToReplica failed in windows | Major | . | Xiao Liang | Xiao Liang
YARN-7598 | Document how to use classpath isolation for aux-services in YARN | Major | . | Xuan Gong | Xuan Gong
HADOOP-15385 | Many tests are failing in hadoop-distcp project in branch-2 | Critical | tools/distcp | Rushabh S Shah | Jason Lowe
MAPREDUCE-7042 | Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled | Major | . | Yesha Vora | Xuan Gong
YARN-8205 | Application State is not updated to ATS if AM launching is delayed. | Critical | . | Sumana Sathish | Rohith Sharma K S
MAPREDUCE-7072 | mapred job -history prints duplicate counter in human output | Major | client | Wilfred Spiegelenburg | Wilfred Spiegelenburg
YARN-8221 | RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps | Major | webapp | Sunil Govindan | Sunil Govindan
HDFS-13509 | Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows | Major | . | Xiao Liang | Xiao Liang
MAPREDUCE-7073 | Optimize TokenCache#obtainTokensForNamenodesInternal | Major | . | Bibin A Chundatt | Bibin A Chundatt
YARN-8232 | RMContainer lost queue name when RM HA happens | Major | resourcemanager | Hu Ziqian | Hu Ziqian
HDFS-13537 | TestHdfsHelper does not generate jceks path properly for relative path in Windows | Major | . | Xiao Liang | Xiao Liang
HADOOP-15446 | WASB: PageBlobInputStream.skip breaks HBASE replication | Major | fs/azure | Thomas Marquardt | Thomas Marquardt
YARN-8244 | TestContainerSchedulerQueuing.testStartMultipleContainers failed | Major | . | Miklos Szegedi | Jim Brennan
HDFS-13586 | Fsync fails on directories on Windows | Critical | datanode, hdfs | Lukas Majercak | Lukas Majercak
HDFS-13590 | Backport HDFS-12378 to branch-2 | Major | datanode, hdfs, test | Lukas Majercak | Lukas Majercak
HADOOP-15478 | WASB: hflush() and hsync() regression | Major | fs/azure | Thomas Marquardt | Thomas Marquardt
HADOOP-15450 | Avoid fsync storm triggered by DiskChecker and handle disk full situation | Blocker | . | Kihwal Lee | Arpit Agarwal
HDFS-13601 | Optimize ByteString conversions in PBHelper | Major | . | Andrew Wang | Andrew Wang
HDFS-13588 | Fix TestFsDatasetImpl test failures on Windows | Major | . | Xiao Liang | Xiao Liang
YARN-8310 | Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats | Major | . | Robert Kanter | Robert Kanter
YARN-8344 | Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
YARN-8327 | Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows | Major | log-aggregation | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
YARN-8346 | Upgrading to 3.1 kills running containers with error “Opportunistic container queue is full” | Blocker | . | Rohith Sharma K S | Jason Lowe
HDFS-13611 | Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient | Major | . | Andrew Wang | Andrew Wang
HDFS-13618 | Fix TestDataNodeFaultInjector test failures on Windows | Major | test | Xiao Liang | Xiao Liang
HADOOP-15473 | Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997 | Critical | kms | Gabor Bota | Gabor Bota
HDFS-13626 | Fix incorrect username when deny the setOwner operation | Minor | namenode | luhuachao | Zsolt Venczel
MAPREDUCE-7103 | Fix TestHistoryViewerPrinter on windows due to a mismatch line separator | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
YARN-8359 | Exclude containermanager.linux test classes on Windows | Major | . | Giovanni Matteo Fumarola | Jason Lowe
HDFS-13664 | Refactor ConfiguredFailoverProxyProvider to make inheritance easier | Minor | hdfs-client | Chao Sun | Chao Sun
YARN-8405 | RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773 | Major | . | Rohith Sharma K S | Íñigo Goiri
MAPREDUCE-7108 | TestFileOutputCommitter fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang
MAPREDUCE-7101 | Add config parameter to allow JHS to alway scan user dir irrespective of modTime | Critical | . | Wangda Tan | Thomas Marquardt
YARN-8404 | Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down | Blocker | . | Rohith Sharma K S | Rohith Sharma K S
HDFS-13675 | Speed up TestDFSAdminWithHA | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak
HDFS-13673 | TestNameNodeMetrics fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang
HDFS-13676 | TestEditLogRace fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang
HADOOP-15523 | Shell command timeout given is in seconds whereas it is taken as millisec while scheduling | Major | . | Bilwa S T | Bilwa S T
YARN-8444 | NodeResourceMonitor crashes on bad swapFree value | Major | . | Jim Brennan | Jim Brennan
YARN-8457 | Compilation is broken with -Pyarn-ui | Major | webapp | Sunil Govindan | Sunil Govindan
YARN-8401 | [UI2] new ui is not accessible with out internet connection | Blocker | . | Bibin A Chundatt | Bibin A Chundatt
YARN-8451 | Multiple NM heartbeat thread created when a slow NM resync with RM | Major | nodemanager | Botong Huang | Botong Huang
HADOOP-15548 | Randomize local dirs | Minor | . | Jim Brennan | Jim Brennan
YARN-8473 | Containers being launched as app tears down can leave containers in NEW state | Major | nodemanager | Jason Lowe | Jason Lowe
HDFS-13729 | Fix broken links to RBF documentation | Minor | documentation | jwhitter | Gabor Bota
YARN-8515 | container-executor can crash with SIGPIPE after nodemanager restart | Major | . | Jim Brennan | Jim Brennan
YARN-8421 | when moving app, activeUsers is increased, even though app does not have outstanding request | Major | . | kyungwan nam |
HADOOP-15614 | TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails | Major | . | Kihwal Lee | Weiwei Yang
YARN-4606 | CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps | Critical | capacity scheduler, capacityscheduler | Karam Singh | Manikandan R
HADOOP-15637 | LocalFs#listLocatedStatus does not filter out hidden .crc files | Minor | fs | Erik Krogen | Erik Krogen
HADOOP-15644 | Hadoop Docker Image Pip Install Fails on branch-2 | Critical | build | Haibo Chen | Haibo Chen
YARN-6966 | NodeManager metrics may return wrong negative values when NM restart | Major | . | Yang Wang | Szilard Nemeth
YARN-8331 | Race condition in NM container launched after done | Major | . | Yang Wang | Pradeep Ambati
HDFS-13758 | DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction | Major | namenode | Wei-Chiu Chuang | chencan
YARN-8612 | Fix NM Collector Service Port issue in YarnConfiguration | Major | ATSv2 | Prabha Manepalli | Prabha Manepalli
HADOOP-15674 | Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite | Major | common | Gabor Bota | Szilard Nemeth
YARN-8640 | Restore previous state in container-executor after failure | Major | . | Jim Brennan | Jim Brennan
YARN-8679 | [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked | Major | . | Rohith Sharma K S | Wangda Tan
HADOOP-14314 | The OpenSolaris taxonomy link is dead in InterfaceClassification.md | Major | documentation | Daniel Templeton | Rui Gao
YARN-8649 | NPE in localizer hearbeat processing if a container is killed while localizing | Major | . | lujie | lujie

### TESTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-13337 | Backport HDFS-4275 to branch-2.9 | Minor | . | Íñigo Goiri | Xiao Liang
HDFS-13503 | Fix TestFsck test failures on Windows | Major | hdfs | Xiao Liang | Xiao Liang
HDFS-13542 | TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13551 | TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-11700 | TestHDFSServerPorts#testBackupNodePorts doesn’t pass on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13548 | TestResolveHdfsSymlink#testFcResolveAfs fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13567 | TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir | Minor | . | Anbang Hu | Anbang Hu
HDFS-13557 | TestDFSAdmin#testListOpenFiles fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13550 | TestDebugAdmin#testComputeMetaCommand fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13559 | TestBlockScanner does not close TestContext properly | Minor | . | Anbang Hu | Anbang Hu
HDFS-13570 | TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13558 | TestDatanodeHttpXFrame does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-13554 | TestDatanodeRegistration#testForcedRegistration does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-13556 | TestNestedEncryptionZones does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-13560 | Insufficient system resources exist to complete the requested service for some tests on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13592 | TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly | Minor | . | Anbang Hu | Anbang Hu
HDFS-13593 | TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows | Minor | test | Anbang Hu | Anbang Hu
HDFS-13587 | TestQuorumJournalManager fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13620 | Randomize the test directory path for TestHDFSFileSystemContract | Minor | . | Anbang Hu | Anbang Hu
HDFS-13591 | TestDFSShell#testSetrepLow fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13632 | Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA | Minor | . | Anbang Hu | Anbang Hu
MAPREDUCE-7102 | Fix TestJavaSerialization for Windows due a mismatch line separator | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
HDFS-13652 | Randomize baseDir for MiniDFSCluster in TestBlockScanner | Minor | . | Anbang Hu | Anbang Hu
YARN-8370 | Some Node Manager tests fail on Windows due to improper path/file separator | Minor | . | Anbang Hu | Anbang Hu
YARN-8422 | TestAMSimulator failing with NPE | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
HADOOP-15532 | TestBasicDiskValidator fails with NoSuchFileException | Minor | . | Íñigo Goiri | Giovanni Matteo Fumarola
HDFS-13563 | TestDFSAdminWithHA times out on Windows | Minor | . | Anbang Hu | Lukas Majercak
HDFS-13681 | Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows | Major | test | Xiao Liang | Xiao Liang

### SUB-TASKS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-14799 | Update nimbus-jose-jwt to 4.41.1 | Major | . | Ray Chiang | Ray Chiang
HADOOP-14997 | Add hadoop-aliyun as dependency of hadoop-cloud-storage | Minor | fs/oss | Genmao Yu | Genmao Yu
HDFS-12801 | RBF: Set MountTableResolver as default file resolver | Minor | . | Íñigo Goiri | Íñigo Goiri
YARN-7430 | Enable user re-mapping for Docker containers by default | Blocker | security, yarn | Eric Yang | Eric Yang
HADOOP-15024 | AliyunOSS: support user agent configuration and include that & Hadoop version information to oss server | Major | fs, fs/oss | Sammi Chen | Sammi Chen
HDFS-12858 | RBF: Add router admin commands usage in HDFS commands reference doc | Minor | documentation | Yiqun Lin | Yiqun Lin
HDFS-12835 | RBF: Fix Javadoc parameter errors | Minor | . | Wei Yan | Wei Yan
HDFS-12396 | Webhdfs file system should get delegation token from kms provider. | Major | encryption, kms, webhdfs | Rushabh S Shah | Rushabh S Shah
YARN-6704 | Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService | Major | . | Botong Huang | Botong Huang
HDFS-12875 | RBF: Complete logic for -readonly option of dfsrouteradmin add command | Major | . | Yiqun Lin | Íñigo Goiri
YARN-7630 | Fix AMRMToken rollover handling in AMRMProxy | Minor | . | Botong Huang | Botong Huang
HDFS-12937 | RBF: Add more unit tests for router admin commands | Major | test | Yiqun Lin | Yiqun Lin
YARN-7032 | [ATSv2] NPE while starting hbase co-processor when HBase authorization is enabled. | Critical | . | Rohith Sharma K S | Rohith Sharma K S
HDFS-12988 | RBF: Mount table entries not properly updated in the local cache | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-12802 | RBF: Control MountTableResolver cache size | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-12934 | RBF: Federation supports global quota | Major | . | Yiqun Lin | Yiqun Lin
HDFS-12972 | RBF: Display mount table quota info in Web UI and admin command | Major | . | Yiqun Lin | Yiqun Lin
YARN-6736 | Consider writing to both ats v1 & v2 from RM for smoother upgrades | Major | timelineserver | Vrushali C | Aaron Gresch
HADOOP-15027 | AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance | Major | fs/oss | wujinhu | wujinhu
HDFS-12973 | RBF: Document global quota supporting in federation | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13028 | RBF: Fix spurious TestRouterRpc#testProxyGetStats | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-12772 | RBF: Federation Router State State Store internal API | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13042 | RBF: Heartbeat Router State | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13049 | RBF: Inconsistent Router OPTS config in branch-2 and branch-3 | Minor | . | Wei Yan | Wei Yan
HDFS-12574 | Add CryptoInputStream to WebHdfsFileSystem read call. | Major | encryption, kms, webhdfs | Rushabh S Shah | Rushabh S Shah
HDFS-13044 | RBF: Add a safe mode for the Router | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13043 | RBF: Expose the state of the Routers in the federation | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13068 | RBF: Add router admin option to manage safe mode | Major | . | Íñigo Goiri | Yiqun Lin
HDFS-13119 | RBF: Manage unavailable clusters | Major | . | Íñigo Goiri | Yiqun Lin
HDFS-13187 | RBF: Fix Routers information shown in the web UI | Minor | . | Wei Yan | Wei Yan
HDFS-13184 | RBF: Improve the unit test TestRouterRPCClientRetries | Minor | test | Yiqun Lin | Yiqun Lin
HDFS-13199 | RBF: Fix the hdfs router page missing label icon issue | Major | federation, hdfs | maobaolong | maobaolong
HADOOP-15090 | Add ADL troubleshooting doc | Major | documentation, fs/adl | Steve Loughran | Steve Loughran
YARN-7919 | Refactor timelineservice-hbase module into submodules | Major | timelineservice | Haibo Chen | Haibo Chen
YARN-8003 | Backport the code structure changes in YARN-7346 to branch-2 | Major | . | Haibo Chen | Haibo Chen
HDFS-13214 | RBF: Complete document of Router configuration | Major | . | Tao Jie | Yiqun Lin
HADOOP-15267 | S3A multipart upload fails when SSE-C encryption is enabled | Critical | fs/s3 | Anis Elleuch | Anis Elleuch
HDFS-13230 | RBF: ConnectionManager’s cleanup task will compare each pool’s own active conns with its total conns | Minor | . | Wei Yan | Chao Sun
HDFS-13233 | RBF: MountTableResolver doesn’t return the correct mount point of the given path | Major | hdfs | wangzhiyuan | wangzhiyuan
HDFS-13212 | RBF: Fix router location cache issue | Major | federation, hdfs | Weiwei Wu | Weiwei Wu
HDFS-13232 | RBF: ConnectionPool should return first usable connection | Minor | . | Wei Yan | Ekanth Sethuramalingam
HDFS-13240 | RBF: Update some inaccurate document descriptions | Minor | . | Yiqun Lin | Yiqun Lin
HDFS-11399 | Many tests fails in Windows due to injecting disk failures | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13241 | RBF: TestRouterSafemode failed if the port 8888 is in use | Major | hdfs, test | maobaolong | maobaolong
HDFS-13253 | RBF: Quota management incorrect parent-child relationship judgement | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13226 | RBF: Throw the exception if mount table entry validated failed | Major | hdfs | maobaolong | maobaolong
HADOOP-15308 | TestConfiguration fails on Windows because of paths | Major | test | Íñigo Goiri | Xiao Liang
HDFS-12773 | RBF: Improve State Store FS implementation | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13198 | RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router | Minor | . | Wei Yan | Wei Yan
HDFS-13224 | RBF: Resolvers to support mount points across multiple subclusters | Major | . | Íñigo Goiri | Íñigo Goiri
HADOOP-15262 | AliyunOSS: move files under a directory in parallel when rename a directory | Major | fs/oss | wujinhu | wujinhu
HDFS-13215 | RBF: Move Router to its own module | Major | . | Íñigo Goiri | Wei Yan
HDFS-13307 | RBF: Improve the use of setQuota command | Major | . | liuhongtong | liuhongtong
HDFS-13250 | RBF: Router to manage requests across multiple subclusters | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13318 | RBF: Fix FindBugs in hadoop-hdfs-rbf | Minor | . | Íñigo Goiri | Ekanth Sethuramalingam
HDFS-12792 | RBF: Test Router-based federation using HDFSContract | Major | . | Íñigo Goiri | Íñigo Goiri
YARN-7581 | HBase filters are not constructed correctly in ATSv2 | Major | ATSv2 | Haibo Chen | Haibo Chen
YARN-7986 | ATSv2 REST API queries do not return results for uppercase application tags | Critical | . | Charan Hebri | Charan Hebri
HDFS-12512 | RBF: Add WebHDFS | Major | fs | Íñigo Goiri | Wei Yan
HDFS-13291 | RBF: Implement available space based OrderResolver | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13204 | RBF: Optimize name service safe mode icon | Minor | . | liuhongtong | liuhongtong
HDFS-13352 | RBF: Add xsl stylesheet for hdfs-rbf-default.xml | Major | documentation | Takanobu Asanuma | Takanobu Asanuma
YARN-8010 | Add config in FederationRMFailoverProxy to not bypass facade cache when failing over | Minor | . | Botong Huang | Botong Huang
HDFS-13347 | RBF: Cache datanode reports | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13289 | RBF: TestConnectionManager#testCleanup() test case need correction | Minor | . | Dibyendu Karmakar | Dibyendu Karmakar
HDFS-13364 | RBF: Support NamenodeProtocol in the Router | Major | . | Íñigo Goiri | Íñigo Goiri
YARN-6936 | [Atsv2] Retrospect storing entities into sub application table from client perspective | Major | . | Rohith Sharma K S | Rohith Sharma K S
HDFS-13353 | RBF: TestRouterWebHDFSContractCreate failed | Major | test | Takanobu Asanuma | Takanobu Asanuma
YARN-8107 | Give an informative message when incorrect format is used in ATSv2 filter attributes | Major | ATSv2 | Charan Hebri | Rohith Sharma K S
YARN-8110 | AMRMProxy recover should catch for all throwable to avoid premature exit | Major | . | Botong Huang | Botong Huang
HDFS-13402 | RBF: Fix java doc for StateStoreFileSystemImpl | Minor | hdfs | Yiran Wu | Yiran Wu
HDFS-13380 | RBF: mv/rm fail after the directory exceeded the quota limit | Major | . | Weiwei Wu | Yiqun Lin
HDFS-13410 | RBF: Support federation with no subclusters | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13384 | RBF: Improve timeout RPC call mechanism | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13045 | RBF: Improve error message returned from subcluster | Minor | . | Wei Yan | Íñigo Goiri
HDFS-13428 | RBF: Remove LinkedList From StateStoreFileImpl.java | Trivial | federation | BELUGA BEHR | BELUGA BEHR
HADOOP-14999 | AliyunOSS: provide one asynchronous multi-part based uploading mechanism | Major | fs/oss | Genmao Yu | Genmao Yu
YARN-7810 | TestDockerContainerRuntime test failures due to UID lookup of a non-existent user | Major | . | Shane Kumpf | Shane Kumpf
HDFS-13435 | RBF: Improve the error loggings for printing the stack trace | Major | . | Yiqun Lin | Yiqun Lin
YARN-7189 | Container-executor doesn’t remove Docker containers that error out early | Major | yarn | Eric Badger | Eric Badger
HDFS-13466 | RBF: Add more router-related information to the UI | Minor | . | Wei Yan | Wei Yan
HDFS-13453 | RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table | Major | . | Dibyendu Karmakar | Dibyendu Karmakar
HDFS-13478 | RBF: Disabled Nameservice store API | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13490 | RBF: Fix setSafeMode in the Router | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13484 | RBF: Disable Nameservices from the federation | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13326 | RBF: Improve the interfaces to modify and view mount tables | Minor | . | Wei Yan | Gang Li
HDFS-13499 | RBF: Show disabled name services in the UI | Minor | . | Íñigo Goiri | Íñigo Goiri
YARN-8215 | ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM | Critical | ATSv2 | Yesha Vora | Rohith Sharma K S
HDFS-13508 | RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries | Minor | . | Ekanth Sethuramalingam | Ekanth Sethuramalingam
HDFS-13434 | RBF: Fix dead links in RBF document | Major | documentation | Akira Ajisaka | Chetna Chaudhari
HDFS-13488 | RBF: Reject requests when a Router is overloaded | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13525 | RBF: Add unit test TestStateStoreDisabledNameservice | Major | . | Yiqun Lin | Yiqun Lin
YARN-8253 | HTTPS Ats v2 api call fails with “bad HTTP parsed” | Critical | ATSv2 | Yesha Vora | Charan Hebri
HADOOP-15454 | TestRollingFileSystemSinkWithLocal fails on Windows | Major | test | Xiao Liang | Xiao Liang
HDFS-13346 | RBF: Fix synchronization of router quota and nameservice quota | Major | . | liuhongtong | Yiqun Lin
YARN-8247 | Incorrect HTTP status code returned by ATSv2 for non-whitelisted users | Critical | ATSv2 | Charan Hebri | Rohith Sharma K S
YARN-8130 | Race condition when container events are published for KILLED applications | Major | ATSv2 | Charan Hebri | Rohith Sharma K S
YARN-7900 | [AMRMProxy] AMRMClientRelayer for stateful FederationInterceptor | Major | . | Botong Huang | Botong Huang
HADOOP-15498 | TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13637 | RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE | Critical | federation | CR Hota | CR Hota
YARN-4781 | Support intra-queue preemption for fairness ordering policy. | Major | scheduler | Wangda Tan | Eric Payne
HADOOP-15506 | Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks | Minor | fs/azure | Esfandiar Manii | Esfandiar Manii
HADOOP-15529 | ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
HADOOP-15533 | Make WASB listStatus messages consistent | Trivial | fs/azure | Esfandiar Manii | Esfandiar Manii
HADOOP-15458 | TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows | Minor | test | Xiao Liang | Xiao Liang
HDFS-13528 | RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries | Major | . | Dibyendu Karmakar | Dibyendu Karmakar
HDFS-13710 | RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable | Major | federation, hdfs | yanghuafeng | yanghuafeng
HDFS-13726 | RBF: Fix RBF configuration links | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma
HDFS-13475 | RBF: Admin cannot enforce Router enter SafeMode | Major | . | Wei Yan | Chao Sun
HDFS-13733 | RBF: Add Web UI configurations and descriptions to RBF document | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma
HDFS-13743 | RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver | Major | . | Takanobu Asanuma | Takanobu Asanuma
HDFS-13583 | RBF: Router admin clrQuota is not synchronized with nameservice | Major | . | Dibyendu Karmakar | Dibyendu Karmakar
HDFS-13750 | RBF: Router ID in RouterRpcClient is always null | Major | . | Takanobu Asanuma | Takanobu Asanuma
YARN-8129 | Improve error message for invalid value in fields attribute | Minor | ATSv2 | Charan Hebri | Abhishek Modi
HDFS-13848 | Refactor NameNode failover proxy providers | Major | ha, hdfs-client | Konstantin Shvachko | Konstantin Shvachko
HDFS-13634 | RBF: Configurable value in xml for async connection request queue size. | Major | federation | CR Hota | CR Hota

### OTHER:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-15149 | CryptoOutputStream should implement StreamCapabilities | Major | fs | Mike Drob | Xiao Chen
HADOOP-15177 | Update the release year to 2018 | Blocker | build | Akira Ajisaka | Bharat Viswanadham
YARN-8412 | Move ResourceRequest.clone logic everywhere into a proper API | Minor | . | Botong Huang | Botong Huang