Issues Fixed in CDH 5.11.x

The following topics describe issues fixed in CDH 5.11.x, from newest to oldest release. You can also review What's New in CDH 5.11.x or Known Issues in CDH 5.

Issues Fixed in CDH 5.11.1

CDH 5.11.1 fixes the following issues:

Upstream Issues Fixed

The following upstream issues are fixed in CDH 5.11.1:

  • FLUME-3080 - Close failure in HDFS Sink might cause data loss
  • FLUME-3085 - HDFS Sink can skip flushing some BucketWriters, might lead to data loss
  • HADOOP-11400 - GraphiteSink does not reconnect to Graphite after 'broken pipe'
  • HADOOP-11599 - Client#getTimeout should use IPC_CLIENT_PING_DEFAULT when IPC_CLIENT_PING_KEY is not configured
  • HADOOP-12672 - RPC timeout should not override IPC ping interval
  • HADOOP-13503 - Improve SaslRpcClient failure logging
  • HADOOP-13926 - S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore
  • HADOOP-14019 - Fix some typos in the s3a docs
  • HADOOP-14028 - S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures
  • HADOOP-14051 - S3Guard: link docs from index, fix typos
  • HADOOP-14059 - typo in s3a rename(self, subdir) error message
  • HADOOP-14092 - Typo in hadoop-aws index.md
  • HADOOP-14104 - Client should always ask namenode for kms provider path
  • HADOOP-14144 - s3guard: CLI diff non-empty after import on new table
  • HADOOP-14172 - S3Guard: import does not import empty directory
  • HADOOP-14195 - CredentialProviderFactory$getProviders is not thread-safe
  • HADOOP-14204 - S3A multipart commit failing, "UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".
  • HADOOP-14215 - DynamoDB client should waitForActive on existing tables
  • HADOOP-14236 - S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries
  • HADOOP-14255 - S3A to delete unnecessary fake directory objects in mkdirs()
  • HADOOP-14256 - [S3A DOC] Correct the format for "Seoul" example
  • HADOOP-14268 - Fix markdown itemization in hadoop-aws documents
  • HADOOP-14282 - S3Guard: DynamoDBMetadata::prune() should self interrupt correctly
  • HADOOP-14417 - Update default SSL cipher list for KMS
  • HDFS-10715 - NPE when applying AvailableSpaceBlockPlacementPolicy
  • HDFS-10797 - Revert "Disk usage summary of snapshots causes renamed blocks to get counted twice
  • HDFS-11499 - Decommissioning stuck because of failing recovery
  • HDFS-11515 - -du throws ConcurrentModificationException
  • HDFS-11689 - New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code
  • HDFS-11816 - Update default SSL cipher list for HttpFS
  • YARN-3251 - Fixed a deadlock in CapacityScheduler when computing absoluteMaxAvailableCapacity in LeafQueue
  • YARN-6042 - Dump scheduler and queue state information into FairScheduler DEBUG log.
  • YARN-6360 - Prevent FS state dump logger from cramming other log files
  • YARN-6432 - FairScheduler: Reserve preempted resources for corresponding applications.
  • YARN-6433 - Only accessible cgroup mount directories should be selected for a controller.
  • YARN-6453 - fairscheduler-statedump.log gets generated regardless of service
  • YARN-6500 - Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler.
  • HBASE-15125 - BackportHBaseFsck's adoptHdfsOrphan function creates region with wrong end key boundary
  • HBASE-15941 - HBCK repair should not unsplit healthy splitted region
  • HBASE-15955 - Disable action in CatalogJanitor#setEnabled should wait for active cleanup scan to finish
  • HBASE-16032 - Possible memory leak in StoreScanner
  • HBASE-16238 - It's useless to catch SESSIONEXPIRED exception and retry in RecoverableZooKeeper
  • HBASE-16350 - Undo server abort from HBASE-14968
  • HBASE-16429 - FSHLog: deadlock if rollWriter called when ring buffer filled with appends
  • HBASE-16663 - JMX ConnectorServer stopped when unauthorized user try to stop HM/RS/cluster
  • HBASE-16721 - Concurrency issue in WAL unflushed seqId tracking
  • HBASE-17265 - Region left unassigned in master failover when region failed to open
  • HBASE-17328 - Properly dispose of looped replication peers
  • HBASE-17460 - enable_table_replication can not perform cyclic replication of a table
  • HBASE-17717 - Explicitly use "sasl" ACL scheme for hbase superuser
  • HBASE-17779 - disable_table_replication returns misleading message and does not turn off replication
  • HBASE-17792 - Use a shared thread pool for AtomicityWriter, AtomicGetReader, AtomicScanReader's connections in TestAcidGuarantees
  • HIVE-11141 - Improve RuleRegExp when the Expression node stack gets huge
  • HIVE-11428 - Performance: Struct IN() clauses are extremely slow
  • HIVE-11671 - Optimize RuleRegExp in DPP codepath
  • HIVE-11842 - Improve RuleRegExp by caching some internal data structures
  • HIVE-12179 - Add option to not add spark-assembly.jar to Hive classpath
  • HIVE-12768 - Thread safety: binary sortable serde decimal deserialization
  • HIVE-14210 - ExecDriver should call jobclient.close() to trigger cleanup
  • HIVE-14380 - Queries on tables with remote HDFS paths fail in "encryption" checks.
  • HIVE-15282 - Different modification times are used when an index is built and when its staleness is checked
  • HIVE-15782 - query on parquet table returns incorrect result when hive.optimize.index.filter is set to true
  • HIVE-15879 - Fix HiveMetaStoreChecker.checkPartitionDirs method
  • HIVE-15997 - Resource leaks when query is cancelled
  • HIVE-16024 - MSCK Repair Requires nonstrict hive.mapred.mode
  • HIVE-16047 - Shouldn't try to get KeyProvider unless encryption is enabled
  • HIVE-16156 - FileSinkOperator should delete existing output target when renaming
  • HIVE-16175 - Possible race condition in InstanceCache
  • HIVE-16205 - Improving type safety in Objectstore
  • HIVE-16297 - Improving hive logging configuration variables
  • HIVE-16394 - HoS does not support queue name change in middle of session
  • HIVE-16459 - Forward channelInactive to RpcDispatcher
  • HIVE-16646 - Alias in transform ... as clause shouldn't be case sensitive
  • HUE-6109 - [core] Remove the restriction on Document2 invalid chars
  • HUE-6144 - [oozie] Add generic XSL template to workflow graph parser
  • HUE-6154 - [core] Ace paste event shouldn't reset the cursor position
  • HUE-6155 - [core] Remove table extender header throttling for Firefox
  • HUE-6158 - [autocomplete] The autocompleter eats characters to the right of the cursor on insert
  • HUE-6193 - [converter] Retain last_executed time when creating doc2 object
  • HUE-6197 - [impala] Fix XSS Vulnerability in the old editors' error messages
  • HUE-6207 - [editor] Avoid to always show the horizontal scroll bar when there's no scrolling needed
  • HUE-6208 - [core] The scroll left anchor should reset the horizontal scrollbar position
  • HUE-6212 - [oozie] Prevent XSS injection in coordinator cron frequency field
  • HUE-6223 - [autocomplete] Fix issue where tables from subsequent statements appear in the autocomplete results
  • HUE-6228 - [core] Disable touchscreen detection on Nicescroll
  • HUE-6251 - [editor] Log warnings and continue on failed bulk delete and copy actions
  • IMPALA-3641 - Fix catalogd RPC responses to DROP IF EXISTS.
  • IMPALA-4088 - Assign fix values to the minicluster server ports
  • IMPALA-4293 - query profile should include error log
  • IMPALA-4544 - ASAN should ignore SEGV and leaks
  • IMPALA-4615 - Fix create_table.sql command order
  • IMPALA-4733 - Change HBase ports to non-ephemeral
  • IMPALA-4787 - Optimize APPX_MEDIAN() memory usage
  • IMPALA-4822 - Implement dynamic log level changes
  • IMPALA-4899 - Fix parquet table writer dictionary leak
  • IMPALA-4902 - Copy parameters map in HdfsPartition.toThrift().
  • IMPALA-4998 - Fix missing table lock acquisition.
  • IMPALA-5028 - Lock table in /catalog_objects endpoint.
  • IMPALA-5055 - Fix DCHECK in parquet-column-readers.cc ReadPageHeader()
  • IMPALA-5088 - Fix heap buffer overflow
  • IMPALA-5115 - Handle status from HdfsTableSink::WriteClusteredRowBatch
  • IMPALA-5145 - Do not constant fold null in CastExprs
  • IMPALA-5156 - Drop VLOG level passed into Kudu client
  • IMPALA-5186 - Handle failed CreateAndOpenScanner() in MT scan.
  • IMPALA-5193 - Initialize decompressor before finding first tuple
  • IMPALA-5251 - Fix propagation of input exprs' types in 2-phase agg
  • IMPALA-5252 - Fix crash in HiveUdfCall::GetStringVal() when mem_limit exceeded
  • IMPALA-5253 - Use appropriate transport for StatestoreSubscriber
  • IMPALA-5322 - Fix a potential crash in Frontend & Catalog JNI startup
  • OOZIE-2739 - Remove property expansion pattern from ShellMain's log4j properties content
  • OOZIE-2818 - Can't overwrite oozie.action.max.output.data on a per-workflow basis
  • OOZIE-2819 - Make Oozie REST API accept multibyte characters for script Actions
  • SENTRY-1508 - Revert ""REVERT: MetastorePlugin.java does not handle properly initialization failure
  • SENTRY-1605 - SENTRY-1508 need to be fixed because of Kerberos initialization issue
  • SENTRY-1683 - MetastoreCacheInitializer has a race condition in handling results list
  • SENTRY-1714 - MetastorePlugin.java should quetly return from renameAuthzObject() when both paths are null
  • SOLR-9836 - Add ability to recover from leader when index corruption is detected on SolrCore creation.
  • SOLR-9848 - Lower solr.cloud.wait-for-updates-with-stale-state-pause back down from 7 seconds.
  • SOLR-10360 - Remove an extra space from Hadoop distcp cmd used by Solr backup/restore
  • SOLR-10430 - Add ls command to ZkCLI for listing sub-dirs
  • SPARK-14930 - [SPARK-13693] Fix race condition in CheckpointWriter.stop()
  • SPARK-19178 - [SQL][Backport-to-1.6] convert string of large numbers to int should return null
  • SPARK-19263 - DAGScheduler should avoid sending conflicting task set.
  • SPARK-19537 - Move pendingPartitions to ShuffleMapStage.
  • SPARK-20435 - [CORE] More thorough redaction of sensitive information
  • SQOOP-3123 - Introduce escaping logic for column mapping parameters (same what Sqoop already uses for the DB column names), thus special column names (e.g. containing '#' character) and mappings realted to those columns can be in the same format (thus not confusing the end users), and also eliminates the related AVRO format clashing issues.
  • SQOOP-3140 - Removing deprecated mapred.map.max.attempts, mapred.reduce.max.attempts entries and using the new constants directly from Hadoop instead
  • SQOOP-3159 - Sqoop (export + --table) with Oracle table_name having '$' fails with error

Issues Fixed in CDH 5.11.0

CDH 5.11.0 fixes the following issues.

Upstream Issues Fixed

The following upstream issues are fixed in CDH 5.11.0:

  • AVRO-1973 - Upgrade netty to the latest 3.x release
  • FLUME-2318 - Make SpoolingDirectorySource able to handle empty files
  • FLUME-2514 - Speed up TestFileChannelRestart tests
  • FLUME-2567 - Remove external Maven repositories
  • FLUME-2593 - ResettableFileInputStream returns negate values from read() method
  • FLUME-2619 - Spooldir source should log channel exceptions
  • FLUME-2704 - Configurable poll delay for spooling directory source
  • FLUME-2718 - HTTP Source to support generic Stream Handler
  • FLUME-2725 - HDFS Sink does not use configured timezone for rounding
  • FLUME-2761 - Move Hive sink out of preview mode
  • FLUME-2798 - Malformed Syslog messages can lead to OutOfMemoryException
  • FLUME-2801 - Performance improvement on TailDir source
  • FLUME-2806 - flume-ng.ps1 Error running script to start an agent on Windows
  • FLUME-2810 - Add static Schema URL to AvroEventSerializer configuration
  • FLUME-2812 - Fix semaphore leak causing java.lang.Error: Maximum permit count exceeded in MemoryChannel
  • FLUME-2835 - Hive Sink tests need to create table with transactional property set
  • FLUME-2857 - Make Kafka Source/Channel/Sink restore default values when live updating config
  • FLUME-2875 - Allow RollingFileSink to specify a file prefix and a file extension.
  • FLUME-2881 - Windows Launch Script fails in plugins dir code
  • FLUME-2889 - Fixes to DateTime computations
  • FLUME-2890 - Typo in Twitter source warning
  • FLUME-2911 - Add include pattern option in SpoolDir source
  • FLUME-2936 - KafkaSource tests: Use loopback instead of host address
  • FLUME-2937 - Integrate checkstyle for non-test classes
  • FLUME-2939 - Update recursive SpoolDir source to use Java 7 APIs
  • FLUME-2941 - Integrate checkstyle for test classes
  • FLUME-2949 - checkstyle: Make suppressions compatible with Windows
  • FLUME-2950 - ReliableSpoolingFileEventReader.rollCurrentFile is broken
  • FLUME-2952 - SyslogAgent: Avoid NPE on stop()
  • FLUME-2955 - Add file path to the header in TaildirSource
  • FLUME-2959 - Fix issues with flume-checkstyle module
  • FLUME-2966 - Fix NPE in JMS Source
  • FLUME-2971 - Add secure Kafka Sink/Source/Channel setup to the User Guide
  • FLUME-2974 - Fix some broken SpoolDirSource unit tests
  • FLUME-2998 - Add a missing configuration parameter to SequenceSource docs
  • FLUME-2999 - Kafka channel and sink should enable statically assigned partition per event via header
  • FLUME-3027 - Change Kafka Channel to clear offsets map after commit
  • FLUME-3049 - Make HDFS sink rotate more reliably in secure mode
  • HADOOP-10823 - TestReloadingX509TrustManager is flaky
  • HADOOP-11447 - Add a more meaningful toString method to SampleStat and MutableStat.
  • HADOOP-11619 - FTPFileSystem should override getDefaultPort.
  • HADOOP-11859 - PseudoAuthenticationHandler fails with httpcomponents v4.4
  • HADOOP-11878 - FileContext#fixRelativePart should check for not null for a more informative exception.
  • HADOOP-12009 - Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus.
  • HADOOP-12655 - TestHttpServer.testBindAddress bind port range is wider than expected.
  • HADOOP-12667 - s3a to support createNonRecursive API
  • HADOOP-12774 - s3a should use UGI.getCurrentUser.getShortname() for username
  • HADOOP-12977 - s3a to handle delete("/", true) robustly
  • HADOOP-13037 - Refactor Azure Data Lake Store as an independent FileSystem
  • HADOOP-13119 - Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu.
  • HADOOP-13164 - Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories
  • HADOOP-13204 - Add support for SSE-KMS and SSE-C in s3a filesystem.
  • HADOOP-13207 - Specify FileSystem listStatus, listFiles and RemoteIterator
  • HADOOP-13257 - Improve Azure Data Lake contract tests
  • HADOOP-13309 - Document S3A known limitations in file ownership and permission model
  • HADOOP-13336 - S3A to support per-bucket configuration (S3Guard specific changes)
  • HADOOP-13345 - S3Guard: Ensure GenericOptionsParser is used for S3Guard CLI
  • HADOOP-13375 - o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky.
  • HADOOP-13433 - Race in UGI.reloginFromKeytab
  • HADOOP-13446 - Support running isolated unit tests separate from AWS integration tests
  • HADOOP-13448 - S3Guard: Define MetadataStore interface.
  • HADOOP-13449 - S3Guard: Implement DynamoDBMetadataStore
  • HADOOP-13452 - S3Guard: Implement access policy for intra-client consistency with in-memory metadata store
  • HADOOP-13455 - S3Guard: Write end user docs, change table autocreate default.
  • HADOOP-13496 - Include file lengths in Mismatch in length error for distcp
  • HADOOP-13502 - Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests
  • HADOOP-13508 - FsPermission string constructor does not recognize sticky bit
  • HADOOP-13540 - improve section on troubleshooting s3a auth problems
  • HADOOP-13560 - S3ABlockOutputStream to support huge (many GB) file writes
  • HADOOP-13573 - S3Guard: create basic contract tests for MetadataStore implementations
  • HADOOP-13589 - S3Guard: Allow execution of all S3A integration tests with S3Guard enabled
  • HADOOP-13599 - s3a close() to be non-synchronized, so avoid risk of deadlock on shutdown
  • HADOOP-13614 - Purge some superfluous/obsolete S3 FS tests that are slowing test runs down
  • HADOOP-13627 - Have an explicit KerberosAuthException for UGI to throw, text from public constants
  • HADOOP-13631 - S3Guard: implement move() for LocalMetadataStore, add unit tests
  • HADOOP-13650 - S3Guard: Provide command line tools to manipulate metadata store.
  • HADOOP-13651 - S3Guard: S3AFileSystem Integration with MetadataStore
  • HADOOP-13674 - S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint
  • HADOOP-13680 - fs.s3a.readahead.range to use getLongBytes
  • HADOOP-13692 - hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts
  • HADOOP-13727 - S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider
  • HADOOP-13735 - ITestS3AFileContextStatistics.testStatistics() failing
  • HADOOP-13742 - Expose NumOpenConnectionsPerUser as a metric.
  • HADOOP-13749 - KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used
  • HADOOP-13782 - Make MutableRates metrics thread-local write, aggregate-on-read
  • HADOOP-13793 - S3guard: add inconsistency injection, integration tests
  • HADOOP-13801 - regression: ITestS3AMiniYarnCluster failing
  • HADOOP-13804 - MutableStat mean loses accuracy if add(long, long) is used
  • HADOOP-13805 - UGI.getCurrentUser() fails if user does not have a keytab associated
  • HADOOP-13823 - s3a rename: fail if dest file exists
  • HADOOP-13826 - S3A Deadlock in multipart copy due to thread pool limits
  • HADOOP-13830 - Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string"
  • HADOOP-13847 - KMSWebApp should close KeyProviderCryptoExtension
  • HADOOP-13850 - s3guard to log choice of metadata store at debug
  • HADOOP-13855 - Fix a couple of the s3a statistic names to be consistent with the rest
  • HADOOP-13857 - S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs
  • HADOOP-13876 - S3Guard: better support for multi-bucket access
  • HADOOP-13877 - S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set
  • HADOOP-13886 - s3guard: ITestS3AFileOperationCost.testFakeDirectoryDeletion failure
  • HADOOP-13893 - dynamodb dependency -> compile
  • HADOOP-13899 - S3Guard: tune dynamodb client & tests
  • HADOOP-13900 - Remove snapshot version of SDK dependency from Azure Data Lake Store File System
  • HADOOP-13903 - Improvements to KMS logging to help debug authorization errors.
  • HADOOP-13904 - DynamoDBMetadataStore to handle DDB throttling
  • HADOOP-13908 - S3Guard: Existing tables may not be initialized correctly in DynamoDBMetadataStore
  • HADOOP-13911 - Remove TRUSTSTORE_PASSWORD related scripts from KMS
  • HADOOP-13914 - S3guard: Improve S3AFileStatus#isEmptyDirectory handling
  • HADOOP-13928 - TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error.
  • HADOOP-13929 - ADLS connector should not check in contract-test-options.xml.
  • HADOOP-13931 - S3AGuard: Use BatchWriteItem in DynamoDBMetadataStore#put()
  • HADOOP-13934 - S3Guard: DynamoDBMetadataStore#move() could be throwing exception due to BatchWriteItem limits
  • HADOOP-13937 - S3Guard: Mock bucket locations in MockS3ClientFactory
  • HADOOP-13953 - Make FTPFileSystem's data connection mode and transfer mode configurable
  • HADOOP-13956 - Read ADLS credentials from Credential Provider.
  • HADOOP-13958 - Bump up release year to 2017
  • HADOOP-13960 - Initialize DynamoDBMetadataStore without associated S3AFileSystem.
  • HADOOP-13962 - Update ADLS SDK to 2.1.4.
  • HADOOP-13976 - Path globbing does not match newlines
  • HADOOP-13985 - s3guard: add a version marker to every table
  • HADOOP-13988 - KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
  • HADOOP-13995 - s3guard cli: make tests easier to run and address failure
  • HADOOP-14001 - Improve delegation token validity checking.
  • HADOOP-14003 - Make additional KMS tomcat settings configurable
  • HADOOP-14013 - S3Guard: fix multi-bucket integration tests
  • HADOOP-14017 - User friendly name for ADLS user and group
  • HADOOP-14020 - Optimize dirListingUnion
  • HADOOP-14027 - S3Guard: Implicitly creating DynamoDB table ignores endpoint config
  • HADOOP-14029 - Fix KMSClientProvider for non-secure proxyuser use case
  • HADOOP-14040 - Use shaded aws-sdk uber-JAR 1.11.86
  • HADOOP-14041 - CLI command to prune old metadata
  • HADOOP-14044 - Synchronization issue in delegation token cancel functionality
  • HADOOP-14046 - Metastore destruction test creates table w/o version
  • HADOOP-14049 - Honour AclBit flag associated to file/folder permission for Azure datalake account
  • HADOOP-14050 - Add process name to kms process
  • HADOOP-14079 - s3guard: fix breaking link in s3guard.md
  • HADOOP-14083 - KMS should support old SSL clients.
  • HADOOP-14085 - s3guard: drop unnecessary type assertion and cast.
  • HADOOP-14094 - Rethink S3GuardTool options
  • HADOOP-14096 - s3guard: regression in dirListingUnion
  • HADOOP-14102 - Relax error message assertion in S3A test ITestS3AEncryptionSSEC
  • HADOOP-14110 - In S3AFileSystem, make getAmazonClient() package private; export getBucketLocation()
  • HADOOP-14113 - Review ADL Docs
  • HADOOP-14114 - S3A can no longer handle unencoded + in URIs
  • HADOOP-14125 - s3guard tool tests aren't isolated; can't run in parallel
  • HADOOP-14129 - ITestS3ACredentialsInURL sometimes fails
  • HADOOP-14130 - Simplify DynamoDBClientFactory for creating Amazon DynamoDB clients
  • HADOOP-14131 - kms.sh creates bogus dir for tomcat logs
  • HADOOP-14168 - S3GuardTool tests should not run if S3Guard is not set up
  • HADOOP-14181 - S3Guard: Add validation of DynamoDB region
  • HDFS-3716 - Purger should remove stale fsimage ckpt files
  • HDFS-4366 - Block Replication Policy Implementation May Skip Higher-Priority Blocks for Lower-Priority Blocks.
  • HDFS-8307 - Spurious DNS Queries from hdfs shell
  • HDFS-8640 - Make reserved RBW space visible through JMX.
  • HDFS-8883 - NameNode Metrics : Add FSNameSystem lock Queue Length
  • HDFS-9145 - Tracking methods that hold FSNamesytemLock for too long
  • HDFS-9205 - Do not schedule corrupt blocks for replication.
  • HDFS-9390 - Block management for maintenance states.
  • HDFS-9391 - Update webUI/JMX to display maintenance state info.
  • HDFS-9467 - Fix data race accessing writeLockHeldTimeStamp in FSNamesystem
  • HDFS-10292 - Add block id when client got Unable to close file exception
  • HDFS-10534 - NameNode WebUI should display DataNode usage histogram
  • HDFS-10559 - DiskBalancer: Use SHA1 for Plan ID
  • HDFS-10713 - Throttle FsNameSystem lock warnings
  • HDFS-10733 - NameNode terminated after full GC thinking QJM is unresponsive
  • HDFS-10798 - Make the threshold of reporting FSNamesystem lock contention configurable
  • HDFS-10808 - DiskBalancer does not execute multi-steps plan-redux
  • HDFS-10817 - Add Logging for Long-held NN Read Locks
  • HDFS-10896 - Move lock logging logic from FSNamesystem into FSNamesystemLock
  • HDFS-10941 - Improve BlockManager#processMisReplicatesAsync log
  • HDFS-10966 - Enhance Dispatcher logic on deciding when to give up a source DataNode
  • HDFS-11087 - NamenodeFsck should check if the output writer is still writable
  • HDFS-11132 - Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files
  • HDFS-11160 - VolumeScanner reports write-in-progress replicas as corrupt incorrectly
  • HDFS-11180 - Intermittent deadlock in NameNode when failover happens.
  • HDFS-11197 - Listing encryption zones fails when deleting a EZ that is on a snapshotted directory
  • HDFS-11259 - Update fsck to display maintenance state info.
  • HDFS-11263 - ClassCastException when we use Bzipcodec for Fsimage compression
  • HDFS-11265 - Extend visualization for Maintenance Mode under Datanode tab in the NameNode UI.
  • HDFS-11271 - Typo in NameNode UI
  • HDFS-11275 - Check groupEntryIndex and throw a helpful exception on failures when removing ACL.
  • HDFS-11280 - Allow WebHDFS to reuse HTTP connections to NN
  • HDFS-11292 - log lastWrittenTxId etc info in logSyncAll
  • HDFS-11296 - Maintenance state expiry should be an epoch time and not jvm monotonic.
  • HDFS-11306 - Print remaining edit logs from buffer if edit log can't be rolled
  • HDFS-11363 - Need more diagnosis info when seeing Slow waitForAckedSeqno.
  • HDFS-11378 - Verify multiple DataNodes can be decommissioned/maintenance at the same time.
  • HDFS-11379 - DFSInputStream may infinite loop requesting block locations
  • HDFS-11390 - Add process name to httpfs process
  • HDFS-11411 - Avoid OutOfMemoryError in TestMaintenanceState test runs.
  • HDFS-11412 - Maintenance minimum replication config value allowable range should be [0, DefaultReplication].
  • HDFS-11418 - HttpFS should support old SSL clients
  • HDFS-11441 - Add escaping to error message in KMS web UI
  • HDFS-11466 - Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms
  • MAPREDUCE-5155 - Race condition in test case TestFetchFailure cause it to fail
  • MAPREDUCE-5335 - Rename Job Tracker terminology in ShuffleSchedulerImpl
  • MAPREDUCE-5883 - "Total megabyte-seconds" in job counters is slightly misleading
  • MAPREDUCE-5981 - Log levels of certain MR logs can be changed to DEBUG
  • MAPREDUCE-6172 - TestDbClasses timeouts are too aggressive
  • MAPREDUCE-6571 - JobEndNotification info logs are missing in AM container syslog
  • MAPREDUCE-6639 - Process hangs in LocatedFileStatusFetcher if FileSystem.get throws.
  • MAPREDUCE-6715 - Fix Several Unsafe Practices
  • MAPREDUCE-6817 - The format of job start time in JHS is different from those of submit and finish time.
  • MAPREDUCE-6829 - Add peak memory usage counter for each task.
  • YARN-1047 - Expose # of pre-emptions as a queue counter
  • YARN-2301 - Improved yarn container command
  • YARN-2306 - Add test for leakage of reservation metrics in fair scheduler.
  • YARN-2336 - Fair scheduler's REST API returns a missing '[' bracket JSON for deep queue tree
  • YARN-2902 - Addendum. Killing a container that is localizing can orphan resources in the DOWNLOADING state
  • YARN-2988 - Graph#save() may leak file descriptors.
  • YARN-3269 - Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path
  • YARN-3396 - Handle URISyntaxException in ResourceLocalizationService.
  • YARN-3933 - FairScheduler: Multiple calls to completedContainer are not safe.
  • YARN-3957 - FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500.
  • YARN-4232 - TopCLI console support for HA mode
  • YARN-4363 - In TestFairScheduler, testcase should not create FairScheduler redundantly.
  • YARN-4544 - All the log messages about rolling monitoring interval are shown with WARN level
  • YARN-4555 - TestDefaultContainerExecutor#testContainerLaunchError fails on non-english locale environment
  • YARN-4719 - Add a helper library to maintain node state and allows common queries.
  • YARN-4752 - Improved preemption in FairScheduler.
  • YARN-4805 - Don't go through all schedulers in ParameterizedTestBase.
  • YARN-4882 - Change the log level to DEBUG for recovering completed applications
  • YARN-5136 - Error in handling event type APP_ATTEMPT_REMOVED to the scheduler
  • YARN-5181 - ClusterNodeTracker: add method to get list of nodes matching a specific resourceName.
  • YARN-5182 - MockNodes.newNodes creates one more node per rack than requested.
  • YARN-5257 - Fix unreleased resources and null dereferences
  • YARN-5308 - FairScheduler: Move continuous scheduling related tests to TestContinuousScheduling
  • YARN-5752 - TestLocalResourcesTrackerImpl#testLocalResourceCache times out
  • YARN-5798 - Set UncaughtExceptionHandler for all FairScheduler threads.
  • YARN-5830 - FairScheduler: Avoid preempting AM containers.
  • YARN-5859 - TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails
  • YARN-5890 - FairScheduler should log information about AM-resource-usage and max-AM-share for queues
  • YARN-5920 - Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang
  • YARN-6061 - Add an UncaughtExceptionHandler for critical threads in RM.
  • YARN-6112 - UpdateCallDuration is calculated only when debug logging is enabled.
  • YARN-6125 - The application attempt's diagnostic message should have a maximum size
  • YARN-6131 - FairScheduler: Lower update interval for faster tests.
  • YARN-6144 - FairScheduler: preempted resources can become negative.
  • YARN-6163 - FS Preemption is a trickle for severely starved applications.
  • YARN-6171 - ConcurrentModificationException on FSAppAttempt.containersToPreempt.
  • YARN-6172 - FSLeafQueue demand update needs to be atomic.
  • YARN-6193 - FairScheduler might not trigger preemption when using DRF.
  • YARN-6194 - Cluster capacity in SchedulingPolicy is updated only on allocation file reload.
  • YARN-6210 - FairScheduler: Node reservations can interfere with preemption.
  • YARN-6215 - Custom Cloudera version. FairScheduler preemption and update should not run concurrently.
  • YARN-6218 - Fix TestAMRMClient when using FairScheduler.
  • YARN-6264 - AM not launched when a single vcore is available on the cluster.
  • YARN-6448 - Continuous scheduling thread crashes while sorting nodes.
  • HBASE-6721 - RegionServer Group based Assignment. Adds new rsgroups feature which allows isol
  • HBASE-12949 - Scanner can be stuck in infinite loop if the HFile is corrupted
  • HBASE-15295 - MutateTableAccess.multiMutate() does not get high priority causing a deadlock
  • HBASE-15315 - Remove always set super user call as high priority
  • HBASE-15322 - Operations using Unsafe path broken for platforms not having sun.misc.Unsafe.
  • HBASE-15325 - ResultScanner allowing partial result will miss the rest of the row if the region is moved between two rpc requests
  • HBASE-15328 - sanity check the redirect used to send master info requests to the embedded regionserver.
  • HBASE-15378 - Scanner cannot handle heartbeat message with no results
  • HBASE-15435 - Add WAL (in bytes) written metric
  • HBASE-15441 - Fix WAL splitting when region has moved multiple times
  • HBASE-15587 - FSTableDescriptors.getDescriptor() logs stack trace erronously
  • HBASE-15645 - hbase.rpc.timeout is not used in operations of HTable
  • HBASE-15931 - Add log for long-running tasks in AsyncProcess
  • HBASE-16062 - Improper error handling in WAL Reader/Writer creation
  • HBASE-16165 - Decrease RpcServer.callQueueSize before writeResponse causes OOM
  • HBASE-16237 - Blocks for hbase:meta table are not cached in L1 cache
  • HBASE-16266 - Do not throw ScannerTimeoutException when catch UnknownScannerException
  • HBASE-16304 - HRegion#RegionScannerImpl#handleFileNotFoundException may lead to deadlock when trying to obtain write lock on updatesLock
  • HBASE-16460 - Can't rebuild the BucketAllocator's data structures when BucketCache uses FileIOEngine
  • HBASE-16527 - IOExceptions from DFS client still can cause CatalogJanitor to delete referenced files
  • HBASE-16604 - Scanner retries on IOException can cause the scans to miss data
  • HBASE-16649 - Truncate table with splits preserved can cause both data loss and truncated data appeared again
  • HBASE-16662 - Fix open POODLE vulnerabilities
  • HBASE-16754 - All WALSplitter OutputSinks should process compaction events
  • HBASE-16765 - backportNew SteppingRegionSplitPolicy, avoid too aggressive spread of regions for small tables. Includesand the amendmentAmend: Make SteppingSplitPolicy the default.
  • HBASE-16807 - , RegionServer will fail to report new active Hmaster until HMaster/RegionServer failover.
  • HBASE-16841 - Data loss in MOB files after cloning a snapshot and deleting that snapshot
  • HBASE-16931 - Setting cell's seqId to zero in compaction flow might cause RS down.
  • HBASE-16960 - RegionServer hang when aborting
  • HBASE-16981 - Expand Mob Compaction Partition policy from daily to weekly, monthly
  • HBASE-17020 - keylen in midkey() dont computed correctly
  • HBASE-17023 - Region left unassigned due to AM and SSH each thinking others would do the assignment work
  • HBASE-17044 - Fix merge failed before creating merged region leaves meta inconsistent
  • HBASE-17069 - RegionServer writes invalid META entries for split daughters in some circumstances
  • HBASE-17206 - FSHLog may roll a new writer successfully with unflushed entries
  • HBASE-17238 - Wrong in-memory hbase:meta location causing SSH failure
  • HBASE-17241 - Avoid compacting already compacted mob files with _del files
  • HBASE-17243 - Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
  • HBASE-17243 - Reuse CompactionPartitionId and avoid creating MobFileName in PartitionedMobCompactor to avoid unnecessary new objects
  • HBASE-17275 - Assign timeout may cause region to be unassigned forever
  • HBASE-17309 - Fix connection leaks in TestAcidGuarantees
  • HBASE-17381 - ReplicationSourceWorkerThread can die due to unhandled exceptions
  • HBASE-17409 - Limit jsonp callback name to prevent xss
  • HBASE-17437 - Support specifying a WAL directory outside of the root directory
  • HBASE-17452 - Failed taking snapshot - region Manifest proto-message too large
  • HBASE-17522 - Handle JVM throwing runtime exceptions when we ask for details on heap usage the same as a correctly returned 'undefined'.
  • HBASE-17558 - ZK dumping jsp should escape HTML.
  • HBASE-17561 - table status page should escape values that may contain arbitrary characters.
  • HBASE-17624 - ADDENDUMincluding missing parts of TestWALObserver changes from
  • HBASE-17640 - Unittest error in TestMobCompactor with different timezone
  • HBASE-17653 - HBASE-17624 rsgroup synchronizations will (distributed) deadlock
  • HBASE-17654 - RSGroup refactoring.
  • HBASE-17675 - ReplicationEndpoint should choose new sinks if a SaslException occurs
  • HBASE-17677 - ServerName parsing from directory name should be more robust to errors from guava's HostAndPort
  • HIVE-7723 - Explain plan for complex query with lots of partitions is slow due to in-efficient collection used to find a matching ReadEntity
  • HIVE-9138 - Add some explain to PTF operator
  • HIVE-9481 - allow column list specification in INSERT statement
  • HIVE-10641 - create CRC32 UDF
  • HIVE-11147 - MetaTool doesn't update FS root location for partitions with space in name
  • HIVE-11293 - HiveConnection.setAutoCommit(true) throws exception
  • HIVE-11376 - Removes legacy code wrt skipping files with codec to HiveInputFormat from CombineHiveInputFormat
  • HIVE-11594 - Analyze Table for column names with embedded spaces
  • HIVE-11849 - NPE in HiveHBaseTableShapshotInputFormat in query with just count(*)
  • HIVE-11882 - Fetch optimizer should stop source files traversal once it exceeds the hive.fetch.task.conversion.threshold
  • HIVE-11920 - ADD JAR failing with URL schemes other than file/ivy/hdfs
  • HIVE-12222 - Define port range in property for RPCServer
  • HIVE-12236 - Enable SimpleFetchOptimizer for more query types
  • HIVE-12261 - schematool version info exit status should depend on compatibility, not equality
  • HIVE-12349 - NPE in ORC SARG for IS NULL queries on Timestamp and Date columns
  • HIVE-12465 - Hive might produce wrong results when (outer) joins are merged
  • HIVE-12780 - Fix the output of the history command in Beeline
  • HIVE-12789 - Fix output twice in the history command of Beeline
  • HIVE-12897 - Improve dynamic partition loading
  • HIVE-12976 - MetaStoreDirectSql doesn't batch IN lists in all cases
  • HIVE-13149 - Remove some unnecessary HMS connections from HS2
  • HIVE-13179 - Allow custom HiveConf to be passed to Authentication Providers
  • HIVE-13240 - GroupByOperator: Drop the hash aggregates when closing operator
  • HIVE-13463 - Fix ImportSemanticAnalyzer to allow for different src/dst filesystems
  • HIVE-13510 - Dynamic partitioning doesn’t work when remote metastore is used
  • HIVE-13726 - Improve dynamic partition loading VI
  • HIVE-13864 - Beeline ignores the command that follows a semicolon and comment
  • HIVE-13866 - flatten callstack for directSQL errors
  • HIVE-13895 - HoS start-up overhead in yarn-client mode
  • HIVE-13901 - Hivemetastore add partitions can be slow depending on filesystems
  • HIVE-13902 - [Refactor] Minimize metastore jar dependencies on task nodes
  • HIVE-13904 - Ignore case when retrieving ColumnInfo from RowResolver
  • HIVE-13966 - DbNotificationListener: can loose DDL operation notifications
  • HIVE-14204 - Optimize loading dynamic partitions
  • HIVE-14423 - S3: Fetching partition sizes from FS can be expensive when stats are not available in metastore
  • HIVE-14511 - Improve MSCK for partitioned table to deal with special cases Updated changed q.out
  • HIVE-14693 - Some paritions will be left out when partition number is the multiple of the option hive.msck.repair.batch.size
  • HIVE-14713 - LDAP Authentication Provider should be covered with unit tests
  • HIVE-14764 - Enabling "hive.metastore.metrics.enabled" throws OOM in HiveMetastore
  • HIVE-14803 - S3: Stats gathering for insert queries can be expensive for partitioned dataset
  • HIVE-14819 - FunctionInfo for permanent functions shows TEMPORARY FunctionType
  • HIVE-14820 - RPC server for spark inside HS2 is not getting server address properly
  • HIVE-14873 - Add UDF for extraction of 'day of week'
  • HIVE-14920 - S3: Optimize SimpleFetchOptimizer::checkThreshold()
  • HIVE-15025 - Secure-Socket-Layer (SSL) support for HMS
  • HIVE-15060 - Remove the autoCommit warning from beeline
  • HIVE-15065 - SimpleFetchOptimizer should decide based on metastore stats when available
  • HIVE-15137 - metastore add partitions background thread should use current username
  • HIVE-15207 - Implement a capability to detect incorrect sequence numbers
  • HIVE-15072 - Schematool should recognize missing tables in metastore
  • HIVE-15073 - Schematool should detect malformed URIs
  • HIVE-12261 - Schematool version info exit status should depend on compatibility, not equality
  • HIVE-15074 - Schematool provides a way to detect invalid entries in VERSION table
  • HIVE-15263 - Detect the values for incorrect NULL values
  • HIVE-15391 - Location validation for table should ignore the values for view
  • HIVE-15392 - Refactoring the validate function of HiveSchemaTool to make the output consistent
  • HIVE-15232 - Add notification events for functions and indexes
  • HIVE-15338 - Wrong result from non-vectorized DATEDIFF with scalar parameter of type DATE/TIMESTAMP
  • HIVE-15346 - "values temp table" should not be an input
  • HIVE-15359 - CDH=48042skip.footer.line.count doesnt work properly for certain situations
  • HIVE-15410 - WebHCat supports get/set table property with its name containing period and hyphen
  • HIVE-15430 - Change SchemaTool table validator to test based on the dbType
  • HIVE-15485 - Investigate the DoAs failure in HoS
  • HIVE-15517 - NOT (x <=> y) returns NULL if x or y is NULL
  • HIVE-15530 - Optimize the column stats update logic in table alteration
  • HIVE-15546 - Optimize Utilities.getInputPaths() so each listStatus of a partition is done in parallel
  • HIVE-15550 - fix arglist logging in schematool
  • HIVE-15551 - memory leak in directsql for mysql+bonecp specific initialization
  • HIVE-15572 - Improve the response time for query canceling when it happens during acquiring locks
  • HIVE-15735 - In some cases, view objects inside a view do not have parents.
  • HIVE-15754 - exchange partition is not generating notifications
  • HIVE-15766 - DBNotificationlistener leaks JDOPersistenceManager : Use same PersistenceManager for metadata and notification
  • HIVE-15778 - DROP INDEX (non-existent) throws NPE when using DbNotificationListener
  • HIVE-15803 - msck can hang when nested partitions are present
  • HIVE-15805 - Some minor improvement on the validation tool
  • HIVE-15815 - Allow to pass some Oozie properties to Spark in HoS
  • HIVE-15872 - The PERCENTILE_APPROX UDAF does not work with empty set
  • HIVE-15879 - Fix HiveMetaStoreChecker.checkPartitionDirs method
  • HIVE-15881 - Use hive.exec.input.listing.max.threads variable name instead of mapred.dfsclient.parallelism.max
  • HIVE-15895 - Use HDFS for stats collection temp dir on blob storage
  • HIVE-16014 - HiveMetastoreChecker should use hive.metastore.fshandler.threads instead of hive.mv.files.thread for pool size
  • HIVE-16019 - Query fails when group by/order by on same column with uppercase name
  • HIVE-16086 - Fix HiveMetaStoreChecker.checkPartitionDirsSingleThreaded method
  • HIVE-16176 - Fixing the non-zero exit code issue with HiveSchemaTool
  • HUE-5167 - Improve SASL error reporting in HUE
  • HUE-5482 - [home] Handle multiple home/trash directories by merging them into one
  • HUE-5594 - [editor] The new autocomplete dropdown should include popular group by and order by clauses
  • HUE-5701 - [security] Remove last references to $.cookie and migrate to $.totalStorage from FB
  • IMPALA-1430 - IMPALA-4878,IMPALA-4879: codegen native UDAs
  • IMPALA-2020 - 4915, 4936: Add rounding for decimal casts
  • IMPALA-2020 - IMPALA-4809: Codegen support for DECIMAL_V2
  • IMPALA-2605 - Omit the sort and mini stress tests
  • IMPALA-4055 - Speed up to_date() with custom implementation.
  • IMPALA-4263 - Fix wrong ommission of agg/analytic hash exchanges.
  • IMPALA-4282 - Remove max length check for type strings.
  • IMPALA-4370 - Divide and modulo result types for DECIMAL version V2
  • IMPALA-4449 - Revisit table locking pattern in the catalog
  • IMPALA-4675 - Case-insensitive matching of Parquet fields.
  • IMPALA-4702 - Fix command line help for webserver_private_key_file
  • IMPALA-4705 - IMPALA-4779, IMPALA-4780: Fix some Expr bugs with codegen
  • IMPALA-4725 - Query option to control Parquet array resolution.
  • IMPALA-4729 - Implement REPLACE()
  • IMPALA-4742 - Change "{}".format() to "{0}".format() for Py 2.6
  • IMPALA-4749 - hit DCHECK in sorter with scratch limit
  • IMPALA-4767 - Workaround for HIVE-15653 to preserve table stats.
  • IMPALA-4808 - old hash join can reference invalid memory
  • IMPALA-4809 - Enable support for DECIMAL_V2 in decimal_casting.py
  • IMPALA-4810 - fix incorrect expr-test decimal types
  • IMPALA-4810 - Make DECIMAL expr-test cases table driven
  • IMPALA-4810 - Add DECIMAL_V2 query option
  • IMPALA-4813 - Round on divide and multiply
  • IMPALA-4821 - Update AVG() for DECIMAL_V2
  • IMPALA-4828 - Alter Kudu schema outside Impala may crash on read
  • IMPALA-4854 - Fix incremental stats with complex types.
  • IMPALA-4916 - Fix maintenance of set of item sets in DisjointSet.
  • IMPALA-4929 - Safe concurrent access to IR function call graph
  • IMPALA-4933 - IMPALA-4931: Simplify SSL initialization on startup
  • IMPALA-4934 - Disable Kudu OpenSSL initialization
  • IMPALA-4981 - Re-enable spilling with MT_DOP.
  • IMPALA-4995 - Fix integer overflow in TopNNode::PrepareForOutput
  • IMPALA-4997 - Fix overflows in Sorter::TupleIterator
  • IMPALA-5005 - Don't allow server to send SASL COMPLETE msg out of order
  • IMPALA-5025 - Update binutils to 2.26.1
  • IMPALA-5027 - make udf headers buildable externally
  • OOZIE-2243 - Kill Command does not kill the child job for java action
  • OOZIE-2519 - Oozie HA with SSL info is slightly incorrect
  • OOZIE-2584 - Eliminate Thread.sleep() calls in TestMemoryLocks
  • OOZIE-2659 - TestPauseTransitService is flaky
  • OOZIE-2720 - Test failure - TestCoordMaterializeTriggerService#testMaxMatThrottleNotPicked
  • OOZIE-2742 - Unable to kill applications based on tag
  • OOZIE-2748 - NPE in LauncherMapper.printArgs()
  • OOZIE-2757 - Malformed xml is Spark action doc page
  • OOZIE-2777 - Config-default.xml longer than 64k results in java.io.UTFDataFormatException
  • OOZIE-2786 - Pass Oozie workflow ID and settings to Spark application configuration
  • OOZIE-2787 - Oozie distributes application jar twice making the spark job fail
  • OOZIE-2792 - Hive2 action is not parsing Spark application ID from log file properly when Hive is on Spark
  • OOZIE-2802 - Spark action failure on Spark 2.1.0 due to duplicate sharelibs
  • OOZIE-2803 - Mask passwords when printing out configs/args in MapReduceMain and SparkMain
  • OOZIE-2811 - Add support for filtering out properties from SparkConfigurationService
  • SENTRY-1476 - SentryStore is subject to JDQL injection. Sentry is subject to JDOQL injection
  • SENTRY-1504 - NPE in log4j.properties parsingSentryMain trips over missing log4j property
  • SENTRY-1505 - CommitContext isn't used by anything and should be removed
  • SENTRY-1507 - Sentry should use Datanucleus version of javax.jdo
  • SENTRY-1508 - MetastorePlugin.java does not handle properly initialization failure
  • SENTRY-1515 - Cleanup exception handling in SentryStore
  • SENTRY-1517 - SentryStore should actually use function getMSentryRole to get roles
  • SENTRY-1518 - Add metrics for SentryStore transactions
  • SENTRY-1520 - Provide mechanism for triggering HMS full snapshot
  • SENTRY-1533 - Sentry console metrics reporting interval should be configurable
  • SENTRY-1534 - Oracle supports serializable instead of repeatable-read
  • SENTRY-1557 - getRolesForGroups(),getRoleNamesForGroups() does too many trips to the DB
  • SENTRY-1564 - Improve error detection and reporting in MetastoreCacheInitializer.java
  • SENTRY-1577 - Support "create function using jar" for hive when Sentry is enabled
  • SENTRY-1581 - Provide Log4J metrics reporter
  • SENTRY-1582 - Additional comments to clarify the intent of string manipulation methods in SentryStore.java
  • SENTRY-1594 - TransactionBlock should become generic
  • SENTRY-1599 - CloseablePersistenceManager is no longer needed
  • SENTRY-1609 - DelegateSentryStore is subject to JDQL injection
  • SENTRY-1615 - SentryStore should not allocate empty objects that are immediately returned
  • SENTRY-1625 - PrivilegeOperatePersistence can use QueryParamBuilder Fix [SQL Injection: JDO] in [sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/service/persistent/PrivilegeOperatePersistence.java]
  • SENTRY-1636 - Remove thrift dependency on fb303
  • SOLR-9284 - The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps grow indefinitely.
  • SOLR-9330 - Fix AlreadyClosedException on admin/mbeans?stats=true
  • SOLR-9389 - HDFS Transaction logs stay open for writes which leaks Xceivers.
  • SOLR-9506 - cache IndexFingerprint for each segment
  • SOLR-9699, SOLR-4668 - fix exception from core status in parallel with core reload
  • SOLR-9819 - Upgrade Apache commons-fileupload to 1.3.2, fixing a security vulnerability
  • SOLR-9859 - backport of replication.properties cannot be updated after being written and neither replication.properties nor index.properties are durable after a crash. Don't log error on NoSuchFileException
  • SOLR-9901 - backport of SOLR-9899 Implement move in HdfsDirectoryFactory. SOLR-9899: StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory.
  • SOLR-10031 - Validation of filename params in ReplicationHandler
  • SOLR-10114, SOLR-9941 - Reordered delete-by-query can delete or omit child documents
  • SOLR-10119 - TestReplicationHandler assertion fixes part of
  • SOLR-10121, SOLR-10116 - BlockCache corruption with high concurrency
  • SPARK-4105 - retry the fetch or stage if shuffle block is corrupt
  • SPARK-12241 - [YARN] Improve failure reporting in Yarn client obtainTokenForHBase()
  • SPARK-12523 - [YARN] Support long-running of the Spark On HBase and hive meta store.
  • SPARK-16654 - [CORE] Add UI coverage for Application Level Blacklisting. Automatically Kill Executors and Nodes when they are Blacklisted
  • SPARK-17874 - [CORE] Add SSL port configuration.
  • SPARK-18372 - [SQL][BRANCH-1.6] Staging directory fail to be removed
  • SPARK-18750 - [YARN] Avoid using "mapValues" when allocating containers.
  • SPARK-19033 - [CORE] Add admin acls for history server
  • SPARK-19220 - [UI] Make redirection to HTTPS apply to all URIs.
  • SPARK-19554 - [UI,YARN] Allow SHS URL to be used for tracking in YARN RM.
  • SPARK-19652 - [UI] Do auth checks for REST API access.
  • SPARK-19720 - [CORE] Redact sensitive information from SparkSubmit console
  • SQOOP-2349 - Add command line option for setting transaction isolation levels for metadata queries
  • SQOOP-2896 - Sqoop exec job fails with SQLException Access denied for user
  • SQOOP-2909 - Oracle related ImportTest fails after SQOOP-2737
  • SQOOP-2911 - Fix failing HCatalogExportTest caused by SQOOP-2863
  • SQOOP-2950 - Sqoop trunk has consistent UT failures - need fixing
  • SQOOP-3053 - Create a cmd line argument for sqoop.throwOnError and use it through SqoopOptions
  • SQOOP-3055 - Fixing MySQL tests failing due to ignored test inputs/configuration
  • SQOOP-3057 - Fixing 3rd party Oracle tests failing due to invalid case of column names
  • SQOOP-3066 - Introduce an option + env variable to enable/disable SQOOP-2737 feature
  • SQOOP-3068 - Enhance error (tool.ImportTool: Encountered IOException running import job: java.io.IOException: Expected schema) to suggest workaround
  • SQOOP-3069 - Get OracleExportTest#testUpsertTestExport in line with SQOOP-3066
  • SQOOP-3071 - Fix OracleManager to apply localTimeZone correctly in case of Date objects too
  • SQOOP-3072 - Reenable escaping in ImportTest#testProductWithWhiteSpaceImport for proper execution
  • SQOOP-3081 - use OracleEscapeUtils.escapeIdentifier in OracleUpsertOutputFormat instead of inline appending quotes
  • SQOOP-3124 - Fix ordering in column list query of PostgreSQL connector to reflect the logical order instead of adhoc ordering
  • ZOOKEEPER-1045 - Backport — Support Quorum Peer mutual authentication via SASL
  • ZOOKEEPER-2044 - Backport — Fix CancelledKeyException on ZooKeeper server (zookeeper branch-3.4)

Cloudera Search

Security vulnerability resolved

An attacker can download any file accessible to Solr superuser using the path manipulation vulnerability in Solr (CVE-2017-3163; see Cloudera Security Bulletins more information). This release provides the fix for it. In addition, attackers can leverage this vulnerability to manipulate files unrelated to Solr index, by launching a “man in the middle” attack between individual Solr servers. To avoid this issue, enable TLS and Kerberos authentication.