Issues Fixed in CDH 5.16.x
Issues Fixed in CDH 5.16.1
Apache Tomcat Vulnerability CVE-2018-11784
Fixed a vulnerability in Apache Tomcat where specially-crafted URLs could be used to redirect to any given URI. CVE-2018-11784.
Cloudera Issue: CDH-73885
Cloudera Search restore operation puts shard replicas on same host
Restoring an Apache Solr collection sometimes places all shard replicas on the same host.
Cloudera Issue: CDH-68828
Missing authorization in Apache Impala may allow data injection
A malicious user who is authenticated with Kerberos may have unauthorized access to internal services used by Impala to transfer intermediate data during query execution. If details of a running query (e.g. query ID, query plan) are available, a user can craft some RPC requests with custom software to inject data into a running query or end query execution prematurely, leading to wrong results of the query.
Cloudera Issue: CDH-72373 / TSB-338
Upstream Issues Fixed
- HADOOP-13426 - Improved IPC performance.
- HADOOP-13483 - Fixed an issue where file-create overwrote directories instead of throwing error messages
- HADOOP-15473 - Configured the serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997
- HADOOP-15655 - Enhanced KMS client retry behavior. Previously, the KMS did not retry upon SocketTimeoutException.
- HDFS-8229 - Fixed an issue where the LAZY_PERSIST file gets deleted after NameNode restart.
- HDFS-10240 - Fixed a race between close/recoverLease leads to missing block
- HDFS-12299 - Fixed a race between update pipeline and DN Re-Registration
- HDFS-13051 - Fixed a dead lock during async editlog rolling if the edit queue is full.
- HDFS-13322 - Fixed an issue where the UID persists when switching between ticket caches.
- HDFS-13486 - Fixed an issue where a faulty node can cause a lease leak and NPE on accessing data.
- HDFS-13601 - Optimized ByteString conversions in PBHelper.
- HDFS-13611 - Fixed an issue where text was used as a ConcurrentHashMap key in PBHelperClient.
- HDFS-13813 - Added a check to see if a child inode exists in the global FSDirectory directory when saving (serializing) INodeDirectorySection.
- MAPREDUCE-7053 - Fixed an issue where Timed out tasks can fail to produce thread dump
- YARN-6966 - Fixed an issue where NodeManager metrics may return wrong negative values when NM restart.
- YARN-6967 - Fixed an issue where the limit for diagnostic message size was not honored
- YARN-8436 - Fixed an issue where the ResourceManager can fail while sorting queues if an update comes in
- HBASE-15232 - Handle region location cache mgmt in AsyncProcess for multi()'s
- HBASE-15390 - Unnecessary MetaCache evictions cause elevated number of requests to meta
- HBASE-18891 - Upgrade to netty-all 4.0.50.Final
- HBASE-19924 - hbase rpc throttling does not work for multi() with request count rater.
- HBASE-20493 - Port HBASE-19994 (Create a new class for RPC throttling exception, make it retryable) to branch-1
- HBASE-20651 - Master, prevents hbck or shell command to reassign the split parent region
- HBASE-20723 - Custom hbase.wal.dir results in data loss because we write recovered edits into a different place than where the recovering region server looks for them
- HBASE-20997 - rebuildUserRegions() does not build ReplicaMapping during master switchover
Code Changes Should Not Be Required
The following fixes should not require code changes, but they contain improvements that might enhance your deployment:
- HIVE-6980 - Drop table by using direct sql
- HIVE-10296 - Cast exception observed when hive runs a multi join query on metastore (postgres), since postgres pushes the filter into the join, and ignores the condition before applying cast
- HIVE-12981 - ThriftCLIService uses incompatible getShortName() implementation
- HIVE-15237 - Propagate Spark job failure to Hive
- HIVE-15860 - RemoteSparkJobMonitor may hang when RemoteDriver exits abnormally
- HIVE-16483 - HoS should populate split related configurations to HiveConf
- HIVE-17213 - HoS file merging doesn't work for union all
- HIVE-18031 - Support replication for Alter Database operation
- HIVE-18283 - Better error message and error code for HoS exceptions
- HIVE-18765 - SparkClientImpl swallows exception messages from the RemoteDriver
- HIVE-18916 - SparkClientImpl doesn't error out if spark-submit fails
- HIVE-19259 - Create view on tables having union all fail with 'Table not found'
- HIVE-19310 - Metastore: MetaStoreDirectSql.ensureDbInit has some slow DN calls which might need to be run only in test env
- HIVE-19371 - Add table ownerType to HMS thrift API
- HIVE-19372 - Add table ownerType to JDO/SQL and ObjectStore
- HIVE-19374 - Parse and process ALTER TABLE SET OWNER command syntax
- HIVE-19605 - TAB_COL_STATS table has no index on db/table name
- HIVE-19668 - Over 30% of the heap wasted by duplicate org.antlr.runtime.CommonToken's and duplicate strings
- HIVE-19783 - Retrieve only locations in HiveMetaStore.dropPartitionsAndGetLocations
- HIVE-20183 - Inserting from bucketed table can cause data loss, if the source table contains empty bucket
- HIVE-20345 - Drop database may hang if the tables get deleted from a different call
- HUE-8118 - [core] Fine grain tracking of the memory usage
- HUE-8118 - [core] The duration of the request is always shown even when instrumentation flag is off
- HUE-8128 - [backend] Force debug logging in server logs does not get all debug
- HUE-8162 - [core] Add delete operation to the right document assist
- HUE-8177 - [oozie] Add a config check for /user/hue/oozie/workspaces
- HUE-8377 - [security] Support new Sentry finer grain privileges
- HUE-8377 - [security] Correctly apply the new permissions to the database scope
- HUE-8451 - [notebook] Many "codec can't decode byte" errors on pig execution if browser language=jp
- HUE-8464 - [core] Fix SAML encryption missing key file passphrase
- HUE-8467 - [jobbrowser] Support impala digest auth for queries
- HUE-8475 - [report] Protect against pivot conflicting with nested facets
- HUE-8476 - [frontend] Fix jQuery Hive autocomplete column mapping
- HUE-8487 - [useradmin] Fix Add Sync LDAP user fails when using DN with special character
- HUE-8505 - [core] Close impala session on logout
- HUE-8519 - [jb] Impala API can now directly return json
- HUE-8558 - [jb] Add tracking URL to Spark Jobs and remove url and killUrl
- HUE-8564 - [useradmin] Fix last activity update for jobbrowser/api/jobs requests
- HUE-8564 - [useradmin] Fix last activity update for notebook/api/check_status
- HUE-8571 - [sentry] navigator_api ERROR for PRIVILEGE_HIERARCHY[hierarchy[server][SENTRY_PRIVILEGE_KEY]['action']]
- HUE-8602 - [sentry] Remove ALTER and DROP in the Hive section
- IMPALA-6086 - Require the SELECT privilege on the database for built-in function calls.
- IMPALA-6451 - Fixed the AuthorizationException in CTAS for Kudu tables.
- IMPALA-6479 - DESCRIBE now respects column level privileges and only shows the columns that the user has the privilege to view.
- IMPALA-6571 - Fixed the NullPointerException in SHOW CREATE TABLE for HBase tables.
- IMPALA-6917 - Limit impalad mem-limit to 12GB.
- IMPALA-7225 - REFRESH..PARTITION no longer reset the number of rows in a partition.
- IMPALA-7272 - Fixed the crash in StringMinMaxFilter.
- IMPALA-7360 - Fixed an issue where Avro scanner sometimes skipped blocks when skip marker was on HDFS block boundary.
- IMPALA-7419 - Fixed the NullPointerException in SimplifyConditionalsRule.
- IMPALA-7483 - impalad/catalogd on JVM deadlock now get aborted.
- IMPALA-7520 - Fixed the NullPointerException in SentryProxy.
- KUDU-2260 - Fixed a rare issue where system failure could leave unexpected null bytes at the end of metadata files, causing Kudu to be unable to restart.
- KUDU-2364 - Fixed an issue when a tablet server was wiped and recreated with the same RPC address, ksck listed it twice, both as healthy, even though only one of them was there.
- KUDU-2412 - The kudu-python client can now compile in environments where __int128 is not supported. This was most commonly el6 environments.
- KUDU-2509 - Fixed an issue that might result in a crash of a tablet server in case of a WAL replay error while bootstrapping a tablet.
- KUDU-2580 - Fixed authentication token reacquisition in the C++ client.
- Fixed an issue that caused the kudu CLI tool to unexpectedly exit when the connection to the master or tserver was abruptly closed.
- SENTRY-1272 - Enable ALTERVIEW_RENAME and ALTERVIEW_AS operation in hive binding
- SENTRY-2194 - Upgrade Sentry hadoop-version dependency to 2.7.5
- SENTRY-2210 - AUTHZ_PATH should have index on the foreign key AUTHZ_OBJ_ID
- SENTRY-2214 - Sentry should not allow URI grants to EMPTY or NULL locations
- SENTRY-2219 - Create index AUTHZ_PATH_FK_IDX at table AUTHZ_PATH only when it does not exist for Oracle
- SENTRY-2238 - Explicitly set Database on SentryHivePrivilegeObjectDesc
- SENTRY-2299 - NPE In Sentry HDFS Sync Plugin
- SENTRY-2310 - Sentry is not be able to fetch full update subsequently, when there is HMS restart in the snapshot process.
- SENTRY-2332 - Load hadoop default configuration when starting sentry service
- SENTRY-2333 - Create index AUTHZ_PATH_FK_IDX at table AUTHZ_PATH for Postgres only when it does not exist
- SENTRY-2403 - Incorrect naming in RollingFileWithoutDeleteAppender
- SENTRY-2406 - Make sure inputHierarchy and outputHierarchy have unique values