Issues Fixed in Cloudera Navigator Data Management

The following sections describe the issues fixed in each Cloudera Navigator release.

Issues Fixed in Cloudera Navigator 2.13.2

Purging Deleted Properties shows "Bad Request" error when viewing Hive entity details

After upgrading to Navigator 2.11.0 (Cloudera Manager 5.12.0) and later versions (to Navigator 2.13.0), removing deleted managed metadata properties with the "Purge deleted Properties" command caused a mismatch between the Solr schema and the data stored for entities. (Administration > Managed Metadata).

The result was that the Navigator console shows the error "Bad Request" when displaying details for Hive entities such as databases, tables, or fields.

When this problem occurs, the Navigator Metadata Server log included a message such as the following:

[CDHExecutor-0-CDHUrlClassLoader@62d310e8]: Internal error while linking. java.lang.IllegalArgumentException: java.lang.ClassCastException@2f145bbc          

Cloudera Bug: NAV-5982

Console description improved for Purge cutoff time

The Navigator Purge Settings page indicated that the purge operation would end at the time specified. However, the specified time is used to stop new purge tasks from starting. Currently running purge tasks may continue to run after the specified end time. The text in the console has been changed to properly reflect the fact that purge tasks may continue to run past the time limit.

Cloudera Bug: NAV-5967

The flag for disabling purge was ignored in the UI

The Administration > Purge Settings page could show scheduled purge operations even if the nav.purge.enabled property was set to false. The displayed purge operations would not run.

This issue has been resolved.

Cloudera Bug: NAV-5966

Less Secure Credentials Protection Policy could expose Azure credentials in audit logs

When you use Cloudera Manager to configure the ADLS Connector service using the Less Secure option for the Credentials Protection Policy, it was possible for Hive audit logs to include Microsoft Azure credentials. If you are using Navigator Audit Server, these credentials could appear in audit reports.

This issue is fixed in this release.

Cloudera Bugs: NAV-5861, CDH-56241

Missing Impala audit events for database-level operations

Impala audit events at the database level were incorrectly recorded, making it seem like they are missing from the Navigator audit data.

This issue is fixed in this release.

Cloudera Bug: NAV-5648

Extracting Spark operation produced error message "Solr Manager can be used by only one thread"

When extracting Spark operations, the following error appeared in the Navigator Metadata Server log:

2018-02-20 09:37:27,869 ERROR com.cloudera.nav.pushextractor.spark.SparkPushExtractor [qtp937744315-126]: com.cloudera.nav.pushextractor.spark.SparkPushExtractor Error extracting Spark operation.java.lang.IllegalStateException: Solr Manager can be used by only one thread.

This error does not have consequences for Navigator handling Spark operations.

Cloudera Bug: NAV-4991

Issues Fixed in Cloudera Navigator 2.13.1

There were no issues fixed in Cloudera Navigator 2.13.1.

Issues Fixed in Cloudera Navigator 2.13.0

Multi-cluster support

Navigator was not able to uniquely identify more than one cluster in installations of Cloudera Manager that supported multiple clusters. This problem only occurs for Cloudera Manager deployments of version 5.12.0, 5.12.1 or 5.13.0. This problem does not affect single cluster deployments or Altus clusters.

This issue has been resolved. The upgrade process to this release of Cloudera Manager resolves the problem.

Cloudera Bug: NAV-5615

Table-to-HDFS links not established when Navigator supported multiple, high-availability clusters

When Navigator extracted metadata for multiple clusters and when the clusters were configured for high availability operation, Navigator did not correctly link tables to their HDFS backing data. The result was that lineage between Hive tables and their physical data were not created. In addition, some Hive table metadata that was derived from the backing files was not available.

You could see errors in the log such as the following:

2018-02-19 09:01:32,999 ERROR com.cloudera.nav.persist.impl.CompositeLinker [CDHExecutor-0-CDHUrlClassLoader@01010d7e]: Internal error while linking.java.lang.IllegalArgumentException: expected one element but was: <com.cloudera.nav.core.model.Source@b22e68ba, com.cloudera.nav.core.model.Source@5b1bac2c>

This issue has been resolved.

Cloudera Bug: NAV-5749

Difficult to know how long Navigator upgrade would take

The upgrade process for Navigator now includes messages to show the estimated time remaining at each upgrade step.

Cloudera Bug: NAV-5639

Not all services were included in log messages during purge operation

The log messages produced during a purge operation are now consistent across sources; they all include counts for the number of entities removed.

Cloudera Bug: NAV-3389

Technical metadata for Hive table partitions was cut off

This release provides a "Show more" link in table detail pages when the list of partition columns doesn't fit in the window.

Cloudera Bug: NAV-4840

Audits Dashboard showed incorrect time period

In certain cases the dashboard link for "Accesses Denied" would not open the correct time period on the Audits page.

This issue has been resolved.

Cloudera Bug: NAV-5689

Duplicate values could be added to dynamic facets

Values added to a dynamic facet were not properly validated. This release includes validation so duplicate values cannot be included in a facet.

Cloudera Bug: NAV-5572

Spark lineage not shown in Navigator for HDFS HA

When HDFS is configured for high availability, Spark lineage didn't show in Navigator. In these cases, the log includes index out of bounds exceptions such as the following:

com.cloudera.nav.pushextractor.spark.SparkPushExtractor Error extracting Spark operation.
java.lang.ArrayIndexOutOfBoundsException

This issue has been resolved.

Cloudera Bug: NAV-4874

Issues Fixed in Cloudera Navigator 2.12.3

Purging Deleted Properties shows "Bad Request" error when viewing Hive entity details

After upgrading to Navigator 2.11.0 (Cloudera Manager 5.12.0) and later versions (to Navigator 2.13.0), removing deleted managed metadata properties with the "Purge deleted Properties" command caused a mismatch between the Solr schema and the data stored for entities. (Administration > Managed Metadata).

The result was that the Navigator console shows the error "Bad Request" when displaying details for Hive entities such as databases, tables, or fields.

When this problem occurs, the Navigator Metadata Server log included a message such as the following:

[CDHExecutor-0-CDHUrlClassLoader@62d310e8]: Internal error while linking. java.lang.IllegalArgumentException: java.lang.ClassCastException@2f145bbc          

Cloudera Bug: NAV-5982

Extracting Spark operation produced error message "Solr Manager can be used by only one thread"

When extracting Spark operations, the following error appeared in the Navigator Metadata Server log:

2018-02-20 09:37:27,869 ERROR com.cloudera.nav.pushextractor.spark.SparkPushExtractor [qtp937744315-126]: com.cloudera.nav.pushextractor.spark.SparkPushExtractor Error extracting Spark operation.java.lang.IllegalStateException: Solr Manager can be used by only one thread.
  

This error does not have consequences for Navigator handling Spark operations.

Cloudera Bug: NAV-4991

Issues Fixed in Cloudera Navigator 2.12.2

Missing Impala audit events for database-level operations

Impala audit events at the database level were incorrectly recorded, making it seem like they were missing from the Navigator audit data.

This issue has been resolved.

Cloudera Bug: NAV-5648

Issues Fixed in Cloudera Navigator 2.12.1

Tags disappeared on secure cluster when a change caused HDFS bulk extraction to run a second time.

When Navigator HDFS extraction occurs for the first time, Navigator performs a bulk extraction. Subsequent bulk extractions should refresh only the changed metadata. However, in previous releases, Navigator extraction refreshed all metadata rather than building on the existing metadata. The result was that tags added before the first bulk extraction were missing after the second bulk extraction. This problem only occurs when some change happens on the cluster to trigger the second bulk extraction, such as resetting the HDFS inotify stream in an Isilon-backed cluster.

This issue has been resolved.

Cloudera Bug: NAV-5620

Multi-cluster support

Navigator was not able to uniquely identify more than one cluster in installations of Cloudera Manager that supported multiple clusters. This problem only occurs for Cloudera Manager deployments of version 5.12.0, 5.12.1 or 5.13.0. This problem does not affect single cluster deployments or Altus clusters.

This issue has been resolved. The upgrade process to this release of Cloudera Manager resolves the problem.

Cloudera Bug: NAV-5615

Navigator did not show metadata content for Isilon

Navigator failed to show file system results in a secure cluster backed by a non-HDFS file system, such as EMC Isilon.

This issue has been resolved.

Cloudera Bug: NAV-5375

Incorrect operation of the Search filter for "Deleted Time"

The Deleted Time search filter did not produce expected results for filtering entities.

This issue has been resolved.

Cloudera Bug: NAV-1168

Support for multiple metadata filters with the same name, different namespaces

Multiple managed metadata filters with same name but different namespaces could not be added as filters to search.

This issue has been resolved.

Cloudera Bug: NAV-5429

User-defined relations did not highlight appropriately in lineage diagrams

In Navigator lineage diagrams, when you highlight a path, all relations that aren't in the highlighted path fade. However, in previous releases, user-defined relations in lineage pictures did not fade when they were not in the highlighted path.

This issue has been resolved.

Cloudera Bug: NAV-5574

Spark lineage not shown in Navigator for HDFS HA

When HDFS is configured for high availability, Spark lineage didn't show in Navigator. In these cases, the log includes index out of bounds exceptions such as the following:

com.cloudera.nav.pushextractor.spark.SparkPushExtractor Error extracting Spark operation.
java.lang.ArrayIndexOutOfBoundsException

This issue has been resolved.

Cloudera Bug: NAV-4874

Issues Fixed in Cloudera Navigator 2.12.0

Limitation on editing certain managed metadata properties

Managed metadata properties (without class associations) that were added using the Navigator API could not be edited after the fact through the Cloudera Navigator console.

This issue has been resolved.

Cloudera Bug: NAV-5363

Hue links were missing from Navigator console

The links to Hue from entities in the Navigator console were not generated. Each time the console attempted to show a link, the Navigator Metadata Server log included an error similar to the following:

10:05:32.539 AM
ERROR
HueUtility
[qtp778450792-57]: Unable to get Hue link from CM.
java.lang.LinkageError: loader constraint violation: when resolving method "com.cloudera.nav.server.NavOptions.getPreferredBaseHueURLForCluster(Ljava/lang/String;)Lcom/google/common/base/Optional;" the class loader (instance of org/eclipse/jetty/webapp/WebAppClassLoader) of the current class, com/cloudera/nav/utils/HueUtility, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for resolved class, com/cloudera/nav/server/NavOptions, have different Class objects for the type ter(Ljava/lang/String;)Lcom/google/common/base/Optional; used in the signature{

This issue has been resolved.

Cloudera Bug: NAV-4756

Issues Fixed in Cloudera Navigator 2.11.2

Multi-cluster support

Navigator was not able to uniquely identify more than one cluster in installations of Cloudera Manager that supported multiple clusters. This problem only occurs for Cloudera Manager deployments of version 5.12.0, 5.12.1 or 5.13.0. This problem does not affect single cluster deployments or Altus clusters.

This issue has been resolved. The upgrade process to this release of Cloudera Manager resolves the problem.

Cloudera Bug: NAV-5615

Navigator did not show metadata content for Isilon

Navigator failed to show file system results in a secure cluster backed by a non-HDFS file system, such as EMC Isilon.

This issue has been resolved.

Cloudera Bug: NAV-5375

Spark lineage not shown in Navigator for HDFS HA

When HDFS is configured for high availability, Spark lineage didn't show in Navigator. In these cases, the log includes index out of bounds exceptions such as the following:

com.cloudera.nav.pushextractor.spark.SparkPushExtractor Error extracting Spark operation.
java.lang.ArrayIndexOutOfBoundsException

This issue has been resolved.

Cloudera Bug: NAV-4874

Issues Fixed in Cloudera Navigator 2.11.1

Navigator Audit Server build version not available

Navigator Audit Server and Navigator Metadata Server have different build versions and numbers. Navigator Metadata Server build information has always been available in the system logs and on the About page of the Cloudera Navigator console. Build information may be needed by Cloudera support to troubleshoot issues.

This issue has been resolved. With this release, Navigator Audit Server also records its build information in its log files.

Cloudera Bug: NAV-5167

Hue links were missing from Navigator console

The links to Hue from entities in the Navigator console were not generated. Each time the console attempted to show a link, the Navigator Metadata Server log included an error similar to the following:

10:05:32.539 AM
ERROR
HueUtility
[qtp778450792-57]: Unable to get Hue link from CM.
java.lang.LinkageError: loader constraint violation: when resolving method "com.cloudera.nav.server.NavOptions.getPreferredBaseHueURLForCluster(Ljava/lang/String;)Lcom/google/common/base/Optional;" the class loader (instance of org/eclipse/jetty/webapp/WebAppClassLoader) of the current class, com/cloudera/nav/utils/HueUtility, and the class loader (instance of sun/misc/Launcher$AppClassLoader) for resolved class, com/cloudera/nav/server/NavOptions, have different Class objects for the type ter(Ljava/lang/String;)Lcom/google/common/base/Optional; used in the signature{

This issue has been resolved.

Cloudera Bug: NAV-4756

Issues Fixed in Cloudera Navigator 2.11.0

Navigator Embedded Solr could reach its limit on number of documents it could store

Navigator Metadata Server extracts HDFS entities by performing a one-time bulk extraction and then switching to incremental extraction. In previous releases, a problem caused HDFS bulk extraction to be run more than one time, resulting in duplicate relations created for HDFS. Over time, embedded Solr runs out of document IDs that it can assign to new relations and fails with following error:

Caused by: java.lang.IllegalArgumentException: Too many documents, composite IndexReaders cannot exceed 2147483519

When this happened, Navigator stopped any more extraction of data as no new documents can be added to Solr.

In addition, see Repairing metadata in the storage directory after upgrading.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11.0

Cloudera Bug: NAV-5600

Solr error message (immense term in queryText field) during upgrade

In some cases, Navigator Metadata Server failed to start after upgrading to Cloudera Navigator 2.10 and displayed messages similar to the following (only relevant details shown):
2017-07-01 14:27:27,905 ERROR org.apache.solr.servlet.SolrDispatchFilter
 [qtp300983713-60]: null:org.apache.solr.common.SolrException:
 Exception writing document id 0c02400d332854fa44a352235b42857c to the index;
. . .
Caused by: java.lang.IllegalArgumentException: Document contains at least one
immense term in field="queryText" (whose UTF8 encoding is longer than the max
length 32766)
. . .

This issue was caused by the underlying Java library that was writing Solr documents (index and searchable entities) during the upgrade. Special characters, such as carriage returns (ASCII 32) were being mishandled and generating more characters than the source element.

This issue has been resolved in this release. The BinaryRequestWriter, which was already being used to write relations, is now being used to handle elements.

Cloudera Bug: NAV-4655

Harmless (but abundant) error messages cause frequent log roll-over

An error message was being generated needlessly by Cloudera Navigator when filtering by source. The error could have been ignored but it was being logged and causing logs to roll-over too quickly.

2017-06-23 10:58:32,967 ERROR com.cloudera.nav.hdfs.client.InotifyClient
[CDHExecutor-0-CDHUrlClassLoader@5177874]: Error handling event (txid: 1627074917):
Renamed /hbase/data/default/ECDS_PROD1_PARTY_RELATED/ccefb... to
/hbase/archive/data/default/ECDS_PROD1_PARTY_RELATED/cbdka... at time 1493302572987

This issue has been resolved in this release. Internally, the logging mechanism has been improved so that it uses a more robust approach to logging, the ThrottlingLogger utility class.

Cloudera Bug: NAV-4654

Disk space issue for Navigator Audit Server instances backed by Oracle

Cloudera Navigator deployments using Oracle database as the backing store for Navigator Audit Server were consuming lots of disk space for example, by the NAV_FILE_INFO_STAGING. The issue was caused by an unclosed connection and is related to the connection issue discussed in Navigator Audit Server (Analytics) table locking issue (HDFS activity loader). However, this issue is specific to Oracle-backed Cloudera Navigator instances.

This issue has been resolved in this release. The connections to Oracle database are now handled properly.

Cloudera Bug: NAV-4467

Audit pipeline test and Oracle error (ORA-24816) issue

Executing queries of a certain size (4000 bytes) in some cases (for example, Impala) would disrupt the Audit Pipeline test thereby registering bad health on the Impalad (Impala daemon) and the Navigator Audit Server logging the following Oracle database error:
[1911239784@NavigatorServer-4858]: SQL Error: 24816, SQLState: 99999
[1911239784@NavigatorServer-4858]: ORA-24816: Expanded non LONG bind data
supplied after actual LONG or LOB column...
This issue was caused by bind value length potentially greater than 4000 bytes followed by binding for LOB or LONG.

This issue has been resolved in this release. Navigator Audit Server now properly handles the order of the binds needed by Oracle databases.

Cloudera Bug: NAV-4194

Navigator Audit Server (Analytics) table locking issue (HDFS activity loader)

Executing multiple statements sometimes resulted in a mismatch between the connection used to execute the statement and the connection used for the commit. The mismatch would result in the server throwing an error message such as the following:

[AnalyticsStagingService]: Could not load HDFS activity data for analytics
org.springframework.dao.CannotAcquireLockException: StatementCallback; SQL
[TRUNCATE TABLE NAV_FILE_INFO_STAGING]...

This issue has been resolved in this release. The same connection is now used to execute queries and for commits.

Cloudera Bug: NAV-4026

Invoking auth/group API with invalid role generates NPE and BAD_REQUEST Error

Invoking the auth/group endpoint with an invalid role as one of the user-provided parameters causes the server to throw an NPE (null pointer exception), resulting in response from server of HTTP 500 (Internal Server Error).

This issue has been resolved in this release. The server code now checks for this condition (avoiding NPE) and replies with HTTP 400 (BAD_REQUEST) instead.

Cloudera Bug: NAV-3910

Bulk extraction being triggered can result in orphaned relations

In some cases, HDFS bulk extraction was being triggered due to an error in incremental extraction. Specifically, Cloudera Navigator was resetting the long identity of HDFS elements resulting in orphaned relations. This issue has been resolved. A new background task looks for and repairs all orphaned relations.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11

Cloudera Bug: NAV-4479

Issues Fixed in Cloudera Navigator 2.10.2

SqoopJobExtractor failures for certain SQL constructs

Certain types of SQL queries that generated sub-operations did not retrieve and display lineage for columns, emitting errors in the Cloudera Navigator console and SQLParserException errors in the Navigator Metadata Server log. For example:
...Extractor com.cloudera.nav.sqoop.extractor.SqoopJobExtractor failed for job_1456104915937_0874
java.lang.RuntimeException: com.akiban.sql.parser.SQLParserException: Encountered " "(" "( "" at line1, column 53.
Was expecting one of: "cs" ...
The root cause of this issue is that the open source parser (akiban) cannot parse certain queries that result in sub-operations, specifically those resulting from Sqoop query IMPORT statements. This issue applies to Sqoop1 query IMPORT statements only, not Sqoop EXPORT and TABLE IMPORT statements.

This issue has been resolved in this release. The SQLParserException is caught and the Cloudera Navigator console renders the message shown below rather than breaking the lineage diagram:


Fixed in Cloudera Navigator 2.10.2, 2.9.2

Cloudera Bugs: NAV-4718, NAV-4233, NAV-3142

Issues Fixed in Cloudera Navigator 2.10.1

Navigator Embedded Solr could reach its limit on number of documents it could store

Navigator Metadata Server extracts HDFS entities by performing a one-time bulk extraction and then switching to incremental extraction. In previous releases, a problem caused HDFS bulk extraction to be run more than one time, resulting in duplicate relations created for HDFS. Over time, embedded Solr runs out of document IDs that it can assign to new relations and fails with following error:

Caused by: java.lang.IllegalArgumentException: Too many documents, composite IndexReaders cannot exceed 2147483519

When this happened, Navigator stopped any more extraction of data as no new documents can be added to Solr.

In addition, see Repairing metadata in the storage directory after upgrading.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11.0

Cloudera Bug: NAV-5600

Bulk extraction being triggered can result in orphaned relations

In some cases, HDFS bulk extraction was being triggered due to an error in incremental extraction. Specifically, Cloudera Navigator was resetting the long identity of HDFS elements resulting in orphaned relations. This issue has been resolved. A new background task looks for and repairs all orphaned relations.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11

Cloudera Bug: NAV-4479

Issues Fixed in Cloudera Navigator 2.10.0

Navigator cannot extract S3 metadata with some databases

Using a database other than MySQL, such as PostgreSQL (the default database) for the Navigator Metadata Server generated Java errors in the extracted data for Amazon S3 because of an incorrect string library. The issue has been resolved.

Cloudera Bug: NAV-4342

Metadata added to Amazon S3 files persists in Navigator

Once added to Amazon S3 data and extracted by Navigator, custom metadata (key-value pairs) remain with the data captured by Navigator, even after that metadata may have been deleted from the Amazon S3 files. Navigator continues to show the deleted metadata for that file. This issue has been resolved.

Cloudera Bug: NAV-3928

Managed metadata properties missing on the page

Clicking managed metadata entities in lineage diagrams to display details silently failed, displaying nothing. This issue has been resolved—the condensed details page now displays properly.

Cloudera Bug: NAV-4320

Cluster selection doesn't work properly

In certain cases, cluster facets on the search page did not work correctly.

Cloudera Bug: NAV-4260

Search results retained after deselecting search facet

After searching for a specific type of data using the selector box (the search facet, for example, selecting HDFS), the results populating the browser pane remain despite navigating away from the page or de-selecting the item. This issue has now been resolved. After searching and navigating away from the results or deselecting the data type, resulting display also refreshes, as it should.

Cloudera Bug: NAV-4190

Deleted managed metadata property still visible on Policy page

In certain cases, deleted Managed metadata properties were visible after deletion.

Cloudera Bug: NAV-3778

Issues Fixed in Cloudera Navigator 2.9.2

Navigator Embedded Solr could reach its limit on number of documents it could store

Navigator Metadata Server extracts HDFS entities by performing a one-time bulk extraction and then switching to incremental extraction. In previous releases, a problem caused HDFS bulk extraction to be run more than one time, resulting in duplicate relations created for HDFS. Over time, embedded Solr runs out of document IDs that it can assign to new relations and fails with following error:

Caused by: java.lang.IllegalArgumentException: Too many documents, composite IndexReaders cannot exceed 2147483519

When this happened, Navigator stopped any more extraction of data as no new documents can be added to Solr.

In addition, see Repairing metadata in the storage directory after upgrading.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11.0

Cloudera Bug: NAV-5600

Bulk extraction being triggered can result in orphaned relations

In some cases, HDFS bulk extraction was being triggered due to an error in incremental extraction. Specifically, Cloudera Navigator was resetting the long identity of HDFS elements resulting in orphaned relations. This issue has been resolved. A new background task looks for and repairs all orphaned relations.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11

Cloudera Bug: NAV-4479

Sqoop job extraction failures

Sqoop job extraction was failing because Sqoop table IMPORT operations were not being retrieved from Solr. This issue has been resolved in this release. Sqoop job extraction is now being processed as expected.

Fixed in Cloudera Navigator 2.9.1, 2.9.2

Cloudera Bug: NAV-4225

Issues Fixed in Cloudera Navigator 2.9.1

Sqoop job extraction failures

Sqoop job extraction was failing because Sqoop table IMPORT operations were not being retrieved from Solr. This issue has been resolved in this release. Sqoop job extraction is now being processed as expected.

Fixed in Cloudera Navigator 2.9.1, 2.9.2

Cloudera Bug: NAV-4225

Viewing deleted properties in managed metadata shows incorrect search results

Managed metadata property that was deleted drilled down to incorrect search results. The results are now displayed correctly.

Cloudera Bug: NAV-4116

Edit Managed Metadata link available for users with insufficient privilege

The Edit Managed Metadata link was available to users without sufficient privilege levels to view it (the link requires AUTH_WRITE_MANAGED_METADATA). The link is now available only to users with the correct privileges.

Cloudera Bug: NAV-3985

Some properties that are URLs in the user interface do not function as links

Previously, managed metadata property values and S3 property values that are URLs did not function as links to the URL. These properties are now formatted as clickable links.

Cloudera Bug: NAV-3960

Purge navigation and messages could be confusing

When purging deleted properties in Navigator, the maintenance page message was confusing in some situations, and the Continue link after purge returned you to the general Search page. The purge messages have been clarified and now, after successful purge, the page on which you started redisplays rather than the general Search page.

Cloudera Bug: NAV-3940

Dashboard average values difficult to read

Previously, the dashboard displayed average values in a long format that could be difficult to read. These values have been converted to a format that is more human-readable. For example, the value 33520.2 now reads 33.5k.

Cloudera Bug: NAV-3930

Audit page filter breadcrumbs missing spaces

There were no spaces in the filter breadcrumbs on the Audit page, making the filter difficult to read. Spaces have been added to improve readability.

Cloudera Bug: NAV-3876

Searching for text in the Name filter shows HTML bold tag

When using the Original Name filter in search, characters that you typed in the name box appeared with HTML bold tags in the drop-down results. The extraneous tags no longer appear.

Cloudera Bug: NAV-3765

Directory contents do not distinguish deleted entities for Amazon S3 entities

In Navigator, search results indicate if an entity has been deleted. However, the Directory Contents information did not indicate that the entity had been deleted; it was indistinguishable from items in Directory Contents that had not been deleted. Directory Contents information now reflects S3 entity deletions.

Cloudera Bug: NAV-3786

Bucket name not included in S3 search results list

In S3 search results, the S3 bucket name was not included with the name of the entity. The S3 bucket name is now displayed.

Cloudera Bug: NAV-3786

Issues Fixed in Cloudera Navigator 2.9.0

Incomplete query strings in Navigator metadata

Storing more than 32K characters in Solr causes an exception, and Navigator previously handled this by truncating any field longer than 32K to prevent the exception.

Now, fields longer than 32K are preserved and no longer truncated.

Cloudera Bugs: NAV-3548, NAV-3384

Lineage options and search items are missing

In the Safari browser, lineage options and lineage search could be empty in certain cases.

Cloudera Bug: NAV-3401

Out-of-date information when using Internet Explorer 11

In Internet Explorer 11, cached headers could cause out-of-date information to display.

Issues Fixed in Cloudera Navigator 2.8.3

Audit Event reports exported as CSV randomly contained quote character

When exporting an Audit Events report using the CSV format, a double-quotation mark (") was sometimes being inserted into the query used to collect audit events. For example, in the audit event shown below, the quote turns _probable into _pro"bable and _beta into _bet"a:
"2016-12-01T12:50:59.966Z",REDACTED,REDACTED,hive,QUERY,
"beta_vat_mart_delta:ds_dup_vrn_pro"bable",true,,,,,,,,,"
INSERT INTO exp_vat_oob_load_bet"a.temp_ds_dup_vrn_probable_delta_subset

The issue arose when exporting Audit Event reports as CSV format only (not JSON). The issue occurred when the string length of SQL query exceeded 35 characters.

The issue's root cause was an error in processing by the underlying Jackson JSON processor library (jackson-dataformat-csv 2.1.0) used to write the CSV files. The 2.2 version of the library fixes the underlying issue. The issue has been resolved in this release of Cloudera Navigator, which uses the 2.2 Jackson processor library.

Cloudera Bug: NAV-4114

Issues Fixed in Cloudera Navigator 2.8.1

Some APIs return incorrect error

In some cases, when a user performs an action with managed metadata that is not permitted because of the user role, Navigator issues an Access Denied error message (500) instead of the expected Forbidden error (403).

Cloudera Bug: NAV-3463

Resource path filtering does not working in audit event filter

Lineage options and search items are missing

In the Safari browser, lineage options and lineage search could be empty in certain cases.

Cloudera Bug: NAV-3401

Navigator logout link does not work when SAML is enabled

When SAML is enabled, clicking on the Logout link in Navigator executes only a local browser logout, without terminating the SAML session. As a result, after logging out, a user can log in again from the login page without providing credentials.

Issues Fixed in Cloudera Navigator 2.8.0

Last access time displayed for Hive tables is incorrect

Navigator no longer displays incorrect access time information for Hive.

Cloudera Bug: NAV-99

Deleted items not shown by default when Deleted filter added

Showing deleted items when the Deleted filter was not enabled by default. Deleted items are now shown by default when the Deleted filter is applied.

Cloudera Bug: NAV-441

No message shown when no Groups with Navigator roles are found

A message is now displayed in Navigator when no Groups with Navigator roles are found.

Cloudera Bug: NAV-836

Hive started and ended time fields not shown in results

Previously, started and ended time fields were not displayed when selected. These fields are now displayed in results.

Cloudera Bug: NAV-1148

Maximum Complexity error message value incorrect

The value in the Maximum Complexity Exceeded error message has been corrected (>3000).

Cloudera Bug: NAV-1246

List of properties in audit filter not sorted

Audit filter properties are now sorted alphabetically.

Cloudera Bug: NAV-1414

Unable to exit Lineage Full Screen mode from Search Results page

Cloudera Bug: NAV-1514

Audit results do not display seconds or fractions seconds

Audit result display granularity has increased to show seconds and fractions of seconds in results.

Cloudera Bug: NAV-1679

Search on lineage page does not work for children

Lineage children are now shown when selected in a lineage search.

Cloudera Bug: NAV-2628

Create property UI does not use class display names

The create property user interface now displays the correct class display name instead of the abbreviated classname.

Cloudera Bug: NAV-2685

Regex entered cannot be confirmed as correct in managed metadata

Navigator now includes regular expression test validation to managed metadata creation.

Cloudera Bug: NAV-2811

Create namespace does not check against reserved names

The Navigator UI now rejects a reserved namespace when creating a new namespace.

Cloudera Bug: NAV-3193

Issues Fixed in Cloudera Navigator 2.7.3

Incomplete query strings in Navigator metadata

Storing more than 32K characters in Solr causes an exception. Navigator now truncates any field longer than 32K to prevent the exception.

Cloudera Bugs: NAV-3548, NAV-3384

Issues Fixed in Cloudera Navigator 2.7.2

Long query text alters the display of the search box

Very long values in search strings did not display properly. The issue is fixed in this release.

Cloudera Bug: NAV-3166

Pop-up error in UI in clicking on hive table entity's schema entity

Cloudera Bug: NAV-3579

Lineage Options and Search items are missing

In the Safari browser, lineage options and lineage search could be empty in certain cases.

Cloudera Bug: NAV-3401

Google Analytics didn't load when Navigator is served over HTTPS/SSL

After upgrading to Navigator 2.6.1, Navigator Metadata Server logs included reference to failed loading of Google Analytics. The issue is resolved in this release.

Cloudera Bug: NAV-3200

Issues Fixed in Cloudera Navigator 2.7.1

Oozie and Yarn metadata is not visible for some jobs

The time window for polling for new jobs didn't account for some lag in the system, leaving out some events. The issue is fixed in this release.

Cloudera Bug: NAV-3268

Issues Fixed in Cloudera Navigator 2.7.0

Linker causing high memory consumption

Due to an incorrect internal setting, all results were being returned again in second batch requests to Search (Solr), resulting in high memory usage. The parameter autoAdjustBatchSize for the AbstractSolrManager.query setting is now correctly set (to false rather than true) so that subsequent batch requests after an initial request batch the proper number of requests.

Cloudera Bug: NAV-3223

Infinite loop during upgrade possible due to deleted entity query

The CheckDeletedEntity::start method was using the *:* query to get entities to mark as deleted. This adversely affected performance and made it possible to cause an upgrade to enter an infinite loop if the server crashed or was restarted during an upgrade. This issue has been resolved in this release.

Cloudera Bug: NAV-3111

Managed metadata does not display in lineage diagram

Lineage diagrams display custom metadata, but not managed metadata. This issue has been resolved in this release.

Cloudera Bug: NAV-2919

User-specified relations not properly marked in lineage

User-defined relations were not displaying correctly in the lineage diagrams. This issue has been resolved in this release.

Cloudera Bug: NAV-3137

Exporting lineage endpoints without direction causes errors

Attempting to export lineage endpoints without providing direction (upstream, downstream) in the query returns a server error (error 500) and adds a null pointer exception (NPE) to the log. This issue has been resolved in this release. Empty direction parameter is prevented from being submitted.

Cloudera Bug: NAV-3155

Placement discrepancy in Cloudera Navigator console for Policies

The Policies Command Action section displays above Metadata Assignments for View but display below Metadata Assignments for Edit. This issue has been resolved in this release.

Cloudera Bug: NAV-3133

Delete VIEW to FILE relations

As part of metadata purge tasks, Navigator failed to delete VIEW to FILE relations. The issue is resolved in this release.

Cloudera Bug: NAV-3106

Logical physical relationships from Hive to YARN should be removed

Upgrading left lineage relationships that incorrectly link Hive entity sources to YARN operation targets. The issue is fixed in this release.

Cloudera Bug: NAV-3105

popover2 binding is positioned incorrectly

The content of the Additional Information pop-over displayed incorrectly in the console pages for creating a new metadata policy. The issue is fixed in this release.

Cloudera Bug: NAV-3071

Show parent when applicable on entity details page

Entities referenced in other entity details pages are now shown with navigation links.

Cloudera Bug: NAV-3034

Entity cannot be deselected in Lineage

Click on a clear section of the lineage canvas to deselect any selected entities or links. Panning or zooming does not deselect the current object.

Cloudera Bug: NAV-3033

Increase zoom out capability in lineage

Controls were added to allow you to zoom further out in the lineage view.

Cloudera Bug: NAV-3032

API/interactive/entities fails when managed metadata properties exist but did not yet have values

Although this problem was not visible in the console, it resulted in a mismatch between what showed in the Navigator console and Navigator stored metadata.

Cloudera Bug: NAV-3012

Entity Details page edit metadata did not show namespace in managed metadata

The namespace name is now available when you hover over the name of the managed metadata property.

Cloudera Bug: NAV-2886

Actions menu should not show Move for deleted entities

The Move menu option is no longer available for deleted entities.

Cloudera Bug: NAV-2727

No Operation/OperationExecution extracted for Hive and HDFS

The previous release introduced a change that caused Navigator communication with services to fail. The issue is fixed in this release.

Cloudera Bug: NAV-2670

numPartitions attribute for Hive should be suppressed

Hive does not produce consistent data for the tp_numPartitions property so Navigator no longer shows that value.

Cloudera Bug: NAV-2548

Inconsistent size and block size units

The Search and Analytics pages in the Navigator console used different units for displaying the size of files, tables, and blocks. The issue is fixed in this release.

Cloudera Bug: NAV-2492

Search drop-down width should be the same as the search input field width

The search drop-down now shows more of the previous queries for selection.

Cloudera Bug: NAV-2208

Heap for Navigator Metadata Server increased

The default heap value for Navigator Metadata Server is increased from 696 MB to 1.5 GB.

Cloudera Bug: NAV-2020

Improve labels for Analytics top users and top commands filter

The titles for filters were updated to make them more readable.

Cloudera Bug: NAV-1917

Cross-site Request Forgery (CSRF) vulnerability

Web client requests to Cloudera Navigator were not protected by default from cross-site request forgery attempts. Additional configuration was needed to protect the cluster. That issue has been resolved in this release. CSRF security is now enabled by default.

Cloudera Bug: NAV-1870

Policy earliest start time should be current time

The Navigator console was updated to allow users to set the current time in a policy start time.

Cloudera Bug: NAV-1733

Policy editor line wrapping divides words in the middle

The formatting was changed for in the policy creation interface.

Cloudera Bug: NAV-1702

Date format is different for facets, breadcrumbs, and full query text

Dates and times are now shown consistently using the current locale.

Cloudera Bug: NAV-1084

Improve rendering for parent elements

This release includes changes in how lineage relationships are shown to improve readability.

Cloudera Bug: NAV-274

"created" date is later than "lastAccessed" date

This release uses the creation time to be the earlier of modified or accessed date when extracting new files.

Cloudera Bug: NAV-39

Added support for managed metadata in the Policy Editor

This release supports policies for managed metadata as well as custom metadata.

Cloudera Bug: NAV-2865

Navigator Metadata Server does not correctly index /hbase/ subpaths

Navigator disabled by default the metadata indexing of /hbase/WALs and /hbase/oldWALs and any entry under /hbase/* entirely.

Previously, /hbase/WALs and /hbase/oldWALs rules were not effective because they were masked by the rule /.*, which was matching everything under /hbase. This has been corrected. Now only nested hidden files and directories under /hbase are discarded.

Cloudera Bug: NAV-3096

Issues Fixed in Cloudera Navigator 2.6.5

Out of memory during file purge

Navigator runs out of memory during purge of files.

Cloudera Bug: NAV-2751

Hive parser issues errors

The Hive parser issued errors during a query in the following cases:
  • Multiple struct attributes in a query
  • Constants are projected as part of a subquery
  • Functions involving constants with an alias

Cloudera Bug: NAV-3402

Incomplete query strings in Navigator metadata

Storing more than 32K characters in Solr causes an exception. Navigator now truncates any field longer than 32K to prevent the exception.

Cloudera Bugs: NAV-3548, NAV-3384

Issues Fixed in Cloudera Navigator 2.6.2

Linker causes high memory consumption

When calculating lineage relations, Navigator was pulling more metadata into memory than required. The issue is resolved in this release.

Cloudera Bug: NAV-3223

Navigator Metadata Server fails with nav.batch and a high amount of heap

Cloudera Bug: NAV-3272

java.lang.IllegalArgumentException error causes Navigator to crash

Cloudera Bug: NAV-3244

Class not visible on Managed Metadata review screen

The fields Class and Additional class are not visible on the Review screen when creating or updating a Managed Metadata property.

Cloudera Bug: NAV-3123

Google Analytics didn't load when Navigator is served over HTTPS/SSL

After upgrading to Navigator 2.6.1, Navigator Metadata Server logs included reference to failed loading of Google Analytics. The issue is resolved in this release.

Cloudera Bug: NAV-3200

Column type for HBase qualifier value in Navigator Audit database should be LONGBLOB

The following errors appeared in the Navigator Audit Server logs when HBase audits were propagated to Navigator Audit Server MySQL Database:

2016-04-02 03:04:20,783 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper: Data truncation: Data too long for column 'QUALIFIER' at row 1
2016-04-02 03:04:20,808 WARN com.cloudera.navigator.NavigatorServer: Error persisting events for service hbase2

The type-conversion issue is resolved in this release.

Cloudera Bug: NAV-3147

Issues Fixed in Cloudera Navigator 2.6.1

Lineage returned deleted entities even if the Hide deleted entities option is selected

The issue is fixed in this release.

Cloudera Bug: NAV-2945

Browser sessions could be unexpectedly logged out

When multiple browser tabs are open to the same Metadata Server, an inactive tab can log you out even though you are active in another tab.

Cloudera Bug: NAV-2757

Navigator Metadata Server does not correctly index /hbase/ subpaths

Navigator disabled by default the metadata indexing of /hbase/WALs and /hbase/oldWALs and any entry under /hbase/* entirely.

Previously, /hbase/WALs and /hbase/oldWALs rules were not effective because they were masked by the rule /.*, which was matching everything under /hbase. This has been corrected. Now only nested hidden files and directories under /hbase are discarded.

Cloudera Bug: NAV-3096

Issues with additional Group facet values in the Search page

If a new value is added to the subfacet SourceType:HDFS->Group, the 'Clear all' actions selects all the values in the Group facet, although they are not included in the query.

The input field expands outside the facet panel.

Adding a new (not suggested) value to the Group facet in Firefox leaves the suggestion drop-down open.

Cloudera Bug: NAV-3017

Logical Physical relationships unexpectedly shown in lineage

HDFS directories are displayed in Hive lineage.

Cloudera Bug: NAV-2984

Lineage rendering sometimes throws NPE when latest operation is selected

Missing start time values in the metadata used to generate lineage relationships causes errors in the Navigator console lineage view. The issue is fixed in this release.

Cloudera Bug: NAV-2969

ClassCastException thrown when a custom entity is encountered

The Navigator API allowed custom entities to be created without some required metadata; the Navigator console produced errors when attempting to display such an entity.

Cloudera Bug: NAV-2946

Mismatch in number of files between Search and Search from Analytics due to deleted files

Search queries invoked from the Analytics page now show the correct number of files.

Cloudera Bug: NAV-2854

On load, the originating entity might not be visible in lineage

If the lineage graph is tall or wide the originating entity might not be initially visible.

Cloudera Bug: NAV-2798

Browser support warning was not working

Internet Explorer 11 was not in the list of supported browsers.

Cloudera Bug: NAV-2780

Lineage could fail or display partial results when related to Workflows entities

Oozie workflows can have multiple parents however lineage cannot display this use case. Workflows are no longer returned for an operation. To get workflow detail, you must launch lineage on the workflow itself.

Cloudera Bug: NAV-2775

Metadata UI could be unreliable when viewing entities with no name

If an entity has no value in the name or originalName properties, they are now set to "(no name)".

Cloudera Bug: NAV-2746

User logging would in some cases be presented with the login screen again

Navigator console logout timing didn't match the server's timing and caused this loop of logout screens. The issue is fixed in this release.

Cloudera Bug: NAV-2731

Lineage load error banner could show on subsequent lineages

Navigator displays two warnings when lineage load fails. The yellow banner displays until dismissed or the page is reloaded even if it does not apply to the current lineage.

Cloudera Bug: NAV-2697

Missing managed metadata property details

Regular expressions and other details are not listed in the detailed managed metadata property view.

Cloudera Bug: NAV-2663

Alignment of logical physical relations does not follow data flow

Logical physical relations should be shown such that they follow data flow. For example, if downstream you hit a logical entity first then it should be logical -> physical. If you hit physical first it should be physical -> logical.

Cloudera Bug: NAV-2639

Input validation could in certain cases show errors after the error was fixed

Native input validation messages on properties in the Edit Metadata dialog keep displaying, for example when working with regular expression patterns.

Cloudera Bug: NAV-2523

Combo box display issues in Internet Explorer

The combo box scroll bar in search facets does not work in Internet Explorer. Clicking it closes the drop-down.

Cloudera Bug: NAV-2522

Hue link missing

When the Cluster Name facet is used, the Hue link is missing in search results.

Cloudera Bug: NAV-1927

Issues Fixed in Cloudera Navigator 2.6.0

Lineage performance suffers when more than 10000 relations are extracted

If more than 10000 relations must be traversed for a lineage diagram, performance suffers. This can occur when there are thousands of files in a directory or hundreds of columns in a table.

Cloudera Bug: NAV-390

In 2.3, the Navigator Search UI is missing the User Defined Property filter

The Navigator Search UI has changed between 2.2 and 2.3 with respect to the User Defined Property filter. In 2.3 it is no longer present.

Workaround: In the Search box type: up_propertyName:value. For example, up_myName:myValue.

Cloudera Bug: NAV-1691

Displaying the lineage of an entity may consume too much memory and the Metadata Server may run out of memory

Memory leaks have been fixed. However, the Navigator Metadata Server embeds Solr server inside its process, and its memory requirement is proportional to the data that is being indexed. For recommendations on how to configure memory, see Memory Sizing Considerations for Navigator Metadata Server.

Cloudera Bug: NAV-1832

After upgrade to 2.4.x, only entries related to deleted files are visible

In upgrading, Navigator failed to properly convert existing Hive entities to the new system. The problem involved how Hive query contents were used for creating entity names. The issue is fixed in this release.

Cloudera Bug: NAV-2542

Issues Fixed in Cloudera Navigator 2.5.0

After upgrade to 2.4.x, only entries related to deleted files are visible

In upgrading, Navigator failed to properly convert existing Hive entities to the new system. The problem involved how Hive query contents were used for creating entity names. The issue is fixed in this release.

Cloudera Bug: NAV-2542

Issues Fixed in Cloudera Navigator 2.4.4

Lineage does not render on MacOS using Chrome 48.0.2564.109

This change anticipates Google's deprecation and removal of the API that Navigator used to render lineage in Chrome browsers.

Cloudera Bug: NAV-2459

After upgrade to 2.4.x, only entries related to deleted files are visible

In upgrading, Navigator failed to properly convert existing Hive entities to the new system. The problem involved how Hive query contents were used for creating entity names. The issue is fixed in this release.

Cloudera Bug: NAV-2542

Issues Fixed in Cloudera Navigator 2.4.2

Searching for strings that start with forward slash ignored sometimes

Searching for a string that starts with a forward slash (/) in conjunction with another filter/facet or followed by AND and another search term will result in the string being ignored completely.

Workaround: Enclose the string starting with forward slash in double quotes, for example "/some/path" AND sourceType:hdfs

Cloudera Bug: NAV-2022

After upgrade, entities are incorrectly marked as deleted

The issue is resolved. However, upgrading from Cloudera Navigator 2.4.0 or 2.4.1 to 2.4.2 requires manual steps after the upgrade, in addition to the process detailed in the upgrade documentation. Specifically, the HDFS extractor state file (on the host system for each HDFS role instance that Cloudera Navigator extracts from) must be modified as detailed below:
  1. Edit the HDFS extractor state file under the Cloudera Navigator storage directory. The default path is:
    /var/lib/cloudera-scm-navigator/extractorState
    The HDFS extractor state file contains a JSON object with an attribute upgrades, which has a list of upgrades and their statuses. For example:
     {"nextTxId":11394669,"lastKnownTransactionTime":1453931555151,"layoutVersion":-60,
    "namespaceId":1613692955,"nextExtractionRunId":"df61cf41183dec8947078c228ec505e6##7453",
    "upgrades":[
    {"taskType":"SET_MISSING_ATTRIBUTES","status":"SUCCEEDED","startTime":1453484370390,"endTime":1453484370410,"attempts":1},
    {"taskType":"UNDELETE","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370418,"attempts":1},
    {"taskType":"BLOCK_SIZE","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370427,"attempts":1},
    {"taskType":"REPLICATION_COUNT","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370434,"attempts":1}]}
  2. Find entries in the upgrades list for "taskType":"UNDELETE", "status":"SUCCEEDED" and change status to failed. For example, this entry:
    {"taskType":"UNDELETE","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370418,"attempts":1}
    should be changed to:
    {"taskType":"UNDELETE","status":"FAILED","startTime":1453484370391,"endTime":1453484370418,"attempts":1} 
  3. Repeat this process for each HDFS extractor state file as needed for your Cloudera Navigator instance.

Cloudera Bug: NAV-2462

On a newly installed cluster, Hive lineage does not work

When Cloudera Manager starts, it does not automatically create the directory /var/log/hive/lineage. Because the Agent cannot detect this directory, it fails to set a watch on it. HiveServer2 creates the directory, but Hive metadata extracted to the directory is never read by Agent and sent to Navigator Metadata Server.

Workaround: Do one of the following:
  • Use the Cloudera Manager Admin Console to restart HiveServer2 (results in downtime).
  • To avoid HiveServer2 downtime, restart the Cloudera Manager Agent on the host running HiveServer2 role instance as follows:
    sudo service cloudera-scm-agent restart

Cloudera Bug: NAV-2111

Entity landing page schema overflow hides name

If the schema type is very long, it can hide the name of the schema entity.

Cloudera Bug: NAV-2083

When Navigator uses the Cloudera Manager authenticator and Cloudera Manager is TLS enabled, authentication fails

When Navigator is configured to use the Cloudera Manager DB for authentication, and Cloudera Manager is TLS enabled, when a user attempts authentication, certificate validation fails with:
Caused by: sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid
 certification path to requested target

Cloudera Bug: NAV-2178

Analytics page of Cloudera Navigator console includes deleted files

The Cloudera Navigator console Analytics page was including deleted files in its counts. This issue has been resolved.

Cloudera Bug: NAV-2182

Policies created from the Analytics page were not working

Policies created from analytics UI were not working because it filtered on clusterName. Now it filters on sourceId.

Cloudera Bug: NAV-2183

No indication when Entity Details page is not completely loaded

Technical metadata displays immediately but it was not always obvious that loading Entity Details page was still loading. A loading spinner now displays on slow loads.

Cloudera Bug: NAV-2286

Tables might not load on Entity Details page

Tables might not load, depending on whether the schema finished loading before or after tables finish loading.

Cloudera Bug: NAV-2288

Issues Fixed in Cloudera Navigator 2.4.1

Potential Security Vulnerability Resolved

This release resolves a potential security vulnerability that may enable an attacker to execute arbitrary code from a remote machine without requiring authentication. Upgrading to the this release resolves the issue. See the Cloudera Security Bulletin, Apache Commons Collections deserialization vulnerability, for complete details.

Cloudera Bug: NAV-2230

Issues Fixed in Cloudera Navigator 2.4.0

After HDFS upgrade, some changes to HDFS entities may not appear in Navigator

After an HDFS upgrade, Navigator might not detect changes to HDFS entities, such as move, rename, and delete operations, that were recorded only in the HDFS edit logs before the upgrade. This may cause an inconsistent view of HDFS entities between Navigator and HDFS.

Workaround: None.

Cloudera Bug: NAV-730

After upgrading TLS/SSL-enabled Navigator to 5.4.0, Navigator starts in non-TLS/SSL

After upgrading an TLS/SSL-enabled Navigator to 5.4.x, Navigator will start in non-TLS/SSL mode. Before the 5.4.x release of Navigator, TLS/SSL for Navigator was turned on using an advanced configuration snippet. In release 5.4.x, this option is exposed in the Cloudera Manager configuration UI. After upgrade to 5.4.x, this setting overrides the setting in the advanced configuration snippet, disabling TLS/SSL.

Workaround: Remove the setting from the advanced configuration snippet and specify it in the configuration UI directly.

Cloudera Bug: NAV-1536

Navigator login using Cloudera Manager credentials does not work if Cloudera Manager has TLS enabled

With Navigator configured to use Cloudera Manager for authentication and Cloudera Manager using TLS, when a user attempts authentication, certificate validation fails with:

Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The issue is fixed in this release.

Cloudera Bug: NAV-2207

Memory leak when creating Impala lineage for very big queries (more than 32K characters)

Cloudera Bug: NAV-1832

Disabling auditing in Hive causes exception in HiveServer2

If you disable auditing for Hive, but do not restart the Hive service, HiveServer2 logs the following exception:
Hive Internal Error: com.cloudera.navigator.shaded.avro.AvroRuntimeException(Field serviceName type:STRING pos:0 does not accept null values)

Cloudera Bug: NAV-1316

Lineage is not collected for Hive on Spark jobs

Cloudera Bug: NAV-1309

Handle HDFS root as special case when generating Hue link

Cloudera Bug: NAV-1817

Entity description does not allow multiple lines

Cloudera Bug: NAV-422

Expanding entity causes unintuitive lineage layout

Cloudera Bug: NAV-1438

Selected Pig field not shown in lineage

When you view lineage for a Hive field, the Hive parent is expanded so you see the field. However, if there is a field of a Pig table inside a Pig operation execution, only the operation execution is expanded, not the table. The field is not visible and it is not obvious to which entity it belongs.

Cloudera Bug: NAV-1440

Pig inside Oozie not shown in lineage view

Cloudera Bug: NAV-1662

Page headers missing after selecting "Show Full Page" in lineage diagram

Cloudera Bug: NAV-1681

Facet does not visually retain selection after refresh

After reloading a page, some selections are not rendered as selected even though the search string itself is still correct.

Cloudera Bug: NAV-1913

Link and parent Hive table not displayed

If you launch a job with two input directories that are also both Hive tables, the Hive table is shown for one of the directories but not the other. If you expand the other directory, the Hive table is shown.

Cloudera Bug: NAV-1794

Links disappear when expanding Pig job run by Oozie

Cloudera Bug: NAV-1819

Expanding a directory folder obscures the Hive table it is linked to

Cloudera Bug: NAV-1908

Source filter missing from filter list

Cloudera Bug: NAV-1159

Subfacets should be enabled and disabled based on values in facet type

Cloudera Bug: NAV-1051

Tags and key-value pairs should be sorted in the policy editor

Cloudera Bug: NAV-1760

Create Policy link should not be available for a policy viewer role

In this release, the Create Policy button does not appear for users who do not have a role that provides permission to create or update policies.

Cloudera Bug: NAV-1972

Impala auditing does not support filtering during event capture

Impala auditing does not support filtering during event capture.

Severity: Low

Workaround: None.

Cloudera Bug: OPSAPS-15372

Issues Fixed in Cloudera Navigator 2.3.9

Apache Commons Collections Deserialization Vulnerability

Cloudera has learned of a potential security vulnerability in a third-party library called the Apache Commons Collections. This library is used in products distributed and supported by Cloudera (“Cloudera Products”), including core Apache Hadoop. The Apache Commons Collections library is also in widespread use beyond the Hadoop ecosystem. At this time, no specific attack vector for this vulnerability has been identified as present in Cloudera Products.

In an abundance of caution, we are currently in the process of incorporating a version of the Apache Commons Collections library with a fix into the Cloudera Products. In most cases, this will require coordination with the projects in the Apache community. One example of this is tracked by HADOOP-12577.

The Apache Commons Collections potential security vulnerability is titled “Arbitrary remote code execution with InvokerTransformer” and is tracked by COLLECTIONS-580. MITRE has not issued a CVE, but related CVE-2015-4852 has been filed for the vulnerability. CERT has issued Vulnerability Note #576313 for this issue.

Releases affected: CDH 5.5.0, CDH 5.4.8 and lower, Cloudera Manager 5.5.0, Cloudera Manager 5.4.8 and lower, Cloudera Navigator 2.4.0, Cloudera Navigator 2.3.8 and lower

Users affected: All

Severity (Low/Medium/High): High

Impact: This potential vulnerability may enable an attacker to execute arbitrary code from a remote machine without requiring authentication.

Immediate action required: Upgrade to Cloudera Navigator 2.3.9.

Issues Fixed in Cloudera Navigator 2.3.8

Disabling auditing in Hive causes exception in HiveServer2

If you disable auditing for Hive, but do not restart the Hive service, HiveServer2 logs the following exception:
Hive Internal Error: com.cloudera.navigator.shaded.avro.AvroRuntimeException(Field serviceName type:STRING pos:0 does not accept null values)

Cloudera Bug: NAV-1316

NPE during YARN extraction

The extractor no longer throws a NPE during YARN extraction.

Cloudera Bug: NAV-2016

Issues Fixed in Cloudera Navigator 2.3.3

Upgrade to 2.3.1 does not create column "NAME" in NAVMS_AUDIT_EVENTS

The upgrade process neglected to include a script to create the "NAME" column.

Cloudera Bug: NAV-1541

Navigator Search UI hangs due to blocked log4j thread

The Metadata Server log4j logger blocked due to a defect in how old logs are deleted. The logger has been replaced and the Cloudera Manager Agent deletes old log files.

Cloudera Bug: NAV-1539

Expanding Pig lineage can cause overlap

When you expand a Pig lineage diagram, relationship lines can sometimes obstruct one another.

Cloudera Bug: NAV-1545

Audit reports subject to cross-site scripting

Navigator does not do input validation on report names, which allows users to enter arbitrary code in this field. The next time a report name is viewed, the script is executed.

Cloudera Bug: NAV-1502

Session timeout results in a redirect using HTTP basic authentication

Upon session timeout, Navigator redirects to HTTP basic authentication instead of redirecting to the main login page.

Cloudera Bug: NAV-1506

“POODLE” vulnerability on TLS/SSL enabled ports

The POODLE (Padding Oracle On Downgraded Legacy Encryption) attack takes advantage of a cryptographic flaw in the obsolete SSLv3 protocol, after first forcing the use of that protocol. The only solution is to disable SSLv3 entirely. This requires changes across a wide variety of components of Cloudera Navigator in 2.3.x and all earlier versions. Cloudera Navigator 2.3.3 provides these changes for Cloudera Navigator 2.3.x deployments. All Cloudera Navigator users should upgrade to 2.3.3 as soon as possible. For more information, see the Cloudera Security Bulletin.

Cloudera Bug: NAV-1501

LDAP group searches appear slow because of web UI behavior

Searching for LDAP groups in the web UI can appear slow because every time a character changes in the search field, an onchange Javascript event triggers a new LDAP search.

Cloudera Bug: NAV-1491

Solr audit fields are incorrectly shown with capital letters in the web UI

Solr audit fields such as sub_operation and entity_id are rendered with capital letters and spaces such as Sub Operation or Entity ID.

Cloudera Bug: NAV-1481

Issues Fixed in Cloudera Navigator 2.3.1

Link disappears when expanded

Sometimes when you expand a lineage diagram, a link disappears.

Cloudera Bug: NAV-1416

Operations on canary files should be filtered out

Cloudera Bug: NAV-1419

Hive views have same icon as Hive fields in lineage

Cloudera Bug: NAV-1365

Facet count should show (0) instead of (-) when there are no matching entities

Facet counts show (-) when the Metadata Server does not return a value. It should show (0).

Workaround: None.

Cloudera Bug: NAV-1271

Column lineage does not display

Column lineage does not display when its parent is automatically expanded.

Workaround: Redisplay the lineage by clicking the parent entity.

Cloudera Bug: NAV-1380

Kite dataset extraction fails when HDFS HA is enabled

Workaround: None.

Cloudera Bug: NAV-1383

Exporting audit reports to CSV does not work

You can export audits to JSON with a limit of 10,000 audits.

Cloudera Bug: NAV-1332

Sentry auditing does not work if the Python version is lower than 2.5

Cloudera Bug: NAV-1080

Sqoop sub-operations don't display in schema view of lineage

Workaround: None.

Cloudera Bug: NAV-1211

Issues Fixed in Cloudera Navigator 2.3.0

There were no issues fixed in Cloudera Navigator 2.3.0.

Issues Fixed in Cloudera Navigator 2.2.9

Apache Commons Collections Deserialization Vulnerability

Cloudera has learned of a potential security vulnerability in a third-party library called the Apache Commons Collections. This library is used in products distributed and supported by Cloudera (“Cloudera Products”), including core Apache Hadoop. The Apache Commons Collections library is also in widespread use beyond the Hadoop ecosystem. At this time, no specific attack vector for this vulnerability has been identified as present in Cloudera Products.

In an abundance of caution, we are currently in the process of incorporating a version of the Apache Commons Collections library with a fix into the Cloudera Products. In most cases, this will require coordination with the projects in the Apache community. One example of this is tracked by HADOOP-12577.

The Apache Commons Collections potential security vulnerability is titled “Arbitrary remote code execution with InvokerTransformer” and is tracked by COLLECTIONS-580. MITRE has not issued a CVE, but related CVE-2015-4852 has been filed for the vulnerability. CERT has issued Vulnerability Note #576313 for this issue.

Releases affected: CDH 5.5.0, CDH 5.4.8 and lower, Cloudera Manager 5.5.0, Cloudera Manager 5.4.8 and lower, Cloudera Navigator 2.4.0, Cloudera Navigator 2.3.8 and lower, Cloudera Navigator 2.2.8 and lower.

Users affected: All

Severity (Low/Medium/High): High

Impact: This potential vulnerability may enable an attacker to execute arbitrary code from a remote machine without requiring authentication.

Immediate action required: Upgrade to Cloudera Navigator 2.2.9.

Issues Fixed in Cloudera Navigator 2.2.4

“POODLE” vulnerability on TLS/SSL enabled ports

The POODLE (Padding Oracle On Downgraded Legacy Encryption) attack takes advantage of a cryptographic flaw in the obsolete SSLv3 protocol, after first forcing the use of that protocol. The only solution is to disable SSLv3 entirely. This requires changes across a wide variety of components of Cloudera Navigator in 2.2.x and all earlier versions. Cloudera Navigator 2.2.4 provides these changes for Cloudera Navigator 2.2.x deployments. All Cloudera Navigator users should upgrade to 2.2.4 as soon as possible. For more information, see the Cloudera Security Bulletin.

Cloudera Bug: NAV-1501

Issues Fixed in Cloudera Navigator 2.2.3

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database. HBase allows null characters in qualifiers, so now Navigator escapes them.

Cloudera Bug: NAV-1286

Oozie extractor throws too many Boolean clauses exception

Cloudera Bug: NAV-1203

Issues Fixed in Cloudera Navigator 2.2.2

The audit reports UI now returns results when there are a large number of audit records

The audit reports UI was not returning results when there were a large number of audit records matching a particular time period, especially when the period included multiple days. The UI is now also much more responsive.

Cloudera Bug: NAV-888

Issues Fixed in Cloudera Navigator 2.2.1

Browser autocomplete no longer enabled before authentication

Form fields before authentication in the application have auto-complete enabled. Any user using the same computer would be able to see information entered by a previous user.

Cloudera Bug: NAV-924

Navigator Web UI no longer exposes paths to directory listing/forceful browsing

The web server is configured to display the list of files contained in this directory. This is not recommended because the directory may contain files that are not normally exposed through links on the web site.

Cloudera Bug: NAV-923

Navigator Audit Server no longer throws OOM for very long Impala queries

Cloudera Bug: NAV-952

Issues Fixed in Cloudera Navigator 2.2.0

If Hue is added after Navigator, search results do not have links to Hue

In the Metadata UI, search results contain links to an appropriate application in Hue. However, if you add a Hue service after Navigator roles, there will be no links to Hue.

Workaround:
  1. Set the cluster's display name and name properties to be the same:
    1. Get cluster's name and display name using following API: http://hostname:7180/api/v6/clusters.
    2. In the Cloudera Manager Admin Console, at the right of the cluster name, click the down arrow and select Rename Cluster. Set the cluster display name to match its name.
  2. Restart the Navigator Metadata server.

Cloudera Bug: NAV-745

Issues Fixed in Cloudera Navigator 2.1.6

When auditing is enabled, the Cloudera Manager Agent may become slow or get stuck when responding to commands and when sending heartbeats to Cloudera Manager Server

This issue can occur when Cloudera Navigator auditing is turned on. The auditing code reads audit logs and sends them to the Audit Server. It acquires a lock to protect the list of roles being audited. The same list is also modified by the Cloudera Manager Agent's main thread when a role is started or stopped. If the Audit thread takes too much time to send audits to the Audit Server (which can happen if there is backlog of audit logs), it starves the main Agent thread. This causes the main Agent thread to not send heartbeats and to not respond to commands from the Cloudera Manager Server.

Issues Fixed in Cloudera Navigator 2.1.5

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database. HBase allows null characters in qualifiers, so now Navigator escapes them.

Cloudera Bug: NAV-1286

Oozie extractor throws too many Boolean clauses exception

Cloudera Bug: NAV-1203

Issues Fixed in Cloudera Navigator 2.1.4

The audit reports UI now returns results when there are a large number of audit records

The audit reports UI was not returning results when there were a large number of audit records matching a particular time period, especially when the period included multiple days. The UI is now also much more responsive.

Cloudera Bug: NAV-888

Issues Fixed in Cloudera Navigator 2.1.2

Search results in Navigator now have links to Hue

In the Metadata UI, search results contain links to an appropriate application in Hue. The links may be missing for either of the following reasons:

  • Hue was added after Navigator Metadata server was started.
  • The cluster name and display name are different.
Workaround:
  1. Set the cluster's display name and name properties to be the same:
    1. Get cluster's name and display name using following API: http://hostname:7180/api/v6/clusters.
    2. In the Cloudera Manager Admin Console, at the right of the cluster name, click the down arrow and select Rename Cluster. Set the cluster display name to match its name.
  2. Restart the Navigator Metadata server.

Cloudera Bugs: NAV-745, NAV-866

Browser autocomplete no longer enabled before authentication

Form fields before authentication in the application have auto-complete enabled. Any user using the same computer would be able to see information entered by a previous user.

Cloudera Bug: NAV-924

Navigator Web UI no longer exposes paths to directory listing/forceful browsing

The web server is configured to display the list of files contained in this directory. This is not recommended because the directory may contain files that are not normally exposed through links on the web site.

Cloudera Bug: NAV-923

Navigator Audit Server no longer throws OOM for very long Impala queries

Cloudera Bug: NAV-952

Issues Fixed in Cloudera Navigator 2.1.1

LDAP lookups in Active Directory to resolve group membership are now working

Cloudera Bug: NAV-784

Dropping a Hive table and creating a view with same name or vice versa no longer raises an error

Cloudera Bug: NAV-759

HDFS extraction now works after upgrading CDH from 5.1 to 5.2

Cloudera Bug: NAV-709

Setting a property in the Hue advanced configuration snippet no longer throws a "too many Boolean clauses" error in Navigator Metadata

Cloudera Bug: NAV-549

Issues Fixed in Cloudera Navigator 2.0.5

Memory leak in Navigator Audit Server due to error during batch operations

Cloudera Bug: NAV-984

Issues Fixed in Cloudera Navigator 2.0.3

Dropping a Hive table and creating a view with same name or vice versa no longer raises an error

Cloudera Bug: NAV-759

Setting a property in the Hue advanced configuration snippet no longer throws a "too many Boolean clauses" error in Navigator Metadata

Cloudera Bug: NAV-549

Issues Fixed in Cloudera Navigator 2.0.2

HBase auditing initialization failure can prevent region opening indefinitely

Cloudera Bug: NAV-512

Issues Fixed in Cloudera Navigator 2.0.1

Hive View to Table lineage is missing

The lineage of the underlying tables does not appear in the lineage view.

Workaround: Launch lineage on the underlying tables directly.

Cloudera Bug: NAV-432

The "allowed" query selector is missing from the audit REST API

Queries such as http://hostname:7180/api/v7/audits?maxResults=10&query=allowed==false are now supported.

Workaround: None.

Cloudera Bug: NAV-404

Metadata in metadata files is not processed

Workaround: None.

Cloudera Bug: NAV-490

Lineage does not work when launched on a field

When you launch lineage on a field (Hive column or Pig field, or a Sqoop sub-operation), the UI displays the initial graph properly. However if you expand the parent item (Hive table in case of a Hive column), then things start disappearing from the lineage diagram.

Workaround: None.

Cloudera Bug: NAV-365

When you specify an end date for the created property, no results are returned.

Workaround: Clear the end date control or specify an end date of TO+*%5D%22%7D.

Cloudera Bug: NAV-335

Navigating back to the parent entity in a Pig lineage diagram sometimes displays the error: Cannot read property 'x' of undefined.

Workaround: None.

Cloudera Bug: NAV-298

Issues Fixed in Cloudera Navigator 2.0.0

The last accessed time for Hive table is incorrect.

Workaround: None.

Cloudera Bug: OPSAPS-17227

Pig job that has relations with self is unreadable in lineage view.

The Metadata UI currently does not handle situation where there is a data-flow relation between elements that are also related via parent-child relation.

Workaround: None.

Cloudera Bug: OPSAPS-18836

If auditing is enabled, during an upgrade from Cloudera Manager 4.6.3 to 4.7, the Impala service won't start.

Impala auditing requires the Audit Log Directory property to be set, but the upgrade process fails to set the property.

Workaround: Do one of the following:
  • Stop the Cloudera Navigator Audit server.
  • Ensure that you have a Cloudera Navigator license and manually set the property to /var/log/impalad/audit.

Cloudera Bug: OPSAPS-15333

Empty box appears over the list of results after adding a tag to a file

When tags are added to an entity, in some cases a white box remains after pressing Enter.

Workaround: Refresh the page to remove the artifact.

Cloudera Bug: NAV-87

Issues Fixed in Cloudera Navigator 1.2.0

Certain complex multi-level lineages, such as directory/file and database/table, may not be fully represented visually.

Workaround: None.

Cloudera Bug: OPSAPS-18207