Issues Fixed in Cloudera Navigator Data Management

The following sections describe the issues fixed in each Cloudera Navigator release.

Issues Fixed in Cloudera Navigator 2.11.0

Solr error message (immense term in queryText field) during upgrade

In some cases, Navigator Metadata Server failed to start after upgrading to Cloudera Navigator 2.10 and displayed messages similar to the following (only relevant details shown):
2017-07-01 14:27:27,905 ERROR org.apache.solr.servlet.SolrDispatchFilter
 [qtp300983713-60]: null:org.apache.solr.common.SolrException:
 Exception writing document id 0c02400d332854fa44a352235b42857c to the index;
. . .
Caused by: java.lang.IllegalArgumentException: Document contains at least one
immense term in field="queryText" (whose UTF8 encoding is longer than the max
length 32766)
. . .

This issue was caused by the underlying Java library that was writing Solr documents (index and searchable entities) during the upgrade. Special characters, such as carriage returns (ASCII 32) were being mishandled and generating more characters than the source element.

This issue has been resolved in this release. The BinaryRequestWriter, which was already being used to write relations, is now being used to handle elements.

Harmless (but abundant) error messages cause frequent log roll-over

An error message was being generated needlessly by Cloudera Navigator when filtering by source. The error could have been ignored but it was being logged and causing logs to roll-over too quickly.

2017-06-23 10:58:32,967 ERROR com.cloudera.nav.hdfs.client.InotifyClient
[CDHExecutor-0-CDHUrlClassLoader@5177874]: Error handling event (txid: 1627074917):
Renamed /hbase/data/default/ECDS_PROD1_PARTY_RELATED/ccefb... to
/hbase/archive/data/default/ECDS_PROD1_PARTY_RELATED/cbdka... at time 1493302572987

This issue has been resolved in this release. Internally, the logging mechanism has been improved so that it uses a more robust approach to logging, the ThrottlingLogger utility class.

Disk space issue for Navigator Audit Server instances backed by Oracle

Cloudera Navigator deployments using Oracle database as the backing store for Navigator Audit Server were consuming lots of disk space for example, by the NAV_FILE_INFO_STAGING. The issue was caused by an unclosed connection and is related to the connection issue discussed in Navigator Audit Server (Analytics) table locking issue (HDFS activity loader). However, this issue is specific to Oracle-backed Cloudera Navigator instances.

This issue has been resolved in this release. The connections to Oracle database are now handled properly.

Audit pipeline test and Oracle error (ORA-24816) issue

Executing queries of a certain size (4000 bytes) in some cases (for example, Impala) would disrupt the Audit Pipeline test thereby registering bad health on the Impalad (Impala daemon) and the Navigator Audit Server logging the following Oracle database error:
[1911239784@NavigatorServer-4858]: SQL Error: 24816, SQLState: 99999
[1911239784@NavigatorServer-4858]: ORA-24816: Expanded non LONG bind data
supplied after actual LONG or LOB column...
This issue was caused by bind value length potentially greater than 4000 bytes followed by binding for LOB or LONG.

This issue has been resolved in this release. Navigator Audit Server now properly handles the order of the binds needed by Oracle databases.

Navigator Audit Server (Analytics) table locking issue (HDFS activity loader)

Executing multiple statements sometimes resulted in a mismatch between the connection used to execute the statement and the connection used for the commit. The mismatch would result in the server throwing an error message such as the following:

[AnalyticsStagingService]: Could not load HDFS activity data for analytics
org.springframework.dao.CannotAcquireLockException: StatementCallback; SQL
[TRUNCATE TABLE NAV_FILE_INFO_STAGING]...

This issue has been resolved in this release. The same connection is now used to execute queries and for commits.

Invoking auth/group API with invalid role generates NPE and BAD_REQUEST Error

Invoking the auth/group endpoint with an invalid role as one of the user-provided parameters causes the server to throw an NPE (null pointer exception), resulting in response from server of HTTP 500 (Internal Server Error).

This issue has been resolved in this release. The server code now checks for this condition (avoiding NPE) and replies with HTTP 400 (BAD_REQUEST) instead.

Bulk extraction being triggered can result in orphaned relations

In some cases, HDFS bulk extraction was being triggered due to an error in incremental extraction. Specifically, Cloudera Navigator was resetting the long identity of HDFS elements resulting in orphaned relations. This issue has been resolved. A new background task looks for and repairs all orphaned relations.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11

Issues Fixed in Cloudera Navigator 2.10.2

SqoopJobExtractor failures for certain SQL constructs

Certain types of SQL queries that generated sub-operations did not retrieve and display lineage for columns, emitting errors in the Cloudera Navigator console and SQLParserException errors in the Navigator Metadata Server log. For example:
...Extractor com.cloudera.nav.sqoop.extractor.SqoopJobExtractor failed for job_1456104915937_0874
java.lang.RuntimeException: com.akiban.sql.parser.SQLParserException: Encountered " "(" "( "" at line1, column 53.
Was expecting one of: "cs" ...
The root cause of this issue is that the open source parser (akiban) cannot parse certain queries that result in sub-operations, specifically those resulting from Sqoop query IMPORT statements. This issue applies to Sqoop1 query IMPORT statements only, not Sqoop EXPORT and TABLE IMPORT statements.

This issue has been resolved in this release. The SQLParserException is caught and the Cloudera Navigator console renders the message shown below rather than breaking the lineage diagram:


Fixed in Cloudera Navigator 2.10.2, 2.9.2

Issues Fixed in Cloudera Navigator 2.10.1

Bulk extraction being triggered can result in orphaned relations

In some cases, HDFS bulk extraction was being triggered due to an error in incremental extraction. Specifically, Cloudera Navigator was resetting the long identity of HDFS elements resulting in orphaned relations. This issue has been resolved. A new background task looks for and repairs all orphaned relations.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11

Issues Fixed in Cloudera Navigator 2.10.0

Navigator cannot extract S3 metadata with some databases

Using a database other than MySQL, such as PostgreSQL (the default database) for the Navigator Metadata Server generated Java errors in the extracted data for Amazon S3 because of an incorrect string library. The issue has been resolved.

Metadata added to Amazon S3 files persists in Navigator

Once added to Amazon S3 data and extracted by Navigator, custom metadata (key-value pairs) remain with the data captured by Navigator, even after that metadata may have been deleted from the Amazon S3 files. Navigator continues to show the deleted metadata for that file. This issue has been resolved.

Managed metadata properties missing on the page

Clicking managed metadata entities in lineage diagrams to display details silently failed, displaying nothing. This issue has been resolved—the condensed details page now displays properly.

Cluster selection doesn't work properly

In certain cases, cluster facets on the search page did not work correctly.

Search results retained after deselecting search facet

After searching for a specific type of data using the selector box (the search facet, for example, selecting HDFS), the results populating the browser pane remain despite navigating away from the page or de-selecting the item. This issue has now been resolved. After searching and navigating away from the results or deselecting the data type, resulting display also refreshes, as it should.

Deleted managed metadata property still visible on Policy page

In certain cases, deleted Managed metadata properties were visible after deletion.

Issues Fixed in Cloudera Navigator 2.9.2

Bulk extraction being triggered can result in orphaned relations

In some cases, HDFS bulk extraction was being triggered due to an error in incremental extraction. Specifically, Cloudera Navigator was resetting the long identity of HDFS elements resulting in orphaned relations. This issue has been resolved. A new background task looks for and repairs all orphaned relations.

Fixed in Cloudera Navigator 2.9.2, 2.10.1, 2.11

Sqoop job extraction failures

Sqoop job extraction was failing because Sqoop table IMPORT operations were not being retrieved from Solr. This issue has been resolved in this release. Sqoop job extraction is now being processed as expected.

Fixed in Cloudera Navigator 2.9.1, 2.9.2

Issues Fixed in Cloudera Navigator 2.9.1

Sqoop job extraction failures

Sqoop job extraction was failing because Sqoop table IMPORT operations were not being retrieved from Solr. This issue has been resolved in this release. Sqoop job extraction is now being processed as expected.

Fixed in Cloudera Navigator 2.9.1, 2.9.2

Viewing deleted properties in managed metadata shows incorrect search results

Managed metadata property that was deleted drilled down to incorrect search results. The results are now displayed correctly.

Edit Managed Metadata link available for users with insufficient privilege

The Edit Managed Metadata link was available to users without sufficient privilege levels to view it (the link requires AUTH_WRITE_MANAGED_METADATA). The link is now available only to users with the correct privileges.

Some properties that are URLs in the user interface do not function as links

Previously, managed metadata property values and S3 property values that are URLs did not function as links to the URL. These properties are now formatted as clickable links.

Purge navigation and messages could be confusing

When purging deleted properties in Navigator, the maintenance page message was confusing in some situations, and the Continue link after purge returned you to the general Search page. The purge messages have been clarified and now, after successful purge, the page on which you started redisplays rather than the general Search page.

Dashboard average values difficult to read

Previously, the dashboard displayed average values in a long format that could be difficult to read. These values have been converted to a format that is more human-readable. For example, the value 33520.2 now reads 33.5k.

Audit page filter breadcrumbs missing spaces

There were no spaces in the filter breadcrumbs on the Audit page, making the filter difficult to read. Spaces have been added to improve readability.

Searching for text in the Name filter shows HTML bold tag

When using the Original Name filter in search, characters that you typed in the name box appeared with HTML bold tags in the drop-down results. The extraneous tags no longer appear.

Directory Contents information does not reflect deletions for S3 entities

In Navigator, search results indicate if an entity has been deleted. However, the Directory Contents information did not indicate that the entity had been deleted; it was indistinguishable from items in Directory Contents that had not been deleted. Directory Contents information now reflects S3 entity deletions.

Bucket name not included in S3 search results list

In S3 search results, the S3 bucket name was not included with the name of the entity. The S3 bucket name is now displayed.

Issues Fixed in Cloudera Navigator 2.9.0

Incomplete query strings in Navigator metadata

Storing more than 32K characters in Solr causes an exception, and Navigator previously handled this by truncating any field longer than 32K to prevent the exception.

Now, fields longer than 32K are preserved and no longer truncated.

Lineage options and search items are missing

In the Safari browser, lineage options and lineage search could be empty in certain cases.

Out-of-date information when using Internet Explorer 11

In Internet Explorer 11, cached headers could cause out-of-date information to display.

Issues Fixed in Cloudera Navigator 2.8.3

Audit Event reports exported as CSV randomly contained quote character

When exporting an Audit Events report using the CSV format, a double-quotation mark (") was sometimes being inserted into the query used to collect audit events. For example, in the audit event shown below, the quote turns _probable into _pro"bable and _beta into _bet"a:
"2016-12-01T12:50:59.966Z",REDACTED,REDACTED,hive,QUERY,
"beta_vat_mart_delta:ds_dup_vrn_pro"bable",true,,,,,,,,,"
INSERT INTO exp_vat_oob_load_bet"a.temp_ds_dup_vrn_probable_delta_subset

The issue arose when exporting Audit Event reports as CSV format only (not JSON). The issue occurred when the string length of SQL query exceeded 35 characters.

The issue's root cause was an error in processing by the underlying Jackson JSON processor library (jackson-dataformat-csv 2.1.0) used to write the CSV files. The 2.2 version of the library fixes the underlying issue. The issue has been resolved in this release of Cloudera Navigator, which uses the 2.2 Jackson processor library.

Issues Fixed in Cloudera Navigator 2.8.1

Some APIs return incorrect error

In some cases, when a user performs an action with managed metadata that is not permitted because of the user role, Navigator issues an Access Denied error message (500) instead of the expected Forbidden error (403).

Resource path filtering does not working in audit event filter

Lineage options and search items are missing

In the Safari browser, lineage options and lineage search could be empty in certain cases.

Navigator logout link does not work when SAML is enabled

When SAML is enabled, clicking on the Logout link in Navigator executes only a local browser logout, without terminating the SAML session. As a result, after logging out, a user can log in again from the login page without providing credentials.

Issues Fixed in Cloudera Navigator 2.8.0

Last access time displayed for Hive tables is incorrect

Navigator no longer displays incorrect access time information for Hive.

Deleted items not shown by default when Deleted filter added

Showing deleted items when the Deleted filter was not enabled by default. Deleted items are now shown by default when the Deleted filter is applied.

No message shown when no Groups with Navigator roles are found

A message is now displayed in Navigator when no Groups with Navigator roles are found.

Hive started and ended time fields not shown in results

Previously, started and ended time fields were not displayed when selected. These fields are now displayed in results.

Maximum Complexity error message value incorrect

The value in the Maximum Complexity Exceeded error message has been corrected (>3000).

List of properties in audit filter not sorted

Audit filter properties are now sorted alphabetically.

Unable to exit Lineage Full Screen mode from Search Results page

Audit results do not display seconds or fractions seconds

Audit result display granularity has increased to show seconds and fractions of seconds in results.

Search on lineage page does not work for children

Lineage children are now shown when selected in a lineage search.

Create property UI does not use class display names

The create property user interface now displays the correct class display name instead of the abbreviated classname.

Regex entered cannot be confirmed as correct in managed metadata

Navigator now includes regex test validation to managed metadata creation.

Create namespace does not check against reserved names

The Navigator UI now rejects a reserved namespace when creating a new namespace.

Issues Fixed in Cloudera Navigator 2.7.3

Incomplete query strings in Navigator metadata

Storing more than 32K characters in Solr causes an exception. Navigator now truncates any field longer than 32K to prevent the exception.

Issues Fixed in Cloudera Navigator 2.7.2

Long query text alters the display of the search box

Pop-up error in UI in clicking on hive table entity's schema entity

Lineage Options and Search items are missing

In the Safari browser, lineage options and lineage search could be empty in certain cases.

Google analytics might not load if Navigator is served over https/ssl

Issues Fixed in Cloudera Navigator 2.7.1

Oozie and Yarn metadata is not visible for some jobs

Issues Fixed in Cloudera Navigator 2.7.0

Linker causing high memory consumption

Due to an incorrect internal setting, all results were being returned again in second batch requests to Search (Solr), resulting in high memory usage. The parameter autoAdjustBatchSize for the AbstractSolrManager.query setting is now correctly set (to false rather than true) so that subsequent batch requests after an initial request batch the proper number of requests.

Infinite loop during upgrade possible due to deleted entity query

The CheckDeletedEntity::start method was using the *:* query to get entities to mark as deleted. This adversely affected performance and made it possible to cause an upgrade to enter an infinite loop if the server crashed or was restarted during an upgrade. This issue has been resolved in this release.

Managed metadata does not display in lineage diagram

Lineage diagrams display custom metadata, but not managed metadata. This issue has been resolved in this release.

User-specified relations not properly marked in lineage

User-defined relations were not displaying correctly in the lineage diagrams. This issue has been resolved in this release.

Exporting lineage endpoints without direction causes errors

Attempting to export lineage endpoints without providing direction (upstream, downstream) in the query returns a server error (error 500) and adds a null pointer exception (NPE) to the log. This issue has been resolved in this release. Empty direction parameter is prevented from being submitted.

Placement discrepancy in Cloudera Navigator console for Policies

The Policies Command Action section displays above Metadata Assignments for View but display below Metadata Assignments for Edit. This issue has been resolved in this release.

Delete VIEW to FILE relations

Logical physical relationships from Hive to YARN should be removed

This issue has been resolved in this release.

This issue has been resolved in this release.

popover2 binding is positioned incorrectly

This issue has been resolved in this release.

Show parent when applicable on entity details page

Entity cannot be deselected in Lineage

Increase zoom out capability in lineage

API/interactive/entities fails when managed metadata properties exist but do not yet have values

Entity Details page edit metadata does not show namespace in managed metadata

Actions menu should not show Move for deleted entities

No Operation/OperationExecution extracted for Hive and HDFS

numPartitions attribute for Hive should be suppressed

Inconsistent size and block size units

Search drop-down width should be the same as the search input field width

Heap for Navigator Metadata Server needs to be increased

Improve labels for Analytics top users and top commands filter

Cross-site Request Forgery (CSRF) vulnerability

Web client requests to Cloudera Navigator were not protected by default from cross-site request forgery attempts. Additional configuration was needed to protect the cluster. That issue has been resolved in this release. CSRF security is now enabled by default.

Policy earliest start time should be current time

Policy editor line wrapping divides words in the middle

Date format is different for facets, breadcrumbs, and full query text

Improve rendering for parent elements

"created" date is later than "lastAccessed" date

Added support for managed metadata in the Policy Editor

Fixed issue with Navigator ignoring all /hbase/ subpaths aside of WALs and oldWALs

Issues Fixed in Cloudera Navigator 2.6.5

Out of memory during file purge

Navigator runs out of memory during purge of files.

Hive parser issues errors

The Hive parser issued errors during a query in the following cases:
  • Multiple struct attributes in a query
  • Constants are projected as part of a subquery
  • Functions involving constants with an alias

Incomplete query strings in Navigator metadata

Storing more than 32K characters in Solr causes an exception. Navigator now truncates any field longer than 32K to prevent the exception.

Issues Fixed in Cloudera Navigator 2.6.2

Linker causes high memory consumption

Navigator Metadata Server fails with nav.batch and a high amount of heap

java.lang.IllegalArgumentException error causes Navigator to crash

Class not visible on Managed Metadata review screen

The fields Class and Additional class are not visible on the Review screen when creating or updating a Managed Metadata property.

Google analytics might not load when Navigator is served over HTTPS/SSL

Column type for HBase qualifier value in Navigator Audit database should be LONGBLOB

Issues Fixed in Cloudera Navigator 2.6.1

Lineage returns deleted entities even if the Hide deleted entities option is selected

Browser sessions could be unexpectedly logged out

When multiple browser tabs are open to the same Metadata Server, an inactive tab can log you out even though you are active in another tab.

Navigator Metadata Server does not correctly index /hbase/ subpaths

Navigator disabled by default the metadata indexing of /hbase/WALs and /hbase/oldWALs and any entry under /hbase/* entirely.

Previously, /hbase/WALs and /hbase/oldWALs rules were not effective because they were masked by the rule /.*, which was matching everything under /hbase. This has been corrected. Now only nested hidden files and directories under /hbase are discarded.

Issues with additional Group facet values in the Search page

If a new value is added to the subfacet SourceType:HDFS->Group, the 'Clear all' actions selects all the values in the Group facet, although they are not included in the query.

The input field expands outside the facet panel.

Adding a new (not suggested) value to the Group facet in Firefox leaves the suggestion drop-down open.

Logical Physical relationships unexpectedly shown in lineage

HDFS directories are displayed in Hive lineage.

Lineage rendering sometimes throws NPE when latest operation is selected

ClassCastException thrown when a custom entity is encountered

Mismatch in number of files between Search and Search from Analytics due to deleted files

Search queries invoked from the Analytics page now show the correct number of files.

On load, the originating entity might not be visible in lineage

If the lineage graph is tall or wide the originating entity might not be initially visible.

Browser support warning was not working

Internet Explorer 11 was not in the list of supported browsers.

Lineage could fail or display partial results when related to Workflows entities

Oozie workflows can have multiple parents however lineage cannot display this use case. Workflows are no longer returned for an operation. To get workflow detail, you must launch lineage on the workflow itself.

Metadata UI could be unreliable when viewing entities with no name

If an entity has no value in the name or originalName properties, they are now set to "(no name)".

User logging would in some cases be presented with the login screen again

Lineage load error banner could show on subsequent lineages

Navigator displays two warnings when lineage load fails. The yellow banner displays until dismissed or the page is reloaded even if it does not apply to the current lineage.

Missing managed metadata property details

Regular expressions and other details are not listed in the detailed managed metadata property view.

Alignment of logical physical relations does not follow data flow

Logical physical relations should be shown such that they follow data flow. For example, if downstream you hit a logical entity first then it should be logical -> physical. If you hit physical first it should be physical -> logical.

Input validation could in certain cases show errors after the error was fixed

Native input validation messages on properties in the Edit Metadata dialog keep displaying, for example when working with regular expression patterns.

Combo box display issues in Internet Explorer

The combo box scroll bar in search facets does not work in Internet Explorer. Clicking it closes the drop-down.

Hue link missing

When the Cluster Name facet is used, the Hue link is missing in search results.

Issues Fixed in Cloudera Navigator 2.6.0

Lineage performance suffers when more than 10000 relations are extracted

If more than 10000 relations must be traversed for a lineage diagram, performance suffers. This can occur when there are thousands of files in a directory or hundreds of columns in a table.

In 2.3, the Navigator Search UI is missing the User Defined Property filter

The Navigator Search UI has changed between 2.2 and 2.3 with respect to the User Defined Property filter. In 2.3 it is no longer present.

Workaround: In the Search box type: up_propertyName:value. For example, up_myName:myValue.

Displaying the lineage of an entity may consume too much memory and the Metadata Server may run out of memory

Memory leaks have been fixed. However, the Navigator Metadata Server embeds Solr server inside its process, and its memory requirement is proportional to the data that is being indexed. For recommendations on how to configure memory, see Memory Sizing Considerations for Navigator Metadata Server.

After upgrade to 2.4.x, only entries related to deleted files are visible

Issues Fixed in Cloudera Navigator 2.5.0

After upgrade to 2.4.x only entries related to deleted files are visible

Issues Fixed in Cloudera Navigator 2.4.4

Lineage does not render on MacOS using Chrome 48.0.2564.109

After upgrade to 2.4.x only entries related to deleted files are visible

Issues Fixed in Cloudera Navigator 2.4.2

Searching for strings that start with forward slash ignored sometimes

Searching for a string that starts with a forward slash (/) in conjunction with another filter/facet or followed by AND and another search term will result in the string being ignored completely.

Workaround: Enclose the string starting with forward slash in double quotes, for example "/some/path" AND sourceType:hdfs

After upgrade, entities are incorrectly marked as deleted

The issue is fixed. However, if you are upgrading from Cloudera Navigator 2.4.0 or 2.4.1 to 2.4.2, after shutting down the Navigator Metadata Server as instructed in the upgrade documentation, you must additionally perform the following steps:
  1. Edit the HDFS extractor state file under the Navigator storage directory (by default, /var/lib/cloudera-scm-navigator), in the extractorState directory. The HDFS extractor state file contains a JSON object with an attribute upgrades, which has a list of upgrades and their statuses. For example:
    {"nextTxId":11394669,"lastKnownTransactionTime":1453931555151,"layoutVersion":-60,
    "namespaceId":1613692955,"nextExtractionRunId":"df61cf41183dec8947078c228ec505e6##7453",
    "upgrades":[
    {"taskType":"SET_MISSING_ATTRIBUTES","status":"SUCCEEDED","startTime":1453484370390,"endTime":1453484370410,"attempts":1},
    {"taskType":"UNDELETE","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370418,"attempts":1},
    {"taskType":"BLOCK_SIZE","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370427,"attempts":1},
    {"taskType":"REPLICATION_COUNT","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370434,"attempts":1}]}
    Under the upgrades list, change the status attribute of taskType:UNDELETE to FAILED. For example, change:
    {"taskType":"UNDELETE","status":"SUCCEEDED","startTime":1453484370391,"endTime":1453484370418,"attempts":1}
    to
    {"taskType":"UNDELETE","status":"FAILED","startTime":1453484370391,"endTime":1453484370418,"attempts":1} 
  2. If your Navigator instance manages multiple HDFS instances, repeat for each HDFS extractor state file.

On a newly installed cluster, Hive lineage does not work

When Cloudera Manager starts, it does not automatically create the directory /var/log/hive/lineage. Because the Agent cannot detect this directory, it fails to set a watch on it. HiveServer2 creates the directory, but Hive metadata extracted to the directory is never read by Agent and sent to Navigator Metadata Server.

Workaround: Do one of the following:
  • In the Cloudera Manager Admin Console UI, restart HiveServer2.
  • To avoid HiveServer2 downtime, restart the Cloudera Manager Agent. On the host where the HiveServer2 role is running:
    sudo service cloudera-scm-agent restart

Entity landing page schema overflow hides name

If the schema type is very long, it can hide the name of the schema entity.

When Navigator uses the Cloudera Manager authenticator and Cloudera Manager is TLS enabled, authentication fails

When Navigator is configured to use the Cloudera Manager DB for authentication, and Cloudera Manager is TLS enabled, when a user attempts authentication, certificate validation fails with:
Caused by: sun.security.validator.ValidatorException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Analytics UI needs to filter out deleted files

The Analytics UI was counting deleted files.

Policies created from the Analytics UI were not working

Policies created from analytics UI were not working because it filtered on clusterName. Now it filters on sourceId.

There is no indication that the Entity Details page is not completely loaded

Technical metadata displays immediately but it was not always obvious that loading Entity Details page was still loading. A loading spinner now displays on slow loads.

Tables might not load on Entity Details page

Tables might not load, depending on whether the schema finished loading before or after tables finish loading.

Searching for strings that start with forward slash ignored sometimes

Searching for a string that starts with a forward slash (/) in conjunction with another filter/facet or followed by AND and another search term results in the string being ignored completely.

Workaround: Enclose the string starting with a forward slash in double quotes, for example "/some/path" AND sourceType:hdfs.

Issues Fixed in Cloudera Navigator 2.4.1

Potential Security Vulnerability Resolved

This release resolves a potential security vulnerability that may enable an attacker to execute arbitrary code from a remote machine without requiring authentication. Upgrading to the this release resolves the issue. See the Cloudera Security Bulletin, Apache Commons Collections deserialization vulnerability, for complete details.

Issues Fixed in Cloudera Navigator 2.4.0

After HDFS upgrade, some changes to HDFS entities may not appear in Navigator

After an HDFS upgrade, Navigator might not detect changes to HDFS entities, such as move, rename, and delete operations, that were recorded only in the HDFS edit logs before the upgrade. This may cause an inconsistent view of HDFS entities between Navigator and HDFS.

Workaround: None.

After upgrading TLS/SSL-enabled Navigator to 5.4.0, Navigator starts in non-TLS/SSL

After upgrading an TLS/SSL-enabled Navigator to 5.4.x, Navigator will start in non-TLS/SSL mode. Before the 5.4.x release of Navigator, TLS/SSL for Navigator was turned on using an advanced configuration snippet. In release 5.4.x, this option is exposed in the Cloudera Manager configuration UI. After upgrade to 5.4.x, this setting overrides the setting in the advanced configuration snippet, disabling TLS/SSL.

Workaround: Remove the setting from the advanced configuration snippet and specify it in the configuration UI directly.

Navigator login using Cloudera Manager credentials does not work if Cloudera Manager has TLS enabled

Memory leak when creating Impala lineage for very big queries (more than 32K characters)

Disabling auditing in Hive causes exception in HiveServer2

If you disable auditing for Hive, but do not restart the Hive service, HiveServer2 logs the following exception:
Hive Internal Error: com.cloudera.navigator.shaded.avro.AvroRuntimeException(Field serviceName type:STRING pos:0 does not accept null values)

Lineage is not collected for Hive on Spark jobs

Handle HDFS root as special case when generating Hue link

Entity description does not allow multiple lines

Expanding entity causes unintuitive lineage layout

Selected Pig field not shown in lineage

When you view lineage for a Hive field, the Hive parent is expanded so you see the field. However, if there is a field of a Pig table inside a Pig operation execution, only the operation execution is expanded, not the table. The field is not visible and it is not obvious to which entity it belongs.

Pig inside Oozie not shown in lineage view

Page headers missing after selecting "Show Full Page" in lineage diagram

Facet does not visually retain selection after refresh

After reloading a page, some selections are not rendered as selected even though the search string itself is still correct.

Link and parent Hive table not displayed

If you launch a job with two input directories that are also both Hive tables, the Hive table is shown for one of the directories but not the other. If you expand the other directory, the Hive table is shown.

Links disappear when expanding Pig job run by Oozie

Expanding a directory folder obscures the Hive table it is linked to

Source filter missing from filter list

Subfacets should be enabled and disabled based on values in facet type

Tags and key-value pairs should be sorted in the policy editor

Create Policy link should not be available for a policy viewer role

Impala auditing does not support filtering during event capture

Impala auditing does not support filtering during event capture.

Severity: Low

Workaround: None.

Issues Fixed in Cloudera Navigator 2.3.9

Apache Commons Collections Deserialization Vulnerability

Cloudera has learned of a potential security vulnerability in a third-party library called the Apache Commons Collections. This library is used in products distributed and supported by Cloudera (“Cloudera Products”), including core Apache Hadoop. The Apache Commons Collections library is also in widespread use beyond the Hadoop ecosystem. At this time, no specific attack vector for this vulnerability has been identified as present in Cloudera Products.

In an abundance of caution, we are currently in the process of incorporating a version of the Apache Commons Collections library with a fix into the Cloudera Products. In most cases, this will require coordination with the projects in the Apache community. One example of this is tracked by HADOOP-12577.

The Apache Commons Collections potential security vulnerability is titled “Arbitrary remote code execution with InvokerTransformer” and is tracked by COLLECTIONS-580. MITRE has not issued a CVE, but related CVE-2015-4852 has been filed for the vulnerability. CERT has issued Vulnerability Note #576313 for this issue.

Releases affected: CDH 5.5.0, CDH 5.4.8 and lower, Cloudera Manager 5.5.0, Cloudera Manager 5.4.8 and lower, Cloudera Navigator 2.4.0, Cloudera Navigator 2.3.8 and lower

Users affected: All

Severity (Low/Medium/High): High

Impact: This potential vulnerability may enable an attacker to execute arbitrary code from a remote machine without requiring authentication.

Immediate action required: Upgrade to Cloudera Navigator 2.3.9.

Issues Fixed in Cloudera Navigator 2.3.8

Disabling auditing in Hive causes exception in HiveServer2

If you disable auditing for Hive, but do not restart the Hive service, HiveServer2 logs the following exception:
Hive Internal Error: com.cloudera.navigator.shaded.avro.AvroRuntimeException(Field serviceName type:STRING pos:0 does not accept null values)

NPE during YARN extraction

The extractor no longer throws a NPE during YARN extraction.

Issues Fixed in Cloudera Navigator 2.3.3

Upgrade to 2.3.1 does not create column "NAME" in NAVMS_AUDIT_EVENTS

The upgrade process neglected to include a script to create the "NAME" column.

Navigator Search UI hangs due to blocked log4j thread

The Metadata Server log4j logger blocked due to a defect in how old logs are deleted. The logger has been replaced and the Cloudera Manager Agent deletes old log files.

Expanding Pig lineage can cause overlap

When you expand a Pig lineage diagram, relationship lines can sometimes obstruct one another.

Audit reports subject to cross-site scripting

Navigator does not do input validation on report names, which allows users to enter arbitrary code in this field. The next time a report name is viewed, the script is executed.

Session timeout results in a redirect using HTTP basic authentication

Upon session timeout, Navigator redirects to HTTP basic authentication instead of redirecting to the main login page.

“POODLE” vulnerability on TLS/SSL enabled ports

The POODLE (Padding Oracle On Downgraded Legacy Encryption) attack takes advantage of a cryptographic flaw in the obsolete SSLv3 protocol, after first forcing the use of that protocol. The only solution is to disable SSLv3 entirely. This requires changes across a wide variety of components of Cloudera Navigator in 2.3.x and all earlier versions. Cloudera Navigator 2.3.3 provides these changes for Cloudera Navigator 2.3.x deployments. All Cloudera Navigator users should upgrade to 2.3.3 as soon as possible. For more information, see the Cloudera Security Bulletin.

LDAP group searches appear slow because of web UI behavior

Searching for LDAP groups in the web UI can appear slow because every time a character changes in the search field, an onchange Javascript event triggers a new LDAP search.

Solr audit fields are incorrectly shown with capital letters in the web UI

Solr audit fields such as sub_operation and entity_id are rendered with capital letters and spaces such as Sub Operation or Entity ID.

Issues Fixed in Cloudera Navigator 2.3.1

Link disappears when expanded

Sometimes when you expand a lineage diagram, a link disappears.

Operations on canary files should be filtered out

Hive views have same icon as Hive fields in lineage

Facet count should show (0) instead of (-) when there are no matching entities

Facet counts show (-) when the Metadata Server does not return a value. It should show (0).

Workaround: None.

Column lineage does not display

Column lineage does not display when its parent is automatically expanded.

Workaround: Redisplay the lineage by clicking the parent entity.

Kite dataset extraction fails when HDFS HA is enabled

Workaround: None.

Exporting audit reports to CSV does not work

You can export audits to JSON with a limit of 10,000 audits.

Sentry auditing does not work if the Python version is lower than 2.5

Sqoop sub-operations don't display in schema view of lineage

Workaround: None.

Issues Fixed in Cloudera Navigator 2.3.0

There were no issues fixed in Cloudera Navigator 2.3.0.

Issues Fixed in Cloudera Navigator 2.2.9

Apache Commons Collections Deserialization Vulnerability

Cloudera has learned of a potential security vulnerability in a third-party library called the Apache Commons Collections. This library is used in products distributed and supported by Cloudera (“Cloudera Products”), including core Apache Hadoop. The Apache Commons Collections library is also in widespread use beyond the Hadoop ecosystem. At this time, no specific attack vector for this vulnerability has been identified as present in Cloudera Products.

In an abundance of caution, we are currently in the process of incorporating a version of the Apache Commons Collections library with a fix into the Cloudera Products. In most cases, this will require coordination with the projects in the Apache community. One example of this is tracked by HADOOP-12577.

The Apache Commons Collections potential security vulnerability is titled “Arbitrary remote code execution with InvokerTransformer” and is tracked by COLLECTIONS-580. MITRE has not issued a CVE, but related CVE-2015-4852 has been filed for the vulnerability. CERT has issued Vulnerability Note #576313 for this issue.

Releases affected: CDH 5.5.0, CDH 5.4.8 and lower, Cloudera Manager 5.5.0, Cloudera Manager 5.4.8 and lower, Cloudera Navigator 2.4.0, Cloudera Navigator 2.3.8 and lower, Cloudera Navigator 2.2.8 and lower.

Users affected: All

Severity (Low/Medium/High): High

Impact: This potential vulnerability may enable an attacker to execute arbitrary code from a remote machine without requiring authentication.

Immediate action required: Upgrade to Cloudera Navigator 2.2.9.

Issues Fixed in Cloudera Navigator 2.2.4

“POODLE” vulnerability on TLS/SSL enabled ports

The POODLE (Padding Oracle On Downgraded Legacy Encryption) attack takes advantage of a cryptographic flaw in the obsolete SSLv3 protocol, after first forcing the use of that protocol. The only solution is to disable SSLv3 entirely. This requires changes across a wide variety of components of Cloudera Navigator in 2.2.x and all earlier versions. Cloudera Navigator 2.2.4 provides these changes for Cloudera Navigator 2.2.x deployments. All Cloudera Navigator users should upgrade to 2.2.4 as soon as possible. For more information, see the Cloudera Security Bulletin.

Issues Fixed in Cloudera Navigator 2.2.3

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database. HBase allows null characters in qualifiers, so now Navigator escapes them.

Oozie extractor throws too many Boolean clauses exception

Issues Fixed in Cloudera Navigator 2.2.2

The audit reports UI now returns results when there are a large number of audit records

The audit reports UI was not returning results when there were a large number of audit records matching a particular time period, especially when the period included multiple days. The UI is now also much more responsive.

Issues Fixed in Cloudera Navigator 2.2.1

Browser autocomplete no longer enabled before authentication

Form fields before authentication in the application have auto-complete enabled. Any user using the same computer would be able to see information entered by a previous user.

Navigator Web UI no longer exposes paths to directory listing/forceful browsing

The web server is configured to display the list of files contained in this directory. This is not recommended because the directory may contain files that are not normally exposed through links on the web site.

Navigator Audit Server no longer throws OOM for very long Impala queries

Issues Fixed in Cloudera Navigator 2.2.0

If Hue is added after Navigator, search results do not have links to Hue

In the Metadata UI, search results contain links to an appropriate application in Hue. However, if you add a Hue service after Navigator roles, there will be no links to Hue.

Workaround:
  1. Set the cluster's display name and name properties to be the same:
    1. Get cluster's name and display name using following API: http://hostname:7180/api/v6/clusters.
    2. In the Cloudera Manager Admin Console, at the right of the cluster name, click the down arrow and select Rename Cluster. Set the cluster display name to match its name.
  2. Restart the Navigator Metadata server.

Issues Fixed in Cloudera Navigator 2.1.6

When auditing is enabled, the Cloudera Manager Agent may become slow or get stuck when responding to commands and when sending heartbeats to Cloudera Manager Server

This issue can occur when Cloudera Navigator auditing is turned on. The auditing code reads audit logs and sends them to the Audit Server. It acquires a lock to protect the list of roles being audited. The same list is also modified by the Cloudera Manager Agent's main thread when a role is started or stopped. If the Audit thread takes too much time to send audits to the Audit Server (which can happen if there is backlog of audit logs), it starves the main Agent thread. This causes the main Agent thread to not send heartbeats and to not respond to commands from the Cloudera Manager Server.

Issues Fixed in Cloudera Navigator 2.1.5

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database

Navigator Audit Server reports invalid null characters in HBase audit events when using the PostgreSQL database. HBase allows null characters in qualifiers, so now Navigator escapes them.

Oozie extractor throws too many Boolean clauses exception

Issues Fixed in Cloudera Navigator 2.1.4

The audit reports UI now returns results when there are a large number of audit records

The audit reports UI was not returning results when there were a large number of audit records matching a particular time period, especially when the period included multiple days. The UI is now also much more responsive.

Issues Fixed in Cloudera Navigator 2.1.2

Search results in Navigator now have links to Hue

In the Metadata UI, search results contain links to an appropriate application in Hue. The links may be missing for either of the following reasons:

  • Hue was added after Navigator Metadata server was started.
  • The cluster name and display name are different.
Workaround:
  1. Set the cluster's display name and name properties to be the same:
    1. Get cluster's name and display name using following API: http://hostname:7180/api/v6/clusters.
    2. In the Cloudera Manager Admin Console, at the right of the cluster name, click the down arrow and select Rename Cluster. Set the cluster display name to match its name.
  2. Restart the Navigator Metadata server.

Browser autocomplete no longer enabled before authentication

Form fields before authentication in the application have auto-complete enabled. Any user using the same computer would be able to see information entered by a previous user.

Navigator Web UI no longer exposes paths to directory listing/forceful browsing

The web server is configured to display the list of files contained in this directory. This is not recommended because the directory may contain files that are not normally exposed through links on the web site.

Navigator Audit Server no longer throws OOM for very long Impala queries

Issues Fixed in Cloudera Navigator 2.1.1

LDAP lookups in Active Directory to resolve group membership are now working

Dropping a Hive table and creating a view with same name or vice versa no longer raises an error

HDFS extraction now works after upgrading CDH from 5.1 to 5.2

Setting a property in the Hue advanced configuration snippet no longer throws a "too many Boolean clauses" error in Navigator Metadata

Issues Fixed in Cloudera Navigator 2.0.5

Memory leak in Navigator Audit Server due to error during batch operations

Issues Fixed in Cloudera Navigator 2.0.3

Dropping a Hive table and creating a view with same name or vice versa no longer raises an error

Setting a property in the Hue advanced configuration snippet no longer throws a "too many Boolean clauses" error in Navigator Metadata

Issues Fixed in Cloudera Navigator 2.0.2

HBase auditing initialization failure can prevent region opening indefinitely

Issues Fixed in Cloudera Navigator 2.0.1

Hive View to Table lineage is missing

The lineage of the underlying tables does not appear in the lineage view.

Workaround: Launch lineage on the underlying tables directly.

The "allowed" query selector is missing from the audit REST API

Queries such as http://hostname:7180/api/v7/audits?maxResults=10&query=allowed==false are now supported.

Workaround: None.

Metadata in metadata files is not processed

Workaround: None.

Lineage does not work when launched on a field

When you launch lineage on a field (Hive column or Pig field, or a Sqoop sub-operation), the UI displays the initial graph properly. However if you expand the parent item (Hive table in case of a Hive column), then things start disappearing from the lineage diagram.

Workaround: None.

When you specify an end date for the created property, no results are returned.

Workaround: Clear the end date control or specify an end date of TO+*%5D%22%7D.

Navigating back to the parent entity in a Pig lineage diagram sometimes displays the error: Cannot read property 'x' of undefined.

Workaround: None.

Issues Fixed in Cloudera Navigator 2.0.0

The last accessed time for Hive table is incorrect.

Workaround: None.

Pig job that has relations with self is unreadable in lineage view.

The Metadata UI currently does not handle situation where there is a data-flow relation between elements that are also related via parent-child relation.

Workaround: None.

If auditing is enabled, during an upgrade from Cloudera Manager 4.6.3 to 4.7, the Impala service won't start.

Impala auditing requires the Audit Log Directory property to be set, but the upgrade process fails to set the property.

Workaround: Do one of the following:
  • Stop the Cloudera Navigator Audit server.
  • Ensure that you have a Cloudera Navigator license and manually set the property to /var/log/impalad/audit.

Empty box appears over the list of results after adding a tag to a file

When tags are added to an entity, in some cases a white box remains after pressing Enter.

Workaround: Refresh the page to remove the artifact.

Issues Fixed in Cloudera Navigator 1.2.0

Certain complex multi-level lineages, such as directory/file and database/table, may not be fully represented visually.

Workaround: None.