New Features in Cloudera Navigator 6.1.0
The following sections describe what's new and what has changed in the data management components of Cloudera Navigator 6.1.0:
- Significant improvements in metadata extraction and purge performance
- Improvements to audit extraction filters
- New Cloudera Manager health alerts for Navigator Metadata Server
- Console shows column metadata in the table entity
- Auto-suggestions in Navigator console search
- New audit events for Fine Grained Privileges and Ownership
- New audit events for HDFS Erasure Coding
Significant improvements in metadata extraction and purge performance
HDFS metadata extraction can take up significant resources, particularly if the initial build extraction is not completed in a timely interval. Extractors running for long periods then cause the metadata purge operation to be cut short or never run at all. This release addresses these problems by significantly improving metadata extraction for HDFS and HMS and reducing the time required to purge metadata. Together, these improvements will make your Navigator Metadata Server more robust and the resource management more predictable.
Improvements to audit extraction filters
This release introduces changes to default HDFS audit filters to better focus the collection of audit events. This change increases the scalability of your Navigator Audit Server system by removing unnecessary audit events.
The following rules were added or updated:
- All HDFS denied access events are accepted. Previously, HDFS denied access events from filtered users and filtered HDFS directories were not recorded.
- HDFS events from the Hive, Spark, and Impala staging directories are discarded. HDFS events from the following job history directories are also discarded:
- All HDFS delete and rename operations from directories that are not already filtered are accepted.
In addition at the end of the filter list, Cloudera Manager provides a rule that filters events from HDFS getfileinfo operations. By default, this filter rule has no affect. Cloudera recommends that you change this rule to Discard to stop capturing events for this HDFS operation that indicates access to file metadata but does not indicate access to file data. This change alone may provide up to 30% reduction in HDFS audit data size.
When upgrading to this release, you may not see the new filters if you have customized your HDFS audit filters previously:
- If HDFS filters have been changed from the default, the new filters will NOT take effect.
- If HDFS filters have not been changed from the default, the new filters will take effect.
New Cloudera Manager health alerts for Navigator Metadata Server
This release includes new checks for Navigator Metadata Server health. If the checks fail, they trigger a health alert in Cloudera Manager for the Navigator Metadata Server. The checks include "Solr Element Count Threshold" and "Solr Relation Count Threshold". These checks trigger an alert if documents in Navigator Metadata Server's embedded Solr collection are threatening to exceed the maximum allowed number for either the element core or the relation core. Typically, if either of these alerts trigger, Navigator has encountered a problem that produces more relations than it should. The triggers let you know that this problem is occurring while it can be addressed efficiently.
For more information, see Navigator Metadata Server Solr Element Count Threshold Test and Navigator Metadata Server Solr Relation Count Threshold Test.
Console shows column metadata in the table entity
This release includes improvements to the Navigator console, including improvements to the layout and information in the columns list. When you are viewing the detail page for a table or view, columns show inline descriptions and managed properties. A popover menu lets you see additional column metadata. The inline search in the Column box now matches for managed metadata and description text in addition to name and type.