Exploring Audit Data Using the Cloudera Navigator Console

Required Role: Auditing Viewer (or Full Administrator)

Logging in to the Cloudera Navigator Console

To access the Cloudera Navigator console:
  1. Open your browser.
  2. Navigate to the host within the cluster running the Cloudera Navigator Metadata Server role. Replace the example host name with the appropriate host name in your cluster, and replace the default port (7187) if necessary.
    http://fqdn-1.example.com:7187/login.html
    The login page displays.
  3. Log in to the Cloudera Navigator console using the credentials assigned by your administrator. The Cloudera Navigator console displays:

By default Cloudera Navigator console opens to the Search menu as shown above.

Viewing Audit Events

  1. Click the Audits tab. By default, the Audit Events report opens, listing all events that occurred within the last hour, with the most recent at the top:

The Audit Events and Recent Denied Accesses reports are available by default. You create your own reports and apply a variety of filters as detailed in the next section.

Filtering Audit Events

You filter audit events by specifying a time range or adding one or more filters containing an audit event field, operator, and value.

Specifying a Time Range

  1. Click the date-time range at the top right of the Audits tab.
  2. Do one of the following:
    • Click a Last n hours link.
    • Specify a custom range:
      1. Click Custom range.
      2. In the Selected Range endpoints, click each endpoint and specify a date and time in the date control fields.
        • Date - Click the down arrow to display a calendar and select a date, or click a field and click the spinner arrows or press the up and down arrow keys.
        • Time - Click the hour, minute, and AM/PM fields and click the spinner arrows or press the up and down arrow keys to specify the value.
        • Move between fields by clicking fields or by using the right and left arrow keys.
  3. Click Apply.

Adding a Filter

  1. Do one of the following:
    • Click the icon that displays next to a field when you hover in one of the event entries.
    • Click the Filters link. The Filters pane displays.
      1. Click Add New Filter to add a filter.
      2. Choose a field in the Select Property... drop-down list. You can search by fields such as username, service name, or operation. The fields vary depending on the service or role. The service name of the Navigator Metadata Server is Navigator.
      3. Choose an operator in the operator drop-down list.
      4. Type a field value in the value text field. To match a substring, use the like operator. For example, to see all the audit events for files created in the folder /user/joe/out, specify Source like /user/joe/out.
    A filter control with field, operation, and value fields is added to the list of filters.
  2. Click Apply. A field, operation, and value breadcrumb is added above the list of audit events and the list of events displays all events that match the filter criteria.

Removing a Filter

  1. Do one of the following:
    • Click the x next to the filter above the list of events. The list of events displays all events that match the filter criteria.
    • Click the Filters link. The Filters pane displays.
      1. Click the at the right of the filter.
      2. Click Apply. The filter is removed from above the list of audit event and the list of events displays all events that match the filter criteria.

Service Audit Event Field Reference

For service audit events—events generated by a given service running on the cluster—audit events can include the fields listed in the table.

Display Name Field Description
Additional Info additional_info JSON text that contains more details about an operation performed on entities in Navigator Metadata Server.
Allowed allowed Indicates whether the request to perform an operation failed or succeeded. A failure occurs if the user is not authorized to perform the action.
Collection Name collection_name The name of the affected Solr collection.
Database Name database_name For Sentry, Hive, and Impala, the name of the database on which the operation was performed.
Delegation Token ID delegation_token_id Delegation token identifier generated by HDFS NameNode that is then used by clients when submitting a job to JobTracker.
Destination dest Path of the final location of an HDFS file in a rename or move operation.
Entity ID entity_id Identifier of a Navigator Metadata Server entity. The ID can be retrieved using the Navigator Metadata Server API.
Event Time timestamp Date and time an action was performed. The Navigator Audit Server stores the timestamp in the timezone of the Navigator Audit Server. The Cloudera Navigator console displays the timestamp converted to the local timezone. Exported audit events contain the stored timestamp.
Family family HBase column family.
Impersonator impersonator Name of user (service) that invokes an action on behalf of another user (service). Impersonator field always displays values when Sentry is not used with the cluster. For clusters that use Sentry, the Impersonator field displays values for all services other than Hive.
IP Address ipAddress The IP address of the host where an action occurred.
Object Type object_type For Sentry, Hive, and Impala, the type of the object (TABLE, VIEW, DATABASE) on which operation was performed.
Operation command Commands executed by component. See Operations by Component for details. For Cloudera Navigator operations, see Navigator Metadata Server Sub Operations.
Operation Params operation_params Solr query or update parameters used when performing the action.
Operation Text operation_text For Sentry, Hive, and Impala, the SQL query that was executed by user. For Hue, the user or group that was added, edited, or deleted.
Permissions permissions HDFS permission of the file or directory on which the HDFS operation was performed.
Privilege privilege Privilege needed to perform an Impala operation.
Qualifier qualifier HBase column qualifier.
Query ID query_id The query ID for an Impala operation.
Resource resource A service-dependent combination of multiple fields generated during fetch. This field is not supported for filtering as it is not persisted.
Resource Path resource_path HDFS URL of Hive objects (TABLE, VIEW, DATABASE, and so on)
Service Name service The name of the service that performed the action.
Session ID session_id Impala session ID.
Solr Version solr_version Solr version number.
Source src Path of the HDFS file or directory present in an HDFS operation.
Status status Status of an Impala operation providing more information on success or failure.
Stored Object Name stored_object_name Name of a policy, saved search, or audit report in Navigator Metadata Server.
Sub Operation sub_operation Operations performed by Navigator Metadata Server are identified by subsystem (authorization, auditing, for example) and by sub-operation within that subsystem. See Navigator Metadata Server Sub Operations for details.
Table Name table_name For Sentry, HBase, Hive, and Impala, the name of the table on which action was performed.
Username username The name of the user that performed the action.

Operations by Component

The Operation field of an audit event includes the actions taken (commands run) on the component. Operations for Cloudera Navigator (and sub-operations) are listed Navigator Metadata Server Sub Operations
Component Action taken
HBase

addColumn, append, assign, balance, balanceSwitch, checkAndDelete, checkAndPut, compact, compactSelection, createTable, delete, deleteColumn, deleteTable, disableTable, enableTable, exists, flush, get, getClosestRowBefore, grant, increment, incrementColumnValue, modifyColumn, modifyTable, move, put, revoke, scannerOpen, shutdown, split, stopMaster, unassign

HDFS append, concat, create, createSymlink, delete, fsck, getfileinfo, listSnapshottableDirectory, listStatus, mkdirs, open, rename, setOwner, setPermission, setReplication, setTimes
Hive ALTER_PARTITION_MERGE, ALTER_TABLE_MERGE, ALTERDATABASE, ALTERINDEX_PROPS, ALTERINDEX_REBUILD, ALTERPARTITION_FILEFORMAT, ALTERPARTITION_LOCATION, ALTERPARTITION_PROTECTMODE, ALTERPARTITION_SERDEPROPERTIES, ALTERPARTITION_SERIALIZER, ALTERTABLE_ADDCOLS, ALTERTABLE_ADDPARTS, ALTERTABLE_ARCHIVE, ALTERTABLE_CLUSTER_SORT, ALTERTABLE_DROPPARTS, ALTERTABLE_FILEFORMAT, ALTERTABLE_LOCATION, ALTERTABLE_PROPERTIES, ALTERTABLE_PROTECTMODE, ALTERTABLE_RENAME, ALTERTABLE_RENAMECOL, ALTERTABLE_RENAMEPART, ALTERTABLE_REPLACECOLS, ALTERTABLE_SERDEPROPERTIES, ALTERTABLE_SERIALIZER, ALTERTABLE_TOUCH, ALTERTABLE_UNARCHIVE, ALTERVIEW_PROPERTIES, CREATEDATABASE, CREATEFUNCTION, CREATEINDEX, CREATEROLE, CREATETABLE_AS_SELECT, CREATETABLE, CREATEVIEW, DESCDATABASE, DESCFUNCTION, DESCTABLE, DROPDATABASE, DROPFUNCTION, DROPINDEX, DROPROLE, DROPTABLE, DROPVIEW, EXPLAIN, EXPORT, GRANT_PRIVILEGE, GRANT_ROLE, IMPORT, LOAD, LOCKTABLE, MSCK, QUERY, REVOKE_PRIVILEGE, REVOKE_ROLE, SHOW_GRANT, SHOW_ROLE_GRANT, SHOW_TABLESTATUS, SHOW_TBLPROPERTIES, SHOWDATABASES, SHOWFUNCTIONS, SHOWINDEXES, SHOWLOCKS, SHOWPARTITIONS, SHOWTABLES, SWITCHDATABASE, UNLOCKTABLE
Hue ADD_LDAP_GROUPS, ADD_LDAP_USERS, CREATE_GROUP, CREATE_USER, DELETE_GROUP, DELETE_USER, EDIT_GROUP, EDIT_PERMISSION, EDIT_USER, SYNC_LDAP_USERS_GROUPS, USER_LOGIN, USER_LOGOUT
Impala

CREATEROLE, Delete, DML (Data Manipulation Language statements), DROPROLE, GRANT_PRIVILEGE, GRANT_ROLE, Insert, Query, REVOKE_PRIVILEGE, REVOKE_ROLE, SHOW_GRANT, SHOW_ROLE_GRANT, Update

Sentry

ADD_ROLE_TO_GROUP, CREATE_ROLE, DELETE_ROLE_FROM_GROUP, DROP_ROLE, GRANT_PRIVILEGE, REVOKE_PRIVILEGE

Solr add, commit, CREATE, CREATEALIAS, CREATESHARD, DELETE, DELETEALIAS, deleteById, deleteByQuery, DELETESHARD, finish, LIST, LOAD_ON_STARTUP, LOAD, MERGEINDEXES, PERSIST, PREPRECOVERY, query, RELOAD, RENAME, REQUESTAPPLYUPDATES, REQUESTRECOVERY, REQUESTSYNCSHARD, rollback, SPLIT, SPLITSHARD, STATUS, SWAP, SYNCSHARD, TRANSIENT, UNLOAD

Navigator Metadata Server Sub Operations

Operation Sub Operation
auditReport createAuditReport, deleteAuditReport, fetchAllReports, fetchAuditReport, updateAuditReport
authorization deleteGroup, fetchGroup, fetchRoles, searchGroup, updateRoles
metadata fetchAllMetadata, fetchMetadata, updateMetadata
policy createPolicy, deletePolicy, deletePolicySchedule, fetchAllPolicies, fetchPolicySchedule, updatePolicy, updatePolicySchedule
savedSearch createSavedSearch, deleteSavedSearch, fetchAllSavedSearches, fetchSavedSearch, updateSavedSearch

Monitoring Navigator Audit Service Health

Cloudera recommends that administrators monitor the Navigator Audit Service to ensure that it is always running. This is especially important when complete and immutable audit records may be needed for corporate governance, legal, and other purposes associated with compliance.

The Navigator Audit Service has a self-check—the Audit Pipeline Health Check—that administrators can enable to generate warning messages when the system slows down or fails. The health check keeps track of bytes—for audits processed, audits remaining to be processed, and count of errors when sending audit data from Cloudera Manager Agent process to Cloudera Manager.

Cloudera Manager generates a warning if:
  • Audit bytes process = 0
  • Audit bytes unprocessed != 0
  • Send errors > 0 and retries unsuccessful

The health check is run for each service role (daemon) that generates events.

Configuring the Audit Pipeline Health Check

Cloudera Manager Required Role: Navigator Administrator (or Full Administrator)

Log in to the Cloudera Manager Admin Console.
  1. Select Clusters > Cloudera Management Service.
  2. Click the Configuration tab.
  3. In the Search field, type "mgmt.navigator" to find the configuration properties, as shown below:

  4. Modify the settings:
    Property Description
    Navigator Audit Pipeline Health Check Check the box to enable the healthcheck. Health check can be enabled for specific groups. By default, the health check is enabled for all groups.
    Monitoring Period for Audit Failures Amount of time within which the counts of audits processed and other metrics are evaluated before generating warnings. The default is 20 minutes.
    Navigator Audit Failure Thresholds Size (in bytes) of audit failure that generates messages. Two different thresholds are available: Warning, and Critical. Set Warning to the number of bytes of unsent audit data at which you want a warning triggered. Critical messages are sent for any failure regardless of size.
  5. Click Save Changes.

For example, as shown in the Cloudera Manager Admin Console, the pipeline health check is enabled for all groups in the service. The failure period is set to 15 minutes, and the health check sends a warning for failures of any size and a critical error when more than 2 KiB of audit events have not been sent.