Diagnostic Data Collection
Cloudera Manager log files provide important information for Cloudera Support to use in analyzing problems or unexpected behavior with Cloudera Manager deployments or CDH clusters. Cloudera Director triggers the collection of diagnostic data for the deployments and clusters it manages. This helps avoid situations where a failed cluster has been terminated but Cloudera Support has no diagnostic data or log files to help them identify the cause of the failure. For users with a Cloudera Enterprise or Cloudera Enterprise Trial license, diagnostic data is collected and sent to Cloudera Support automatically upon cluster bootstrap or update failure. By default, diagnostic data is also downloaded to the Cloudera Director instance.
In addition to the automatic collection of diagnostic data, you can initiate diagnostic data collection manually through the Cloudera Director web UI or API. You can trigger manual collection of diagnostic data for an entire Cloudera Manager deployment or for a specific CDH cluster, as described below.
For more information on how diagnostic data collection works on the Cloudera Manager side, see the Cloudera Manager documentation page Sending Usage and Diagnostic Data to Cloudera.
Manual Collection of Diagnostic Data
You can manually trigger the collection of diagnostic data using either the Cloudera Director web UI or the Cloudera Director API.
Using the Web UI
To trigger diagnostic data collection for a Cloudera Manager deployment or cluster, go to the web UI page for the deployment or cluster, and click the down arrow on the dropdown list to the right of the deployment or cluster name. In the dropdown list, click Collect Diagnostic Data.
Cloudera Director will make an API call to the Cloudera Manager API collectDiagnosticData. If successful, Cloudera Manager will send the diagnostic data to Cloudera Support and, if the download diagnostic data property is set to true in the Cloudera Director application.properties file, will also download a zip file containing the diagnostic data for the deployment or cluster to the Cloudera Director EC2 instance. If diagnostic data collection is unsuccessful, and the download diagnostic data property is set to true, Cloudera Manager will download the Cloudera Manager service logs to Cloudera Director.
Manually Triggering Collection of Diagnostic Data at Cluster Termination
When you terminate a Cloudera Manager deployment or CDH cluster in the web UI, the screen for confirming the termination includes a button that triggers collection of diagnostic data before terminating.
Here is the button as seen on the screen for terminating a Cloudera Manager deployment:
Using the API
To manually trigger collection of diagnostic data for Cloudera Manager deployments, use the API at http://cloudera_director_ip:port_number/api-console/index.html#!/deployments/collectDiagnosticData.
To manually trigger collection of diagnostic data for CDH clusters, use the API at http://cloudera_director_ip:port_number/api-console/index.html#!/clusters/collectDiagnosticData.
Status for Data Collection
While diagnostic data collection is in progress, the status of the deployment or cluster changes from its current state, for example, Ready or
Bootstrap Failed, to Updating: Collecting diagnostic data when you mouse-over the Status
The process does not involve an update of the cluster status; the updating message is displayed simply to inform you that diagnostic data collection is in progress. Diagnostic data collection does not change the status of the cluster, so when the data collection is complete, the deployment or cluster status changes back to what it was before diagnostic data collection began.
If you click View Log Status on the deployment or cluster screen, the Diagnostic Log Summary is displayed, showing
information about the last diagnostic data collection:
If diagnostic data has never been collected for the deployment or cluster, the Diagnostic Log Status value is Not Collected and there is no link to open the Diagnostic Log Status screen.
Configuring Diagnostic Data Collection
By default, Cloudera Manager sends diagnostic data to Cloudera Support and to Cloudera Director. Whether the collected diagnostic data is sent to Cloudera Support is controlled by a configuration property in Cloudera Manager. Whether diagnostic data (or the Cloudera Manager logs, if diagnostic data collection is unsuccessful) are downloaded to Cloudera Director is controlled by Cloudera Director server configuration properties. You can configure diagnostic data collection on Cloudera Manager and Cloudera Director using the procedures described in this section.
Configuring the Upload of Diagnostic Data to Cloudera Support
- In Cloudera Manager, click .
- In the list of Filters in the lefthand pane, click Support.
- Click the checkbox for the property Send Diagnostic Data to Cloudera Automatically to toggle the setting between true and false.
For more information on the phone_home property, see the table in the Support section of Cloudera Manager Server Properties.
Configuring the Download of Diagnostic Data to Cloudera Director
- lp.debug.collectDiagnosticDataOnFailure: Determines whether automatic collection of diagnostic data will occur for cluster bootstrap or update failures. The default value is true.
- lp.debug.downloadDiagnosticData: Determines whether or not, in addition to being sent to Cloudera Support, the diagnostic data is also downloaded to the Cloudera Director instance. The default value is true.
- lp.remote.terminate.assumeYes: Determines whether Cloudera Director skips prompting the user to confirm termination when the terminate-remote command is invoked. If you set the property to true, termination proceeds even if diagnostic data collection has failed. The default setting is false
- lp.debug.diagnosticDataDownloadDirectory: Sets a non-default path for the download of diagnostic data for deployments and clusters. The default location is /tmp. The directory where diagnostic data has been downloaded appears in the File Path field in the Diagnostic Log Summary.
- lp.debug.createDiagnosticDataDownloadDirectory: A setting of true allows Cloudera Director to create the non-default download directory specified in lp.debug.diagnosticDataDownloadDirectory if it does not exist. The default value is true.
For information about setting Cloudera Director properties by using the CLI or editing the application.properties file, see Setting Cloudera Director Properties.