Defining Managed Metadata

Minimum Required Role: Metadata Administrator (also provided by Full Administrator)

You can use managed metadata to add typed metadata to classes of entities. You can add namespaces and properties.

A namespace is a container for properties. Four namespaces are reserved:
  • nav for Navigator metadata classes (for example, fselement and user-defined custom fields)
  • up (custom metadata)
  • tp (technical properties)
  • xt (partner applications)
The combination of namespace and property name must be unique.
A property can be one of the following types:
  • Boolean
  • date
  • integer
  • long
  • float
  • double
  • text (with optional maximum length and regular expression validation criteria)
  • enum (of string)
A property can be single-valued or assume multiple values.

Once you have created properties and assigned property values to specific entities, you can create search filters for property values.

Creating Custom Properties Using the Metadata UI

To create custom properties:

  1. Start and log into the Cloudera Navigator data management component UI with the credentials of a user having one of the following user roles:
    • Cloudera Manager Full Administrator
    • Cloudera Manager Navigator Administrator
    • Cloudera Navigator Full Administrator
    • Cloudera Navigator Metadata Administrator
  2. Click the Administration link in the upper right. The Managed Metadata tab displays the list of namespaces and the properties defined in the namespaces.
  3. Click the Create... button.
  4. In the Class field, click or type the beginning of a Navigator entity classname.
  5. Select the class of entities to which the property applies. To clear the field, hover over the field and click the that displays at the right of the field.
  6. Click the Namespace field and select a namespace. If the Namespace drop-down list is empty, click Create Namespace....
    1. Specify a namespace name and optional description.
    2. Click Continue.
  7. Add the name for the property.

    The name can contain letters, numbers, underscores, and hyphens and can be up to 50 characters long.

  8. Specify an optional description.
  9. Select the Multivalued Enable checkbox if the property can have more than one value. For example, an emailFrom property should accept only one value, but an emailTo property could accept more than one value.
  10. In the Type drop-down list, select the property type and specify constraints on the value.
    • Boolean - Boolean: true or false.
    • Date - Date and time.
    • Enum - A set of values. In the Enumeration field, type valid enumeration values and press Enter or Tab.
    • Number - A number. In the Number Type field, select the type of the number: Integer, Long, Float, Double.
    • Text - A string.
      • Maximum Length - The maximum length of the string.
      • Regular Expression - A regular expression that determines whether a string is a valid value.
  11. Click Continue to Review. The Review and Save screen displays.
  12. Click Create to commit the change or Back to Edit Property to continue editing the property.

Example Properties

The following figure shows two properties in the namespace MailAnnotation that apply to entities of the fselement class (that is, HDFS files and directories). The emailFrom property is of type TEXT and can be assigned a single value. The emailTo property is also of type TEXT but can have multiple values.

Editing Custom Properties Using the Metadata UI

The only changes you can make to a custom property are to add classes to which the property applies and enumeration values.

  1. Start and log into the Cloudera Navigator data management component UI with the credentials of a user having one of the following roles:
    • Cloudera Manager Full Administrator
    • Cloudera Manager Navigator Administrator
    • Cloudera Navigator Full Administrator
    • Cloudera Navigator Metadata Administrator
  2. In the Properties table, click a property link. The Edit propertyname dialog box displays.
  3. In the Additional Class field, click or type the beginning of a Navigator entity classname.
  4. Select the class of entities to which the property applies. To clear the field, hover over the field and click the that displays at the right of the field.
  5. In the Additional Enumeration Values field, type valid enumeration values and press Enter or Tab.
  6. Click Continue to Review. The Review and Save screen displays.
  7. Click Update to commit the change or Back to Edit Property to continue editing the property.

Navigator Built-in Classes

Class Description
hdfs_dataset Logical dataset backed by a path in HDFS.
hdfs_dataset_field Field in an HDFS dataset.
fselement DFS file or directory.
hv_column Column in a Hive table.
hv_database Hive database.
hv_partition Partition of a Hive table.
hv_query Hive query template.
hv_query_exec Instance of a Hive query.
hv_query_part Component of a Hive query that maps specific input columns to output columns.
hv_table A Hive table.
hv_view View on one or more Hive tables.
impala_query Impala query template.
impala_query_exec Instance of an Impala query.
impala_query_part Component of an Impala query that maps specific input columns to output columns.
mrjobinstance Instance of a MapReduce, YARN, or Spark job.
mrjobspec Template for a MapReduce, YARN, or Spark job.
oozie_workflow Template for an Oozie workflow.
oozie_wf_instance Instance of an Oozie workflow.
sq_export_sub_oper Sqoop export component that connects specific columns.
sq_qry_import Sqoop import job with query options.
sq_import_sub_oper Sqoop import component that connects specific columns.
sq_oper_exec Instance of a Sqoop job.
sq_tbl_import Sqoop table import operation template.
sq_tbl_export Sqoop table export operation template.
pig_field Field for a relation in Pig; similar to a column in a Hive table.
pig_operation Template for a Pig transformation.
pig_op_exec Instance of a Pig transformation.
pig_relation Pig relation; similar to a Hive table.
userexpression User-specified sub-operation of a MapReduce or YARN job; used for specifying custom column-level lineage.

Defining Metadata with the Navigator API and Navigator SDK

In addition to defining metadata using features provided by the Navigator Metadata UI, you can also define metadata using the Navigator API and Navigator SDK.

For information on the Navigator API, see Cloudera Navigator Data Management API.

For information on the SDK, see the Navigator SDK documentation.