This is the documentation for CDH 5.1.x. Documentation for other versions is available at Cloudera Documentation.

Flume Morphline Solr Sink Configuration Options

You can use the flume.conf configuration file to configure one or more Flume agents, including their sources, sinks, and channels. This is the standard Flume configuration mechanism. For more information about the flume.conf configuration file, see the Flume User Guide.

Flume Morphline Solr Sink provides the following configuration options in the flume.conf file:

Property Name

Default

Description

type

 

The FQCN of this class:
org.apache.flume.sink.solr.
morphline.MorphlineSolrSink

batchSize

100

The maximum number of events to take per flume transaction.

batchDurationMillis

1000

The maximum duration per flume transaction (ms). The transaction commits after this duration or when batchSize is exceeded, whichever comes first.

indexerClass

org.apache.flume.sink.solr.
morphline.MorphlineSolrIndexer
The FQCN of a class implementing
org.apache.flume.sink.solr.
morphline.SolrIndexer

morphlineFile

n/a

The location of the morphline configuration file.
  • In a Cloudera Managed Deployment, use:
    agent.sinks.solrSink.
    morphlineFile=morphlines.conf
  • In unmanaged deployments, provide the relative or absolute path on the local file system to the morphline configuration file. For example:
    /etc/flume-ng/conf/
    tutorialReadAvroContainer.conf

morphlineId

null

Name used to identify a morphline if there are multiple morphlines in a morphline configuration file.

For example, here is a flume.conf section for a SolrSink for the agent named "agent":
agent.sinks.solrSink.type = org.apache.flume.sink.solr.morphline.MorphlineSolrSink
agent.sinks.solrSink.channel = memoryChannel
agent.sinks.solrSink.batchSize = 100
agent.sinks.solrSink.batchDurationMillis = 1000
agent.sinks.solrSink.morphlineFile = /etc/flume-ng/conf/morphline.conf
agent.sinks.solrSink.morphlineId = morphline1
  Note: The examples in this document use a Flume MemoryChannel to easily get started. For production use it is often more appropriate to configure a Flume FileChannel instead, which is a high performance transactional persistent queue.
Page generated September 3, 2015.