Flume Solr BlobHandler Configuration Options

Flume accepts Flume events by HTTP POST and GET with the help of HTTPSource.

By default, HTTPSource splits JSON input into Flume events. As an alternative, Flume Solr BlobHandler for HTTPSource returns an event that contains the request parameters as well as the Binary Large Object (BLOB) uploaded with this request. This approach is not suitable for very large objects because it buffers the entire BLOB.

Flume Solr BlobHandler provides the following configuration options in the flume.conf file:

Property Name

Default

Description

handler

 

The FQCN of this class:
org.apache.flume.sink.
solr.morphline.BlobHandler

handler.maxBlobLength

100000000 (100 MB)

The maximum number of bytes to read and buffer for a request.

This example shows a flume.conf section for a HTTPSource with a BlobHandler for the agent named agent:
agent.sources.httpSrc.type = org.apache.flume.source.http.HTTPSource
agent.sources.httpSrc.port = 5140
agent.sources.httpSrc.handler = org.apache.flume.sink.solr.morphline.BlobHandler
agent.sources.httpSrc.handler.maxBlobLength = 2000000000
agent.sources.httpSrc.interceptors = uuidinterceptor
agent.sources.httpSrc.interceptors.uuidinterceptor.type = org.apache.flume.sink.solr.morphline.UUIDInterceptor$Builder
agent.sources.httpSrc.interceptors.uuidinterceptor.headerName = id
#agent.sources.httpSrc.interceptors.uuidinterceptor.preserveExisting = false
#agent.sources.httpSrc.interceptors.uuidinterceptor.prefix = myhostname
agent.sources.httpSrc.channels = memoryChannel