IT虾米网

01 Flume系列(一)安装配置详解

xmjava 2020年02月20日 大数据 279 0

01 Flume系列(一)安装配置

Flume(http://flume.apache.org/) is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.

Flume 是 Cloudera 提供的一个高可用的,高可靠的,分布式的海量日志采集、聚合和传输的系统。Flume 是基于流数据的架构,用于在线日志分析。

Flume 架构

一、Flume 安装及配置

Flume 启动教程:http://flume.apache.org/FlumeUserGuide.html#starting-an-agent

(1) Flume 下载

tar -zxvf apache-flume-1.8.0-bin

(2) Flume 配置

vi conf/example.conf 
 
# 声明三种组件 
a1.sources = r1 
a1.sinks = k1 
a1.channels = c1 
 
# 定义source信息 
a1.sources.r1.type = netcat 
a1.sources.r1.bind = localhost 
a1.sources.r1.port = 44444 
 
# 定义sinks信息 
a1.sinks.k1.type = logger 
 
# 定义channels信息 
a1.channels.c1.type = memory 
 
# 绑定在一起 
a1.sources.r1.channels = c1 
a1.sinks.k1.channel = c1

(3) Flume 启动

bin/flume-ng agent -f conf/example.conf -n a1 -Dflume.root.logger=INFO,console

补充:netcat 简介

yum -y install netcat 
# nc 监听 
nc -lk localhost 8888 
# nc 登陆 
nc localhost 8888

每天用心记录一点点。内容也许不重要,但习惯很重要!

发布评论

分享到:

IT虾米网

微信公众号号:IT虾米 (左侧二维码扫一扫)欢迎添加!

Greeplum 系列(七) 权限管理详解
你是第一个吃螃蟹的人
发表评论

◎欢迎参与讨论,请在这里发表您的看法、交流您的观点。