报错解决：Flume实现多节点Avro Source报错

Flume与Avro数据采集问题排查与解决

最新推荐文章于 2024-08-07 19:26:10 发布

邵奈一

最新推荐文章于 2024-08-07 19:26:10 发布

阅读量2.1k

点赞数

CC 4.0 BY-SA版权

分类专栏：大数据文章标签： flume apache java

本文链接：https://blog.csdn.net/shaock2018/article/details/123136171

大数据同时被 2 个专栏收录

89 篇文章

订阅专栏

错误解决

71 篇文章

订阅专栏

在尝试使用Flume的AvroSource从slave1节点采集数据到master时，遇到`Failed to set up server socket`错误。问题源于master配置错误的IP地址或端口，以及启动顺序不正确。解决方案是确保配置文件中master的IP和端口正确，并先启动Flume Agent，再启动avro-client。按照此步骤操作，成功解决了数据采集的问题。

实操背景：其实很简单，就是想要使用 Avro Source ，在 master 上启动 Flume 的 Agent ，然后让 slave1 节点发送信息过来，让 master 采集到。

有小伙伴报了以下错误：

org.apache.flume.FlumeException: Failed to set up server socket
	at org.apache.flume.source.AvroSource.start(AvroSource.java:248)
	at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
	at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: /192.168.128.132:4141
	at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
	at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
	at org.apache.flume.source.AvroSource.start(AvroSource.java:243)
	... 9 more
Caused by: java.net.BindException: Cannot assign requested address
	at sun.nio.ch.Net.bind0(Native Method)

在这里插入图片描述
打开 slave1
启动 avro-client ，报错如下：

22/02/25 16:31:26 WARN api.NettyAvroRpcClient: Using default maxIOWorkers
22/02/25 16:31:26 ERROR avro.AvroCLIClient: Unable to open connection to Flume. Exception follows.
org.apache.flume.FlumeException: NettyAvroRpcClient { host: master, port: 4141 }: RPC connection error
	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:181)
	at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:120)
	at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:638)