一次奇葩Hama问题记录

最新推荐文章于 2023-01-05 10:52:01 发布

原创最新推荐文章于 2023-01-05 10:52:01 发布 · 684 阅读

0 ·

CC 4.0 BY-SA版权

hadoop 同时被 2 个专栏收录

118 篇文章

订阅专栏

Java多线程

95 篇文章

订阅专栏

本文讨论了在使用Hama进行改进时，由于引入的类A继承自线程类并在特定方法中启动线程，导致GraphJobRunner中IPC连接频繁断开的问题。通过将启动线程的代码移至加载顶点的方法内部，问题得到解决。分析了日志信息，解释了可能的原因，并提供了解决方案。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

对Hama进行改进，引用了一个类A(A继承了线程类)，当该类实现如下时，:

GraphJobRunner 中

@Override
public final void setup(
BSPPeer<Writable, Writable, Writable, Writable, GraphJobMessage> peer)
throws IOException, SyncException, InterruptedException {

setupFields(peer);

loadVertices(peer);

countGlobalVertexCount(peer);

doInitialSuperstep(peer);

}

private void setupFields(

.....

A a = new A;

a.start() ;

vertices.init(this, conf, peer.getTaskId());

}

运行时就IPC就开始不稳定了，mu02就会持续掉线再连上，子节点的日志则无此问题。。。。不知道是咋回事。

日志显示如下:

15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.100:37703. Number of active connections: 20
15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.100:37710. Number of active connections: 19
15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.100:37718. Number of active connections: 18
15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.100:37724. Number of active connections: 17
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to c02b01/192.168.0.21:61001 from liuqiang2: closed
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to c02b01/192.168.0.21:61001 from liuqiang2: stopped, remaining connections 15
15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.30:41874. Number of active connections: 16
15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.21:42530. Number of active connections: 15
15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.100:37733. Number of active connections: 14
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to c02b01/192.168.0.21:61002 from liuqiang2: closed
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to c02b01/192.168.0.21:61002 from liuqiang2: stopped, remaining connections 14
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to mu02/192.168.0.100:61006 from liuqiang2: closed
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to mu02/192.168.0.100:61006 from liuqiang2: stopped, remaining connections 13
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to mu02/192.168.0.100:61005 from liuqiang2: closed
15/03/05 21:52:49 DEBUG ipc.Client: IPC Client (47) connection to mu02/192.168.0.100:61005 from liuqiang2: stopped, remaining connections 12
15/03/05 21:52:49 DEBUG ipc.Server: IPC Server listener on 61005: disconnecting client 192.168.0.100:37738. Number of active connections: 13

非常奇怪！

后来尝试将

a.start() ; 放到loadVertices中

loadVertices{

a.start() ;

...

}

就好了。。。。这尼玛能差几行代码啊?????? 哪位知道是咋回事啊。。。。