hadoop HA 集群namenode无法自动切换为active

lvdongjie 2022年11月07日大数据 144 0

今天在学习HA自动化配置的时候，按照网上的教程去配置并启动HA即两台namenode后，看到一台为active模式，一台为standby模式，并且文件也可以正常写入。在做测试时，用 kill -9 端口号杀死active的namenode后，standby的namenode并没有自动启动。
检查配置文件确保无误后，重启集群，发现依然无果。查看logs->hadoop-root-zkfc-master.log,发现有如下报错信息：

2019-05-18 17:18:35,812 WARN org.apache.hadoop.ha.FailoverController: Unable to gracefully make NameNode at master1/10.253.3.101:53310 standby (unable to connect)
java.net.ConnectException: Call From master2/10.253.3.102 to master1:53310 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
   at sun.reflect.GeneratedConstructorAccessor10.newInstance(Unknown Source)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:791)
   at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:731)
   at org.apache.hadoop.ipc.Client.call(Client.java:1474)
   at org.apache.hadoop.ipc.Client.call(Client.java:1401)
   at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
   at com.sun.proxy.$Proxy9.transitionToStandby(Unknown Source)
   at org.apache.hadoop.ha.protocolPB.HAServiceProtocolClientSideTranslatorPB.transitionToStandby(HAServiceProtocolClientSideTranslatorPB.java:112)
   at org.apache.hadoop.ha.FailoverController.tryGracefulFence(FailoverController.java:172)
   at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:514)
   at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:505)
   at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:61)
   at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:892)
   at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:902)
   at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:801)
   at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:416)
   at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:599)
   at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Caused by: java.net.ConnectException: Connection refused
   at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
   at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530)
   at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494)
   at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:609)
   at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:707)
   at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:370)
   at org.apache.hadoop.ipc.Client.getConnection(Client.java:1523)
   at org.apache.hadoop.ipc.Client.call(Client.java:1440)
   ... 14 more
原来是未找到fuster程序，导致无法进行fence，所以可以通过如下命令来安装，Psmisc软件包中包含了fuster程序：