cluster網(wǎng)絡(luò)異常發(fā)生資源組切換，故障。

yulemi

Sun Jun 30 11:03:35 CST 2013
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource zbjk status msg on node wgv8902 change to <LogicalHostname offline.>
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8902 change to R_OFFLINE
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8902 change to R_POSTNET_STOPPING
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8902 change to R_OFFLINE
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource zbjk-storageplus status on node wgv8902 change to R_FM_OFFLINE
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource zbjk-storageplus status msg on node wgv8902 change to <>
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 529407 daemon.notice] resource group zbjk-rg state on node wgv8902 change to RG_OFFLINE_START_FAILED
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 529407 daemon.notice] resource group zbjk-rg state on node wgv8902 change to RG_OFFLINE
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 447451 daemon.error] Not attempting to start resource group <zbjk-rg> on node <wgv8901> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 447451 daemon.error] Not attempting to start resource group <zbjk-rg> on node <wgv8902> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jun 23 10:43:49 wgv8901 Cluster.RGM.rgmd: [ID 674214 daemon.notice] rebalance: no primary node is currently found for resource group <zbjk-rg>.
Jun 23 10:43:51 wgv8901 in.mpathd[2332]: [ID 620804 daemon.error] Successfully failed back to NIC ce3
Jun 23 10:43:51 wgv8901 in.mpathd[2332]: [ID 299542 daemon.error] NIC repair detected on ce3 of group ipmp1
Jun 23 10:43:51 wgv8901 in.mpathd[2332]: [ID 237757 daemon.error] At least 1 interface (ce3) of group ipmp1 has repaired
Jun 23 10:43:51 wgv8901 in.mpathd[2332]: [ID 832587 daemon.error] Successfully failed over from NIC ce0 to NIC ce3
Jun 23 10:43:51 wgv8901 Cluster.PNM: [ID 890413 daemon.notice] ipmp1: state transition from DOWN to OK.
Jun 23 22:54:12 wgv8901 Cluster.RGM.rgmd: [ID 529407 daemon.notice] resource group zbjk-rg state on node wgv8901 change to RG_PENDING_ONLINE
Jun 23 22:54:12 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8901 change to R_PRENET_STARTING
Jun 23 22:54:12 wgv8901 Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hafoip_prenet_start> for resource <zbjk>, resource group <zbjk-rg>, timeout <300> seconds
Jun 23 22:54:12 wgv8901 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource zbjk status on node wgv8901 change to R_FM_UNKNOWN
Jun 23 22:54:12 wgv8901 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource zbjk status msg on node wgv8901 change to <Starting>
Jun 23 22:54:13 wgv8901 Cluster.RGM.rgmd: [ID 736390 daemon.notice] method <hafoip_prenet_start> completed successfully for resource <zbjk>, resource group <zbjk-rg>, time used: 0% of timeout <300 seconds>
Jun 23 22:54:13 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8901 change to R_PRENET_STARTED
Jun 23 22:54:13 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8901 change to R_PRENET_STARTING
Jun 23 22:54:13 wgv8901 Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hastorageplus_prenet_start> for resource <zbjk-storageplus>, resource group <zbjk-rg>, timeout <1800> seconds
Jun 23 22:54:13 wgv8901 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource zbjk-storageplus status on node wgv8901 change to R_FM_UNKNOWN
Jun 23 22:54:13 wgv8901 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource zbjk-storageplus status msg on node wgv8901 change to <Starting>
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 736390 daemon.notice] method <hastorageplus_prenet_start> completed successfully for resource <zbjk-storageplus>, resource group <zbjk-rg>, time used: 0% of timeout <1800 seconds>
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8901 change to R_PRENET_STARTED
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8901 change to R_STARTING
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hafoip_start> for resource <zbjk>, resource group <zbjk-rg>, timeout <500> seconds
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource zbjk status on node wgv8901 change to R_FM_ONLINE
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource zbjk status msg on node wgv8901 change to <LogicalHostname online.>
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 736390 daemon.notice] method <hafoip_start> completed successfully for resource <zbjk>, resource group <zbjk-rg>, time used: 0% of timeout <500 seconds>
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8901 change to R_JUST_STARTED
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8901 change to R_ONLINE_UNMON
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8901 change to R_STARTING
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8901 change to R_MON_STARTING
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hafoip_monitor_start> for resource <zbjk>, resource group <zbjk-rg>, timeout <300> seconds
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hastorageplus_start> for resource <zbjk-storageplus>, resource group <zbjk-rg>, timeout <90> seconds
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 736390 daemon.notice] method <hastorageplus_start> completed successfully for resource <zbjk-storageplus>, resource group <zbjk-rg>, time used: 0% of timeout <90 seconds>
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8901 change to R_JUST_STARTED
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8901 change to R_ONLINE_UNMON
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 784560 daemon.notice] resource zbjk-storageplus status on node wgv8901 change to R_FM_ONLINE
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 922363 daemon.notice] resource zbjk-storageplus status msg on node wgv8901 change to <>
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8901 change to R_MON_STARTING
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 529407 daemon.notice] resource group zbjk-rg state on node wgv8901 change to RG_PENDING_ON_STARTED
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 707948 daemon.notice] launching method <hastorageplus_monitor_start> for resource <zbjk-storageplus>, resource group <zbjk-rg>, timeout <90> seconds
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 736390 daemon.notice] method <hastorageplus_monitor_start> completed successfully for resource <zbjk-storageplus>, resource group <zbjk-rg>, time used: 0% of timeout <90 seconds>
Jun 23 22:54:16 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk-storageplus state on node wgv8901 change to R_ONLINE
Jun 23 22:54:17 wgv8901 Cluster.RGM.rgmd: [ID 736390 daemon.notice] method <hafoip_monitor_start> completed successfully for resource <zbjk>, resource group <zbjk-rg>, time used: 0% of timeout <300 seconds>
Jun 23 22:54:17 wgv8901 Cluster.RGM.rgmd: [ID 443746 daemon.notice] resource zbjk state on node wgv8901 change to R_ONLINE
Jun 23 22:54:17 wgv8901 Cluster.RGM.rgmd: [ID 529407 daemon.notice] resource group zbjk-rg state on node wgv8901 change to RG_ONLINE
Jun 30 10:43:17 wgv8901 in.mpathd[2332]: [ID 168056 daemon.error] All Interfaces in group ipmp1 have failed


2次都是IPMP發(fā)生了切換導(dǎo)致cluster出現(xiàn)問題，如果在主節(jié)點(diǎn)或從節(jié)點(diǎn)手動切換做實(shí)驗(yàn)都可以切換，但是cluster不自動切換
如何才能解決這個問題？IPMP配置的沒有問題。只連接了一個路由。難道是路由引起的問題？請教高手。

東方蜘蛛

All Interfaces in group ipmp1 have failed

東方蜘蛛

別用probo base方式配置ipmp，改成link base試試～

yulemi

東方蜘蛛 發(fā)表于 2013-06-30 18:49
別用probo base方式配置ipmp，改成link base試試～

謝謝蜘蛛大俠哈，這個是solaris9的系統(tǒng)，是不是可以在rc3.d這個目錄下添加幾個路由地址就能防止這樣的問題？

東方蜘蛛

如果網(wǎng)關(guān)不穩(wěn)定，建議配置下：/etc/init.d/ipmp.targets
http://wenku.baidu.com/view/725083155f0e7cd184253641.html

yulemi

東方蜘蛛 發(fā)表于 2013-07-01 12:22
如果網(wǎng)關(guān)不穩(wěn)定，建議配置下：/etc/init.d/ipmp.targets
http://wenku.baidu.com/view/725083155f0e7cd184 ...

我的pong值是默認(rèn)的3600.我想修改，但是提示報錯，不知道是哪的原因
root@wgv8901 # scrgadm -c -g zbjk-rg -y Pingpong_interval=60
wgv8902 - Entry in /etc/vfstab for file system mount point /var/mqm is incorrect: File system mount point should specify 'mount at boot' as 'no'.

VALIDATE on resource zbjk-storageplus, resource group zbjk-rg, exited with non-zero exit status.
Validation of resource zbjk-storageplus in resource group zbjk-rg on node wgv8902 failed.
root@wgv8901 # scrgadm -pvv | grep pong
  (zbjk-rg) Res Group Pingpong_interval:           3600
root@wgv8901 # more /etc/vfstab
#device device  mount   FS      fsck    mount   mount
#to     mount   to      fsck            point           type    pass    at boot options
#                       
fd      -       /dev/fd fd      -       no      -
/proc   -       /proc   proc    -       no      -
#/dev/dsk/c1t0d0s1      -       -       swap    -       no      -
/dev/md/dsk/d20 -       -       swap    -       no      -
/dev/md/dsk/d10 /dev/md/rdsk/d10        /       ufs     1       no      -
#/dev/dsk/c1t0d0s4      /dev/rdsk/c1t0d0s4      /var    ufs     1       no      -
/dev/md/dsk/d40 /dev/md/rdsk/d40        /var    ufs     1       no      -
#/dev/dsk/c1t0d0s7      /dev/rdsk/c1t0d0s7      /globaldevices  ufs     2       yes     -
#/dev/dsk/c1t0d0s3      /dev/rdsk/c1t0d0s3      /opt    ufs     2       yes     -
/dev/md/dsk/d30 /dev/md/rdsk/d30        /opt    ufs     2       yes     -
/dev/dsk/c1t3d0s0       /dev/rdsk/c1t3d0s0      /mqmtemp        ufs     1       yes     -
swap    -       /tmp    tmpfs   -       yes     -
#/dev/did/dsk/d11s7 /dev/did/rdsk/d11s7 /global/.devices/node@2 ufs 2 no global
/dev/md/dsk/d50 /dev/md/rdsk/d50        /global/.devices/node@2 ufs     2       no      global
#/dev/md/zbjkset2/dsk/d101      /dev/md/zbjkset2/rdsk/d101      /var/mqm        ufs     2       no      logging
#/dev/md/zbjkset2/dsk/d102      /dev/md/zbjkset2/rdsk/d102      /opt/mqm        ufs     2       no      logging
#/dev/md/zbjkset2/dsk/d103      /dev/md/zbjkset2/rdsk/d103      /u1/informix    ufs     2       no      logging
#/dev/md/zbjkset2/dsk/d104      /dev/md/zbjkset2/rdsk/d104      /u1/tuxedo      ufs     2       no      logging
#add sbjkset
/dev/md/sbjkset/dsk/d121        /dev/md/sbjkset/rdsk/d121      /opt/mqm        ufs     2       no      logging
/dev/md/sbjkset/dsk/d122        /dev/md/sbjkset/rdsk/d122      /u1/informix     ufs     2       no      logging
/dev/md/sbjkset/dsk/d123        /dev/md/sbjkset/rdsk/d123      /u1/tuxedo      ufs     2       no      logging
/dev/md/sbjkset/dsk/d124        /dev/md/sbjkset/rdsk/d124      /var/mqm        ufs     2       no      logging
root@wgv8901 #
另外一個節(jié)點(diǎn)的vfstab和主節(jié)點(diǎn)的vfstab是一樣的，現(xiàn)在要如何才能把pong值改��？

東方蜘蛛

# scrgadm -c -g xxxx -y Pingpong_interval=xxx
或者在WEB界面改

yulemi

東方蜘蛛 發(fā)表于 2013-07-01 15:29
# scrgadm -c -g xxxx -y Pingpong_interval=xxx
或者在WEB界面改

我是這樣改的，可是報錯

東方蜘蛛

yulemi 發(fā)表于 2013-07-01 16:03
我是這樣改的，可是報錯

報錯不是很明顯了嗎～～～

wgv8902 - Entry in /etc/vfstab for file system mount point /var/mqm is incorrect: File system mount point should specify 'mount at boot' as 'no'.

yulemi

東方蜘蛛 發(fā)表于 2013-07-01 16:06
報錯不是很明顯了嗎～～～

wgv8902 - Entry in /etc/vfstab for file system mount point /v ...

我看了它是no的，不過之前是注釋掉的，我現(xiàn)在把注釋去掉了，還是一樣報錯。