Redhat NFS 集群問題

wjemail

有一個集群的問題，搞了幾天不得要領，望高人指點。

概況：
2個節(jié)點共享一個SAN存儲，系統(tǒng)是RHEL5.3，用redhat自帶的集群軟件配置NFS服務，沒有配置GFS.

cluster.conf 如下：
<?xml version="1.0"?>
<cluster alias="NFSCluster" config_version="123" name="NFSCluster">
        <fence_daemon post_fail_delay="0" post_join_delay="3"/>
        <clusternodes>
                <clusternode name="elgar" nodeid="1" votes="1">
                </clusternode>
                <clusternode name="chopin" nodeid="2" votes="1">
                </clusternode>
        </clusternodes>
        <cman expected_votes="1" two_node="1"/>
        <fencedevices>
        </fencedevices>
        <rm log_level="7">
                <failoverdomains>
                        <failoverdomain name="NFSCDomain" ordered="0" restricted="0">
                                <failoverdomainnode name="elgar" priority="1"/>
                                <failoverdomainnode name="chopin" priority="1"/>
                        </failoverdomain>
                </failoverdomains>
                <resources>
                        <ip address="10.217.212.238" monitor_link="1"/>
                        <fs device="/dev/mapper/nfscserver-proj" force_fsck="0" force_unmount="1" fsid="53751" fstype="ext3" mountpoint="/proj" name="fs-proj" options="usrquota,grpquota" self_fence="1"/>
                        <fs device="/dev/mapper/nfscserver-export" force_fsck="0" force_unmount="1" fsid="38296" fstype="ext3" mountpoint="/export" name="fs-export" options="usrquota,grpquota" self_fence="1"/>
                        <fs device="/dev/mapper/nfscserver-alpha" force_fsck="0" force_unmount="1" fsid="25724" fstype="ext3" mountpoint="/alpha" name="fs-alpha" options="usrquota,grpquota" self_fence="1"/>
                        <fs device="/dev/mapper/nfscserver-sim" force_fsck="0" force_unmount="1" fsid="47898" fstype="ext3" mountpoint="/sim" name="fs-sim" options="usrquota,grpquota" self_fence="1"/>
                        <fs device="/dev/mapper/nfscserver-cad" force_fsck="0" force_unmount="1" fsid="10950" fstype="ext3" mountpoint="/cad" name="fs-cad" options="usrquota,grpquota" self_fence="1"/>
                        <nfsexport name="NFS-E2K"/>
                        <nfsclient name="nfsclt-proj" options="rw" path="/proj" target="*"/>
                        <nfsclient name="nfsclt-sim" options="rw" path="/sim" target="*"/>
                        <nfsclient name="nfsclt-cad" options="rw" path="/cad" target="*"/>
                        <nfsclient name="nfsclt-home1" options="rw" path="/export/home1" target="*"/>
                        <nfsclient name="nfsclt-alpha" options="rw" path="/alpha" target="*"/>
                </resources>
                <service autostart="1" domain="NFSCDomain" name="NFCServices" recovery="relocate" nfslock="1">
                        <ip ref="10.217.212.238">
                                <fs ref="fs-export">
                                        <nfsexport ref="NFS-E2K">
                                                <nfsclient ref="nfsclt-home1"/>
                                        </nfsexport>
                                </fs>
                                <fs ref="fs-alpha">
                                        <nfsexport ref="NFS-E2K">
                                                <nfsclient ref="nfsclt-alpha"/>
                                        </nfsexport>
                                </fs>
                                <fs ref="fs-sim">
                                        <nfsexport ref="NFS-E2K">
                                                <nfsclient ref="nfsclt-sim"/>
                                        </nfsexport>
                                </fs>
                                <fs ref="fs-cad">
                                        <nfsexport ref="NFS-E2K">
                                                <nfsclient ref="nfsclt-cad"/>
                                        </nfsexport>
                                </fs>
                                <fs ref="fs-proj">
                                        <nfsexport ref="NFS-E2K">
                                                <nfsclient ref="nfsclt-proj"/>
                                        </nfsexport>
                                </fs>
                        </ip>
                </service>
        </rm>
</cluster>

當self_fence設為0時，不能正常切換。相應的log如下 （正在重啟的active節(jié)點）：
Mar 26 20:25:41 chopin rpc.statd[11846]: unlink (/tmp/statd-apollo.z11831/sm.bak/10.217.212.222): Permission denied
Mar 26 20:25:41 chopin rpc.statd[11846]: unlink (/tmp/statd-apollo.z11831/sm.bak/10.217.212.254): Permission denied
Mar 26 20:25:41 chopin rpc.statd[11846]: unlink (/tmp/statd-apollo.z11831/sm.bak/10.217.212.207): Permission denied
Mar 26 20:25:43 chopin rpc.statd[11846]: Caught signal 15, un-registering and exiting.
Mar 26 20:25:43 chopin clurgmgrd: [5845]: <err> 'umount /export' failed, error=0

當self_fence設為1時，也不能正常切換。相應的log如下 （正在重啟的active節(jié)點）：
Mar 26 20:25:41 chopin rpc.statd[11846]: unlink (/tmp/statd-apollo.z11831/sm.bak/10.217.212.254): Permission denied
Mar 26 20:25:41 chopin rpc.statd[11846]: unlink (/tmp/statd-apollo.z11831/sm.bak/10.217.212.207): Permission denied
Mar 26 20:25:43 chopin rpc.statd[11846]: Caught signal 15, un-registering and exiting.
Mar 26 20:25:43 chopin clurgmgrd: [5845]: <err> 'umount /export' failed, error=0
Mar 26 20:25:43 chopin clurgmgrd: [5845]: <alert> umount failed - REBOOTING
此時，備份節(jié)點的log如下：
Mar 29 12:26:47 elgar kernel: dlm: closing connection to node 2
Mar 29 12:26:47 elgar fenced[5086]: chopin not a cluster member after 0 sec post_fail_delay
Mar 29 12:26:47 elgar openais[5066]: [CLM  ]         r(0) ip(10.217.212.236)  
Mar 29 12:26:47 elgar clurgmgrd[6647]: <info> State change: chopin DOWN
Mar 29 12:26:47 elgar fenced[5086]: fencing node "chopin"
Mar 29 12:26:47 elgar openais[5066]: [CLM  ] Members Left:
Mar 29 12:26:47 elgar fenced[5086]: fence "chopin" failed
Mar 29 12:26:47 elgar openais[5066]: [CLM  ]         r(0) ip(10.217.212.237)  
Mar 29 12:26:47 elgar openais[5066]: [CLM  ] Members Joined:
Mar 29 12:26:47 elgar openais[5066]: [CLM  ] CLM CONFIGURATION CHANGE
Mar 29 12:26:47 elgar openais[5066]: [CLM  ] New Configuration:
Mar 29 12:26:47 elgar openais[5066]: [CLM  ]         r(0) ip(10.217.212.236)  
Mar 29 12:26:47 elgar openais[5066]: [CLM  ] Members Left:
Mar 29 12:26:47 elgar openais[5066]: [CLM  ] Members Joined:
Mar 29 12:26:47 elgar openais[5066]: [SYNC ] This node is within the primary component and will provide service.
Mar 29 12:26:47 elgar openais[5066]: [TOTEM] entering OPERATIONAL state.
Mar 29 12:26:47 elgar openais[5066]: [CLM  ] got nodejoin message 10.217.212.236
Mar 29 12:26:47 elgar openais[5066]: [CPG  ] got joinlist message from node 1
Mar 29 12:26:48 elgar kernel: bnx2: eth1 NIC Copper Link is Up, 1000 Mbps full duplex, receive & transmit flow control ON
Mar 29 12:26:52 elgar fenced[5086]: fencing node "chopin"
Mar 29 12:26:52 elgar fenced[5086]: fence "chopin" failed
Mar 29 12:26:53 elgar mountd[5610]: export request from 10.217.212.230 failed.
Mar 29 12:26:53 elgar last message repeated 3 times
Mar 29 12:26:57 elgar fenced[5086]: fencing node "chopin"
Mar 29 12:26:57 elgar fenced[5086]: fence "chopin" failed
Mar 29 12:27:02 elgar fenced[5086]: fencing node "chopin"
Mar 29 12:27:02 elgar fenced[5086]: fence "chopin" failed

不知道能否告知：
1. 為什么有些filesystem不能卸載？有什么辦法去卸載嗎？
2. 為什么備份節(jié)點嘗試著去fencing？這個情況不論我們有沒有配置fencing device都會出現(xiàn)。

謝謝