RAC中三臺(tái)中的兩臺(tái)會(huì)重啟

jsnjycy

主機(jī)是三臺(tái)HP DL580 G5都是雙光纖卡連接到兩臺(tái)IBM的光交1和2，兩臺(tái)光交分別連接到一臺(tái)DS4700的AB兩個(gè)控制器 ，也就是說光交1連控制器A，光交2連控制器B
多路徑裝的是“l(fā)inuxrdac-09.03.0B05.0331”
操作系統(tǒng)是“CentOS release 4.8 (Final)”
oracle 版本如下

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi
PL/SQL Release 10.2.0.4.0 - Production
CORE        10.2.0.4.0        Production
TNS for Linux: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production


看了一下/var/log/messages
有如下信息
Jul 26 07:22:11 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 07:22:11 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 07:22:11 oraclerac3 kernel:
Jul 26 07:32:11 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 07:32:11 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 07:32:11 oraclerac3 kernel:
Jul 26 07:42:12 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 07:42:12 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 07:42:12 oraclerac3 kernel:
Jul 26 07:52:12 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 07:52:12 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 07:52:12 oraclerac3 kernel:
Jul 26 08:02:11 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 08:02:11 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 08:02:11 oraclerac3 kernel:
Jul 26 08:12:14 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 08:12:14 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 08:12:14 oraclerac3 kernel:
Jul 26 08:22:12 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 08:22:12 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 08:22:12 oraclerac3 kernel:
Jul 26 08:32:12 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 08:32:12 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 08:32:12 oraclerac3 kernel:
Jul 26 08:42:14 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 08:42:14 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 08:42:14 oraclerac3 kernel:
Jul 26 08:52:10 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 08:52:10 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 08:52:10 oraclerac3 kernel:
Jul 26 09:02:14 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 09:02:14 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 09:02:14 oraclerac3 kernel:
Jul 26 09:12:12 oraclerac3 kernel:       blocks= 286677119 block_size= 512
Jul 26 09:12:12 oraclerac3 kernel:       heads= 255, sectors= 32, cylinders= 35132
Jul 26 09:12:12 oraclerac3 kernel:
Jul 26 09:13:30 oraclerac3 kernel: 122 [RAIDarray.mpp]DS4700_01:0:0:1 Controller IO time expired. Delta 304 secs
Jul 26 09:13:30 oraclerac3 kernel: 497 [RAIDarray.mpp]DS4700_01:0:0:1 Failed controller to 1. retry. vcmnd SN 27916509 pdev H4:C0:T0:L1 0x00/0x00/0x00 0x0600
0000 mpp_status:8
Jul 26 09:13:30 oraclerac3 kernel: 10 [RAIDarray.mpp]DS4700_01:1 Failover command issued
Jul 26 09:13:31 oraclerac3 su(pam_unix)[25741]: session closed for user oracle
Jul 26 09:17:23 oraclerac3 syslogd 1.4.1: restart.
Jul 26 09:17:23 oraclerac3 syslog: syslogd 啟動(dòng) succeeded
Jul 26 09:17:23 oraclerac3 kernel: klogd 1.4.1, log source = /proc/kmsg started.
Jul 26 09:17:23 oraclerac3 kernel: Bootdata ok (command line is ro root=/dev/VolGroup00/LogVol00 rhgb quiet)
Jul 26 09:17:23 oraclerac3 kernel: Linux version 2.6.9-89.ELlargesmp (mockbuild@builder10.centos.org) (gcc version 3.4.6 20060404 (Red Hat 3.4.6-11)) #1 SMP
Mon Jun 22 12:46:58 EDT 2009

jsnjycy

是主機(jī)連接盤柜的問題嗎，但是盤柜好像沒有報(bào)錯(cuò)

Date/Time: 12-7-26 9:14:26
Sequence number: 359628
Event type: 2023
Event category: Internal
Priority: Informational
Description: Media scan (scrub) completed
Event specific codes: 0/0/0
Component type: Logical Drive
Component location: Logical Drive 1
Logged by: Controller in slot B

Raw data:
4d 45 4c 48 03 00 00 00 cc 7c 05 00 00 00 00 00
23 20 4d 00 f2 99 10 50 00 00 00 10 00 00 00 00
00 00 00 00 04 00 00 00 0d 00 00 00 0d 00 00 00
02 00 00 00 00 31 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 01 01 14 00 00 00 10 00 18 06
05 00 00 00 09 00 00 00 00 00 00 00 00 00 00 00



Date/Time: 12-7-26 9:14:25
Sequence number: 359627
Event type: 210A
Event category: Internal
Priority: Informational
Description: Controller cache not enabled or was internally disabled
Event specific codes: 0/0/0
Component type: Controller
Component location: Enclosure 85, Slot 2
Logged by: Controller in slot B

Raw data:
4d 45 4c 48 03 00 00 00 cb 7c 05 00 00 00 00 00
0a 21 48 00 f1 99 10 50 00 00 00 00 00 00 00 00
00 00 00 00 04 00 00 00 22 00 00 00 22 00 00 00
08 00 00 00 55 00 00 00 02 00 00 00 01 00 00 00
0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 01 00 00 00 00 00


Date/Time: 12-7-26 9:13:55
Sequence number: 359626
Event type: 2024
Event category: Internal
Priority: Informational
Description: Media scan (scrub) resumed
Event specific codes: 0/0/0
Component type: Logical Drive
Component location: Logical Drive 1
Logged by: Controller in slot B

Raw data:
4d 45 4c 48 03 00 00 00 ca 7c 05 00 00 00 00 00
24 20 4d 00 d3 99 10 50 00 00 00 10 00 00 00 00
00 00 00 00 04 00 00 00 0d 00 00 00 0d 00 00 00
02 00 00 00 00 31 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 01 01 10 00 00 00 0c 00 13 06
01 00 00 00 00 00 00 00 00 e0 4e 49


Date/Time: 12-7-26 9:13:55
Sequence number: 359625
Event type: 300D
Event category: Command
Priority: Informational
Description: Mode select for redundant controller page 2C received
Event specific codes: 0/0/0
Component type: Controller
Component location: Enclosure 85, Slot 2
Logged by: Controller in slot B

Raw data:
4d 45 4c 48 03 00 00 00 c9 7c 05 00 00 00 00 00
0d 30 38 00 d3 99 10 50 00 00 00 00 06 00 00 00
00 00 00 00 03 00 00 00 22 00 00 00 22 00 00 00
08 00 00 00 55 00 00 00 02 00 00 00 01 00 00 00
0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 01 04 7c 00 00 00 20 00 1a 06
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20 00 1a 86 00 01 00 00 14 02 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 20 00 1a 86 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 0a 00 1a 86 00 00 00 00
00 00 00 00 00 00 00 00


Date/Time: 12-7-26 9:13:53
Sequence number: 359624
Event type: 210A
Event category: Internal
Priority: Informational
Description: Controller cache not enabled or was internally disabled
Event specific codes: 0/0/0
Component type: Controller
Component location: Enclosure 85, Slot 2
Logged by: Controller in slot B

Raw data:
4d 45 4c 48 03 00 00 00 c8 7c 05 00 00 00 00 00
0a 21 48 00 d1 99 10 50 00 00 00 00 00 00 00 00
00 00 00 00 04 00 00 00 22 00 00 00 22 00 00 00
08 00 00 00 55 00 00 00 02 00 00 00 01 00 00 00
0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 01 00 00 00 00 00


Date/Time: 12-7-26 9:14:26
Sequence number: 359623
Event type: 2024
Event category: Internal
Priority: Informational
Description: Media scan (scrub) resumed
Event specific codes: 0/0/0
Component type: Logical Drive
Component location: Logical Drive 1
Logged by: Controller in slot A

Raw data:
4d 45 4c 48 03 00 00 00 c7 7c 05 00 00 00 00 00
24 20 4d 00 f2 99 10 50 00 00 00 10 00 00 00 00
00 00 00 00 04 00 00 00 0d 00 00 00 0d 00 00 00
02 00 00 00 00 31 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 00 01 10 00 00 00 0c 00 13 06
01 00 00 00 00 00 00 00 00 40 4f 49


Date/Time: 12-7-26 9:14:26
Sequence number: 359622
Event type: 300D
Event category: Command
Priority: Informational
Description: Mode select for redundant controller page 2C received
Event specific codes: 0/0/0
Component type: Controller
Component location: Enclosure 85, Slot 1
Logged by: Controller in slot A

Raw data:
4d 45 4c 48 03 00 00 00 c6 7c 05 00 00 00 00 00
0d 30 38 00 f2 99 10 50 00 00 00 00 07 00 00 00
04 00 00 00 03 00 00 00 22 00 00 00 22 00 00 00
08 00 00 00 55 00 00 00 01 00 00 00 01 00 00 00
0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 00 04 7c 00 00 00 20 00 1a 06
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20 00 1a 86 00 02 00 00 14 00 81 81 81 81 81 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 20 00 1a 86 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 0a 00 1a 86 00 00 00 00
00 00 00 00 00 00 00 00


Date/Time: 12-7-26 9:14:25
Sequence number: 359621
Event type: 210A
Event category: Internal
Priority: Informational
Description: Controller cache not enabled or was internally disabled
Event specific codes: 0/0/0
Component type: Controller
Component location: Enclosure 85, Slot 1
Logged by: Controller in slot A

Raw data:
4d 45 4c 48 03 00 00 00 c5 7c 05 00 00 00 00 00
0a 21 48 00 f1 99 10 50 00 00 00 00 00 00 00 00
00 00 00 00 04 00 00 00 22 00 00 00 22 00 00 00
08 00 00 00 55 00 00 00 01 00 00 00 01 00 00 00
0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 00 00 00 00 00 00


Date/Time: 12-7-26 9:13:55
Sequence number: 359620
Event type: 2023
Event category: Internal
Priority: Informational
Description: Media scan (scrub) completed
Event specific codes: 0/0/0
Component type: Logical Drive
Component location: Logical Drive 1
Logged by: Controller in slot A

Raw data:
4d 45 4c 48 03 00 00 00 c4 7c 05 00 00 00 00 00
23 20 4d 00 d3 99 10 50 00 00 00 10 00 00 00 00
00 00 00 00 04 00 00 00 0d 00 00 00 0d 00 00 00
02 00 00 00 00 31 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 00 01 14 00 00 00 10 00 18 06
05 00 00 00 09 00 00 00 00 00 00 00 00 00 00 00



Date/Time: 12-7-26 9:13:53
Sequence number: 359619
Event type: 210A
Event category: Internal
Priority: Informational
Description: Controller cache not enabled or was internally disabled
Event specific codes: 0/0/0
Component type: Controller
Component location: Enclosure 85, Slot 1
Logged by: Controller in slot A

Raw data:
4d 45 4c 48 03 00 00 00 c3 7c 05 00 00 00 00 00
0a 21 48 00 d1 99 10 50 00 00 00 00 00 00 00 00
00 00 00 00 04 00 00 00 22 00 00 00 22 00 00 00
08 00 00 00 55 00 00 00 01 00 00 00 01 00 00 00
0a 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 00 00 00 00 00 00 00 00 00 00 00

flutter

RAC重啟，ORACLE官方給的幾個(gè)主要原因：
1，心跳網(wǎng)絡(luò)問題
2，時(shí)間不同步
3，負(fù)載過重

重啟是/etc/init.d/init.cssd引發(fā)的，看看CRS的日志吧，會(huì)有詳細(xì)的問題記錄。
偶的RAC前幾天也反復(fù)重啟，1、2都碰到了。。。。三個(gè)節(jié)點(diǎn)莫名其妙就成了2012年、2010年、2000年