請教一個問題：F4800宕機原因分析

dawningzh

我這兒有一臺F4800，近期經(jīng)常宕機，有些規(guī)律，只要執(zhí)行一個Q3程序，系統(tǒng)就會宕機，讓人頭痛�？戳丝慈罩荆�看不太懂，好象是線程之間互相死鎖，也不知道是硬件故障還是程序問題，更不知道怎么解決。請大家?guī)兔η魄啤Ｈ罩驹斠娤旅妫?br /> ......
Feb  2 17:51:18 sxmd6 unix: [ID 836849 kern.notice]
Feb  2 17:51:18 sxmd6 ^Mpanic[cpu11]/thread=300122826e0:
Feb  2 17:51:18 sxmd6 unix: [ID 783603 kern.notice] Deadlock: cycle in blocking chain
Feb  2 17:51:18 sxmd6 unix: [ID 100000 kern.notice]
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a102191070 genunix:turnstile_block+360 (300122826e0, 0, 10420808, 30011744030, 0, 300137b7d10)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 000000001004980c 0000030011496420 00000300124dd680 0000000000000000
Feb  2 17:51:18 sxmd6   %l4-7: 0000000010420808 0000000000000000 00000300117f5140 00000300124dd680
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a102191140 unix:mutex_vector_enter+3b8 (300137b7d10, 104266d0, 104647e0, 300000c0000, 0, 1d2)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: ffff14f051a593d0 00000300137b7d10 000003000574aab0 0000000000000640
Feb  2 17:51:18 sxmd6   %l4-7: 00000300124dd680 000003000574aab0 00000300117f5141 000002a102191ba0
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a102191200 genunix:set_qend+c (300137b7d10, 300137b7ca8, 30012e2d128, 30012e2d1a8, 30012e2d128, 300137de4d0)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 0000000000000000 0000000000000000 000003001389d550 0000000000007fff
Feb  2 17:51:18 sxmd6   %l4-7: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a1021912b0 genunix:removeq+230 (0, 300137de310, 300137de4d0, 30012e2d128, 3000bf60810, 300137de3f0)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 0000000000000001 000003000a9e1378 0000000000010000 0000000000000000
Feb  2 17:51:18 sxmd6   %l4-7: 000000007846e908 0000000000000000 0000000000000000 0000000000000000
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a102191360 genunix:link_rempassthru+10 (300137de310, 300122826e0, 20, 3000a2b2cd0, 30010a556e0, 10)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 0000000000000040 0000030010a55658 000002a102191558 0000000000000040
Feb  2 17:51:18 sxmd6   %l4-7: 0000030010a556d8 0000000000003a98 000003000a248d80 0000000000000000
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a102191410 genunix:mlink+40c (5000, 0, 104f04e0, ffffffff, 300106f7d88, 2a102191aec)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 0000030010a556d8 0000030008c57b80 0000030010a55658 000000000000530c
Feb  2 17:51:18 sxmd6   %l4-7: 0000000000000005 0000030012e2d128 00000300137b7ca8 0000000000002000
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a102191570 genunix:strioctl+2cc4 (30010a556d8, 0, 2a102191aec, 3000a2b2cd0, 1, 300135b2910)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 0000000000000005 0000030008c57b80 000000000000530c 0000000000202003
Feb  2 17:51:18 sxmd6   %l4-7: 0000030010a556d8 0000030010a55658 000002a102191aec 00000300135b2830
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a1021918e0 specfs:spec_ioctl+8c (30008c57b80, 530c, 5, 202003, 3000a2b2cd0, 2a102191aec)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 0000004482001204 0000000000000016 0000000000000000 0000000000000004
Feb  2 17:51:18 sxmd6   %l4-7: 0000000000000000 0000030005968000 00000300122826e0 000002a102191ba0
Feb  2 17:51:18 sxmd6 genunix: [ID 723222 kern.notice] 000002a1021919a0 genunix:ioctl+1e4 (300138737e0, 4, 5, ffffffff7eeb6f60, 530c, 0)
Feb  2 17:51:18 sxmd6 genunix: [ID 179002 kern.notice]   %l0-3: 0000000010160600 0000000000000000 0000000000000000 0000000000000000
Feb  2 17:51:18 sxmd6   %l4-7: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
Feb  2 17:51:18 sxmd6 unix: [ID 100000 kern.notice]
Feb  2 17:51:18 sxmd6 genunix: [ID 672855 kern.notice] syncing file systems...
Feb  2 17:51:21 sxmd6 genunix: [ID 733762 kern.notice]  162
Feb  2 17:51:23 sxmd6 genunix: [ID 733762 kern.notice]  110
Feb  2 17:51:25 sxmd6 genunix: [ID 733762 kern.notice]  83
Feb  2 17:52:00 sxmd6 last message repeated 20 times
Feb  2 17:52:01 sxmd6 genunix: [ID 622722 kern.notice]  done (not all i/o completed)
Feb  2 17:52:02 sxmd6 genunix: [ID 353387 kern.notice] dumping to /dev/dsk/c0t0d0s1, offset 859111424
Feb  2 17:53:02 sxmd6 genunix: [ID 409368 kern.notice] ^M100% done: 142029 pages dumped, compression ratio 3.02,
Feb  2 17:53:02 sxmd6 genunix: [ID 851671 kern.notice] dump succeeded
Feb  2 17:56:53 sxmd6 genunix: [ID 540533 kern.notice] ^MSunOS Release 5.8 Version Generic_117350-16 64-bit
Feb  2 17:56:53 sxmd6 genunix: [ID 913632 kern.notice] Copyright 1983-2003 Sun Microsystems, Inc.  All rights reserved.
Feb  2 17:56:53 sxmd6 genunix: [ID 678236 kern.info] Ethernet address = 0:3:ba:4:1e:ec
......

謝謝�。�
祝DJ新年快樂！

abcU

建議先作系統(tǒng)最大化監(jiān)測 來確定是不是硬件問題，假如不是硬件問題，那么應(yīng)該考慮是不是軟件兼容性問題？

ga0feng

運行這個程序，立刻察看 mpstat 5的輸出的第smtx（這個就是死鎖）項，看是不是很快的增大？
一般的如果機子的內(nèi)存和cpu很強的話，這個值肯定到30是沒關(guān)系的。像我的Fire v120,一般的時候smtx都是0。

然后有一個命令是lockstat，你man一下，我也不太會用。


如果問題以后解決了，無論如何請告訴我們解決的方法，讓大家都學(xué)習(xí)一下。

sophia_dotty

1\如果系統(tǒng)每次都報cpu11出現(xiàn)panic錯,那就是此cpu 硬件有問題,可以考慮把他拔出來，讓其他幾個CPU工作,看是否還出問題。
2\ 如果每次都是在Q3運行的時候出問題，那應(yīng)該是軟件的問題，
3\如果以前運行Q3都沒問題的話,那還是要懷疑是CPU硬件問題.

Philmoon

也可能是內(nèi)存。尤其是供CPU11的內(nèi)存條�？纯磎essages中有沒有關(guān)于內(nèi)存的報錯記錄。

sophia_dotty

樓上的,會分析crash 文件嗎?

june_cai

把補丁打到最新的看看，另外，如果以前運行該程序沒有問題的話，那么就應(yīng)該是硬件的問題。

lisuit

99% is a software problem. try to update your patch level.
If problem still exists, let program developer to debug your application.

dawningzh

重新安裝了Q3軟件，然后再運行，系統(tǒng)不再crash。前幾次crash時，日志顯示都是因為CPU11的線程死鎖，我想找個機會，裝個CAT軟件再分析一下前幾次的core文件。