碰到一個死鎖的情況，大俠們幫看看我的分析思路對嗎？怎么越分析越糊涂

FlankerSky

新手，沒有深入的分析過死鎖，以前碰到過類似嵌套的那種死鎖，能看出來，這次這個真是死活都看不出來，下面是我的分析，各位大俠指點下：

1.基本情況描述：
紅色的那部分是一個內(nèi)核線程(kks_tx)，它的作用是處理上層下來的數(shù)據(jù)包(上層下來的包是在netfilter的local_out處被我加到隊列里去的)，上層應用發(fā)下來的數(shù)據(jù)包，出于業(yè)務(wù)需要，有可能會被我寫的內(nèi)核模塊直接響應，所以下面會調(diào)用netif_receive_skb給上層應用發(fā)送數(shù)據(jù)包。在tcp的入口tcp_v4_rcv這里是有一個自旋鎖的，kks_tx在獲得自旋鎖后，繼續(xù)執(zhí)行，但是接下來我就看不懂了(紅色上面的那部分)，是被e1000e的中斷給中斷掉了嗎？然后接著往下執(zhí)行到tcp_v4_rcv，就會造成死鎖？？？
2.但是我的代碼是跑在vmware里的,不管我把vmware設(shè)置成單核還是多核都會出現(xiàn)這種情況，根據(jù)LDD3中說的，在非smp上，自旋鎖不是沒有任何作用嗎？
而且在LDD3中也說，自旋鎖期間是禁止本地CPU中斷的啊，那這樣就說不通了啊。

我是越分析越糊涂，大俠們幫我看看我的思路對不對啊?是不是方向錯了？謝謝了！




[ 3182.546342] BUG: soft lockup - CPU#0 stuck for 22s! [KksTx:2742]
[ 3182.547942] Modules linked in: kksfilter(O) e1000e(O) vmhgfs(O) vsock(O) acpiphp vmwgfx ttm drm snd_ens1371 gameport snd_ac97_codec ac97_bus snd_pcm vmw_balloon snd_seq_midi snd_rawmidi snd_seq_midi_event psmouse snd_seq serio_raw snd_timer snd_seq_device snd joydev soundcore snd_page_alloc bnep rfcomm bluetooth parport_pc ppdev vmci(O) shpchp mac_hid i2c_piix4 lp parport usbhid hid floppy mptspi mptscsih mptbase vmxnet(O) vmw_pvscsi vmxnet3 [last unloaded: e1000e]
[ 3182.560267] irq event stamp: 0
[ 3182.561060] hardirqs last  enabled at (0): [<  (null)>]   (null)
[ 3182.562649] hardirqs last disabled at (0): [<c1058564>] copy_process+0x484/0x1170
[ 3182.564637] softirqs last  enabled at (0): [<c1058564>] copy_process+0x484/0x1170
[ 3182.566563] softirqs last disabled at (0): [<  (null)>]   (null)
[ 3182.568115] Modules linked in: kksfilter(O) e1000e(O) vmhgfs(O) vsock(O) acpiphp vmwgfx ttm drm snd_ens1371 gameport snd_ac97_codec ac97_bus snd_pcm vmw_balloon snd_seq_midi snd_rawmidi snd_seq_midi_event psmouse snd_seq serio_raw snd_timer snd_seq_device snd joydev soundcore snd_page_alloc bnep rfcomm bluetooth parport_pc ppdev vmci(O) shpchp mac_hid i2c_piix4 lp parport usbhid hid floppy mptspi mptscsih mptbase vmxnet(O) vmw_pvscsi vmxnet3 [last unloaded: e1000e]
[ 3182.580028]
[ 3182.580431] Pid: 2742, comm: KksTx Tainted: G           O 3.2.6 #18 VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform
[ 3182.583631] EIP: 0060:[<c12ba6df>] EFLAGS: 00000282 CPU: 0
[ 3182.585043] EIP is at delay_tsc+0x1f/0x70
[ 3182.586068] EAX: 7170b643 EBX: f0a98d34 ECX: f0a98d34 EDX: 000007db
[ 3182.587670] ESI: 00000000 EDI: 00000001 EBP: f680dbb4 ESP: f680dba4
[ 3182.589252]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
[ 3182.590632] Process KksTx (pid: 2742, ti=f680c000 task=e604ea80 task.ti=f4de6000)
[ 3182.592516] Stack:
[ 3182.593046]  7170b5d7 f0a98d34 280c08d0 00000000 f680dbbc c12ba62e f680dbdc c12c150b
[ 3182.595284]  00000001 a087b7e8 00000001 f0a98d34 c1e896c0 f0a98d00 f680dbf8 c159eefd
[ 3182.597516]  00000000 00000002 00000000 c14f6f2a f4c91600 f680dc4c c14f6f2a 00000050
[ 3182.599751] Call Trace:
[ 3182.600390]  [<c12ba62e>] __delay+0xe/0x10
[ 3182.601454]  [<c12c150b>] do_raw_spin_lock+0xab/0xf0
[ 3182.602706]  [<c159eefd>] _raw_spin_lock_nested+0x3d/0x50
[ 3182.604087]  [<c14f6f2a>] ? tcp_v4_rcv+0x76a/0xaa0
[ 3182.605285]  [<c14f6f2a>] tcp_v4_rcv+0x76a/0xaa0
[ 3182.606451]  [<c14d4b8f>] ip_local_deliver_finish+0xdf/0x380
[ 3182.607866]  [<c14d4aec>] ? ip_local_deliver_finish+0x3c/0x380
[ 3182.609318]  [<c14d4fbf>] ip_local_deliver+0x7f/0x90
[ 3182.610578]  [<c14d46be>] ip_rcv_finish+0x16e/0x560
[ 3182.611804]  [<c14d522a>] ip_rcv+0x25a/0x320
[ 3182.612880]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.614193]  [<c14a63d2>] __netif_receive_skb+0x4c2/0x570
[ 3182.615547]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.616918]  [<c14a707b>] netif_receive_skb+0xcb/0xe0
[ 3182.618187]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.619517]  [<f848e33e>] SendSkb2Middle+0x162/0x176 [kksfilter]
[ 3182.621019]  [<f848f7ac>] KksNatHandler+0x612/0x86d [kksfilter]
[ 3182.622516]  [<c12ba291>] ? sscanf+0x11/0x14
[ 3182.623592]  [<f84903a4>] ? kks_inet_addr+0x37/0x4f [kksfilter]
[ 3182.625069]  [<f848fad6>] kks_rx+0x4e/0xb9 [kksfilter]
[ 3182.626362]  [<f848fc52>] hook_local_in+0x111/0x1d7 [kksfilter]
[ 3182.627838]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.629145]  [<c14cd663>] nf_iterate+0x63/0x90
[ 3182.630274]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.631582]  [<c14cd722>] nf_hook_slow+0x92/0x150
[ 3182.632761]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.634069]  [<c14d521a>] ip_rcv+0x24a/0x320
[ 3182.635155]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.636465]  [<c14a63d2>] __netif_receive_skb+0x4c2/0x570
[ 3182.638624]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.639996]  [<c14a707b>] netif_receive_skb+0xcb/0xe0
[ 3182.641250]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.642564]  [<c14a71a7>] napi_skb_finish+0x37/0x50
[ 3182.643781]  [<c14a7701>] napi_gro_receive+0xa1/0xb0
[ 3182.645060]  [<f9d41726>] e1000_receive_skb+0xc6/0x170 [e1000e]
[ 3182.646577]  [<c149a19d>] ? __kfree_skb+0x3d/0x90
[ 3182.647779]  [<f9d43166>] e1000_clean_rx_irq+0x206/0x340 [e1000e]
[ 3182.649419]  [<f9d49b74>] e1000e_poll+0x64/0x2d0 [e1000e]
[ 3182.650768]  [<c14a790d>] net_rx_action+0x12d/0x240
[ 3182.651988]  [<c1060e30>] ? local_bh_enable+0xd0/0xd0
[ 3182.653245]  [<c1060ec9>] __do_softirq+0x99/0x1d0
[ 3182.654429]  [<c1060e30>] ? local_bh_enable+0xd0/0xd0
[ 3182.655688]  <IRQ>
[ 3182.656264]  [<c106123e>] ? irq_exit+0x7e/0xa0
[ 3182.657406]  [<c15a6f2b>] ? do_IRQ+0x4b/0xc0
[ 3182.658496]  [<c15a6d75>] ? common_interrupt+0x35/0x3c
[ 3182.659796]  [<c14f00d8>] ? tcp_connect+0x318/0x490
[ 3182.661011]  [<c14e6841>] ? tcp_validate_incoming+0x71/0x340
[ 3182.662421]  [<c14f8010>] ? tcp_check_req+0x260/0x4b0
[ 3182.663676]  [<c14ecaf7>] ? tcp_rcv_state_process+0x47/0xb80
[ 3182.665078]  [<c14f8093>] ? tcp_check_req+0x2e3/0x4b0
[ 3182.666338]  [<c14f7ced>] ? tcp_child_process+0x8d/0x150
[ 3182.667660]  [<c14f5d8a>] ? tcp_v4_do_rcv+0x2aa/0x3c0
[ 3182.668922]  [<c12c149b>] ? do_raw_spin_lock+0x3b/0xf0
[ 3182.670202]  [<c159eefd>] ? _raw_spin_lock_nested+0x3d/0x50
[ 3182.671603]  [<c14f6f4a>] ? tcp_v4_rcv+0x78a/0xaa0
[ 3182.672800]  [<c14d4b8f>] ? ip_local_deliver_finish+0xdf/0x380
[ 3182.674263]  [<c14d4aec>] ? ip_local_deliver_finish+0x3c/0x380
[ 3182.675713]  [<c14d4fbf>] ? ip_local_deliver+0x7f/0x90
[ 3182.676991]  [<c14d46be>] ? ip_rcv_finish+0x16e/0x560
[ 3182.678259]  [<c14d522a>] ? ip_rcv+0x25a/0x320
[ 3182.679371]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.680670]  [<c14a63d2>] ? __netif_receive_skb+0x4c2/0x570
[ 3182.682057]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.683436]  [<c14a707b>] ? netif_receive_skb+0xcb/0xe0
[ 3182.684730]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.686027]  [<f848e33e>] ? SendSkb2Middle+0x162/0x176 [kksfilter]
[ 3182.687568]  [<f848f867>] ? KksNatHandler+0x6cd/0x86d [kksfilter]
[ 3182.689094]  [<c1095000>] ? futex_wait_requeue_pi+0x1c0/0x380
[ 3182.690526]  [<f848fa67>] ? kks_tx+0x60/0x81 [kksfilter]
[ 3182.691859]  [<f848fa07>] ? KksNatHandler+0x86d/0x86d [kksfilter]
[ 3182.693386]  [<c1079df8>] ? kthread+0x78/0x80
[ 3182.694488]  [<c1079d80>] ? __init_kthread_worker+0x60/0x60
[ 3182.695876]  [<c15a6d82>] ? kernel_thread_helper+0x6/0x10
[ 3182.697220] Code: c3 8d 74 26 00 8d bc 27 00 00 00 00 55 89 e5 57 56 53 83 ec 04 3e 8d 74 26 00 64 8b 35 ec 5e 91 c1 89 c7 8d 76 00 0f ae e8 0f 31 <8d> 74 26 00 89 45 f0 eb 0d f3 90 64 8b 1d ec 5e 91 c1 39 de 75
[ 3182.704103] Call Trace:
[ 3182.704733]  [<c12ba62e>] __delay+0xe/0x10
[ 3182.705761]  [<c12c150b>] do_raw_spin_lock+0xab/0xf0
[ 3182.707016]  [<c159eefd>] _raw_spin_lock_nested+0x3d/0x50
[ 3182.708359]  [<c14f6f2a>] ? tcp_v4_rcv+0x76a/0xaa0
[ 3182.709555]  [<c14f6f2a>] tcp_v4_rcv+0x76a/0xaa0
[ 3182.710715]  [<c14d4b8f>] ip_local_deliver_finish+0xdf/0x380
[ 3182.712119]  [<c14d4aec>] ? ip_local_deliver_finish+0x3c/0x380
[ 3182.713565]  [<c14d4fbf>] ip_local_deliver+0x7f/0x90
[ 3182.714807]  [<c14d46be>] ip_rcv_finish+0x16e/0x560
[ 3182.716023]  [<c14d522a>] ip_rcv+0x25a/0x320
[ 3182.717090]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.718410]  [<c14a63d2>] __netif_receive_skb+0x4c2/0x570
[ 3182.719756]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.721120]  [<c14a707b>] netif_receive_skb+0xcb/0xe0
[ 3182.722391]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.723693]  [<f848e33e>] SendSkb2Middle+0x162/0x176 [kksfilter]
[ 3182.725185]  [<f848f7ac>] KksNatHandler+0x612/0x86d [kksfilter]
[ 3182.726665]  [<c12ba291>] ? sscanf+0x11/0x14
[ 3182.727788]  [<f84903a4>] ? kks_inet_addr+0x37/0x4f [kksfilter]
[ 3182.729257]  [<f848fad6>] kks_rx+0x4e/0xb9 [kksfilter]
[ 3182.730574]  [<f848fc52>] hook_local_in+0x111/0x1d7 [kksfilter]
[ 3182.732044]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.733344]  [<c14cd663>] nf_iterate+0x63/0x90
[ 3182.734457]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.735758]  [<c14cd722>] nf_hook_slow+0x92/0x150
[ 3182.736979]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.738293]  [<c14d521a>] ip_rcv+0x24a/0x320
[ 3182.739365]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.740660]  [<c14a63d2>] __netif_receive_skb+0x4c2/0x570
[ 3182.741997]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.743364]  [<c14a707b>] netif_receive_skb+0xcb/0xe0
[ 3182.744622]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.745921]  [<c14a71a7>] napi_skb_finish+0x37/0x50
[ 3182.747146]  [<c14a7701>] napi_gro_receive+0xa1/0xb0
[ 3182.748385]  [<f9d41726>] e1000_receive_skb+0xc6/0x170 [e1000e]
[ 3182.749856]  [<c149a19d>] ? __kfree_skb+0x3d/0x90
[ 3182.751043]  [<f9d43166>] e1000_clean_rx_irq+0x206/0x340 [e1000e]
[ 3182.752559]  [<f9d49b74>] e1000e_poll+0x64/0x2d0 [e1000e]
[ 3182.753901]  [<c14a790d>] net_rx_action+0x12d/0x240
[ 3182.755126]  [<c1060e30>] ? local_bh_enable+0xd0/0xd0
[ 3182.756384]  [<c1060ec9>] __do_softirq+0x99/0x1d0
[ 3182.757555]  [<c1060e30>] ? local_bh_enable+0xd0/0xd0
[ 3182.758823]  <IRQ>  [<c106123e>] ? irq_exit+0x7e/0xa0
[ 3182.760124]  [<c15a6f2b>] ? do_IRQ+0x4b/0xc0
[ 3182.761192]  [<c15a6d75>] ? common_interrupt+0x35/0x3c
[ 3182.762478]  [<c14f00d8>] ? tcp_connect+0x318/0x490
[ 3182.763694]  [<c14e6841>] ? tcp_validate_incoming+0x71/0x340
[ 3182.765101]  [<c14f8010>] ? tcp_check_req+0x260/0x4b0
[ 3182.766370]  [<c14ecaf7>] ? tcp_rcv_state_process+0x47/0xb80
[ 3182.767775]  [<c14f8093>] ? tcp_check_req+0x2e3/0x4b0
[ 3182.769032]  [<c14f7ced>] ? tcp_child_process+0x8d/0x150
[ 3182.770366]  [<c14f5d8a>] ? tcp_v4_do_rcv+0x2aa/0x3c0
[ 3182.771626]  [<c12c149b>] ? do_raw_spin_lock+0x3b/0xf0
[ 3182.772905]  [<c159eefd>] ? _raw_spin_lock_nested+0x3d/0x50
[ 3182.774297]  [<c14f6f4a>] ? tcp_v4_rcv+0x78a/0xaa0
[ 3182.775494]  [<c14d4b8f>] ? ip_local_deliver_finish+0xdf/0x380
[ 3182.776945]  [<c14d4aec>] ? ip_local_deliver_finish+0x3c/0x380
[ 3182.778409]  [<c14d4fbf>] ? ip_local_deliver+0x7f/0x90
[ 3182.779693]  [<c14d46be>] ? ip_rcv_finish+0x16e/0x560
[ 3182.780952]  [<c14d522a>] ? ip_rcv+0x25a/0x320
[ 3182.782064]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.783379]  [<c14a63d2>] ? __netif_receive_skb+0x4c2/0x570
[ 3182.784762]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.786125]  [<c14a707b>] ? netif_receive_skb+0xcb/0xe0
[ 3182.787536]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.788836]  [<f848e33e>] ? SendSkb2Middle+0x162/0x176 [kksfilter]
[ 3182.790382]  [<f848f867>] ? KksNatHandler+0x6cd/0x86d [kksfilter]
[ 3182.791890]  [<c1095000>] ? futex_wait_requeue_pi+0x1c0/0x380
[ 3182.793313]  [<f848fa67>] ? kks_tx+0x60/0x81 [kksfilter]
[ 3182.794645]  [<f848fa07>] ? KksNatHandler+0x86d/0x86d [kksfilter]
[ 3182.796155]  [<c1079df8>] ? kthread+0x78/0x80
[ 3182.797244]  [<c1079d80>] ? __init_kthread_worker+0x60/0x60
[ 3182.798634]  [<c15a6d82>] ? kernel_thread_helper+0x6/0x10
[ 3182.799988] Kernel panic - not syncing: softlockup: hung tasks
[ 3182.801440] Pid: 2742, comm: KksTx Tainted: G           O 3.2.6 #18
[ 3182.803002] Call Trace:
[ 3182.803629]  [<c1594f9f>] ? printk+0x1d/0x1f
[ 3182.804696]  [<c1594e75>] panic+0x5c/0x169
[ 3182.805746]  [<c10bd4d1>] watchdog_timer_fn+0x151/0x160
[ 3182.807060]  [<c107e28d>] __run_hrtimer+0x6d/0x1b0
[ 3182.808255]  [<c10bd380>] ? __touch_watchdog+0x20/0x20
[ 3182.809533]  [<c107ecd5>] hrtimer_interrupt+0xe5/0x260
[ 3182.810826]  [<c107ed51>] ? hrtimer_interrupt+0x161/0x260
[ 3182.812169]  [<c15a6ff4>] smp_apic_timer_interrupt+0x54/0x88
[ 3182.813575]  [<c12bb8d8>] ? trace_hardirqs_off_thunk+0xc/0x14
[ 3182.815016]  [<c159fe02>] apic_timer_interrupt+0x36/0x3c
[ 3182.816336]  [<c12ba6df>] ? delay_tsc+0x1f/0x70
[ 3182.817465]  [<c12ba62e>] __delay+0xe/0x10
[ 3182.818497]  [<c12c150b>] do_raw_spin_lock+0xab/0xf0
[ 3182.819733]  [<c159eefd>] _raw_spin_lock_nested+0x3d/0x50
[ 3182.821081]  [<c14f6f2a>] ? tcp_v4_rcv+0x76a/0xaa0
[ 3182.822287]  [<c14f6f2a>] tcp_v4_rcv+0x76a/0xaa0
[ 3182.823440]  [<c14d4b8f>] ip_local_deliver_finish+0xdf/0x380
[ 3182.824856]  [<c14d4aec>] ? ip_local_deliver_finish+0x3c/0x380
[ 3182.826314]  [<c14d4fbf>] ip_local_deliver+0x7f/0x90
[ 3182.827549]  [<c14d46be>] ip_rcv_finish+0x16e/0x560
[ 3182.828763]  [<c14d522a>] ip_rcv+0x25a/0x320
[ 3182.829831]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.831143]  [<c14a63d2>] __netif_receive_skb+0x4c2/0x570
[ 3182.832484]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.833848]  [<c14a707b>] netif_receive_skb+0xcb/0xe0
[ 3182.835108]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.836408]  [<f848e33e>] SendSkb2Middle+0x162/0x176 [kksfilter]
[ 3182.837899]  [<f848f7ac>] KksNatHandler+0x612/0x86d [kksfilter]
[ 3182.839384]  [<c12ba291>] ? sscanf+0x11/0x14
[ 3182.840453]  [<f84903a4>] ? kks_inet_addr+0x37/0x4f [kksfilter]
[ 3182.841913]  [<f848fad6>] kks_rx+0x4e/0xb9 [kksfilter]
[ 3182.843194]  [<f848fc52>] hook_local_in+0x111/0x1d7 [kksfilter]
[ 3182.844659]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.845954]  [<c14cd663>] nf_iterate+0x63/0x90
[ 3182.847065]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.848361]  [<c14cd722>] nf_hook_slow+0x92/0x150
[ 3182.849531]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.850840]  [<c14d521a>] ip_rcv+0x24a/0x320
[ 3182.851909]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.853209]  [<c14a63d2>] __netif_receive_skb+0x4c2/0x570
[ 3182.854561]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.855921]  [<c14a707b>] netif_receive_skb+0xcb/0xe0
[ 3182.857175]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.858483]  [<c14a71a7>] napi_skb_finish+0x37/0x50
[ 3182.859698]  [<c14a7701>] napi_gro_receive+0xa1/0xb0
[ 3182.860939]  [<f9d41726>] e1000_receive_skb+0xc6/0x170 [e1000e]
[ 3182.862420]  [<c149a19d>] ? __kfree_skb+0x3d/0x90
[ 3182.863594]  [<f9d43166>] e1000_clean_rx_irq+0x206/0x340 [e1000e]
[ 3182.865114]  [<f9d49b74>] e1000e_poll+0x64/0x2d0 [e1000e]
[ 3182.866474]  [<c14a790d>] net_rx_action+0x12d/0x240
[ 3182.867692]  [<c1060e30>] ? local_bh_enable+0xd0/0xd0
[ 3182.868950]  [<c1060ec9>] __do_softirq+0x99/0x1d0
[ 3182.870123]  [<c1060e30>] ? local_bh_enable+0xd0/0xd0
[ 3182.871397]  <IRQ>  [<c106123e>] ? irq_exit+0x7e/0xa0
[ 3182.872700]  [<c15a6f2b>] ? do_IRQ+0x4b/0xc0
[ 3182.873767]  [<c15a6d75>] ? common_interrupt+0x35/0x3c
[ 3182.875060]  [<c14f00d8>] ? tcp_connect+0x318/0x490
[ 3182.876276]  [<c14e6841>] ? tcp_validate_incoming+0x71/0x340
[ 3182.877682]  [<c14f8010>] ? tcp_check_req+0x260/0x4b0
[ 3182.878945]  [<c14ecaf7>] ? tcp_rcv_state_process+0x47/0xb80
[ 3182.880351]  [<c14f8093>] ? tcp_check_req+0x2e3/0x4b0
[ 3182.881613]  [<c14f7ced>] ? tcp_child_process+0x8d/0x150
[ 3182.882945]  [<c14f5d8a>] ? tcp_v4_do_rcv+0x2aa/0x3c0
[ 3182.884199]  [<c12c149b>] ? do_raw_spin_lock+0x3b/0xf0
[ 3182.885473]  [<c159eefd>] ? _raw_spin_lock_nested+0x3d/0x50
[ 3182.886867]  [<c14f6f4a>] ? tcp_v4_rcv+0x78a/0xaa0
[ 3182.888062]  [<c14d4b8f>] ? ip_local_deliver_finish+0xdf/0x380
[ 3182.889510]  [<c14d4aec>] ? ip_local_deliver_finish+0x3c/0x380
[ 3182.890966]  [<c14d4fbf>] ? ip_local_deliver+0x7f/0x90
[ 3182.892245]  [<c14d46be>] ? ip_rcv_finish+0x16e/0x560
[ 3182.893504]  [<c14d522a>] ? ip_rcv+0x25a/0x320
[ 3182.894626]  [<c14d4550>] ? inet_del_protocol+0x30/0x30
[ 3182.895925]  [<c14a63d2>] ? __netif_receive_skb+0x4c2/0x570
[ 3182.897308]  [<c14a5fec>] ? __netif_receive_skb+0xdc/0x570
[ 3182.898680]  [<c14a707b>] ? netif_receive_skb+0xcb/0xe0
[ 3182.899980]  [<c14a6fcf>] ? netif_receive_skb+0x1f/0xe0
[ 3182.901281]  [<f848e33e>] ? SendSkb2Middle+0x162/0x176 [kksfilter]
[ 3182.902820]  [<f848f867>] ? KksNatHandler+0x6cd/0x86d [kksfilter]
[ 3182.904330]  [<c1095000>] ? futex_wait_requeue_pi+0x1c0/0x380
[ 3182.905777]  [<f848fa67>] ? kks_tx+0x60/0x81 [kksfilter]
[ 3182.907106]  [<f848fa07>] ? KksNatHandler+0x86d/0x86d [kksfilter]
[ 3182.908614]  [<c1079df8>] ? kthread+0x78/0x80
[ 3182.909703]  [<c1079d80>] ? __init_kthread_worker+0x60/0x60
[ 3182.911102]  [<c15a6d82>] ? kernel_thread_helper+0x6/0x10

FlankerSky

求指點

FlankerSky

版主，求指點迷津！

chenyu105

X86先把內(nèi)核配置上CONFIG_STACKFRAME，不然回溯可能不準。
第一句的soft lockup，應該是spinlock關(guān)了搶占后一直不退出 ，導致這個核上優(yōu)先級為99的fifo watchdog調(diào)度不到打印的回溯吧。

whaaat

應該是死鎖。
1. 調(diào)用spin_lock就不能調(diào)用帶休眠的函數(shù)了
2. 內(nèi)核線程和中斷共享數(shù)據(jù)用spin_lock_irq，和軟中斷用spin_lock_bh

humjb_1983

FlankerSky 發(fā)表于 2013-10-31 19:00
新手，沒有深入的分析過死鎖，以前碰到過類似嵌套的那種死鎖，能看出來，這次這個真是死活都看不出來，下面 ...

個人分析，做參考：
死鎖原因為：內(nèi)核線程(kks_tx)直接調(diào)用了netif_receive_skb最終在tcp_v4_rcv處需要獲取spinlock，在獲取到spinlock后，來了一個中斷(可能是時鐘中斷或是其他的)，按中斷正常處理邏輯，在中斷結(jié)束后會進入軟中斷，而網(wǎng)絡(luò)收包軟中斷最終也會調(diào)用netif_receive_skb，導致該函數(shù)重入，最終也會死等同一把spinlock，而導致死鎖，最終導致softlockup。
死鎖根本原因應該在于內(nèi)核線程(kks_tx)不應該直接調(diào)用netif_receive_skb函數(shù)(應該有一定的保護措施)，該函數(shù)應該是不可安全重入的。
另外，再解釋下樓主的疑問：
“2.但是我的代碼是跑在vmware里的,不管我把vmware設(shè)置成單核還是多核都會出現(xiàn)這種情況，根據(jù)LDD3中說的，在非smp上，自旋鎖不是沒有任何作用嗎？
而且在LDD3中也說，自旋鎖期間是禁止本地CPU中斷的啊，那這樣就說不通了啊。”
在這種情況下，單核同樣是會死鎖的，自旋鎖不起作用并不代表使用自旋鎖后不會死鎖，當臨界資源在內(nèi)核態(tài)，而且在考慮中斷的情況下，單核情況下的互斥保護應該也是需要的。
“自旋鎖期間是禁止本地CPU中斷”的說法也是不準確的，僅有部分spinlock接口禁用中斷(比如spin_lock_irq)，普通的spin_lock接口是不禁用中斷的(通常只禁用搶占)。
而內(nèi)核自身的軟中斷機制是保證了其自身是不可重入的，因為在do_softirq執(zhí)行的開始就判斷了當前是否處于中斷上下文(包括軟中斷)，如果是的話，會直接退出，所以其自身不會導致netif_receive_skb的重入。

FlankerSky

回復 4# chenyu105

謝謝，你說的回溯的配置項是CONFIG_STACKFRAME這個嗎?我怎么在.config文件里沒有找到�。�

FlankerSky

回復 5# whaaat

恩恩，謝謝。這個出錯的鎖是系統(tǒng)定義的，，所以，我想可能是函數(shù)調(diào)用不合理造成的，
   

FlankerSky

回復 6# humjb_1983

謝謝，看了你的回復頓時明白了很多，也看了下do_softirq的實現(xiàn)。
現(xiàn)在還想請教下，是不是模仿do_softirq函數(shù)，在我自己調(diào)用 netif_receive_skb的地方，檢查下是不是有中斷，沒有的話，禁用所有中斷之后再調(diào)用netif_receive_skb是不是就可以了？
   

humjb_1983

FlankerSky 發(fā)表于 2013-11-01 11:02
回復 6# humjb_1983

謝謝，看了你的回復頓時明白了很多，也看了下do_softirq的實現(xiàn)。

這樣應該可以解決重入死鎖問題，但禁用中斷會導致效率問題，同時如果禁用時間稍長，可能會導致其他問題，需要仔細考慮和驗證。
也可以嘗試不直接調(diào)用netif_receive_skb，而通過其他方式實現(xiàn)~