panic: spin lock held too long (RELENG_8 from today)
Attilio Rao
attilio at freebsd.org
Sat Sep 3 10:05:50 UTC 2011
This should be enough for someone NFS-aware to look into it.
Were you also able to get a core?
I'll try to look into it in the next days, in particular about the
softclock state.
Attilio
2011/9/3 Hiroki Sato <hrs at freebsd.org>:
> Hiroki Sato <hrs at freebsd.org> wrote
> in <20110820.105229.834911491934932780.hrs at allbsd.org>:
>
> hr> Attilio Rao <attilio at freebsd.org> wrote
> hr> in <CAJ-FndDHmwa+=LNGgU+5MK2Xmtj8kWHB10JsoytkMGEtVgncYw at mail.gmail.com>:
> hr>
> hr> at> If nobody complains about it earlier, I'll propose the patch to re@ in 8 hours.
> hr>
> hr> Running fine for 45 hours so far. Please go ahead!
>
> The NFS server was working fine with no panic for a week, but after
> that I noticed it sometimes got stuck. When it occurred, all of
> processes seemed to stop working though I was able to break it into
> ddb. I am still not sure of what triggered it, but this symptom is
> reproducible within three days now. Does anyone suffer from this?
>
> The attached file is a result of show allpcpu, show threads, ps, info
> thread, and bt for all threads. I guess all of CPUs became idle due
> to some deadlock, but how do I debug this?
>
> -- Hiroki
>
> KDB: enter: Break sequence on console
> [thread pid 11 tid 100003 ]
> Stopped at kdb_enter+0x3b: movq $0,0x6a4102(%rip)
> db> show allpcpu
> Current CPU: 1
>
> cpuid = 0
> dynamic pcpu = 0x4b3380
> curthread = 0xffffff00033fe000: pid 11 "idle: cpu0"
> curpcb = 0xffffff8000043d10
> fpcurthread = none
> idlethread = 0xffffff00033fe000: tid 100004 "idle: cpu0"
> curpmap = 0xffffffff80d00250
> tssp = 0xffffffff80d6d200
> commontssp = 0xffffffff80d6d200
> rsp0 = 0xffffff8000043d10
> gs32p = 0xffffffff80d6c038
> ldt = 0xffffffff80d6c078
> tss = 0xffffffff80d6c068
>
> cpuid = 1
> dynamic pcpu = 0xffffff807f36b380
> curthread = 0xffffff00033fe460: pid 11 "idle: cpu1"
> curpcb = 0xffffff800003ed10
> fpcurthread = none
> idlethread = 0xffffff00033fe460: tid 100003 "idle: cpu1"
> curpmap = 0xffffffff80d00250
> tssp = 0xffffffff80d6d268
> commontssp = 0xffffffff80d6d268
> rsp0 = 0xffffff800003ed10
> gs32p = 0xffffffff80d6c0a0
> ldt = 0xffffffff80d6c0e0
> tss = 0xffffffff80d6c0d0
>
> db> show threads
> 100530 (0xffffff000662f8c0) sched_switch() at sched_switch+0x102
> 100538 (0xffffff004caa28c0) sched_switch() at sched_switch+0x102
> 100524 (0xffffff004c022460) sched_switch() at sched_switch+0x102
> 100536 (0xffffff00066898c0) sched_switch() at sched_switch+0x102
> 100527 (0xffffff004c115460) sched_switch() at sched_switch+0x102
> 100238 (0xffffff00066848c0) sched_switch() at sched_switch+0x102
> 100526 (0xffffff004c1158c0) sched_switch() at sched_switch+0x102
> 100236 (0xffffff0006685460) sched_switch() at sched_switch+0x102
> 100087 (0xffffff0006284000) sched_switch() at sched_switch+0x102
> 100242 (0xffffff0006683460) sched_switch() at sched_switch+0x102
> 100516 (0xffffff002eba2000) sched_switch() at sched_switch+0x102
> 100515 (0xffffff002eba2460) sched_switch() at sched_switch+0x102
> 100514 (0xffffff002eba28c0) sched_switch() at sched_switch+0x102
> 100513 (0xffffff002eba3000) sched_switch() at sched_switch+0x102
> 100512 (0xffffff002eba3460) sched_switch() at sched_switch+0x102
> 100511 (0xffffff002eba38c0) sched_switch() at sched_switch+0x102
> 100510 (0xffffff002eba4000) sched_switch() at sched_switch+0x102
> 100509 (0xffffff002eba4460) sched_switch() at sched_switch+0x102
> 100508 (0xffffff002eba48c0) sched_switch() at sched_switch+0x102
> 100507 (0xffffff002eba5000) sched_switch() at sched_switch+0x102
> 100506 (0xffffff002eba5460) sched_switch() at sched_switch+0x102
> 100505 (0xffffff002eb96000) sched_switch() at sched_switch+0x102
> 100504 (0xffffff002eb96460) sched_switch() at sched_switch+0x102
> 100503 (0xffffff002eb968c0) sched_switch() at sched_switch+0x102
> 100502 (0xffffff002eb97000) sched_switch() at sched_switch+0x102
> 100501 (0xffffff002eb97460) sched_switch() at sched_switch+0x102
> 100500 (0xffffff002eb978c0) sched_switch() at sched_switch+0x102
> 100499 (0xffffff002eb99000) sched_switch() at sched_switch+0x102
> 100498 (0xffffff002eb99460) sched_switch() at sched_switch+0x102
> 100497 (0xffffff002eb998c0) sched_switch() at sched_switch+0x102
> 100496 (0xffffff002eb9a000) sched_switch() at sched_switch+0x102
> 100495 (0xffffff002eb9a460) sched_switch() at sched_switch+0x102
> 100494 (0xffffff002eb9a8c0) sched_switch() at sched_switch+0x102
> 100493 (0xffffff002eb9b000) sched_switch() at sched_switch+0x102
> 100492 (0xffffff002eb9b460) sched_switch() at sched_switch+0x102
> 100491 (0xffffff002eb9b8c0) sched_switch() at sched_switch+0x102
> 100490 (0xffffff002eb9d000) sched_switch() at sched_switch+0x102
> 100489 (0xffffff002eb8f8c0) sched_switch() at sched_switch+0x102
> 100488 (0xffffff002eb90000) sched_switch() at sched_switch+0x102
> 100487 (0xffffff002eb90460) sched_switch() at sched_switch+0x102
> 100486 (0xffffff002eb908c0) sched_switch() at sched_switch+0x102
> 100485 (0xffffff002eb91000) sched_switch() at sched_switch+0x102
> 100484 (0xffffff002eb91460) sched_switch() at sched_switch+0x102
> 100483 (0xffffff002eb918c0) sched_switch() at sched_switch+0x102
> 100482 (0xffffff002eb92000) sched_switch() at sched_switch+0x102
> 100481 (0xffffff002eb92460) sched_switch() at sched_switch+0x102
> 100480 (0xffffff002eb928c0) sched_switch() at sched_switch+0x102
> 100479 (0xffffff002eb93000) sched_switch() at sched_switch+0x102
> 100478 (0xffffff002eb93460) sched_switch() at sched_switch+0x102
> 100477 (0xffffff002eb938c0) sched_switch() at sched_switch+0x102
> 100476 (0xffffff002eb94000) sched_switch() at sched_switch+0x102
> 100475 (0xffffff002eb94460) sched_switch() at sched_switch+0x102
> 100474 (0xffffff002eb948c0) sched_switch() at sched_switch+0x102
> 100473 (0xffffff002eb84460) sched_switch() at sched_switch+0x102
> 100472 (0xffffff002eb848c0) sched_switch() at sched_switch+0x102
> 100471 (0xffffff002eb88000) sched_switch() at sched_switch+0x102
> 100470 (0xffffff002eb88460) sched_switch() at sched_switch+0x102
> 100469 (0xffffff002eb888c0) sched_switch() at sched_switch+0x102
> 100468 (0xffffff002eb89000) sched_switch() at sched_switch+0x102
> 100467 (0xffffff002eb89460) sched_switch() at sched_switch+0x102
> 100466 (0xffffff002eb898c0) sched_switch() at sched_switch+0x102
> 100465 (0xffffff002eb8b000) sched_switch() at sched_switch+0x102
> 100464 (0xffffff002eb8b460) sched_switch() at sched_switch+0x102
> 100463 (0xffffff002eb8b8c0) sched_switch() at sched_switch+0x102
> 100462 (0xffffff002eb8c000) sched_switch() at sched_switch+0x102
> 100461 (0xffffff002eb8c460) sched_switch() at sched_switch+0x102
> 100460 (0xffffff002eb8c8c0) sched_switch() at sched_switch+0x102
> 100459 (0xffffff002eb8f000) sched_switch() at sched_switch+0x102
> 100458 (0xffffff002eb8f460) sched_switch() at sched_switch+0x102
> 100457 (0xffffff002eb7f000) sched_switch() at sched_switch+0x102
> 100456 (0xffffff002eb7f460) sched_switch() at sched_switch+0x102
> 100455 (0xffffff002eb7f8c0) sched_switch() at sched_switch+0x102
> 100454 (0xffffff002eb80000) sched_switch() at sched_switch+0x102
> 100453 (0xffffff002eb80460) sched_switch() at sched_switch+0x102
> 100452 (0xffffff002eb808c0) sched_switch() at sched_switch+0x102
> 100451 (0xffffff002eb81000) sched_switch() at sched_switch+0x102
> 100450 (0xffffff002eb81460) sched_switch() at sched_switch+0x102
> 100449 (0xffffff002eb818c0) sched_switch() at sched_switch+0x102
> 100448 (0xffffff002eb82000) sched_switch() at sched_switch+0x102
> 100447 (0xffffff002eb82460) sched_switch() at sched_switch+0x102
> 100446 (0xffffff002eb828c0) sched_switch() at sched_switch+0x102
> 100445 (0xffffff002eb83000) sched_switch() at sched_switch+0x102
> 100444 (0xffffff002eb83460) sched_switch() at sched_switch+0x102
> 100443 (0xffffff002eb838c0) sched_switch() at sched_switch+0x102
> 100442 (0xffffff002eb84000) sched_switch() at sched_switch+0x102
> 100441 (0xffffff002eb768c0) sched_switch() at sched_switch+0x102
> 100440 (0xffffff002eb77000) sched_switch() at sched_switch+0x102
> 100439 (0xffffff002eb77460) sched_switch() at sched_switch+0x102
> 100438 (0xffffff002eb778c0) sched_switch() at sched_switch+0x102
> 100437 (0xffffff002eb78000) sched_switch() at sched_switch+0x102
> 100436 (0xffffff002eb78460) sched_switch() at sched_switch+0x102
> 100435 (0xffffff002eb788c0) sched_switch() at sched_switch+0x102
> 100434 (0xffffff002eb7a000) sched_switch() at sched_switch+0x102
> 100433 (0xffffff002eb7a460) sched_switch() at sched_switch+0x102
> 100432 (0xffffff002eb7a8c0) sched_switch() at sched_switch+0x102
> 100431 (0xffffff002eb7b000) sched_switch() at sched_switch+0x102
> 100430 (0xffffff002eb7b460) sched_switch() at sched_switch+0x102
> 100429 (0xffffff002eb7b8c0) sched_switch() at sched_switch+0x102
> 100428 (0xffffff002eb7c000) sched_switch() at sched_switch+0x102
> 100427 (0xffffff002eb7c460) sched_switch() at sched_switch+0x102
> 100426 (0xffffff002eb7c8c0) sched_switch() at sched_switch+0x102
> 100425 (0xffffff002eb71460) sched_switch() at sched_switch+0x102
> 100424 (0xffffff002eb718c0) sched_switch() at sched_switch+0x102
> 100423 (0xffffff002eb72000) sched_switch() at sched_switch+0x102
> 100422 (0xffffff002eb72460) sched_switch() at sched_switch+0x102
> 100421 (0xffffff002eb728c0) sched_switch() at sched_switch+0x102
> 100420 (0xffffff002eb73000) sched_switch() at sched_switch+0x102
> 100419 (0xffffff002eb73460) sched_switch() at sched_switch+0x102
> 100418 (0xffffff002eb738c0) sched_switch() at sched_switch+0x102
> 100417 (0xffffff002eb74000) sched_switch() at sched_switch+0x102
> 100416 (0xffffff002eb74460) sched_switch() at sched_switch+0x102
> 100415 (0xffffff002eb748c0) sched_switch() at sched_switch+0x102
> 100414 (0xffffff002eb75000) sched_switch() at sched_switch+0x102
> 100413 (0xffffff002eb75460) sched_switch() at sched_switch+0x102
> 100412 (0xffffff002eb758c0) sched_switch() at sched_switch+0x102
> 100411 (0xffffff002eb76000) sched_switch() at sched_switch+0x102
> 100410 (0xffffff002eb76460) sched_switch() at sched_switch+0x102
> 100409 (0xffffff002eb6a000) sched_switch() at sched_switch+0x102
> 100408 (0xffffff002eb6a460) sched_switch() at sched_switch+0x102
> 100407 (0xffffff002eb6a8c0) sched_switch() at sched_switch+0x102
> 100406 (0xffffff002eb6b000) sched_switch() at sched_switch+0x102
> 100405 (0xffffff002eb6b460) sched_switch() at sched_switch+0x102
> 100404 (0xffffff002eb6b8c0) sched_switch() at sched_switch+0x102
> 100403 (0xffffff002eb6d000) sched_switch() at sched_switch+0x102
> 100402 (0xffffff002eb6d460) sched_switch() at sched_switch+0x102
> 100401 (0xffffff002eb6d8c0) sched_switch() at sched_switch+0x102
> 100400 (0xffffff002eb6e000) sched_switch() at sched_switch+0x102
> 100399 (0xffffff002eb6e460) sched_switch() at sched_switch+0x102
> 100398 (0xffffff002eb6e8c0) sched_switch() at sched_switch+0x102
> 100397 (0xffffff002eb6f000) sched_switch() at sched_switch+0x102
> 100396 (0xffffff002eb6f460) sched_switch() at sched_switch+0x102
> 100395 (0xffffff002eb6f8c0) sched_switch() at sched_switch+0x102
> 100394 (0xffffff002eb71000) sched_switch() at sched_switch+0x102
> 100393 (0xffffff002eb608c0) sched_switch() at sched_switch+0x102
> 100392 (0xffffff002eb62000) sched_switch() at sched_switch+0x102
> 100391 (0xffffff002eb62460) sched_switch() at sched_switch+0x102
> 100390 (0xffffff002eb628c0) sched_switch() at sched_switch+0x102
> 100389 (0xffffff002eb63000) sched_switch() at sched_switch+0x102
> 100388 (0xffffff002eb63460) sched_switch() at sched_switch+0x102
> 100387 (0xffffff002eb638c0) sched_switch() at sched_switch+0x102
> 100386 (0xffffff002eb64000) sched_switch() at sched_switch+0x102
> 100385 (0xffffff002eb64460) sched_switch() at sched_switch+0x102
> 100384 (0xffffff002eb648c0) sched_switch() at sched_switch+0x102
> 100383 (0xffffff002eb65000) sched_switch() at sched_switch+0x102
> 100382 (0xffffff002eb65460) sched_switch() at sched_switch+0x102
> 100381 (0xffffff002eb658c0) sched_switch() at sched_switch+0x102
> 100380 (0xffffff002eb66000) sched_switch() at sched_switch+0x102
> 100379 (0xffffff002eb66460) sched_switch() at sched_switch+0x102
> 100378 (0xffffff002eb668c0) sched_switch() at sched_switch+0x102
> 100377 (0xffffff002eb58460) sched_switch() at sched_switch+0x102
> 100376 (0xffffff002eb588c0) sched_switch() at sched_switch+0x102
> 100375 (0xffffff002eb59000) sched_switch() at sched_switch+0x102
> 100374 (0xffffff002eb59460) sched_switch() at sched_switch+0x102
> 100373 (0xffffff002eb598c0) sched_switch() at sched_switch+0x102
> 100372 (0xffffff002eb5a000) sched_switch() at sched_switch+0x102
> 100371 (0xffffff002eb5a460) sched_switch() at sched_switch+0x102
> 100370 (0xffffff002eb5a8c0) sched_switch() at sched_switch+0x102
> 100369 (0xffffff002eb5c000) sched_switch() at sched_switch+0x102
> 100368 (0xffffff002eb5c460) sched_switch() at sched_switch+0x102
> 100367 (0xffffff002eb5c8c0) sched_switch() at sched_switch+0x102
> 100366 (0xffffff002eb5d000) sched_switch() at sched_switch+0x102
> 100365 (0xffffff002eb5d460) sched_switch() at sched_switch+0x102
> 100364 (0xffffff002eb5d8c0) sched_switch() at sched_switch+0x102
> 100363 (0xffffff002eb60000) sched_switch() at sched_switch+0x102
> 100362 (0xffffff002eb60460) sched_switch() at sched_switch+0x102
> 100361 (0xffffff002eb53000) sched_switch() at sched_switch+0x102
> 100360 (0xffffff002eb53460) sched_switch() at sched_switch+0x102
> 100359 (0xffffff002eb538c0) sched_switch() at sched_switch+0x102
> 100358 (0xffffff002eb54000) sched_switch() at sched_switch+0x102
> 100357 (0xffffff002eb54460) sched_switch() at sched_switch+0x102
> 100356 (0xffffff002eb548c0) sched_switch() at sched_switch+0x102
> 100355 (0xffffff002eb55000) sched_switch() at sched_switch+0x102
> 100354 (0xffffff002eb55460) sched_switch() at sched_switch+0x102
> 100353 (0xffffff002eb558c0) sched_switch() at sched_switch+0x102
> 100352 (0xffffff002eb56000) sched_switch() at sched_switch+0x102
> 100351 (0xffffff002eb56460) sched_switch() at sched_switch+0x102
> 100350 (0xffffff002eb568c0) sched_switch() at sched_switch+0x102
> 100349 (0xffffff002eb57000) sched_switch() at sched_switch+0x102
> 100348 (0xffffff002eb57460) sched_switch() at sched_switch+0x102
> 100347 (0xffffff002eb578c0) sched_switch() at sched_switch+0x102
> 100346 (0xffffff002eb58000) sched_switch() at sched_switch+0x102
> 100345 (0xffffff002eb498c0) sched_switch() at sched_switch+0x102
> 100344 (0xffffff002eb4a000) sched_switch() at sched_switch+0x102
> 100343 (0xffffff002eb4a460) sched_switch() at sched_switch+0x102
> 100342 (0xffffff002eb4a8c0) sched_switch() at sched_switch+0x102
> 100341 (0xffffff002eb4b000) sched_switch() at sched_switch+0x102
> 100340 (0xffffff002eb4b460) sched_switch() at sched_switch+0x102
> 100339 (0xffffff002eb4b8c0) sched_switch() at sched_switch+0x102
> 100338 (0xffffff002eb4d000) sched_switch() at sched_switch+0x102
> 100337 (0xffffff002eb4d460) sched_switch() at sched_switch+0x102
> 100336 (0xffffff002eb4d8c0) sched_switch() at sched_switch+0x102
> 100335 (0xffffff002eb4e000) sched_switch() at sched_switch+0x102
> 100334 (0xffffff002eb4e460) sched_switch() at sched_switch+0x102
> 100333 (0xffffff002eb4e8c0) sched_switch() at sched_switch+0x102
> 100332 (0xffffff002eb4f000) sched_switch() at sched_switch+0x102
> 100331 (0xffffff002eb4f460) sched_switch() at sched_switch+0x102
> 100330 (0xffffff002eb4f8c0) sched_switch() at sched_switch+0x102
> 100329 (0xffffff002eb41460) sched_switch() at sched_switch+0x102
> 100328 (0xffffff002eb418c0) sched_switch() at sched_switch+0x102
> 100327 (0xffffff002eb45000) sched_switch() at sched_switch+0x102
> 100326 (0xffffff002eb45460) sched_switch() at sched_switch+0x102
> 100325 (0xffffff002eb458c0) sched_switch() at sched_switch+0x102
> 100324 (0xffffff002eb46000) sched_switch() at sched_switch+0x102
> 100323 (0xffffff002eb46460) sched_switch() at sched_switch+0x102
> 100322 (0xffffff002eb468c0) sched_switch() at sched_switch+0x102
> 100321 (0xffffff002eb47000) sched_switch() at sched_switch+0x102
> 100320 (0xffffff002eb47460) sched_switch() at sched_switch+0x102
> 100319 (0xffffff002eb478c0) sched_switch() at sched_switch+0x102
> 100318 (0xffffff002eb48000) sched_switch() at sched_switch+0x102
> 100317 (0xffffff002eb48460) sched_switch() at sched_switch+0x102
> 100316 (0xffffff002eb488c0) sched_switch() at sched_switch+0x102
> 100315 (0xffffff002eb49000) sched_switch() at sched_switch+0x102
> 100314 (0xffffff002eb49460) sched_switch() at sched_switch+0x102
> 100313 (0xffffff002eb38000) sched_switch() at sched_switch+0x102
> 100312 (0xffffff002eb38460) sched_switch() at sched_switch+0x102
> 100311 (0xffffff002eb388c0) sched_switch() at sched_switch+0x102
> 100310 (0xffffff002eb39000) sched_switch() at sched_switch+0x102
> 100309 (0xffffff002eb39460) sched_switch() at sched_switch+0x102
> 100308 (0xffffff002eb398c0) sched_switch() at sched_switch+0x102
> 100307 (0xffffff002eb3c000) sched_switch() at sched_switch+0x102
> 100306 (0xffffff002eb3c460) sched_switch() at sched_switch+0x102
> 100305 (0xffffff002eb3c8c0) sched_switch() at sched_switch+0x102
> 100304 (0xffffff002eb3d000) sched_switch() at sched_switch+0x102
> 100303 (0xffffff002eb3d460) sched_switch() at sched_switch+0x102
> 100302 (0xffffff002eb3d8c0) sched_switch() at sched_switch+0x102
> 100301 (0xffffff002eb3e000) sched_switch() at sched_switch+0x102
> 100300 (0xffffff002eb3e460) sched_switch() at sched_switch+0x102
> 100299 (0xffffff002eb3e8c0) sched_switch() at sched_switch+0x102
> 100298 (0xffffff002eb41000) sched_switch() at sched_switch+0x102
> 100297 (0xffffff002eb318c0) sched_switch() at sched_switch+0x102
> 100296 (0xffffff002eb33000) sched_switch() at sched_switch+0x102
> 100295 (0xffffff002eb33460) sched_switch() at sched_switch+0x102
> 100294 (0xffffff002eb338c0) sched_switch() at sched_switch+0x102
> 100293 (0xffffff002eb34000) sched_switch() at sched_switch+0x102
> 100292 (0xffffff002eb34460) sched_switch() at sched_switch+0x102
> 100291 (0xffffff002eb348c0) sched_switch() at sched_switch+0x102
> 100290 (0xffffff002eb35000) sched_switch() at sched_switch+0x102
> 100289 (0xffffff002eb35460) sched_switch() at sched_switch+0x102
> 100288 (0xffffff002eb358c0) sched_switch() at sched_switch+0x102
> 100287 (0xffffff002eb36000) sched_switch() at sched_switch+0x102
> 100286 (0xffffff002eb36460) sched_switch() at sched_switch+0x102
> 100285 (0xffffff002eb368c0) sched_switch() at sched_switch+0x102
> 100284 (0xffffff002eb37000) sched_switch() at sched_switch+0x102
> 100283 (0xffffff002eb37460) sched_switch() at sched_switch+0x102
> 100282 (0xffffff002eb378c0) sched_switch() at sched_switch+0x102
> 100281 (0xffffff001de40460) sched_switch() at sched_switch+0x102
> 100280 (0xffffff001de408c0) sched_switch() at sched_switch+0x102
> 100279 (0xffffff002eb12000) sched_switch() at sched_switch+0x102
> 100278 (0xffffff002eb12460) sched_switch() at sched_switch+0x102
> 100277 (0xffffff002eb128c0) sched_switch() at sched_switch+0x102
> 100276 (0xffffff002eb13000) sched_switch() at sched_switch+0x102
> 100275 (0xffffff002eb13460) sched_switch() at sched_switch+0x102
> 100274 (0xffffff002eb138c0) sched_switch() at sched_switch+0x102
> 100273 (0xffffff002eb15000) sched_switch() at sched_switch+0x102
> 100272 (0xffffff002eb15460) sched_switch() at sched_switch+0x102
> 100271 (0xffffff002eb158c0) sched_switch() at sched_switch+0x102
> 100270 (0xffffff002eb16000) sched_switch() at sched_switch+0x102
> 100269 (0xffffff002eb16460) sched_switch() at sched_switch+0x102
> 100268 (0xffffff002eb168c0) sched_switch() at sched_switch+0x102
> 100267 (0xffffff002eb31000) sched_switch() at sched_switch+0x102
> 100266 (0xffffff002eb31460) sched_switch() at sched_switch+0x102
> 100265 (0xffffff001de3b000) sched_switch() at sched_switch+0x102
> 100264 (0xffffff001de3b460) sched_switch() at sched_switch+0x102
> 100263 (0xffffff001de3b8c0) sched_switch() at sched_switch+0x102
> 100262 (0xffffff001de3c000) sched_switch() at sched_switch+0x102
> 100088 (0xffffff00062838c0) sched_switch() at sched_switch+0x102
> 100239 (0xffffff0006684460) sched_switch() at sched_switch+0x102
> 100255 (0xffffff001de3e460) sched_switch() at sched_switch+0x102
> 100241 (0xffffff00066838c0) sched_switch() at sched_switch+0x102
> 100232 (0xffffff0003fa0460) sched_switch() at sched_switch+0x102
> 100254 (0xffffff001de3e8c0) sched_switch() at sched_switch+0x102
> 100091 (0xffffff00062808c0) sched_switch() at sched_switch+0x102
> 100261 (0xffffff001de3c460) sched_switch() at sched_switch+0x102
> 100252 (0xffffff001de3f460) sched_switch() at sched_switch+0x102
> 100078 (0xffffff0003f9e000) sched_switch() at sched_switch+0x102
> 100074 (0xffffff0003f9e8c0) sched_switch() at sched_switch+0x102
> 100073 (0xffffff0003fa0000) sched_switch() at sched_switch+0x102
> 100072 (0xffffff0003ed5000) sched_switch() at sched_switch+0x102
> 100071 (0xffffff0003ed5460) sched_switch() at sched_switch+0x102
> 100070 (0xffffff0003ed58c0) sched_switch() at sched_switch+0x102
> 100069 (0xffffff0003ed7000) sched_switch() at sched_switch+0x102
> 100068 (0xffffff0003ed7460) sched_switch() at sched_switch+0x102
> 100067 (0xffffff0003ed78c0) sched_switch() at sched_switch+0x102
> 100066 (0xffffff0003ed9000) sched_switch() at sched_switch+0x102
> 100231 (0xffffff0003fa08c0) sched_switch() at sched_switch+0x102
> 100230 (0xffffff0006280000) sched_switch() at sched_switch+0x102
> 100065 (0xffffff0003ed9460) sched_switch() at sched_switch+0x102
> 100064 (0xffffff0003ed98c0) sched_switch() at sched_switch+0x102
> 100076 (0xffffff0006285460) sched_switch() at sched_switch+0x102
> 100075 (0xffffff00062858c0) sched_switch() at sched_switch+0x102
> 100058 (0xffffff0003c40000) sched_switch() at sched_switch+0x102
> 100057 (0xffffff0003c40460) sched_switch() at sched_switch+0x102
> 100056 (0xffffff0003c408c0) sched_switch() at sched_switch+0x102
> 100055 (0xffffff0003c41000) sched_switch() at sched_switch+0x102
> 100054 (0xffffff0003c41460) sched_switch() at sched_switch+0x102
> 100053 (0xffffff0003c418c0) sched_switch() at sched_switch+0x102
> 100052 (0xffffff0003c42000) sched_switch() at sched_switch+0x102
> 100051 (0xffffff0003c42460) sched_switch() at sched_switch+0x102
> 100049 (0xffffff0003900000) sched_switch() at sched_switch+0x102
> 100048 (0xffffff0003900460) sched_switch() at sched_switch+0x102
> 100047 (0xffffff00039008c0) sched_switch() at sched_switch+0x102
> 100046 (0xffffff0003901000) sched_switch() at sched_switch+0x102
> 100044 (0xffffff00039018c0) sched_switch() at sched_switch+0x102
> 100043 (0xffffff0003903000) sched_switch() at sched_switch+0x102
> 100042 (0xffffff0003903460) sched_switch() at sched_switch+0x102
> 100041 (0xffffff00039038c0) sched_switch() at sched_switch+0x102
> 100039 (0xffffff00035f1000) sched_switch() at sched_switch+0x102
> 100038 (0xffffff00035f1460) sched_switch() at sched_switch+0x102
> 100037 (0xffffff00035f18c0) sched_switch() at sched_switch+0x102
> 100036 (0xffffff00035f2000) sched_switch() at sched_switch+0x102
> 100032 (0xffffff00035f3460) sched_switch() at sched_switch+0x102
> 100031 (0xffffff00035f38c0) sched_switch() at sched_switch+0x102
> 100029 (0xffffff000357d8c0) sched_switch() at sched_switch+0x102
> 100028 (0xffffff0003581000) sched_switch() at sched_switch+0x102
> 100013 (0xffffff00034138c0) sched_switch() at sched_switch+0x102
> 100011 (0xffffff0003414460) sched_switch() at sched_switch+0x102
> 100010 (0xffffff00034148c0) sched_switch() at sched_switch+0x102
> 100009 (0xffffff00033ff460) sched_switch() at sched_switch+0x102
> 100061 (0xffffff0003eda8c0) sched_switch() at sched_switch+0x102
> 100060 (0xffffff0003904460) fork_trampoline() at fork_trampoline
> 100059 (0xffffff00039048c0) sched_switch() at sched_switch+0x102
> 100050 (0xffffff0003c428c0) sched_switch() at sched_switch+0x102
> 100045 (0xffffff0003901460) fork_trampoline() at fork_trampoline
> 100040 (0xffffff0003904000) sched_switch() at sched_switch+0x102
> 100035 (0xffffff00035f2460) sched_switch() at sched_switch+0x102
> 100030 (0xffffff000357d460) sched_switch() at sched_switch+0x102
> 100027 (0xffffff0003581460) sched_switch() at sched_switch+0x102
> 100024 (0xffffff0003582460) fork_trampoline() at fork_trampoline
> 100022 (0xffffff000357a000) sched_switch() at sched_switch+0x102
> 100017 (0xffffff000357b8c0) sched_switch() at sched_switch+0x102
> 100015 (0xffffff0003413000) sched_switch() at sched_switch+0x102
> 100014 (0xffffff0003413460) fork_trampoline() at fork_trampoline
> 100008 (0xffffff00033ff8c0) fork_trampoline() at fork_trampoline
> 100007 (0xffffff0003410000) sched_switch() at sched_switch+0x102
> 100006 (0xffffff0003410460) sched_switch() at sched_switch+0x102
> 100005 (0xffffff00034108c0) sched_switch() at sched_switch+0x102
> 100004 (0xffffff00033fe000) cpustop_handler() at cpustop_handler+0x3a
> 100003 (0xffffff00033fe460) kdb_enter() at kdb_enter+0x3b
> 100002 (0xffffff00033fe8c0) sched_switch() at sched_switch+0x102
> 100001 (0xffffff00033ff000) sched_switch() at sched_switch+0x102
> 100250 (0xffffff00066388c0) sched_switch() at sched_switch+0x102
> 100249 (0xffffff0006639000) sched_switch() at sched_switch+0x102
> 100248 (0xffffff0006639460) sched_switch() at sched_switch+0x102
> 100247 (0xffffff00066398c0) sched_switch() at sched_switch+0x102
> 100246 (0xffffff000663a000) sched_switch() at sched_switch+0x102
> 100245 (0xffffff000663a460) sched_switch() at sched_switch+0x102
> 100244 (0xffffff001de40000) sched_switch() at sched_switch+0x102
> 100243 (0xffffff000663a8c0) sched_switch() at sched_switch+0x102
> 100229 (0xffffff0006280460) sched_switch() at sched_switch+0x102
> 100228 (0xffffff0006650460) sched_switch() at sched_switch+0x102
> 100227 (0xffffff000665a460) sched_switch() at sched_switch+0x102
> 100226 (0xffffff000667d000) sched_switch() at sched_switch+0x102
> 100225 (0xffffff000665c460) sched_switch() at sched_switch+0x102
> 100224 (0xffffff000667f000) sched_switch() at sched_switch+0x102
> 100223 (0xffffff0006689460) sched_switch() at sched_switch+0x102
> 100222 (0xffffff0006680000) sched_switch() at sched_switch+0x102
> 100221 (0xffffff000667d460) sched_switch() at sched_switch+0x102
> 100220 (0xffffff0006681000) sched_switch() at sched_switch+0x102
> 100219 (0xffffff000667f8c0) sched_switch() at sched_switch+0x102
> 100218 (0xffffff0006682000) sched_switch() at sched_switch+0x102
> 100217 (0xffffff00066808c0) sched_switch() at sched_switch+0x102
> 100216 (0xffffff0006683000) sched_switch() at sched_switch+0x102
> 100215 (0xffffff00066818c0) sched_switch() at sched_switch+0x102
> 100214 (0xffffff0006675460) sched_switch() at sched_switch+0x102
> 100213 (0xffffff00066828c0) sched_switch() at sched_switch+0x102
> 100212 (0xffffff0006676460) sched_switch() at sched_switch+0x102
> 100211 (0xffffff0006675000) sched_switch() at sched_switch+0x102
> 100210 (0xffffff0006678460) sched_switch() at sched_switch+0x102
> 100209 (0xffffff00066758c0) sched_switch() at sched_switch+0x102
> 100208 (0xffffff0006679000) sched_switch() at sched_switch+0x102
> 100207 (0xffffff00066768c0) sched_switch() at sched_switch+0x102
> 100206 (0xffffff000667a000) sched_switch() at sched_switch+0x102
> 100205 (0xffffff00066788c0) sched_switch() at sched_switch+0x102
> 100204 (0xffffff000666b460) sched_switch() at sched_switch+0x102
> 100203 (0xffffff00066798c0) sched_switch() at sched_switch+0x102
> 100202 (0xffffff000666e460) sched_switch() at sched_switch+0x102
> 100201 (0xffffff000667a460) sched_switch() at sched_switch+0x102
> 100200 (0xffffff0006671460) sched_switch() at sched_switch+0x102
> 100199 (0xffffff000666b8c0) sched_switch() at sched_switch+0x102
> 100198 (0xffffff0006672460) sched_switch() at sched_switch+0x102
> 100197 (0xffffff000666e8c0) sched_switch() at sched_switch+0x102
> 100196 (0xffffff0006673460) sched_switch() at sched_switch+0x102
> 100195 (0xffffff00066718c0) sched_switch() at sched_switch+0x102
> 100194 (0xffffff0006674460) sched_switch() at sched_switch+0x102
> 100193 (0xffffff00066728c0) sched_switch() at sched_switch+0x102
> 100192 (0xffffff0006665460) sched_switch() at sched_switch+0x102
> 100191 (0xffffff00066738c0) sched_switch() at sched_switch+0x102
> 100190 (0xffffff0006665000) sched_switch() at sched_switch+0x102
> 100189 (0xffffff0006666000) sched_switch() at sched_switch+0x102
> 100188 (0xffffff0006666460) sched_switch() at sched_switch+0x102
> 100187 (0xffffff0006667000) sched_switch() at sched_switch+0x102
> 100186 (0xffffff0006667460) sched_switch() at sched_switch+0x102
> 100185 (0xffffff00066678c0) sched_switch() at sched_switch+0x102
> 100184 (0xffffff0006669460) sched_switch() at sched_switch+0x102
> 100183 (0xffffff00066698c0) sched_switch() at sched_switch+0x102
> 100182 (0xffffff000666a460) sched_switch() at sched_switch+0x102
> 100181 (0xffffff000666a8c0) sched_switch() at sched_switch+0x102
> 100180 (0xffffff000665c8c0) sched_switch() at sched_switch+0x102
> 100179 (0xffffff000665f000) sched_switch() at sched_switch+0x102
> 100178 (0xffffff000665f8c0) sched_switch() at sched_switch+0x102
> 100177 (0xffffff0006660000) sched_switch() at sched_switch+0x102
> 100176 (0xffffff00066608c0) sched_switch() at sched_switch+0x102
> 100175 (0xffffff0006661000) sched_switch() at sched_switch+0x102
> 100174 (0xffffff00066618c0) sched_switch() at sched_switch+0x102
> 100173 (0xffffff0006663000) sched_switch() at sched_switch+0x102
> 100172 (0xffffff00066638c0) sched_switch() at sched_switch+0x102
> 100171 (0xffffff0006664000) sched_switch() at sched_switch+0x102
> 100170 (0xffffff0006664460) sched_switch() at sched_switch+0x102
> 100169 (0xffffff0006656460) sched_switch() at sched_switch+0x102
> 100168 (0xffffff00066568c0) sched_switch() at sched_switch+0x102
> 100167 (0xffffff0006657460) sched_switch() at sched_switch+0x102
> 100166 (0xffffff00066578c0) sched_switch() at sched_switch+0x102
> 100165 (0xffffff0006658460) sched_switch() at sched_switch+0x102
> 100164 (0xffffff00066588c0) sched_switch() at sched_switch+0x102
> 100163 (0xffffff000665a000) sched_switch() at sched_switch+0x102
> 100162 (0xffffff000665a8c0) sched_switch() at sched_switch+0x102
> 100161 (0xffffff000665b000) sched_switch() at sched_switch+0x102
> 100160 (0xffffff000665b8c0) sched_switch() at sched_switch+0x102
> 100159 (0xffffff000665c000) sched_switch() at sched_switch+0x102
> 100158 (0xffffff0006650000) sched_switch() at sched_switch+0x102
> 100157 (0xffffff00066508c0) sched_switch() at sched_switch+0x102
> 100156 (0xffffff0006651460) sched_switch() at sched_switch+0x102
> 100155 (0xffffff0006652000) sched_switch() at sched_switch+0x102
> 100154 (0xffffff0006651000) sched_switch() at sched_switch+0x102
> 100153 (0xffffff00066518c0) sched_switch() at sched_switch+0x102
> 100152 (0xffffff0006652460) sched_switch() at sched_switch+0x102
> 100151 (0xffffff0006653000) sched_switch() at sched_switch+0x102
> 100150 (0xffffff0006654000) sched_switch() at sched_switch+0x102
> 100149 (0xffffff00066548c0) sched_switch() at sched_switch+0x102
> 100148 (0xffffff00066528c0) sched_switch() at sched_switch+0x102
> 100147 (0xffffff0006653460) sched_switch() at sched_switch+0x102
> 100146 (0xffffff00066538c0) sched_switch() at sched_switch+0x102
> 100145 (0xffffff0006654460) sched_switch() at sched_switch+0x102
> 100144 (0xffffff0006656000) sched_switch() at sched_switch+0x102
> 100143 (0xffffff0006646460) sched_switch() at sched_switch+0x102
> 100142 (0xffffff0006649000) sched_switch() at sched_switch+0x102
> 100141 (0xffffff0006646000) sched_switch() at sched_switch+0x102
> 100140 (0xffffff00066468c0) sched_switch() at sched_switch+0x102
> 100139 (0xffffff0006649460) sched_switch() at sched_switch+0x102
> 100138 (0xffffff00066498c0) sched_switch() at sched_switch+0x102
> 100137 (0xffffff000664b000) sched_switch() at sched_switch+0x102
> 100136 (0xffffff000664b460) sched_switch() at sched_switch+0x102
> 100135 (0xffffff000664b8c0) sched_switch() at sched_switch+0x102
> 100134 (0xffffff000664c460) sched_switch() at sched_switch+0x102
> 100133 (0xffffff000664d460) sched_switch() at sched_switch+0x102
> 100132 (0xffffff0003f9b000) sched_switch() at sched_switch+0x102
> 100131 (0xffffff000664c000) sched_switch() at sched_switch+0x102
> 100130 (0xffffff000664c8c0) sched_switch() at sched_switch+0x102
> 100129 (0xffffff000664d000) sched_switch() at sched_switch+0x102
> 100128 (0xffffff000664d8c0) sched_switch() at sched_switch+0x102
> 100127 (0xffffff0003f9b460) sched_switch() at sched_switch+0x102
> 100126 (0xffffff0003f9b8c0) sched_switch() at sched_switch+0x102
> 100125 (0xffffff0003f9c000) sched_switch() at sched_switch+0x102
> 100124 (0xffffff0006687460) sched_switch() at sched_switch+0x102
> 100123 (0xffffff00066878c0) sched_switch() at sched_switch+0x102
> 100122 (0xffffff0006688000) sched_switch() at sched_switch+0x102
> 100121 (0xffffff0006688460) sched_switch() at sched_switch+0x102
> 100120 (0xffffff00066888c0) sched_switch() at sched_switch+0x102
> 100119 (0xffffff0006689000) sched_switch() at sched_switch+0x102
> 100118 (0xffffff000667d8c0) sched_switch() at sched_switch+0x102
> 100117 (0xffffff0006680460) sched_switch() at sched_switch+0x102
> 100116 (0xffffff0006682460) sched_switch() at sched_switch+0x102
> 100115 (0xffffff000667f460) sched_switch() at sched_switch+0x102
> 100114 (0xffffff0006676000) sched_switch() at sched_switch+0x102
> 100113 (0xffffff0006681460) sched_switch() at sched_switch+0x102
> 100112 (0xffffff0006679460) sched_switch() at sched_switch+0x102
> 100111 (0xffffff00066748c0) sched_switch() at sched_switch+0x102
> 100110 (0xffffff000666e000) sched_switch() at sched_switch+0x102
> 100109 (0xffffff0006678000) sched_switch() at sched_switch+0x102
> 100108 (0xffffff0006672000) sched_switch() at sched_switch+0x102
> 100107 (0xffffff000667a8c0) sched_switch() at sched_switch+0x102
> 100106 (0xffffff0006674000) sched_switch() at sched_switch+0x102
> 100105 (0xffffff0006671000) sched_switch() at sched_switch+0x102
> 100104 (0xffffff00066668c0) sched_switch() at sched_switch+0x102
> 100103 (0xffffff0006673000) sched_switch() at sched_switch+0x102
> 100102 (0xffffff000666a000) sched_switch() at sched_switch+0x102
> 100101 (0xffffff00066658c0) sched_switch() at sched_switch+0x102
> 100100 (0xffffff000665f460) sched_switch() at sched_switch+0x102
> 100099 (0xffffff0006669000) sched_switch() at sched_switch+0x102
> 100098 (0xffffff0006661460) sched_switch() at sched_switch+0x102
> 100097 (0xffffff000666b000) sched_switch() at sched_switch+0x102
> 100096 (0xffffff00066648c0) sched_switch() at sched_switch+0x102
> 100095 (0xffffff0006660460) sched_switch() at sched_switch+0x102
> 100094 (0xffffff0006658000) sched_switch() at sched_switch+0x102
> 100093 (0xffffff0006663460) sched_switch() at sched_switch+0x102
> 100092 (0xffffff000665b460) sched_switch() at sched_switch+0x102
> 100063 (0xffffff0003eda000) sched_switch() at sched_switch+0x102
> 100062 (0xffffff0003eda460) sched_switch() at sched_switch+0x102
> 100034 (0xffffff00035f28c0) sched_switch() at sched_switch+0x102
> 100033 (0xffffff00035f3000) sched_switch() at sched_switch+0x102
> 100026 (0xffffff00035818c0) sched_switch() at sched_switch+0x102
> 100025 (0xffffff0003582000) sched_switch() at sched_switch+0x102
> 100023 (0xffffff00035828c0) sched_switch() at sched_switch+0x102
> 100021 (0xffffff000357a460) sched_switch() at sched_switch+0x102
> 100020 (0xffffff000357a8c0) sched_switch() at sched_switch+0x102
> 100019 (0xffffff000357b000) sched_switch() at sched_switch+0x102
> 100018 (0xffffff000357b460) sched_switch() at sched_switch+0x102
> 100016 (0xffffff000357d000) sched_switch() at sched_switch+0x102
> 100012 (0xffffff0003414000) sched_switch() at sched_switch+0x102
> 100000 (0xffffffff80cffcf0) sched_switch() at sched_switch+0x102
>
> db>ps
> pid ppid pgrp uid state wmesg wchan cmd
> 5002 2389 5002 20001 SL+ pfault 0xffffffff80d33adc top
> 2389 2388 2389 20001 Ss+ pause 0xffffff004ca3a0a0 tcsh
> 2388 2386 2386 20001 S vmwait 0xffffffff80d33adc sshd
> 2386 1195 2386 0 Ss sbwait 0xffffff01ef77fe8c sshd
> 1345 1 1345 0 SLs+ pfault 0xffffffff80d33adc getty
> 1344 1 1344 0 Ss+ ttyin 0xffffff0003ede4a8 getty
> 1248 1 1248 0 ?s cron
> 1227 1 1227 25 ?s sendmail
> 1211 1 1211 0 SLs pfault 0xffffffff80d33adc sendmail
> 1195 1 1195 0 SLs pfault 0xffffffff80d33adc sshd
> 1055 1 1055 0 SLs pfault 0xffffffff80d33adc perl5.10.1
> 1035 1 1035 1 ?s rwhod
> 1005 1 1005 0 SLs pfault 0xffffffff80d33adc ntpd
> 939 1 939 0 Ss rpcsvc 0xffffff001de7a4a0 NLM: master
> 933 1 933 0 SLs pfault 0xffffffff80d33adc rpc.statd
> 927 926 926 0 S (threaded) nfsd
> 100516 D vmwait 0xffffffff80d33adc nfsd: service
> 100515 S rpcsvc 0xffffff002dba9520 nfsd: service
> 100514 D vmwait 0xffffffff80d33adc nfsd: service
> 100513 D zio->io_ 0xffffff01187c0320 nfsd: service
> 100512 D zfs 0xffffff00b55f17f8 nfsd: service
> 100511 D vmwait 0xffffffff80d33adc nfsd: service
> 100510 D zfs 0xffffff00b55f17f8 nfsd: service
> 100509 D zfs 0xffffff00c495aba8 nfsd: service
> 100508 D db->db_c 0xffffff008d7a4950 nfsd: service
> 100507 S rpcsvc 0xffffff002d2ba720 nfsd: service
> 100506 D tx->tx_q 0xffffff0006abf240 nfsd: service
> 100505 S rpcsvc 0xffffff002d728ba0 nfsd: service
> 100504 D zio->io_ 0xffffff0185b36320 nfsd: service
> 100503 S rpcsvc 0xffffff002d2ba7a0 nfsd: service
> 100502 S rpcsvc 0xffffff002d3165a0 nfsd: service
> 100501 S rpcsvc 0xffffff002d728c20 nfsd: service
> 100500 D tx->tx_q 0xffffff0006abf240 nfsd: service
> 100499 D zfsvfs-> 0xffffff001ddc4788 nfsd: service
> 100498 D zfs 0xffffff009b810ba8 nfsd: service
> 100497 S rpcsvc 0xffffff002d728ca0 nfsd: service
> 100496 D zfs 0xffffff00bdab0098 nfsd: service
> 100495 D tx->tx_q 0xffffff0006abf240 nfsd: service
> 100494 D vmwait 0xffffffff80d33adc nfsd: service
> 100493 S rpcsvc 0xffffff002d728d20 nfsd: service
> 100492 S rpcsvc 0xffffff002d316620 nfsd: service
> 100491 S rpcsvc 0xffffff002d2ba9a0 nfsd: service
> 100490 D vmwait 0xffffffff80d33adc nfsd: service
> 100489 D zfsvfs-> 0xffffff0011243828 nfsd: service
> 100488 D zfs 0xffffff00bdab0098 nfsd: service
> 100487 D zfs 0xffffff00bdab0098 nfsd: service
> 100486 D zfs 0xffffff00bdab0098 nfsd: service
> 100485 D zio->io_ 0xffffff01046c2320 nfsd: service
> 100484 S rpcsvc 0xffffff002d316720 nfsd: service
> 100483 D zfs 0xffffff00b62b9d80 nfsd: service
> 100482 S rpcsvc 0xffffff002d728a20 nfsd: service
> 100481 S rpcsvc 0xffffff002d728e20 nfsd: service
> 100480 S rpcsvc 0xffffff002d3167a0 nfsd: service
> 100479 S rpcsvc 0xffffff002d2baaa0 nfsd: service
> 100478 D zfs 0xffffff00bdab0098 nfsd: service
> 100477 D zfs 0xffffff00bdab0098 nfsd: service
> 100476 D zio->io_ 0xffffff00e076ed70 nfsd: service
> 100475 D zfs 0xffffff00b62b9d80 nfsd: service
> 100474 D zfs 0xffffff00bdab0098 nfsd: service
> 100473 D zfs 0xffffff00bdab0098 nfsd: service
> 100472 D vmwait 0xffffffff80d33adc nfsd: service
> 100471 D zfs 0xffffff00b62b9d80 nfsd: service
> 100470 D zfs 0xffffff00bdab0098 nfsd: service
> 100469 D zfs 0xffffff00c495aba8 nfsd: service
> 100468 D zfs 0xffffff00bdab0098 nfsd: service
> 100467 D zfs 0xffffff00b9501ba8 nfsd: service
> 100466 S rpcsvc 0xffffff002d3169a0 nfsd: service
> 100465 D zfs 0xffffff00b5979620 nfsd: service
> 100464 D zfs 0xffffff00b62b9d80 nfsd: service
> 100463 D zfs 0xffffff00bdab0098 nfsd: service
> 100462 S rpcsvc 0xffffff002d316920 nfsd: service
> 100461 D zfs 0xffffff00c495aba8 nfsd: service
> 100460 D zfs 0xffffff00b55f17f8 nfsd: service
> 100459 S rpcsvc 0xffffff002d935320 nfsd: service
> 100458 S rpcsvc 0xffffff002d318aa0 nfsd: service
> 100457 D zfsvfs-> 0xffffff001ddc4788 nfsd: service
> 100456 S rpcsvc 0xffffff002d318a20 nfsd: service
> 100455 D zfs 0xffffff00b9467098 nfsd: service
> 100454 S rpcsvc 0xffffff002d316aa0 nfsd: service
> 100453 S rpcsvc 0xffffff002d2bae20 nfsd: service
> 100452 S rpcsvc 0xffffff002d3189a0 nfsd: service
> 100451 D zfs 0xffffff00c495aba8 nfsd: service
> 100450 D zfs 0xffffff00850f77f8 nfsd: service
> 100449 D zfs 0xffffff00c495aba8 nfsd: service
> 100448 D vmwait 0xffffffff80d33adc nfsd: service
> 100447 D zfs 0xffffff00bdab0098 nfsd: service
> 100446 D vmwait 0xffffffff80d33adc nfsd: service
> 100445 S rpcsvc 0xffffff002d315020 nfsd: service
> 100444 D zfs 0xffffff00b62b9d80 nfsd: service
> 100443 D zfs 0xffffff00c495aba8 nfsd: service
> 100442 S rpcsvc 0xffffff002dfa6720 nfsd: service
> 100441 S rpcsvc 0xffffff002d3150a0 nfsd: service
> 100440 D vmwait 0xffffffff80d33adc nfsd: service
> 100439 S rpcsvc 0xffffff002d9355a0 nfsd: service
> 100438 D zfs 0xffffff00bdab0098 nfsd: service
> 100437 D zfs 0xffffff00bdab0098 nfsd: service
> 100436 S rpcsvc 0xffffff002dba96a0 nfsd: service
> 100435 D zfs 0xffffff00c495aba8 nfsd: service
> 100434 S rpcsvc 0xffffff002dfa6820 nfsd: service
> 100433 S rpcsvc 0xffffff002d935720 nfsd: service
> 100432 D zfs 0xffffff00c495aba8 nfsd: service
> 100431 D zfs 0xffffff00b55f17f8 nfsd: service
> 100430 D vmwait 0xffffffff80d33adc nfsd: service
> 100429 S rpcsvc 0xffffff002d315220 nfsd: service
> 100428 D zio->io_ 0xffffff01b4551690 nfsd: service
> 100427 D vmwait 0xffffffff80d33adc nfsd: service
> 100426 S rpcsvc 0xffffff002dbab6a0 nfsd: service
> 100425 D vmwait 0xffffffff80d33adc nfsd: service
> 100424 S rpcsvc 0xffffff002dba97a0 nfsd: service
> 100423 D zio->io_ 0xffffff002d979320 nfsd: service
> 100422 D zfs 0xffffff00bdab0098 nfsd: service
> 100421 D zfs 0xffffff00bdab0098 nfsd: service
> 100420 D zfs 0xffffff00b55f17f8 nfsd: service
> 100419 S rpcsvc 0xffffff002dbab620 nfsd: service
> 100418 D vmwait 0xffffffff80d33adc nfsd: service
> 100417 D zfs 0xffffff00b9501ba8 nfsd: service
> 100416 D zfs 0xffffff00bdab0098 nfsd: service
> 100415 D zio->io_ 0xffffff01d4611d70 nfsd: service
> 100414 S rpcsvc 0xffffff002dbaa6a0 nfsd: service
> 100413 S rpcsvc 0xffffff002dba98a0 nfsd: service
> 100412 D zfs 0xffffff00bb36eba8 nfsd: service
> 100411 S rpcsvc 0xffffff002dbab7a0 nfsd: service
> 100410 S rpcsvc 0xffffff002ddbe620 nfsd: service
> 100409 S rpcsvc 0xffffff002dba9920 nfsd: service
> 100408 D zfs 0xffffff00c495aba8 nfsd: service
> 100407 S rpcsvc 0xffffff002dbab820 nfsd: service
> 100406 D zfs 0xffffff00850f77f8 nfsd: service
> 100405 S rpcsvc 0xffffff002dbab8a0 nfsd: service
> 100404 D zio->io_ 0xffffff0028d8da00 nfsd: service
> 100403 S rpcsvc 0xffffff002dba9a20 nfsd: service
> 100402 D zio->io_ 0xffffff0157902d70 nfsd: service
> 100401 S rpcsvc 0xffffff002dba99a0 nfsd: service
> 100400 D zfs 0xffffff009b810ba8 nfsd: service
> 100399 D zfs 0xffffff00bd5e5448 nfsd: service
> 100398 S rpcsvc 0xffffff002dbaa920 nfsd: service
> 100397 D zio->io_ 0xffffff002ec82a00 nfsd: service
> 100396 S rpcsvc 0xffffff002d935d20 nfsd: service
> 100395 D zfs 0xffffff00bdab0098 nfsd: service
> 100394 D zfs 0xffffff00c495aba8 nfsd: service
> 100393 D zfs 0xffffff00bdab0098 nfsd: service
> 100392 D zfs 0xffffff00bdab0098 nfsd: service
> 100391 D zfs 0xffffff00b55f17f8 nfsd: service
> 100390 D zfs 0xffffff00bdab0098 nfsd: service
> 100389 D zfs 0xffffff00bb36eba8 nfsd: service
> 100388 D tx->tx_q 0xffffff0006abf240 nfsd: service
> 100387 D zfs 0xffffff00bb36eba8 nfsd: service
> 100386 D vmwait 0xffffffff80d33adc nfsd: service
> 100385 S rpcsvc 0xffffff002dbabaa0 nfsd: service
> 100384 D zfs 0xffffff00c1e12620 nfsd: service
> 100383 D zfs 0xffffff009b810ba8 nfsd: service
> 100382 D vmwait 0xffffffff80d33adc nfsd: service
> 100381 D zfs 0xffffff00c495aba8 nfsd: service
> 100380 D vmwait 0xffffffff80d33adc nfsd: service
> 100379 D vmwait 0xffffffff80d33adc nfsd: service
> 100378 D zio->io_ 0xffffff01005ada00 nfsd: service
> 100377 D zfs 0xffffff00bdab0098 nfsd: service
> 100376 D vmwait 0xffffffff80d33adc nfsd: service
> 100375 D zfs 0xffffff00bdab0098 nfsd: service
> 100374 S rpcsvc 0xffffff002db1a1a0 nfsd: service
> 100373 D zfs 0xffffff00bdab0098 nfsd: service
> 100372 D vmwait 0xffffffff80d33adc nfsd: service
> 100371 D zfs 0xffffff00850f77f8 nfsd: service
> 100370 D vmwait 0xffffffff80d33adc nfsd: service
> 100369 D zfs 0xffffff00c1e12620 nfsd: service
> 100368 D zfsvfs-> 0xffffff001ddc4788 nfsd: service
> 100367 D zfs 0xffffff00b9467098 nfsd: service
> 100366 S rpcsvc 0xffffff002db1a2a0 nfsd: service
> 100365 D zfs 0xffffff00850f77f8 nfsd: service
> 100364 S rpcsvc 0xffffff002dbaad20 nfsd: service
> 100363 S rpcsvc 0xffffff002dbabda0 nfsd: service
> 100362 D zfs 0xffffff00b9501ba8 nfsd: service
> 100361 D zfs 0xffffff00b62b9d80 nfsd: service
> 100360 S rpcsvc 0xffffff002dbaaca0 nfsd: service
> 100359 D zfs 0xffffff00b55f17f8 nfsd: service
> 100358 D zio->io_ 0xffffff013395a320 nfsd: service
> 100357 D zfs 0xffffff00bd5e5448 nfsd: service
> 100356 D vmwait 0xffffffff80d33adc nfsd: service
> 100355 D vmwait 0xffffffff80d33adc nfsd: service
> 100354 D zfs 0xffffff00b55f17f8 nfsd: service
> 100353 S rpcsvc 0xffffff002dbaa220 nfsd: service
> 100352 D zfs 0xffffff00850f77f8 nfsd: service
> 100351 D zfs 0xffffff00b55f17f8 nfsd: service
> 100350 S rpcsvc 0xffffff002dbab020 nfsd: service
> 100349 D zio->io_ 0xffffff00b9edfa00 nfsd: service
> 100348 D tx->tx_q 0xffffff0006abf240 nfsd: service
> 100347 S rpcsvc 0xffffff002ddbe120 nfsd: service
> 100346 D zfs 0xffffff00bb36eba8 nfsd: service
> 100345 D vmwait 0xffffffff80d33adc nfsd: service
> 100344 S rpcsvc 0xffffff002db1a620 nfsd: service
> 100343 D zfs 0xffffff00c495aba8 nfsd: service
> 100342 S rpcsvc 0xffffff002dbab120 nfsd: service
> 100341 D zio->io_ 0xffffff002c859690 nfsd: service
> 100340 D zfs 0xffffff00850f77f8 nfsd: service
> 100339 D vmwait 0xffffffff80d33adc nfsd: service
> 100338 S rpcsvc 0xffffff002dbab1a0 nfsd: service
> 100337 D zfs 0xffffff00c1e12620 nfsd: service
> 100336 D zfs 0xffffff00bdab0098 nfsd: service
> 100335 S rpcsvc 0xffffff002ca84a20 nfsd: service
> 100334 D zio->io_ 0xffffff017b42a690 nfsd: service
> 100333 D buf_hash 0xffffffff8107d600 nfsd: service
> 100332 D zio->io_ 0xffffff002f6ccd70 nfsd: service
> 100331 D zfs 0xffffff009b810ba8 nfsd: service
> 100330 S rpcsvc 0xffffff002dbab320 nfsd: service
> 100329 D zio->io_ 0xffffff0161a66690 nfsd: service
> 100328 S rpcsvc 0xffffff002db1a720 nfsd: service
> 100327 D zio->io_ 0xffffff0038042320 nfsd: service
> 100326 D vmwait 0xffffffff80d33adc nfsd: service
> 100325 S rpcsvc 0xffffff002e1ac7a0 nfsd: service
> 100324 D zio->io_ 0xffffff016d9b0690 nfsd: service
> 100323 D zio->io_ 0xffffff002a0b7a00 nfsd: service
> 100322 D zfs 0xffffff00c31037f8 nfsd: service
> 100321 D zio->io_ 0xffffff0205a61320 nfsd: service
> 100320 S rpcsvc 0xffffff002dbab3a0 nfsd: service
> 100319 D vmwait 0xffffffff80d33adc nfsd: service
> 100318 D vmwait 0xffffffff80d33adc nfsd: service
> 100317 S rpcsvc 0xffffff002dba8520 nfsd: service
> 100316 S rpcsvc 0xffffff002dbab4a0 nfsd: service
> 100315 S rpcsvc 0xffffff002e1ac920 nfsd: service
> 100314 D zio->io_ 0xffffff018542ed70 nfsd: service
> 100313 D zio->io_ 0xffffff01e156f320 nfsd: service
> 100312 D zfs 0xffffff00b55f17f8 nfsd: service
> 100311 D zfs 0xffffff00bdab0098 nfsd: service
> 100310 D zfs 0xffffff017689cd80 nfsd: service
> 100309 D zfs 0xffffff00850f77f8 nfsd: service
> 100308 D zfs 0xffffff00bdab0098 nfsd: service
> 100307 D zfs 0xffffff00bdab0098 nfsd: service
> 100306 D tx->tx_q 0xffffff0006abf240 nfsd: service
> 100305 S rpcsvc 0xffffff002dba86a0 nfsd: service
> 100304 D zio->io_ 0xffffff00b7c48690 nfsd: service
> 100303 D zfs 0xffffff00850f77f8 nfsd: service
> 100302 D zio->io_ 0xffffff01c7f41690 nfsd: service
> 100301 D zio->io_ 0xffffff01f44aed70 nfsd: service
> 100300 D vmwait 0xffffffff80d33adc nfsd: service
> 100299 D zfs 0xffffff00bb36eba8 nfsd: service
> 100298 S rpcsvc 0xffffff002db1aaa0 nfsd: service
> 100297 D zfs 0xffffff00bb36eba8 nfsd: service
> 100296 D zfs 0xffffff00c495aba8 nfsd: service
> 100295 D zfs 0xffffff00bdab0098 nfsd: service
> 100294 S rpcsvc 0xffffff002db1ab20 nfsd: service
> 100293 S rpcsvc 0xffffff002ddbe8a0 nfsd: service
> 100292 D vmwait 0xffffffff80d33adc nfsd: service
> 100291 S rpcsvc 0xffffff002dba88a0 nfsd: service
> 100290 D zfs 0xffffff00bdab0098 nfsd: service
> 100289 D zfs 0xffffff00c495aba8 nfsd: service
> 100288 D zfs 0xffffff00c495aba8 nfsd: service
> 100287 D zfs 0xffffff00b55f17f8 nfsd: service
> 100286 D zfs 0xffffff00bb36eba8 nfsd: service
> 100285 D zfs 0xffffff00bb36eba8 nfsd: service
> 100284 D zio->io_ 0xffffff0115b28a00 nfsd: service
> 100283 D zfs 0xffffff00bdab0098 nfsd: service
> 100282 D zfs 0xffffff00b55f17f8 nfsd: service
> 100281 D zfs 0xffffff00bdab0098 nfsd: service
> 100280 D vmwait 0xffffffff80d33adc nfsd: service
> 100279 S rpcsvc 0xffffff002dba89a0 nfsd: service
> 100278 D zfs 0xffffff00b62b9d80 nfsd: service
> 100277 D zfs 0xffffff00b5979620 nfsd: service
> 100276 D zfs 0xffffff00c31037f8 nfsd: service
> 100275 S rpcsvc 0xffffff002dba8a20 nfsd: service
> 100274 D vq->vq_l 0xffffff0006692600 nfsd: service
> 100273 S rpcsvc 0xffffff002ddbeaa0 nfsd: service
> 100272 S rpcsvc 0xffffff002dfa6520 nfsd: service
> 100271 S rpcsvc 0xffffff002dba8b20 nfsd: service
> 100270 D vmwait 0xffffffff80d33adc nfsd: service
> 100269 S rpcsvc 0xffffff002ddbeba0 nfsd: service
> 100268 S rpcsvc 0xffffff002dfa65a0 nfsd: service
> 100267 D zfs 0xffffff00bd5e5448 nfsd: service
> 100266 D zfs 0xffffff00bdab0098 nfsd: service
> 100265 D zio->io_ 0xffffff0190d08690 nfsd: service
> 100264 D zfsvfs-> 0xffffff001ddc4788 nfsd: service
> 100263 S rpcsvc 0xffffff002dba8ba0 nfsd: service
> 100262 D zfs 0xffffff00b55f17f8 nfsd: service
> 100088 S rpcsvc 0xffffff002dba8c20 nfsd: master
> 926 1 926 0 SLs pfault 0xffffffff80d33adc nfsd
> 924 920 920 0 SL pfault 0xffffffff80d33adc nfsuserd
> 923 920 920 0 SL pfault 0xffffffff80d33adc nfsuserd
> 922 920 920 0 SL pfault 0xffffffff80d33adc nfsuserd
> 921 920 920 0 SL pfault 0xffffffff80d33adc nfsuserd
> 920 1 920 0 Ss pause 0xffffff002e9bc0a0 nfsuserd
> 901 1 901 0 Ss select 0xffffff002d317440 mountd
> 820e 1 820 0 SLs pfault 0xffffffff80d33adc rpcbind
> 795 1 795 0 SLs pfault 0xffffffff80d33adc syslogd
> 92 0 0 0 SL mdwait 0xffffff0011141000 [md0]
> 23 0 0 0 SL sdflush 0xffffffff80d33058 [softdepflush]
> 22 0 0 0 SL zio->io_ 0xffffff00b49fca00 [syncer]
> 21 0 0 0 SL vlruwt 0xffffff0003f53470 [vnlru]
> 20 0 0 0 SL psleep 0xffffffff80d27608 [bufdaemon]
> 19 0 0 0 SL pgzero 0xffffffff80d34b2c [pagezero]
> 18 0 0 0 SL psleep 0xffffffff80d33ec8 [vmdaemon]
> 17 0 0 0 SL psleep 0xffffffff80d33e8c [pagedaemon]
> 16 0 0 0 SL ccb_scan 0xffffffff80cc46e0 [xpt_thrd]
> 15 0 0 0 SL waiting_ 0xffffffff80d2d3a0 [sctp_iterator]
> 9 0 0 0 SL (threaded) zfskern
> 100231 D vmwait 0xffffffff80d33adc [txg_thread_enter]
> 100230 D tx->tx_q 0xffffff0006abf230 [txg_thread_enter]
> 100065 D l2arc_fe 0xffffffff81081940 [l2arc_feed_thread]
> 100064 D arc_recl 0xffffffff81071aa0 [arc_reclaim_thread]
> 14 0 0 0 SL (threaded) usb
> 100076 D - 0xffffff0006226810 [ucom]
> 100075 D - 0xffffff0006225c10 [ucom]
> 100058 D - 0xffffff800034ae18 [usbus4]
> 100057 D - 0xffffff800034adc0 [usbus4]
> 100056 D - 0xffffff800034ad68 [usbus4]
> 100055 D - 0xffffff800034ad10 [usbus4]
> 100054 D - 0xffffff8000341ef0 [usbus3]
> 100053 D - 0xffffff8000341e98 [usbus3]
> 100052 D - 0xffffff8000341e40 [usbus3]
> 100051 D - 0xffffff8000341de8 [usbus3]
> 100049 D - 0xffffff8000338ef0 [usbus2]
> 100048 D - 0xffffff8000338e98 [usbus2]
> 100047 D - 0xffffff8000338e40 [usbus2]
> 100046 D - 0xffffff8000338de8 [usbus2]
> 100044 D - 0xffffff800032fef0 [usbus1]
> 100043 D - 0xffffff800032fe98 [usbus1]
> 100042 D - 0xffffff800032fe40 [usbus1]
> 100041 D - 0xffffff800032fde8 [usbus1]
> 100039 D - 0xffffff8000326ef0 [usbus0]
> 100038 D - 0xffffff8000326e98 [usbus0]
> 100037 D - 0xffffff8000326e40 [usbus0]
> 100036 D - 0xffffff8000326de8 [usbus0]
> Mo8e 0 0 0 SL idle 0xffffff80002d7300 [mpt_raid1]
> 7 0 0 0 SL idle 0xffffff80002d7000 [mpt_recovery1]
> 6 0 0 0 SL idle 0xffffff80002c2300 [mpt_raid0]
> 5 0 0 0 SL idle 0xffffff80002c2000 [mpt_recovery0]
> 13 0 0 0 SL - 0xffffffff80d031a4 [yarrow]
> 4 0 0 0 SL - 0xffffffff80cff748 [g_down]
> 3 0 0 0 SL - 0xffffffff80cff740 [g_up]
> 2 0 0 0 SL - 0xffffffff80cff730 [g_event]
> 12 0 0 0 WL (threaded) intr
> 100061 I [swi0: uart]
> 100060 I [irq1: atkbd0]
> 100059 I [irq14: ata0]
> 100050 I [irq16: uhci3]
> 100045 I [irq18: uhci2]
> 100040 I [irq19: uhci1+]
> 100035 I [irq23: uhci0 ehci0]
> 100030 I [irq49: mpt1]
> 100027 I [irq48: mpt0]
> 100024 I [irq9: acpi0]
> 100022 I [swi5: +]
> 100017 I [swi2: cambio]
> 100015 I [swi6: task queue]
> 100014 I [swi6: Giant taskq]
> 100008 I [swi3: vm]
> 100007 I [swi4: clock]
> 100006 I [swi4: clock]
> 100005 I [swi1: netisr 0]
> 11 0 0 0 RL (threaded) idle
> 100004 Run CPU 0 [idle: cpu0]
> 100003 Run CPU 1 [idle: cpu1]
> 1 0 1 0 SLs wait 0xffffff00033fb8e0 [init]
> 10 0 0 0 SL audit_wo 0xffffffff80d32390 [audit]
> 0 0 0 0 SLs (threaded) kernel
> 100250 D - 0xffffff001d741b00 [zil_clean]
> 100249 D - 0xffffff001de7b500 [zil_clean]
> 100248 D - 0xffffff001de6b580 [zil_clean]
> 100247 D - 0xffffff001de66080 [zil_clean]
> 100246 D - 0xffffff001de51e00 [zil_clean]
> 100245 D - 0xffffff001de51400 [zil_clean]
> 100244 D - 0xffffff001d74e580 [zil_clean]
> 100243 D - 0xffffff001de04100 [zil_clean]
> 100229 D - 0xffffff0003f14980 [zfs_vn_rele_taskq]
> 100228 D - 0xffffff0006301b80 [zio_ioctl_intr]
> 100227 D - 0xffffff0006301b00 [zio_ioctl_issue]
> 100226 D - 0xffffff0006301a80 [zio_claim_intr]
> 100225 D - 0xffffff0006301a00 [zio_claim_issue]
> 100224 D - 0xffffff0006301980 [zio_free_intr]
> 100223 D - 0xffffff0006301900 [zio_free_issue_99]
> 100222 D - 0xffffff0006301900 [zio_free_issue_98]
> 100221 D - 0xffffff0006301900 [zio_free_issue_97]
> 100220 D - 0xffffff0006301900 [zio_free_issue_96]
> 100219 D - 0xffffff0006301900 [zio_free_issue_95]
> 100218 D - 0xffffff0006301900 [zio_free_issue_94]
> 100217 D - 0xffffff0006301900 [zio_free_issue_93]
> 100216 D - 0xffffff0006301900 [zio_free_issue_92]
> 100215 D - 0xffffff0006301900 [zio_free_issue_91]
> 100214 D - 0xffffff0006301900 [zio_free_issue_90]
> 100213 D - 0xffffff0006301900 [zio_free_issue_89]
> 100212 D - 0xffffff0006301900 [zio_free_issue_88]
> 100211 D - 0xffffff0006301900 [zio_free_issue_87]
> 100210 D - 0xffffff0006301900 [zio_free_issue_86]
> 100209 D - 0xffffff0006301900 [zio_free_issue_85]
> 100208 D - 0xffffff0006301900 [zio_free_issue_84]
> 100207 D - 0xffffff0006301900 [zio_free_issue_83]
> 100206 D - 0xffffff0006301900 [zio_free_issue_82]
> 100205 D - 0xffffff0006301900 [zio_free_issue_81]
> 100204 D - 0xffffff0006301900 [zio_free_issue_80]
> 100203 D - 0xffffff0006301900 [zio_free_issue_79]
> 100202 D - 0xffffff0006301900 [zio_free_issue_78]
> 100201 D - 0xffffff0006301900 [zio_free_issue_77]
> 100200 D - 0xffffff0006301900 [zio_free_issue_76]
> 100199 D - 0xffffff0006301900 [zio_free_issue_75]
> 100198 D - 0xffffff0006301900 [zio_free_issue_74]
> 100197 D - 0xffffff0006301900 [zio_free_issue_73]
> 100196 D - 0xffffff0006301900 [zio_free_issue_72]
> 100195 D - 0xffffff0006301900 [zio_free_issue_71]
> 100194 D - 0xffffff0006301900 [zio_free_issue_70]
> 100193 D - 0xffffff0006301900 [zio_free_issue_69]
> 100192 D - 0xffffff0006301900 [zio_free_issue_68]
> 100191 D - 0xffffff0006301900 [zio_free_issue_67]
> 100190 D - 0xffffff0006301900 [zio_free_issue_66]
> 100189 D - 0xffffff0006301900 [zio_free_issue_65]
> 100188 D - 0xffffff0006301900 [zio_free_issue_64]
> 100187 D - 0xffffff0006301900 [zio_free_issue_63]
> 100186 D - 0xffffff0006301900 [zio_free_issue_62]
> 100185 D - 0xffffff0006301900 [zio_free_issue_61]
> 100184 D - 0xffffff0006301900 [zio_free_issue_60]
> 100183 D - 0xffffff0006301900 [zio_free_issue_59]
> 100182 D - 0xffffff0006301900 [zio_free_issue_58]
> 100181 D - 0xffffff0006301900 [zio_free_issue_57]
> 100180 D - 0xffffff0006301900 [zio_free_issue_56]
> 100179 D - 0xffffff0006301900 [zio_free_issue_55]
> 100178 D - 0xffffff0006301900 [zio_free_issue_54]
> 100177 D - 0xffffff0006301900 [zio_free_issue_53]
> 100176 D - 0xffffff0006301900 [zio_free_issue_52]
> 100175 D - 0xffffff0006301900 [zio_free_issue_51]
> 100174 D - 0xffffff0006301900 [zio_free_issue_50]
> 100173 D - 0xffffff0006301900 [zio_free_issue_49]
> 100172 D - 0xffffff0006301900 [zio_free_issue_48]
> 100171 D - 0xffffff0006301900 [zio_free_issue_47]
> 100170 D - 0xffffff0006301900 [zio_free_issue_46]
> 100169 D - 0xffffff0006301900 [zio_free_issue_45]
> 100168 D - 0xffffff0006301900 [zio_free_issue_44]
> 100167 D - 0xffffff0006301900 [zio_free_issue_43]
> 100166 D - 0xffffff0006301900 [zio_free_issue_42]
> 100165 D - 0xffffff0006301900 [zio_free_issue_41]
> 100164 D - 0xffffff0006301900 [zio_free_issue_40]
> 100163 D - 0xffffff0006301900 [zio_free_issue_39]
> 100162 D - 0xffffff0006301900 [zio_free_issue_38]
> 100161 D - 0xffffff0006301900 [zio_free_issue_37]
> 100160 D - 0xffffff0006301900 [zio_free_issue_36]
> 100159 D - 0xffffff0006301900 [zio_free_issue_35]
> 100158 D - 0xffffff0006301900 [zio_free_issue_34]
> 100157 D - 0xffffff0006301900 [zio_free_issue_33]
> 100156 D - 0xffffff0006301900 [zio_free_issue_32]
> 100155 D - 0xffffff0006301900 [zio_free_issue_31]
> 100154 D - 0xffffff0006301900 [zio_free_issue_30]
> 100153 D - 0xffffff0006301900 [zio_free_issue_29]
> 100152 D - 0xffffff0006301900 [zio_free_issue_28]
> 100151 D - 0xffffff0006301900 [zio_free_issue_27]
> 100150 D - 0xffffff0006301900 [zio_free_issue_26]
> 100149 D - 0xffffff0006301900 [zio_free_issue_25]
> 100148 D - 0xffffff0006301900 [zio_free_issue_24]
> 100147 D - 0xffffff0006301900 [zio_free_issue_23]
> 100146 D - 0xffffff0006301900 [zio_free_issue_22]
> 100145 D - 0xffffff0006301900 [zio_free_issue_21]
> 100144 D - 0xffffff0006301900 [zio_free_issue_20]
> 100143 D - 0xffffff0006301900 [zio_free_issue_19]
> 100142 D - 0xffffff0006301900 [zio_free_issue_18]
> 100141 D - 0xffffff0006301900 [zio_free_issue_17]
> 100140 D - 0xffffff0006301900 [zio_free_issue_16]
> 100139 D - 0xffffff0006301900 [zio_free_issue_15]
> 100138 D - 0xffffff0006301900 [zio_free_issue_14]
> 100137 D - 0xffffff0006301900 [zio_free_issue_13]
> 100136 D - 0xffffff0006301900 [zio_free_issue_12]
> 100135 D - 0xffffff0006301900 [zio_free_issue_11]
> 100134 D - 0xffffff0006301900 [zio_free_issue_10]
> 100133 D - 0xffffff0006301900 [zio_free_issue_9]
> 100132 D - 0xffffff0006301900 [zio_free_issue_8]
> 100131 D - 0xffffff0006301900 [zio_free_issue_7]
> 100130 D - 0xffffff0006301900 [zio_free_issue_6]
> 100129 D - 0xffffff0006301900 [zio_free_issue_5]
> 100128 D - 0xffffff0006301900 [zio_free_issue_4]
> 100127 D - 0xffffff0006301900 [zio_free_issue_3]
> 100126 D - 0xffffff0006301900 [zio_free_issue_2]
> 100125 D - 0xffffff0006301900 [zio_free_issue_1]
> 100124 D - 0xffffff0006301900 [zio_free_issue_0]
> 100123 D - 0xffffff0006301880 [zio_write_intr_high]
> 100122 D - 0xffffff0006301880 [zio_write_intr_high]
> 100121 D - 0xffffff0006301880 [zio_write_intr_high]
> 100120 D - 0xffffff0006301880 [zio_write_intr_high]
> 100119 D - 0xffffff0006301880 [zio_write_intr_high]
> 100118 D - 0xffffff0006301800 [zio_write_intr_7]
> 100117 D - 0xffffff0006301800 [zio_write_intr_6]
> 100116 D - 0xffffff0006301800 [zio_write_intr_5]
> 100115 D - 0xffffff0006301800 [zio_write_intr_4]
> 100114 D - 0xffffff0006301800 [zio_write_intr_3]
> 100113 D - 0xffffff0006301800 [zio_write_intr_2]
> 100112 D - 0xffffff0006301800 [zio_write_intr_1]
> 100111 D - 0xffffff0006301800 [zio_write_intr_0]
> 100110 D - 0xffffff0006301780 [zio_write_issue_hig]
> 100109 D - 0xffffff0006301780 [zio_write_issue_hig]
> 100108 D - 0xffffff0006301780 [zio_write_issue_hig]
> 100107 D - 0xffffff0006301780 [zio_write_issue_hig]
> 100106 D - 0xffffff0006301780 [zio_write_issue_hig]
> 100105 D vmwait 0xffffffff80d33adc [zio_write_issue_1]
> 100104 D vmwait 0xffffffff80d33adc [zio_write_issue_0]
> 100103 D vmwait 0xffffffff80d33adc [zio_read_intr_1]
> 100102 D vq->vq_l 0xffffff0006692600 [zio_read_intr_0]
> 100101 D - 0xffffff0006301600 [zio_read_issue_7]
> 100100 D - 0xffffff0006301600 [zio_read_issue_6]
> 100099 D - 0xffffff0006301600 [zio_read_issue_5]
> 100098 D - 0xffffff0006301600 [zio_read_issue_4]
> 100097 D - 0xffffff0006301600 [zio_read_issue_3]
> 100096 D - 0xffffff0006301600 [zio_read_issue_2]
> 100095 D - 0xffffff0006301600 [zio_read_issue_1]
> 100094 D - 0xffffff0006301600 [zio_read_issue_0]
> 100093 D - 0xffffff0006064e00 [zio_null_intr]
> 100092 D - 0xffffff0006084000 [zio_null_issue]
> 100063 D - 0xffffff0003ee2880 [system_taskq_1]
> 100062 D - 0xffffff0003ee2880 [system_taskq_0]
> 100034 D - 0xffffff0003620500 [em3 taskq]
> 100033 D - 0xffffff0003611000 [em2 taskq]
> 100026 D - 0xffffff00035d1d00 [em1 taskq]
> 100025 D - 0xffffff00035d0e00 [em0 taskq]
> 100023 D - 0xffffff000357f180 [thread taskq]
> 100021 D - 0xffffff000357f300 [acpi_task_2]
> 100020 D - 0xffffff000357f300 [acpi_task_1]
> 100019 D - 0xffffff000357f300 [acpi_task_0]
> 100018 D - 0xffffff000357f380 [kqueue taskq]
> 100016 D - 0xffffff000352b900 [ffs_trim taskq]
> 100012 D - 0xffffff00033f7d00 [firmware taskq]
> 100000 D vmwait 0xffffffff80d33adc [swapper]
> db> call doadump
> Dumping 8191 out of 8173 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
> Dump complete
> = 0
> db> reset
>
>
--
Peace can only be achieved by understanding - A. Einstein
More information about the freebsd-stable
mailing list