Intermittent hangs in vmem_periodic()

[hiren panchasara]

On Wed, Sep 11, 2013 at 9:51 PM, hiren panchasara <
hiren.panchasara at gmail.com> wrote:

> I am seeing really weird system freeze for 2-3 seconds every minute or so.
>
> % uname -a
> FreeBSD flymockour-l7.corp.yahoo.com 10.0-CURRENT FreeBSD 10.0-CURRENT #2
> r255423M: Mon Sep  9 12:29:22 PDT 2013     root at flymocko
> ur-l7.corp.yahoo.com:/usr/obj/usr/home/hirenp/head/sys/GENERIC  amd64
>
> Simple comands like "man top" hangs and this is what I see in ctrl+T
>  % man top
>  load: 1.59  cmd: zcat 76574 [*kmem arena] 2.16r 0.00u 1.82s 16% 2000k
>  load: 1.70  cmd: zcat 76574 [*kmem arena] 7.19r 0.00u 1.82s 7% 2000k
>  load: 1.89  cmd: zcat 76574 [*kmem arena] 12.58r 0.00u 1.82s 3% 2000k
>  load: 1.90  cmd: zcat 76574 [*kmem arena] 14.58r 0.00u 1.82s 2% 2000k
>
> I see similar things while trying to "exit" a terminal window.
>
> I see following in pmcstat:
>
> PMC: [INSTR_RETIRED_ANY] Samples: 117647 (100.0%) , 6885 unresolved
>
> %SAMP CALLTREE
>  75.9 fork_exit at kernel taskqueue_thread_loop taskqueue_run_locked
> vmem_periodic(75.9%)
>

At any point in time whether system is misbehaving or not, I see this on
top of the graph. Which seems wrong.

Providing a few samples over the time:

PMC: [INSTR_RETIRED_ANY] Samples: 97253 (100.0%) , 6080 unresolved

%SAMP CALLTREE
 78.1 fork_exit at kernel taskqueue_thread_loop taskqueue_run_locked
vmem_periodic(78.1%)
  1.0 bsearch at libc.so.7(1.0%)
  0.7 0xe7b661 at libnvidia-glcore.s clock_gettime at libc.so.7 0x128c6e
0x128a80(0.6%)
  0.2 amd64_syscall at kernel sys_select kern_select
  0.2 lapic_handle_timer at kernel timercb handleevents hardclock_cnt
pmc_hook_handler at hwpmc.ko pmc_process_samples
  0.0 amd64_syscall at kernel sys_poll
  0.0 free at libc.so.7
  0.0 0x212ff56 at libxul.so 0x2130835 0x212f907 0x212f1e7 0x212d2fa 0x22b6777
0x22b6478

PMC: [INSTR_RETIRED_ANY] Samples: 35320 (100.0%) , 3663 unresolved

%SAMP CALLTREE
 32.4 fork_exit at kernel taskqueue_thread_loop taskqueue_run_locked
vmem_periodic(32.4%)
  1.6 free at libc.so.7 0x9ad75 0x9a597(1.4%)
  1.3 intr_event_execute_handlers at kernel hdac_intr_handler hdaa_stream_intr
chn_intr chn_intr_locked sndbuf_feed
      feed_mixer_feed z_resampler_feed z_feed_linear_S16LE(1.0%)
  1.3 hdac_intr_handler at kernel hdaa_stream_intr chn_intr chn_intr_locked
sndbuf_feed feed_mixer_feed z_resampler_feed
      z_feed_linear_S16LE(1.0%)
  1.2 amd64_syscall at kernel sys___sysctl userland_sysctl sysctl_root
cpufreq_curr_sysctl cf_set_method est_set DELAY(0.6%)
      get_tsc(0.5%)
  1.1 bsearch at libc.so.7(1.1%)

PMC: [INSTR_RETIRED_ANY] Samples: 118799 (100.0%) , 7859 unresolved

%SAMP CALLTREE
 70.5 fork_exit at kernel taskqueue_thread_loop taskqueue_run_locked
vmem_periodic(70.5%)
  1.1 bsearch at libc.so.7(1.1%)
  0.2 lapic_handle_timer at kernel timercb handleevents hardclock_cnt
pmc_hook_handler at hwpmc.ko pmc_process_samples
  0.1 free at libc.so.7
  0.1 0xe7b661 at libnvidia-glcore.s clock_gettime at libc.so.7
  0.1 amd64_syscall at kernel sys_select kern_select
  0.1 __realloc at libc.so.7
  0.1 amd64_syscall at kernel sys_poll

cheers,
Hiren

>
> I am not running anything funny and I do not see anything weird in "top"
> too.
>
> Any help is appreciated.
>
> cheers,
> Hiren
>