Help ZFS FreeBSD 8.0 RC2 Write performance issue
Sam Fourman Jr.
sfourman at gmail.com
Wed Nov 11 21:26:17 UTC 2009
On Wed, Nov 11, 2009 at 2:49 PM, Dan Nelson <dnelson at allantgroup.com> wrote:
> In the last episode (Nov 11), Ivan Voras said:
>> Sam Fourman Jr. wrote:
>> > I am running FreeBSD 8.0RC2 and I dont understand why my ZFS/NFS is
>> > acting weird on writes. I get ~150mbit writes idk if this is good or
>> > not? but it paused for a few seconds every once and awhile.
>>
>> You didn't give any "iostat" statistics - I suspect that if you
>> correlate ifstat and iostat output that you will see that network
>> "pauses" happen during spikes in IO. You should check for this and post
>> your results.
>
> Yes, iostat would be useful here. "iostat -zxC 2" will give you per-disk
> stats plus CPU usage every 2 seconds (CPU may be a factor if you have
> compression enabled).
>
> On a Solaris box I admin, setting zfs_write_limit_override helped stuttering
> while doing heavy writes. It's not exported on FreeBSD, but it should be
> easy to add it as a RW sysctl; it lives in dsl_pool.c and can be tweaked at
> runtime. Start big and tune it down so each write burst takes under a
> second; it looks like you're writing solid for around 6-8 seconds now. The
> number will vary depending on your disk speed and how much ARC you have.
here are some iostats for you. I do not believe I have compression enabled
am I mistaken? isn'y SATA2 300MB/s? and I am doing ~6MB/s per disk?
I built this machine with 4GB of memory because I thought ZFS would like it.
now maybe a re(4) interface isnt the best choice. if that is the
problem here I can change it.
We spent ~$800 on a hardware RAID card thinking that it would help performance
Why is it that with sftp we do not see the pauses in Network transfer?
# iostat -zxC 1 (with NFS and ZFS)
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.1 0.4 2.0 6.1 0 3.1 0 0 0 1 1 98
da0 2.2 7.2 127.0 246.7 0 120.3 6
da1 2.0 7.5 117.3 246.2 0 120.4 6
da2 2.2 7.3 127.1 246.7 0 119.8 6
da3 2.0 7.6 115.5 246.2 0 119.8 5
da4 2.2 7.3 126.1 246.7 0 119.9 5
da5 2.0 7.5 115.6 246.3 0 119.8 5
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 99
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.0 11.0 0.0 175.6 0 1.4 0 2 0 3 0 95
da0 0.0 47.9 0.0 94.3 0 1.0 1
da1 0.0 54.9 0.0 84.8 0 1.4 2
da2 0.0 44.9 0.0 92.8 0 1.0 1
da3 0.0 48.9 0.0 83.3 0 1.4 2
da4 0.0 47.9 0.0 95.3 0 1.0 1
da5 0.0 53.9 0.0 87.8 0 1.2 2
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 99
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
0 0 0 0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.0 28.9 0.0 462.8 0 3.5 1 0 0 0 0 99
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 131.7 0.0 8392.0 0.0 35 73.6 60 0 0 4 3 93
da1 118.8 0.0 7561.7 0.0 35 74.9 62
da2 127.7 0.0 8136.5 0.0 35 77.7 63
da3 103.8 0.0 6603.7 0.0 29 137.6 79
da4 106.8 0.0 6795.8 0.0 33 112.9 74
da5 123.7 0.0 7919.4 0.0 35 126.9 83
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 51.9 89.9 3322.5 1487.5 4 77.4 38 0 0 6 5 89
da1 55.9 109.8 3578.1 1469.6 4 76.4 49
da2 58.9 88.9 3769.8 1604.8 1 75.3 46
da3 69.9 105.8 4472.6 1652.8 0 117.0 82
da4 66.9 85.9 4280.9 1490.0 3 112.4 70
da5 66.9 108.8 4280.9 1469.6 3 105.3 69
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 107.7 0.0 5069.1 10 53.0 80 0 0 9 8 83
da1 0.0 107.7 0.0 5069.1 10 54.4 81
da2 0.0 103.7 0.0 4953.9 10 50.4 73
da3 0.0 102.7 0.0 4890.1 10 51.7 74
da4 0.0 108.7 0.0 5299.5 6 50.0 74
da5 0.0 108.7 0.0 5299.5 6 51.6 76
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.7 0.0 5196.4 4 50.5 76 0 0 8 8 85
da1 0.0 106.7 0.0 5196.4 4 52.1 78
da2 0.0 103.7 0.0 5196.4 4 49.3 73
da3 0.0 103.7 0.0 5196.4 4 52.1 75
da4 0.0 103.7 0.0 4991.5 3 51.7 78
da5 0.0 103.7 0.0 4991.5 5 52.8 80
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 114.8 0.0 5404.5 0 47.5 81 0 0 9 7 84
da1 0.0 111.8 0.0 5250.9 3 49.4 78
da2 0.0 113.8 0.0 5404.5 0 48.3 73
da3 0.0 113.8 0.0 5404.5 0 50.1 75
da4 0.0 111.8 0.0 5379.1 0 48.0 78
da5 0.0 111.8 0.0 5379.1 0 49.1 81
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 102.7 0.0 4890.1 13 50.2 70 0 0 8 9 83
da1 0.0 105.7 0.0 5043.7 13 53.0 77
da2 0.0 105.7 0.0 5120.5 9 47.7 73
da3 0.0 105.7 0.0 5120.5 9 50.5 75
da4 0.0 107.7 0.0 5120.1 9 49.7 77
da5 0.0 107.7 0.0 5120.1 9 51.6 79
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 5353.4 0 52.6 81 0 0 7 7 85
da1 0.0 109.8 0.0 5353.4 0 53.9 81
da2 0.0 103.8 0.0 5122.3 0 53.0 78
da3 0.0 103.8 0.0 5122.3 0 54.5 78
da4 0.0 105.8 0.0 5122.8 0 52.8 80
da5 0.0 105.8 0.0 5122.8 0 54.7 81
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 107.8 0.0 5150.4 8 50.0 77 0 0 9 8 83
da1 0.0 107.8 0.0 5150.4 8 52.3 79
da2 0.0 107.8 0.0 5176.4 8 49.2 77
da3 0.0 107.8 0.0 5176.4 8 51.5 78
da4 0.0 108.8 0.0 5175.9 8 49.9 77
da5 0.0 108.8 0.0 5175.9 8 52.7 77
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 107.7 0.0 5299.6 0 53.5 78 0 0 7 7 86
da1 0.0 107.7 0.0 5299.6 0 56.1 78
da2 0.0 105.7 0.0 5146.0 2 54.0 82
da3 0.0 103.7 0.0 5043.3 4 54.1 77
da4 0.0 107.7 0.0 5248.7 1 54.1 78
da5 0.0 107.7 0.0 5248.7 1 56.2 78
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.8 0.0 5124.5 8 48.7 77 0 0 9 9 82
da1 0.0 102.8 0.0 4893.9 12 52.0 76
da2 0.0 105.8 0.0 5022.2 13 51.7 75
da3 0.0 107.8 0.0 5125.0 13 54.9 81
da4 0.0 107.8 0.0 5150.5 8 50.2 78
da5 0.0 107.8 0.0 5150.5 8 50.7 79
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 105.7 0.0 5171.9 1 53.0 78 0 0 9 7 84
da1 0.0 109.7 0.0 5402.3 1 54.8 81
da2 0.0 107.7 0.0 5274.1 3 51.5 81
da3 0.0 105.7 0.0 5171.4 5 52.0 77
da4 0.0 105.7 0.0 5146.0 1 53.8 76
da5 0.0 105.7 0.0 5146.0 1 55.1 78
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 107.8 0.0 5124.6 9 49.8 79 0 0 9 11 80
da1 0.0 105.8 0.0 4996.8 11 52.0 78
da2 0.0 105.8 0.0 5022.3 13 51.3 73
da3 0.0 107.8 0.0 5125.1 13 54.5 79
da4 0.0 107.8 0.0 5150.6 9 49.4 76
da5 0.0 107.8 0.0 5150.6 9 51.3 77
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 105.8 0.0 5148.1 0 53.2 79 0 0 6 7 87
da1 0.0 107.8 0.0 5275.8 0 54.7 79
da2 0.0 108.8 0.0 5378.6 0 52.2 81
da3 0.0 108.8 0.0 5378.6 0 54.1 81
da4 0.0 106.8 0.0 5148.1 0 53.9 82
da5 0.0 106.8 0.0 5148.1 0 55.6 82
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.0 4.0 0.0 12.0 0 0.5 0 0 0 16 6 77
da0 0.0 108.8 0.0 4988.2 64 69.9 65
da1 0.0 109.8 0.0 5091.5 64 91.7 73
da2 0.0 107.8 0.0 5014.1 64 48.3 57
da3 0.0 86.8 0.0 3733.3 43 71.4 73
da4 0.0 109.8 0.0 5088.0 64 70.6 71
da5 0.0 104.8 0.0 4578.0 62 59.0 61
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 98.8 0.0 5724.0 69 490.3 109 0 0 2 1 97
da1 0.0 86.8 0.0 5007.5 69 562.7 102
da2 0.0 125.7 0.0 7222.9 69 465.5 114
da3 0.0 92.8 0.0 5303.4 70 538.2 100
da4 0.0 113.8 0.0 6557.3 70 486.6 104
da5 0.0 110.8 0.0 6339.2 69 468.6 114
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 104.8 0.0 6032.5 68 706.3 100 0 0 2 0 98
da1 0.0 103.8 0.0 5993.6 62 654.4 99
da2 0.0 104.8 0.0 6057.4 67 657.3 100
da3 0.0 104.8 0.0 6057.4 62 654.0 100
da4 0.0 104.8 0.0 6057.4 67 644.2 100
da5 0.0 103.8 0.0 5993.6 61 683.7 99
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 104.8 0.0 6057.3 67 624.9 100 0 0 1 1 98
da1 0.0 104.8 0.0 6057.3 63 531.2 100
da2 0.0 104.8 0.0 6057.3 68 625.4 100
da3 0.0 104.8 0.0 6032.3 63 551.6 100
da4 0.0 104.8 0.0 6057.3 68 625.7 100
da5 0.0 105.8 0.0 6121.1 61 510.1 101
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.8 0.0 6160.2 68 616.8 100 0 0 2 1 97
da1 0.0 106.8 0.0 6160.2 62 522.8 100
da2 0.0 106.8 0.0 6160.2 67 617.1 100
da3 0.0 106.8 0.0 6185.1 62 522.8 100
da4 0.0 106.8 0.0 6160.2 67 617.4 100
da5 0.0 106.8 0.0 6160.2 62 503.4 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 107.7 0.0 6221.3 68 607.9 100 0 0 0 2 98
da1 0.0 108.7 0.0 6285.1 63 515.2 100
da2 0.0 108.7 0.0 6285.1 68 608.3 101
da3 0.0 108.7 0.0 6260.2 63 515.3 100
da4 0.0 107.7 0.0 6221.3 67 608.6 100
da5 0.0 107.7 0.0 6221.3 62 496.0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 6354.5 68 599.2 101 0 0 1 1 98
da1 0.0 109.8 0.0 6329.5 63 507.8 100
da2 0.0 108.8 0.0 6290.6 65 599.4 99
da3 0.0 108.8 0.0 6290.6 62 507.9 99
da4 0.0 109.8 0.0 6354.5 66 599.7 100
da5 0.0 109.8 0.0 6329.5 62 489.0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 112.8 0.0 6518.3 31 586.1 100 0 0 1 1 98
da1 0.0 112.8 0.0 6518.3 53 480.6 100
da2 0.0 112.8 0.0 6493.4 5 573.1 100
da3 0.0 112.8 0.0 6174.5 63 677.5 100
da4 0.0 112.8 0.0 6493.4 15 576.6 100
da5 0.0 111.8 0.0 5690.1 42 846.4 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 159.6 0.0 5581.4 1 208.5 86 0 0 7 7 86
da1 0.0 188.5 0.0 6181.4 1 346.6 87
da2 0.0 131.7 0.0 4023.3 2 152.0 76
da3 0.0 191.5 0.0 7381.9 5 244.5 86
da4 0.0 140.6 0.0 4442.3 5 177.2 80
da5 0.0 164.6 0.0 5976.9 5 149.8 84
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 103.8 0.0 4945.3 13 50.8 78 0 0 12 7 81
da1 0.0 103.8 0.0 4945.3 13 52.6 79
da2 0.0 105.8 0.0 4983.7 13 52.0 77
da3 0.0 108.8 0.0 5150.5 13 54.2 80
da4 0.0 107.8 0.0 5150.5 13 52.5 79
da5 0.0 107.8 0.0 5150.5 13 54.9 79
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.8 0.0 5250.5 4 51.8 80 0 0 7 7 86
da1 0.0 106.8 0.0 5250.5 4 53.5 79
da2 0.0 107.8 0.0 5225.1 5 52.8 81
da3 0.0 107.8 0.0 5225.1 5 54.9 82
da4 0.0 102.8 0.0 5020.5 8 51.7 74
da5 0.0 102.8 0.0 5020.5 8 53.2 74
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 102.7 0.0 5069.2 0 53.3 77 0 0 7 7 85
da1 0.0 102.7 0.0 5069.2 0 56.2 80
da2 0.0 104.7 0.0 5094.7 0 54.2 80
da3 0.0 104.7 0.0 5094.7 0 55.7 80
da4 0.0 108.7 0.0 5299.6 0 55.0 83
da5 0.0 108.7 0.0 5299.6 0 57.2 83
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 5304.2 5 48.8 77 0 0 7 8 85
da1 0.0 109.8 0.0 5304.2 5 50.9 79
da2 0.0 105.8 0.0 5048.1 9 49.2 75
da3 0.0 105.8 0.0 5048.1 9 50.7 77
da4 0.0 106.8 0.0 5073.6 9 49.7 74
da5 0.0 106.8 0.0 5073.6 9 50.4 74
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 105.7 0.0 5095.2 1 55.4 80 0 0 10 8 83
da1 0.0 105.7 0.0 5095.2 1 57.4 80
da2 0.0 108.7 0.0 5376.5 1 54.4 81
da3 0.0 108.7 0.0 5376.5 1 56.5 82
da4 0.0 104.7 0.0 5120.1 5 53.7 79
da5 0.0 108.7 0.0 5350.5 1 56.6 86
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.0 13.0 0.0 207.7 0 1.8 0 0 0 10 10 80
da0 0.0 107.8 0.0 5124.5 6 49.7 78
da1 0.0 107.8 0.0 5124.5 6 51.6 79
da2 0.0 104.8 0.0 4970.7 8 49.4 73
da3 0.0 102.8 0.0 4868.4 10 52.2 70
da4 0.0 110.8 0.0 5330.1 6 50.6 80
da5 0.0 106.8 0.0 5099.5 6 51.1 75
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.0 1.0 0.0 2.0 0 0.2 0 0 0 8 10 82
da0 0.0 102.7 0.0 5069.0 0 51.9 78
da1 0.0 102.7 0.0 5069.0 0 53.1 78
da2 0.0 105.7 0.0 5197.2 0 52.0 77
da3 0.0 107.7 0.0 5299.4 0 53.1 81
da4 0.0 105.7 0.0 5069.0 0 54.6 79
da5 0.0 105.7 0.0 5069.0 0 55.1 81
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 105.8 0.0 5099.8 7 49.0 74 0 0 9 7 84
da1 0.0 105.8 0.0 5099.8 7 49.6 75
da2 0.0 110.8 0.0 5329.9 3 49.2 80
da3 0.0 110.8 0.0 5329.9 3 51.2 80
da4 0.0 109.8 0.0 5240.5 4 50.6 78
da5 0.0 106.8 0.0 5073.8 7 51.5 77
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.8 0.0 5198.3 0 53.9 80 0 0 6 7 87
da1 0.0 106.8 0.0 5198.3 0 56.4 82
da2 0.0 101.8 0.0 4968.3 0 53.5 77
da3 0.0 101.8 0.0 4968.3 0 54.4 80
da4 0.0 103.8 0.0 5057.6 0 53.3 75
da5 0.0 106.8 0.0 5224.3 0 55.6 77
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.8 0.0 5098.2 7 48.8 73 0 0 7 9 84
da1 0.0 106.8 0.0 5098.2 7 50.7 76
da2 0.0 110.8 0.0 5328.2 3 49.2 78
da3 0.0 110.8 0.0 5328.2 3 51.2 80
da4 0.0 108.8 0.0 5238.9 4 49.8 75
da5 0.0 105.8 0.0 5072.2 7 50.7 73
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 137.7 0.0 7561.1 70 223.8 94 0 0 13 2 85
da1 0.0 116.8 0.0 6087.7 70 241.2 94
da2 0.0 92.8 0.0 4976.1 70 297.9 91
da3 0.0 111.8 0.0 5906.1 70 277.4 72
da4 0.0 80.8 0.0 4320.9 70 279.3 88
da5 0.0 106.8 0.0 5625.7 70 316.1 91
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 111.7 0.0 6451.7 70 643.3 100 0 0 1 1 98
da1 0.0 111.7 0.0 6451.7 70 643.6 100
da2 0.0 111.7 0.0 6451.7 70 595.3 100
da3 0.0 104.7 0.0 6054.7 69 641.7 118
da4 0.0 112.7 0.0 6515.5 69 620.6 101
da5 0.0 103.7 0.0 5990.8 70 602.5 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 108.8 0.0 6287.9 69 635.6 100 0 0 2 2 97
da1 0.0 109.8 0.0 6326.8 70 636.0 100
da2 0.0 109.8 0.0 6326.8 70 635.7 100
da3 0.0 109.8 0.0 6351.7 69 635.9 101
da4 0.0 108.8 0.0 6262.9 70 635.4 100
da5 0.0 109.8 0.0 6326.8 70 635.3 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.0 8.0 0.0 127.7 0 1.1 0 0 0 3 0 96
da0 0.0 109.8 0.0 6326.8 65 616.5 100
da1 0.0 108.8 0.0 6287.9 62 630.3 100
da2 0.0 108.8 0.0 6287.9 66 635.8 99
da3 0.0 109.8 0.0 6326.8 60 634.5 100
da4 0.0 109.8 0.0 6351.7 66 635.5 100
da5 0.0 108.8 0.0 6287.9 61 635.4 99
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 6351.7 65 562.5 100 0 0 4 1 95
da1 0.0 109.8 0.0 6351.7 62 507.9 100
da2 0.0 109.8 0.0 6351.7 66 586.2 100
da3 0.0 108.8 0.0 6287.9 61 471.3 99
da4 0.0 108.8 0.0 6287.9 65 576.8 99
da5 0.0 109.8 0.0 6351.7 61 503.6 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 108.8 0.0 6263.0 66 562.1 99 0 0 2 0 97
da1 0.0 108.8 0.0 6263.0 63 507.6 99
da2 0.0 109.8 0.0 6326.8 66 580.3 100
da3 0.0 109.8 0.0 6326.8 61 471.1 100
da4 0.0 109.8 0.0 6326.8 65 561.7 100
da5 0.0 109.8 0.0 6326.8 61 488.7 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 129.8 0.0 4390.7 0 380.0 70 0 0 1 4 95
da1 0.0 154.7 0.0 5853.3 0 440.6 91
da2 0.0 166.7 0.0 6683.4 0 410.9 96
da3 0.0 169.7 0.0 6133.9 0 439.4 95
da4 0.0 176.7 0.0 7427.7 0 424.0 98
da5 0.0 170.7 0.0 6736.9 0 449.1 97
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 111.8 0.0 5148.2 8 44.7 75 0 0 7 7 85
da1 0.0 111.8 0.0 5148.2 8 47.3 75
da2 0.0 109.8 0.0 5148.2 8 45.2 75
da3 0.0 109.8 0.0 5148.2 8 47.2 76
da4 0.0 107.8 0.0 4917.7 13 46.1 77
da5 0.0 107.8 0.0 4917.7 13 48.2 76
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 105.7 0.0 5196.9 0 52.8 80 0 0 8 9 83
da1 0.0 105.7 0.0 5196.9 0 55.7 83
da2 0.0 106.7 0.0 5171.9 1 53.8 78
da3 0.0 106.7 0.0 5171.9 1 55.9 78
da4 0.0 109.7 0.0 5427.8 0 52.2 81
da5 0.0 109.7 0.0 5427.8 0 54.8 82
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.8 0.0 5227.4 4 47.3 75 0 0 12 8 80
da1 0.0 106.8 0.0 5227.4 4 48.9 75
da2 0.0 107.8 0.0 4996.7 9 49.3 80
da3 0.0 107.8 0.0 4996.7 9 52.0 81
da4 0.0 104.8 0.0 4996.7 8 47.2 65
da5 0.0 104.8 0.0 4996.7 8 50.0 68
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 102.7 0.0 4966.8 0 53.2 76 0 0 8 9 83
da1 0.0 102.7 0.0 4966.8 0 56.1 77
da2 0.0 105.7 0.0 5222.6 0 52.4 76
da3 0.0 105.7 0.0 5222.6 0 55.4 78
da4 0.0 106.7 0.0 5196.7 0 53.6 81
da5 0.0 106.7 0.0 5196.7 0 56.1 81
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 111.8 0.0 5279.0 6 50.9 82 0 0 6 10 84
da1 0.0 113.8 0.0 5406.7 4 52.7 82
da2 0.0 106.8 0.0 5150.7 9 49.9 76
da3 0.0 106.8 0.0 5150.7 9 51.4 77
da4 0.0 106.8 0.0 5176.6 8 49.1 76
da5 0.0 106.8 0.0 5176.6 8 50.2 78
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 103.7 0.0 5146.0 0 53.0 77 0 0 7 9 84
da1 0.0 101.7 0.0 5018.3 0 55.6 79
da2 0.0 108.7 0.0 5273.7 0 54.7 82
da3 0.0 108.7 0.0 5273.7 0 55.2 82
da4 0.0 107.7 0.0 5248.2 0 54.3 80
da5 0.0 107.7 0.0 5248.2 0 56.9 80
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 106.8 0.0 5073.6 9 49.1 74 0 0 10 9 82
da1 0.0 106.8 0.0 5073.6 9 51.1 74
da2 0.0 108.8 0.0 5304.2 5 48.1 76
da3 0.0 108.8 0.0 5304.2 5 50.3 78
da4 0.0 106.8 0.0 5048.1 9 49.3 78
da5 0.0 106.8 0.0 5048.1 9 51.2 78
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 107.8 0.0 5122.3 11 50.3 79 0 0 8 8 83
da1 0.0 107.8 0.0 5122.3 11 52.7 83
da2 0.0 103.8 0.0 4892.3 11 50.4 71
da3 0.0 103.8 0.0 4892.3 11 51.7 71
da4 0.0 109.8 0.0 5378.8 6 48.5 74
da5 0.0 109.8 0.0 5378.8 6 50.0 77
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 108.5 0.0 5188.6 4 51.0 78 0 0 7 9 84
da1 0.0 108.5 0.0 5188.6 4 53.0 80
da2 0.0 106.6 0.0 5188.6 4 51.6 72
da3 0.0 106.6 0.0 5188.6 4 52.6 73
da4 0.0 99.6 0.0 4958.1 4 50.5 76
da5 0.0 99.6 0.0 4958.1 4 52.4 77
^C
#
Here is a quick sample with sftp and ZFS (keep in mind ssh has
encryption NFS does not)
# ifstat -b -i re0
re0
Kbps in Kbps out
260931.9 8590.13
205778.7 6825.03
285794.0 9565.88
242095.0 8016.92
238060.4 7924.14
262288.7 8660.99
260094.6 8660.25
268791.7 8869.08
303607.5 10010.25
263964.4 8736.56
261743.6 8659.45
246390.6 8191.83
300105.3 10305.06
255019.9 8291.85
283355.3 9390.99
277390.9 9198.74
279070.0 9218.81
278103.5 9339.65
258649.8 8492.39
251343.6 8320.49
283036.9 9346.05
283302.4 9277.87
278886.2 9384.52
300472.5 9772.91
292509.8 9669.22
299081.4 9816.50
313643.0 10299.40
289208.9 9747.82
313738.8 10228.35
207670.6 6837.77
248762.1 8224.13
194959.1 6457.88
265009.2 9495.61
249181.9 8228.59
272940.6 8945.51
226037.8 7485.62
273701.2 9048.08
236890.4 7834.88
# iostat -zxC 1
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.1 0.4 1.8 6.1 0 3.0 0 1 0 1 1 97
da0 2.0 13.4 117.8 551.1 16 203.8 10
da1 1.9 13.7 108.8 550.6 16 200.5 10
da2 2.1 13.4 117.8 551.1 20 203.0 10
da3 1.9 13.8 107.1 550.6 20 200.1 9
da4 2.0 13.4 116.8 551.1 16 203.4 9
da5 1.9 13.8 107.3 550.6 16 199.8 9
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 108.8 0.0 6289.0 69 315.8 101 21 0 18 13 48
da1 0.0 107.8 0.0 6225.1 70 321.8 102
da2 0.0 108.8 0.0 6289.0 69 331.0 100
da3 0.0 111.8 0.0 6455.7 70 322.7 101
da4 0.0 104.8 0.0 6058.4 69 326.7 101
da5 0.0 108.8 0.0 6289.0 69 327.1 101
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 6326.8 69 633.3 100 15 0 8 9 68
da1 0.0 109.8 0.0 6326.8 70 627.8 100
da2 0.0 109.8 0.0 6351.8 69 624.3 100
da3 0.0 108.8 0.0 6287.9 69 629.0 99
da4 0.0 109.8 0.0 6351.8 69 640.0 100
da5 0.0 109.8 0.0 6351.8 69 625.0 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 6351.9 44 634.9 100 3 0 3 3 91
da1 0.0 108.8 0.0 6288.0 46 634.5 99
da2 0.0 109.8 0.0 6326.9 41 634.8 100
da3 0.0 109.8 0.0 6351.9 39 634.6 100
da4 0.0 109.8 0.0 6326.9 48 635.2 100
da5 0.0 108.8 0.0 6263.0 45 635.2 99
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 111.8 0.0 6163.1 68 356.9 98 9 0 18 6 67
da1 0.0 127.7 0.0 7109.6 68 354.4 98
da2 0.0 106.8 0.0 5829.3 68 399.2 91
da3 0.0 104.8 0.0 5701.1 67 349.2 84
da4 0.0 113.8 0.0 6291.3 68 377.4 96
da5 0.0 110.8 0.0 6124.2 68 353.0 79
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 129.8 0.0 6805.8 70 519.5 100 19 0 14 19 49
da1 0.0 119.8 0.0 5371.1 70 502.2 100
da2 0.0 93.8 0.0 5408.1 58 607.6 104
da3 0.0 139.8 0.0 6706.4 70 457.6 111
da4 0.0 132.8 0.0 7582.5 60 476.8 102
da5 0.0 130.8 0.0 6156.3 69 511.3 117
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 6326.8 70 535.4 100 17 0 12 12 59
da1 0.0 109.8 0.0 6351.8 70 523.0 100
da2 0.0 109.8 0.0 6351.8 58 396.3 100
da3 0.0 109.8 0.0 6326.8 70 521.1 100
da4 0.0 109.8 0.0 6351.8 60 431.1 100
da5 0.0 109.8 0.0 6326.8 69 515.6 101
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 6351.7 70 632.7 100 18 0 11 15 56
da1 0.0 109.8 0.0 6326.8 70 632.7 100
da2 0.0 109.8 0.0 6326.8 58 414.1 100
da3 0.0 110.8 0.0 6415.6 69 632.4 101
da4 0.0 109.8 0.0 6326.8 60 468.3 100
da5 0.0 109.8 0.0 6351.7 69 632.5 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 104.8 0.0 5996.4 0 631.6 93 22 0 8 14 57
da1 0.0 123.7 0.0 6598.7 0 616.9 99
da2 0.0 135.7 0.0 7264.3 5 668.5 101
da3 0.0 115.8 0.0 6149.6 0 623.4 97
da4 0.0 99.8 0.0 5323.8 0 746.0 89
da5 0.0 125.7 0.0 6735.4 0 615.4 99
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 126.7 0.0 4006.1 70 115.8 43 17 0 18 12 54
da1 0.0 126.7 0.0 3634.9 70 115.3 51
da2 0.0 109.8 0.0 3090.1 70 100.1 52
da3 0.0 109.8 0.0 2708.9 70 107.4 48
da4 0.0 115.8 0.0 3163.9 70 92.1 52
da5 0.0 118.8 0.0 3196.3 70 111.5 52
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 103.8 0.0 5993.5 67 592.8 108 20 0 12 15 53
da1 0.0 114.8 0.0 6646.1 62 547.2 100
da2 0.0 117.8 0.0 6812.8 67 554.3 100
da3 0.0 108.8 0.0 6287.9 64 566.6 103
da4 0.0 108.8 0.0 6287.9 66 595.2 100
da5 0.0 104.8 0.0 6057.4 62 597.1 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.8 0.0 6326.6 67 597.7 100 23 0 10 15 52
da1 0.0 109.8 0.0 6326.6 62 546.1 100
da2 0.0 109.8 0.0 6326.6 67 601.6 100
da3 0.0 110.8 0.0 6390.5 62 563.9 101
da4 0.0 109.8 0.0 6351.5 66 603.0 100
da5 0.0 109.8 0.0 6326.6 62 566.4 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
ad4 0.0 1.0 0.0 6.0 0 0.4 0 18 0 10 13 59
da0 0.0 109.7 0.0 6349.1 67 578.3 100
da1 0.0 109.7 0.0 6349.1 62 506.0 100
da2 0.0 109.7 0.0 6349.1 67 578.2 100
da3 0.0 109.7 0.0 6324.2 62 486.9 100
da4 0.0 109.7 0.0 6324.2 66 578.2 100
da5 0.0 109.7 0.0 6349.1 61 505.5 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 110.8 0.0 6367.5 37 577.3 101 18 0 10 12 60
da1 0.0 109.8 0.0 6303.6 40 505.1 100
da2 0.0 109.8 0.0 6303.6 44 577.4 100
da3 0.0 109.8 0.0 6303.6 60 486.3 100
da4 0.0 109.8 0.0 6303.6 47 577.3 100
da5 0.0 109.8 0.0 6316.6 58 500.2 100
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 145.7 0.0 5188.2 70 280.0 85 19 0 16 10 55
da1 0.0 146.7 0.0 4827.9 70 360.9 89
da2 0.0 154.7 0.0 6099.2 70 300.0 83
da3 0.0 173.6 0.0 6462.0 70 371.0 84
da4 0.0 163.7 0.0 6266.4 70 297.1 91
da5 0.0 176.6 0.0 6327.8 70 352.1 89
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 122.7 0.0 7082.2 69 507.1 102 25 0 11 13 51
da1 0.0 117.8 0.0 6812.8 70 514.6 99
da2 0.0 104.8 0.0 6057.3 69 589.2 108
da3 0.0 102.8 0.0 5929.6 69 591.3 109
da4 0.0 107.8 0.0 6224.0 70 561.0 101
da5 0.0 105.8 0.0 6096.3 70 566.3 104
extended device statistics cpu
device r/s w/s kr/s kw/s wait svc_t %b us ni sy in id
da0 0.0 109.7 0.0 6348.9 69 632.2 100 20 0 12 11 57
da1 0.0 109.7 0.0 6324.0 70 633.1 100
da2 0.0 109.7 0.0 6324.0 69 632.3 100
da3 0.0 109.7 0.0 6348.9 69 633.1 100
da4 0.0 109.7 0.0 6348.9 70 633.1 100
da5 0.0 109.7 0.0 6348.9 70 632.3 100
^C
#
More information about the freebsd-current
mailing list