FreeBSD ZFS file server with SSD HDD
Kate Dawson
k4t at 3msg.es
Thu Oct 12 10:38:22 UTC 2017
On Wed, Oct 11, 2017 at 09:41:13AM -0500, Adam Vande More wrote:
> On Wed, Oct 11, 2017 at 8:05 AM, Kate Dawson <k4t at 3msg.es> wrote:
>
> > Hi,
> >
> > If I look at the output of `zpool iostat -v tank 1 ` I see that every
> > five seconds or so, the numner of write operation go to > 2k
> >
> > I think this shows that the I'm hitting the limit that the spinning disk
> > can provide in this workload.
> >
>
> I doubt that is the cause. It is more likely you have
>
> vfs.zfs.txg.timeout
This is set to 5
I may try decreasing it and seeing how things go for a few hours.
>
> set to the default. Have you tried any other zfs or nfs tuning? If so,
> please share those details.
vfs.zfs.l2arc_noprefetch=0
zfs get sync tank/exports/xendomu
NAME PROPERTY VALUE SOURCE
tank/exports/xendomu sync disabled local
>
> Does gstat reveal anything useful?
>
Thanks for this tool... on regular occasions drives show high percentage busy
dT: 1.001s w: 1.000s filter: eli L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
5 247 0 0 0.0 247 15160 16.2 69.1| da2.eli
0 304 0 0 0.0 304 16219 14.1 71.6| da3.eli
0 240 0 0 0.0 240 15108 16.3 66.6| da4.eli
2 231 0 0 0.0 231 16695 16.1 65.0| da5.eli
1 227 0 0 0.0 227 16703 16.5 66.7| da9.eli
0 304 0 0 0.0 304 16219 13.8 72.7| da8.eli
0 193 0 0 0.0 193 15872 20.2 65.8| ada0.eli
3 263 0 0 0.0 263 13878 15.7 70.7| ada2.eli
0 193 0 0 0.0 193 15872 20.4 64.6| ada1.eli
3 269 0 0 0.0 269 13914 14.7 69.9| ada3.eli
0 256 0 0 0.0 256 16139 16.4 71.5| ada4.eli
0 258 0 0 0.0 258 16139 16.6 72.5| ada5.eli
dT: 1.001s w: 1.000s filter: eli
L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name
0 265 6 739 36.8 257 19425 17.5 103.0| da2.eli
0 260 1 108 18.0 258 21643 25.8 87.1| da3.eli
0 268 8 927 29.4 258 19437 20.2 95.0| da4.eli
1 305 14 1754 32.6 289 21379 17.6 101.2| da5.eli
0 300 8 871 34.8 290 21391 20.1 98.4| da9.eli
0 273 15 1786 30.2 257 21635 19.7 94.1| da8.eli
0 292 13 1378 29.5 278 21731 22.9 103.0| ada0.eli
0 285 10 1187 30.2 273 21003 16.5 77.7| ada2.eli
0 290 10 1091 40.0 279 21731 20.6 97.7| ada1.eli
0 269 4 384 33.4 263 21015 19.7 112.2| ada3.eli
0 251 5 619 23.0 245 21343 21.4 89.0| ada4.eli
0 250 7 791 31.7 242 21199 21.7 88.3| ada5.eli
Here is some 30s or so of `zpool iostat tank 1 `
root at blue:/tank/exports/xendomu/restore # zpool iostat tank 1
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
tank 3.39T 2.05T 80 521 7.04M 13.8M
tank 3.39T 2.05T 2 0 160K 0
tank 3.39T 2.05T 8 1.74K 461K 45.4M
tank 3.39T 2.05T 107 1.04K 11.9M 25.0M
tank 3.39T 2.05T 29 0 2.48M 0
tank 3.39T 2.05T 27 0 1.38M 0
tank 3.39T 2.05T 4 0 116K 0
tank 3.39T 2.05T 15 1.52K 1.27M 23.4M
tank 3.39T 2.05T 12 0 846K 0
tank 3.39T 2.05T 901 0 108M 0
tank 3.39T 2.05T 14 0 814K 0
tank 3.39T 2.05T 1 1.34K 95.8K 103M
tank 3.39T 2.05T 249 1.90K 29.2M 114M
tank 3.39T 2.05T 530 0 59.4M 0
tank 3.39T 2.05T 3 0 180K 0
tank 3.39T 2.05T 83 0 7.47M 0
tank 3.39T 2.05T 4 2.80K 355K 106M
tank 3.39T 2.05T 5 229 263K 3.09M
tank 3.39T 2.05T 2 0 116K 0
tank 3.39T 2.05T 4 0 180K 0
tank 3.39T 2.05T 7 0 439K 0
tank 3.39T 2.05T 5 1.46K 447K 24.0M
tank 3.39T 2.05T 1 0 43.9K 0
tank 3.39T 2.05T 106 0 11.7M 0
tank 3.39T 2.05T 15 0 814K 0
tank 3.39T 2.05T 12 0 974K 0
tank 3.39T 2.05T 10 1.74K 675K 29.7M
tank 3.39T 2.05T 6 0 236K 0
tank 3.39T 2.05T 37 0 1.67M 0
tank 3.39T 2.05T 7 0 747K 0
tank 3.39T 2.05T 2 0 67.9K 0
tank 3.39T 2.05T 0 2.58K 0 90.9M
tank 3.39T 2.05T 19 1.38K 1.49M 20.6M
tank 3.39T 2.05T 1 0 140K 0
tank 3.39T 2.05T 0 0 43.9K 0
tank 3.39T 2.05T 3 0 164K 0
tank 3.39T 2.05T 61 3.36K 3.55M 78.4M
regards,
Kate Dawson
--
"The introduction of a coordinate system to geometry is an act of violence"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20171012/6016faf9/attachment.sig>
More information about the freebsd-questions
mailing list