Re: measuring swap partition speed
- In reply to: Ronald Klop : "Re: measuring swap partition speed"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Fri, 22 Dec 2023 09:59:26 UTC
Van: Ronald Klop <ronald-lists@klop.ws>
Datum: vrijdag, 22 december 2023 10:37
Aan: void <void@f-m.fm>
CC: freebsd-fs <freebsd-fs@freebsd.org>
Onderwerp: Re: measuring swap partition speed
>
>
> Van: void <void@f-m.fm>
> Datum: vrijdag, 22 december 2023 01:05
> Aan: freebsd-fs <freebsd-fs@freebsd.org>
> Onderwerp: Re: measuring swap partition speed
>>
>> On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote:
>> > A bit weird that a simple dd is so slow.
>> >
>> > Just a quick thought.
>> > Are your partitions aligned properly?
>>
>> no idea. How would I check? The system when installed,
>> the auto-zfs option was selected, block size set to 4k. The swap
>> was changed from the default 2gb to 12gb. geli encryption is
>> active for the zfs filesystem but not for swap. It's only the swap
>> that shows performance issues.
>>
>> > Is other IO going on on the same time?
>>
>> not really, the tests were done with cron jobs deactivated, almost completely idle,
>> swap was not being used at all.
>>
>> > What does gstat say about %util, queue length and all the other stats
>> > while running the dd? Or "iostat -x -d 1".
>>
>> I'll try that next, thanks
>>
>> > Could you try if another disk has the same issues?
>>
>> no spare or equivalent disk for this arch.
>>
>> > How is your disk connected? USB-to-SATA-adapter? Any output of dmesg?
>>
>> Regular usb3 connection.
>>
>> da0: 400.000MB/s transfers
>> da0: 953869MB (1953525168 512 byte sectors)
>> da0: quirks=0x2<NO_6_BYTE>
>> da0: Delete methods: <NONE(*),ZERO>
>> --
>>
>>
>>
>>
>
>
> Can you provide more concrete information? Like:
>
> # usbconfig list
> ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
> ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA)
> ugen0.3: <Silicon Labs CP2102 USB to UART Bridge Controller> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
> ugen0.4: <USB 3.0 Device USB 3.0 Device> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
> ugen0.5: <Western Digital Elements 25A2> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
>
> And the output of "devinfo -v". That will give people much more information to work with instead of "anecdotal evidence" about your hardware being usb3.
> And the complete output of dmesg instead of only the "da0:" lines. There is more information in there which can be valuable for people to help you.
>
> About the alignment. You wrote in your first mail:
> "# gpart show
>
> => 40 1953525088 da0 GPT (932G)
> 40 532480 1 efi (260M)
> 532520 2008 - free - (1.0M)
> 534528 4194304 2 freebsd-swap (2.0G)
> 4728832 4194304 4 freebsd-swap (2.0G)
> 8923136 4194304 5 freebsd-swap (2.0G)
> 13117440 4194304 6 freebsd-swap (2.0G)
> 17311744 4194304 7 freebsd-swap (2.0G)
> 21506048 4194304 8 freebsd-swap (2.0G)
> 25700352 1927823360 3 freebsd-zfs (920G)
> 1953523712 1416 - free - (708K)
> "
>
> Your dd tests were on da0p4 which starts on block 4728832.
> Assuming "gpart list" shows "Sectorsize: 512".
> (4728832 * 512) mod 4096 = 0, so the partitiion is aligned on a 4KB page size. And the calculation is also 0 for 8KB (is this the page size of arm64?) which is good.
> AFAIK: this is properly aligned such that the hardware does not need to double write or read-before-write when paging out.
>
> Up until now I haven't seen anything which would explain the slow dd speed on of=/dev/da0p4 vs the quick dd on of=/mnt/test8k.bin.
>
> Regards,
> Ronald.
>
Just did the same test on my RPI4 running FreeBSD 15.0-CURRENT of Dec 14.
Used an unused partition on spinning disk. My used swap is on an ssd nowadays.
[root@rpi4 ~]# dd if=/dev/urandom of=/dev/da1p2 bs=8k count=25000 conv=sync status=progress
193200128 bytes (193 MB, 184 MiB) transferred 10.024s, 19 MB/s
25000+0 records in
25000+0 records out
204800000 bytes transferred in 10.592984 secs (19333551 bytes/sec)
NB: using bs=128k makes it go to 69MB/s.
# gpart show
=> 40 1953458096 da1 GPT (931G)
40 102400 1 efi (50M)
102440 8388608 2 freebsd-swap (4.0G)
8491048 1944967088 3 freebsd-zfs (927G)
ugen0.5: <Western Digital Elements 25A2> at usbus0
umass1 on uhub0
umass1: <Western Digital Elements 25A2, class 0/0, rev 3.00/10.04, addr 4> on usbus0
da1 at umass-sim1 bus 1 scbus1 target 0 lun 0
da1: <WD Elements 25A2 1004> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number 575843314538354834414536
da1: 400.000MB/s transfers
da1: 953837MB (1953458176 512 byte sectors)
da1: quirks=0x2<NO_6_BYTE>
# usbconfig list
ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA)
ugen0.3: <Silicon Labs CP2102 USB to UART Bridge Controller> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
ugen0.4: <USB 3.0 Device USB 3.0 Device> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
ugen0.5: <Western Digital Elements 25A2> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
Found some more things to check:
Do these settings have something non-default?
# sysctl kern.geom.disk
# sysctl kern.cam
And your disk prints this in dmesg. How does that influence things? I haven't seen this before anywhere.
da0: Delete methods: <NONE(*),ZERO>
Because of this I'm particularly interested in:
sysctl kern.cam.da.0.delete_method
and
sysctl kern.cam.da.0.trim_count
I might be totally of in my thinking here, but you never know with such unpredictable problems.
Regards,
Ronald.