IO appears unlimited on read, limited on write

[Biglebowski]

I've got an account with IO limit set to 1024.

During write tests with dd, I get throughput of up to 1.3MB/s.
With read tests the figures vary, although I'm getting up to 1GB/s quite regularly.

The CL documentation suggests the IO limit applies to both read and write and is a combination of both.
If this is the case then how am I getting 1GB/s?

The read test uses:

dd if=./test.img of=/dev/null bs=4k iflag=fullblock,count_bytes count=10M oflag=dsync

The write test uses:

dd if=/dev/zero of=./test.img bs=4k iflag=fullblock,count_bytes count=10M oflag=dsync

Thanks

[Biglebowski]

OK, the reason this read test us ineffective is because the kernel is I/O caching reads. If I set vm_drop_caches=3 with sysctl then this reduces the read throughout temporarily. The read io then appears in lvetop, again temporarily - caching appears to deploy itself again even with the flag still set at 3.

Can anyone think of a more suitable read test ? I had to reboot my vm to reset vm_drop_caches to 0 again and I don't want a test to affect other server users.

Thanks

[bogdan.sh]

Hi there,

Indeed, dropping cache will affect it, however, the vm_drop_caches is not a regular kernel option, it's a one-time action to actually drop caches. I just tested on lab server and it works perfectly, e.g.:

As a user with 1024 io limit:

$ dd if=./test.img of=/dev/null bs=4k iflag=fullblock,count_bytes count=10M oflag=dsync
2560+0 records in
2560+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 0.00532637 s, 2.0 GB/s

Now dropping cache as a root:

sync; echo 3 > /proc/sys/vm/drop_caches

And as a user again:

$ dd if=./test.img of=/dev/null bs=4k iflag=fullblock,count_bytes count=10M oflag=dsync
2560+0 records in
2560+0 records out
10485760 bytes (10 MB, 10 MiB) copied, 10.0407 s, 1.0 MB/s​

There is no need to change vm_drop_caches to 0 again. The value is not considered after the initial action.

[Biglebowski]

Bogdan, is your test server a server or vm?

I'm using vms. The results are unreliable as presumably this kernel option is managed by the hypervisor.

There's a similar problem apparently with sysbench fileio when reading.

[bogdan.sh]

This is virtual machine, KVM based:

# virt-what
redhat
kvm