use lve wrapper to reduce I/O for cpbackup

[steven_elvisda]

As many cPanel administrators know, the backup process (cpbackup) can be a real resource hog.
Especially the disk i/o can bring a busy server to a grinding halt.

Would it be possible to use lve_wrapper or lve_suwrapper to wrap the cpbackup process within an LVE, and thus gaining control of the choking I/O usage of the backup process?
If so, how do I change the cron for cpbackup to allow for lve wrapping?

information about the wrappers on http://docs.cloudlinux.com/index.html?lve_wrappers.html

[iseletsk]

What does cpbackup cron job line looks like?
In most cases it should be possible.

[info]

cron job is /usr/local/cpanel/scripts/cpbackup

One thing to note: this script calls all kinds of other scripts, including pkgacct which calls

/usr/local/cpanel/3rdparty/bin/pigz -6 --processes 3 --blocksize 128

pigz is a gzip replacement which can utilitize multiple CPUs (in my case set to 3), thus speeding up the zip process. So I think I should allow the cpbackup script to use at least 3 CPUs at 100%?

<edit>
pigz is a new option in cPanel 11.38, and is meant to speed up the backup (which it certainly does, except that disk I/O stays high, only for a shorter period of time).

</edit>

[iseletsk]

This should be fine, all called scripts will be under same LVE.
Try changing the line to:

lve_suwrapper XX /usr/local/cpanel/scripts/cpbackup

Where XX is LVE ID. You can use any number you want, like 50. Then use that number to monitor/adjust limits, like:
lvectp set 50 --cpu=50 --ncpu=100 --io=2048
etc..

And tell us if it works for you

[info]

Thanks, will try this.

If this works, it will be a real life save for me and many other cPanel administrators

[info]

Its working indeed!

I will be watching Munin closely while experimenting with the --io parameter to find the optimal setting for my server. Currently using --io=12000 with with disk I/O monitor in Munin never exeeding 60% (was peaking at 100% several times a night before during backups)

Will the lvectl settings be lost after server reboot?

[iseletsk]

The settings will persist after reboot.

[webmaster]

Will this command limit pigz? I have noticed that pigz is not using all resources to kae faster backups I currently have a 16 core processor, is there any way to optimize this? I used the command lve_suwrapper 100 /usr/local/cpanel/bin/backup but do not know if this is right I figured that 400 will be the % of cpu for each core so pigz can use.

Can anyone help?
Regards

[bogdan.sh]

Yes that command will limit pigz also, as we use SPEED for limits all your cores are 1600% . However as far as I remember pigz has their own method to limit CPU core usage called
umber of threads in WHM interface, maybe it will be better to tune it with that parameter (8 threads = half of your server power).

So, the best what I can propose:
1. Limit CPU usage by pigz with their
umber of threads
2. Limit IO usage with lve_suwrappers, so pigz will not hammer disk IOwait

You may want to play with settings to get better acceptable results. And we will appreciate if you come back with them from a real server

[bogdan.sh]

Hi, is possible use lve_wrapper in cPanel user panel. From cPanel ->backups - create

[vmarchuk]

Hello,

According to our documentation:
/bin/lve_wrapper – can be used by any non-root user, as long as that user is in group lve (see /etc/groups file).

Please note, that LVE Wrappers tool available only from command line interface.