server load arrive 650% (cause is a website)

[naja7host]

the server load arrive 650% , the cause is a website !!!

[naja7host]

Hello

is the last two days , i found the server is overloaded fr om 80% to 650% !!!! , when i have make some recherche and checks, i found the cause is a website !!!

as i know the cloudlinux limit the cpu usage for useres , but how the user can sature all the cpu .

i made limit

CPU 10
memory 4Gb .
Concurrent Connections: 20
Number of Cores Per LVE*: 8

how i can check that really the cpu lim it is working ?

is some website , i see the cpu usage in the cpanel controle that is used .

or is there any error with my confuguration ?

i have cagefs enabled also .

[naja7host]

i have found hundred of this thread in the apache status

2-4 182847 0/36/1030 W 17.03 30 0 0.0 0.35 23.91 173.236.110.130 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
3-4 181741 0/82/1012 W 37.10 28 0 0.0 1.84 20.29 62.162.58.41 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
4-4 182904 0/20/859 W 1.11 34 0 0.0 0.27 14.38 134.0.11.123 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
5-4 181743 0/87/851 W 34.67 30 0 0.0 1.39 17.12 187.17.98.9 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
7-4 182244 0/94/852 W 5.12 37 0 0.0 1.91 17.46 182.50.130.194 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
8-4 183333 2/2/871 W 0.12 32 0 0.0 0.00 16.57 182.50.130.194 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
9-4 182869 0/34/809 W 2.09 36 0 0.0 0.42 14.35 72.167.232.190 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
10-4 179635 0/143/814 W 22.36 35 0 0.0 2.47 15.30 117.121.222.41 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
11-4 183339 1/5/930 W 0.19 27 0 0.0 0.11 17.81 85.214.239.113 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0
12-4 182903 0/21/787 W 6.97 31 0 0.0 0.14 21.55 69.163.242.222 userdomain.com POST /spectra/plugins/system/dvmessages/dvmessages.php HTTP/1.0

[naja7host]

is this related ?

http://www.cloudlinux.com/solutions/forum/forum18/topic512/

[iseletsk]

If you see hundreds of such requests -- it seems your CloudLinux serer is not setup bad. Most likely: mod_hostinglimits module is missing
What is the output of apachectl -M|grep hosting?

[j]

Ive seen that on one server also, with apache 2.4 and mod_fcgi.
I could confirm that mod_hostinglimits was working, but the load gone wild. The only way to stop that was to stop apache, and delete that file.

I found that the client lost is ftp password, and a hacker, have entered is joomla site and installed that plugin, which was malicious.

The component was trying to continuous uploading files, and mod_security was continuously stopping the uploads (virus/trojan)

[j]

I forgot to say, this was before the latest two beta CL6 kernels.

[iseletsk]

I would love to see lveps -p
when this happens -- there is clearly something wrong, mod_hostinglimits shouldn have allowed 100s of apache connections if everything was properly setup.

[j]

In my case was not 100s of connections, it was 5 different ips using always the same connection.
Sometimes, only one connection to that file, was enough.
Could be some kind of bug of apache 2.4, i don know, but since it was resolved, i cannot give anymore info, just the experience i had.

I never saw apache doing that, only at that time with dvmessages

[naja7host]

mod_hostinglimits is exist and loaded by apache .

the request is sent from defferent ip not the same ip .

lvetop show the usage of the websites .

[iseletsk]

What is the output of top (the whole page)?
What is the output of uname -a
what is the output of lveps -?

[j]

In my case, i forgot to say, that this happened with apache 2.4.3 .
Until now, i never saw this problem anymore.

[jp]

I just experienced this problem just now. Temporary solution was to block the dvmessages.php file using .htaccess (in order not to suspend the whole site).

Ive seen hundreds of this file being executed all at once from different IP addresses, and yes, its still not yet entering in the process list as PHP, thats why EP lim its is not being enforced. Its still in the Apache side. I believe this file is receiving huge POST data, thats why Apache is having a hard time processing it.

httpd processes are the processed that consumes 100%+ CPU all at ones, which are owned by "nobody".

Igor, is there any way we can prevent this kind of attack without using mod_ruid2?

[jp]

> I could confirm that mod_hostinglimits was working, but the load gone wild. The only way to stop that was to stop apache, and delete that file.

Dont just delete it. Block it using .htaccess. You might not want it to restore by itself, because we dont know hot it got there in the first place.

Deny from all

[iseletsk]

I was told by another client that the issue is related to a particular mod_security rule and apache2.4.
Could you confirm that you are running both?