Number of Cores, lCPU & CloudLinux 6.1

[michael]

Anomalous value reported for lCPU with CL 6.1

[michael]

Hello.

We have a few servers with running CloudLinux 5.7.1 running on them and one server which has CloudLinux 6.1 running on it. All of the servers have the following Default LVE settings:

CPU: 25
MEM: 1.0GB
Concurrent Connections: 20
Number of Cores Per LV: 1

(and persists is enabled). Now with regards to the CL 5.7.1 servers: A couple of those are dual dual core machines = 4 cores whilst the remainder are dual quad core machines = 8 cores. So as expected, with all LVEs using the default settings, lCPU for the 4 core machines is 25 whilst lCPU for the 8 core machines is 12.5 and of course, unless we allocate more cores (either globally or to a specific LVE), then this value cannot be increased.

Now with regards to the CL 6.1 server: It too is a dual quad core server and it has the same global defaults of as all the other server. However, on the CL 6.1 server, with a global Number of cores setting per LVE = 1, lCPU is 25 and in fact can also be increased beyond that without allocating more cores. For example, one of my colleagues at work assigned mCPU of 50 to one LVE on that server and lveinfo reports that for that LVE, lCPU for is now 50 and that sometimes mCPU hits 50 yet the number of cores allocated to this LVE is still 1.

So how is it possible for this particular LVE to be able to use 50% and for the others to be able to use 25% if they all have only 1 core allocated to them? I would have thought that on an 8 core machine that the LVE with lCPU 50 would need 4 cores allocated to it whilst for the others to have lCPU of 25 they would each need 2 cores allocated to them.

As such, keeping in mind that all of the LVEs have only 1 core allocated to them, then wouldn the lCPU for all of the LVEs actually be 12.5?

Thanks,
Michael

[accounts]

Heres some info I saved from some web site I cant remember, but we found it very useful on this subject:

> The number of cores settings takes precedence. For example if you have 25% CPU lim it, 1 core per LVE, and 8 core server -- each LVE will be lim ited by 100/8 =~ 12%, instead of 25% CPU, as at most one core will be used by each LVE (1 core on 8 core server ~= 12% of total CPU capacity)
> So, here is the way this works –
>
> If you have 4 cores available and you assign one core to the LVE, the absolute most this core can consume is 25% (1core x 100) / 4 (total cores) of a single core.
>
> If you have 8 cores available and you assign one core to the LVE, the absolute most this one core can consume is 12.5% (1core x100) / 8 (total cores) of a single core.
>
> However, if you increase the nCPU value to two cores – this now changes. For the examples above these are the changes;
>
> If you have 4 cores available and you assign TWO cores to the LVE, the absolute most each core can consume is 50% (2cores x 100) / 4 (total cores) of each core.
>
> If you have 8 cores available and you assign TWO cores to the LVE, the absolute most each core can consume is 25% (2cores x 100) / 8 (total cores) for each core.
>
> So, the first example demonstrates that the most CPU you can assign to the LVE is either 25% or 12.5% depending on the number of cores available to your system.
>
> Raising the CPU % above the 25% or 12.5% will have no effect – the documentation states explicitly;
> The number of cores settings takes precedence

[michael]

Hi Kernow.

Thanks for replying The anomalous behaviour that Im enquiring about though is that with CloudLinux 6.1, then on our server that has 8 cores (this probably would also apply to a 4 core machine as well machine though) and with an LVE that has only 1 core assigned to it, LVE Manager will permit the Max CPU for that LVE to be set to 25, 50 - even up to 100 - with lveinfo then displaying whatever value as chosen as being the value for lCPU for that LVE.

Thanks,
Michael

[accounts]

Hmmm.....Interesting. I guess we will have to wait for Igor Seletskiy for an answer, but Im guessing that its just reporting a higher value not actually using it. You could run a test like:
#ab -n 500 -c 100 http://yourdomain.com/some_file.html
and watch lvetop to see what cpu level it kicks in and stops it.

[michael]

Thanks for the tip However, to run ab it seems I have to tune a few sysctl parameters on my PC so I just took the easy way out and referred to LVE stats instead. So over the past 30 days, there have been a fair few sites that have hit mCPU 25 and the one site that has been configured for lCPU 50 has hit mCPU 50 a few times as well.

[iseletsk]

We are yet to be able to replicate the issue in house. It might be some bug in lve-stats, or in how ncpu parameter is treated. Something we are researching.

[michael]

Ok. Just out of curiosity, to see if perhaps lveinfo was not correctly reporting mCPU and lCPU, I tried running

ab -n 500 -c 100 http://xxx.yyy.com/

(obviously the real domain is not xxx.yyy.com) with the homepage first being index.html and then index.php with index.php simply containing an echo statement. However, even with:

MaxClients 512
MaxRequestsPerChild 10000

I can get above 30 concurrent connections from *any* server and -c 30 happens so fast that the LVE does not appear in lvetop. I tried leaving concurrent at 30 and increasing the number of connections to ridiculously high values but still didn see the LVE appear in lvetop.

In fact, lvetop itself seems to be extremely slow to update even with the -d flag and when I tried running it with the -r flag, I got this:

warning: rt priority is not available

Which is a pity because I would have liked to see what lvetop reported as compared to lveinfo.

Thanks,
Michael

[iseletsk]

HTML pages are not in LVE. Make sure you are hitting php page.

[michael]

Hi Igor.

Yes, I assumed as much which is why I then tried it with a PHP page:

> ...with the homepage first being index.html and then index.php with index.php simply containing an echo statement.

Thanks,
Michael

[iseletsk]

It doesn sound right. If you are hitting php page -- with 30 concurrent connections -- you should see it in lvetop, or something is wrong with that setup.

Are you hitting user account, or are you hitting php on some "system" portion of website (like domain pointed to /var/www or some similar place).
Make sure that virtual host has SuexecUserGroup in its configuration.

[michael]

It is an actual user account, yes - being one that we created using cPanel and which we use for simultaneously monitoring both MySQL and HTTPD using Nagios.

Thanks,
Michael

[iseletsk]

Does lvetop show other users?
Try hitting this file:

It takes some considerable amount of time to run. Hit it via browser first, as if that user/site is not in LVE for some reason -- hitting it with 30 concurrent connections will kill the server.

<?php
function getTime()
{
$a = explode ( ,microtime());
return(double) $a[0] + $a[1];
}
$Start = getTime();
$i = 1.1;
$j = 0.1;
$k = 10000000;
while ($k > 0) {
$j=$i*$i;
$k--;
}
?>
<?php
$End = getTime();
echo "Time taken = ".number_format(($End - $Start),2)." secs";
?>

[michael]

Yes it does.

lol ok. Please Standby

[michael]

> Hit it via browser first,

Ok. Done. Response was:

Time taken = 0.68 secs

I then ran:

ab -n 1000 -c 30 http://xxx.yyy.com.au/

from another server in the data centre (because if I do it from my PC, connectivity to the server is lost for a while) and saw the LVEs mCPU sit at 12% - although it did hit 13% once in one run (I actually performed 3 runs) but could that have been due to the LVE actually sitting at 12.5% and an extra .1% or so "snuck past" and so was rounded up?

So it would appear from this test that lveinfo (and hence lve-stats) is reporting incorrect values for lCPU and mCPU.