[sr-dev] ASYNC module

Discussion:

[sr-dev] ASYNC module - CPU load

Olle E. Johansson

2015-04-20 07:58:12 UTC

Good morning!

In one installation where Kamailio runs in a VMware virtual machine we use Async in a registrar.

There's no traffic - only one SIP device registred - but the CPU load goes up.

If we change the workers to 1 CPU load goes down, if we set it to 8 CPU load average quickly goes up to 3. If we add a second CPU, nothing changes. Again, with no traffic. The CPU load is directly related to number of workers, but not if they are used or not.

My conclusion is that something in the Async module is causing CPU load regardless of traffic - primarily depending on the number of workers.

I don't think that's the correct behaviour ;-)

Kamailio 4.2.0

Cheers,
/O

Alex Balashov

2015-04-20 09:24:36 UTC

Permalink

I encountered the identical condition inside VirtualBox and remarked upon it to the list previously. This is why I had to revert to rtimer + mqueue + tm_suspend/continue manual approaches.

--
Alex Balashov | Principal | Evariste Systems LLC
303 Perimeter Center North, Suite 300
Atlanta, GA 30346
United States

Tel: +1-800-250-5920 (toll-free) / +1-678-954-0671 (direct)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/

Sent from my BlackBerry.
Original Message
From: Olle E. Johansson
Sent: Monday, April 20, 2015 03:58
To: Kamailio (SER) - Development Mailing List
Reply To: Kamailio (SER) - Development Mailing List
Subject: [sr-dev] ASYNC module - CPU load

Good morning!

In one installation where Kamailio runs in a VMware virtual machine we use Async in a registrar.

There's no traffic - only one SIP device registred - but the CPU load goes up.

If we change the workers to 1 CPU load goes down, if we set it to 8 CPU load average quickly goes up to 3. If we add a second CPU, nothing changes. Again, with no traffic. The CPU load is directly related to number of workers, but not if they are used or not.

My conclusion is that something in the Async module is causing CPU load regardless of traffic - primarily depending on the number of workers.

I don't think that's the correct behaviour ;-)

Kamailio 4.2.0

Cheers,
/O

Daniel-Constantin Mierla

2015-04-20 09:49:12 UTC

Permalink

IIRC, the issue appeared to be related to virtualization, because the
the process should have been blocked in I/O state by recvfrom().

Anyhow, to confirm this, look at the process using lot of cpu with gdb
and get the backtrace:

gdb /path/to/kamailio PID
bt

Cheers,
Daniel

Post by Alex Balashov
I encountered the identical condition inside VirtualBox and remarked upon it to the list previously. This is why I had to revert to rtimer + mqueue + tm_suspend/continue manual approaches.
--
Alex Balashov | Principal | Evariste Systems LLC
303 Perimeter Center North, Suite 300
Atlanta, GA 30346
United States
Tel: +1-800-250-5920 (toll-free) / +1-678-954-0671 (direct)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Sent from my BlackBerry.
Original Message
From: Olle E. Johansson
Sent: Monday, April 20, 2015 03:58
To: Kamailio (SER) - Development Mailing List
Reply To: Kamailio (SER) - Development Mailing List
Subject: [sr-dev] ASYNC module - CPU load
Good morning!
In one installation where Kamailio runs in a VMware virtual machine we use Async in a registrar.
There's no traffic - only one SIP device registred - but the CPU load goes up.
If we change the workers to 1 CPU load goes down, if we set it to 8 CPU load average quickly goes up to 3. If we add a second CPU, nothing changes. Again, with no traffic. The CPU load is directly related to number of workers, but not if they are used or not.
My conclusion is that something in the Async module is causing CPU load regardless of traffic - primarily depending on the number of workers.
I don't think that's the correct behaviour ;-)
Kamailio 4.2.0
Cheers,
/O
_______________________________________________
sr-dev mailing list
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
_______________________________________________
sr-dev mailing list
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev

--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio World Conference, May 27-29, 2015
Berlin, Germany - http://www.kamailioworld.com

Alex Balashov

2015-04-20 10:04:28 UTC

Permalink

‎I agree that it seems to be a peculiarity of virtualisation; I haven't reproduced this on bare metal. But it's a bit odd when it appears in KVM, Xen, and, evidently, per Olle's report, VMware, despite the fact that these are all rather different implementations.

--
Alex Balashov | Principal | Evariste Systems LLC
303 Perimeter Center North, Suite 300
Atlanta, GA 30346
United States

Tel: +1-800-250-5920 (toll-free) / +1-678-954-0671 (direct)
Web: http://www.evaristesys.com/, http://www.csrpswitch.com/

Sent from my BlackBerry.
Original Message
From: Daniel-Constantin Mierla
Sent: Monday, April 20, 2015 11:49
To: Kamailio (SER) - Development Mailing List
Reply To: ***@gmail.com
Subject: Re: [sr-dev] ASYNC module - CPU load

IIRC, the issue appeared to be related to virtualization, because the
the process should have been blocked in I/O state by recvfrom().

Anyhow, to confirm this, look at the process using lot of cpu with gdb
and get the backtrace:

gdb /path/to/kamailio PID
bt

Cheers,
Daniel

--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio World Conference, May 27-29, 2015
Berlin, Germany - http://www.kamailioworld.com

_______________________________________________
sr-dev mailing list
sr-***@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev

Olle E. Johansson

2015-04-20 11:09:37 UTC

Permalink

Post by Daniel-Constantin Mierla
IIRC, the issue appeared to be related to virtualization, because the
the process should have been blocked in I/O state by recvfrom().
Anyhow, to confirm this, look at the process using lot of cpu with gdb

Funny enough we don't see any process with a specific high load, just the load average. We'll try
chasing a particular process. Running top kamailio doesn't even show.

/O

Post by Daniel-Constantin Mierla
gdb /path/to/kamailio PID
bt
Cheers,
Daniel

--
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Kamailio World Conference, May 27-29, 2015
Berlin, Germany - http://www.kamailioworld.com
_______________________________________________
sr-dev mailing list
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev