PDNS Recursor Alpha - RAM usage

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

PDNS Recursor Alpha - RAM usage

shthead
Hi all,

I have been running the alpha of PowerDNS recursor
(4.0.0~alpha2-1pdns.jessie) for a while but seem to be having issues
with the memory usage of it after a while.

My DNS servers all have 16GB of RAM, about 40% of which is allocated to
MySQL (3 instances of PowerDNS auth running against 3 reasonably sized
DB's), 10% which is taken up by everything else except PowerDNS recursor.

After about 2-3 weeks of running the memory usage of pdns recursor
slowly creeps up until it is using all the memory + swap causing
oomkiller to kill things off. This isn't really a problem as I use a
health checker on each DNS server to verify everything works before
announcing the IP's via BGP, but it would be nice if it just ran
correctly without it restarting the rescursor so often. I have checked
manually when monitoring starts alarming about the memory usage and by
that time the recursor is using about 9GB of RAM + 4GB swap.

Here is the settings in my recursor.conf:

allow-from-file=/etc/powerdns/recursor_allowed_ips
api-key=...
api-readonly=no
carbon-ourname=dns1-recursor
carbon-server=...
daemon=yes
entropy-source=/dev/urandom
export-etc-hosts=off
forward-zones-file=/etc/powerdns/recursor_forward_zones
hint-file=/etc/powerdns/named.cache
local-address=10.254.1.10 10.254.1.11 fd12:3456:ffff:1::1
fd12:3456:ffff:1::2
local-port=53
max-cache-entries=1000000
max-mthreads=8192
max-negative-ttl=360
max-qperq=100
max-tcp-clients=2048
max-total-msec=600000
network-timeout=3000
pdns-distributes-queries=yes
query-local-address=0.0.0.0
query-local-address6=::
setgid=pdns
setuid=pdns
threads=24
version-string=PowerDNS Recursor
webserver-address=...
webserver-allow-from=0.0.0.0/0,::/0
webserver-password=...
webserver-port=5333
webserver=yes

I have it reporting to a metronome instance I run myself and the memory
used graph is a constantly increasing curve but I am not exactly sure
why.... maybe I will need to reduce the max-cache-entries to something
much lower?

Is anyone else having difficulties running recursor? What does your
config look like/how much RAM does it use?

Thanks
_______________________________________________
Pdns-users mailing list
[hidden email]
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Reply | Threaded
Open this post in threaded view
|

Re: PDNS Recursor Alpha - RAM usage

bert hubert-3
On Mon, May 09, 2016 at 09:06:01AM +0800, Chris wrote:
> Hi all,
>
> I have been running the alpha of PowerDNS recursor
> (4.0.0~alpha2-1pdns.jessie) for a while but seem to be having issues with
> the memory usage of it after a while.

Hi Chris,

We solved a memory leak in 4.0.0 recently, you might find that the packages
from https://repo.powerdns.com/ don't leak.

commit b275b9171d3fcf272d6b3fc94b8c5b2af478939e
Author: Remi Gacogne <[hidden email]>
Date:   Mon May 2 17:24:08 2016 +0200

    rec: Fix a memory leak in DNSSEC validation
   
    `DNSCryptoKeyEngine::makeFromPublicKeyString()` returns a naked
    pointer to a new object.

Can you check if it helps? The master packages are better than the alpha
packages in all ways we know about.

        Bert

>
> My DNS servers all have 16GB of RAM, about 40% of which is allocated to
> MySQL (3 instances of PowerDNS auth running against 3 reasonably sized
> DB's), 10% which is taken up by everything else except PowerDNS recursor.
>
> After about 2-3 weeks of running the memory usage of pdns recursor slowly
> creeps up until it is using all the memory + swap causing oomkiller to kill
> things off. This isn't really a problem as I use a health checker on each
> DNS server to verify everything works before announcing the IP's via BGP,
> but it would be nice if it just ran correctly without it restarting the
> rescursor so often. I have checked manually when monitoring starts alarming
> about the memory usage and by that time the recursor is using about 9GB of
> RAM + 4GB swap.
>
> Here is the settings in my recursor.conf:
>
> allow-from-file=/etc/powerdns/recursor_allowed_ips
> api-key=...
> api-readonly=no
> carbon-ourname=dns1-recursor
> carbon-server=...
> daemon=yes
> entropy-source=/dev/urandom
> export-etc-hosts=off
> forward-zones-file=/etc/powerdns/recursor_forward_zones
> hint-file=/etc/powerdns/named.cache
> local-address=10.254.1.10 10.254.1.11 fd12:3456:ffff:1::1
> fd12:3456:ffff:1::2
> local-port=53
> max-cache-entries=1000000
> max-mthreads=8192
> max-negative-ttl=360
> max-qperq=100
> max-tcp-clients=2048
> max-total-msec=600000
> network-timeout=3000
> pdns-distributes-queries=yes
> query-local-address=0.0.0.0
> query-local-address6=::
> setgid=pdns
> setuid=pdns
> threads=24
> version-string=PowerDNS Recursor
> webserver-address=...
> webserver-allow-from=0.0.0.0/0,::/0
> webserver-password=...
> webserver-port=5333
> webserver=yes
>
> I have it reporting to a metronome instance I run myself and the memory used
> graph is a constantly increasing curve but I am not exactly sure why....
> maybe I will need to reduce the max-cache-entries to something much lower?
>
> Is anyone else having difficulties running recursor? What does your config
> look like/how much RAM does it use?
>
> Thanks
> _______________________________________________
> Pdns-users mailing list
> [hidden email]
> https://mailman.powerdns.com/mailman/listinfo/pdns-users
_______________________________________________
Pdns-users mailing list
[hidden email]
https://mailman.powerdns.com/mailman/listinfo/pdns-users
Reply | Threaded
Open this post in threaded view
|

Re: PDNS Recursor Alpha - RAM usage

shthead
On 09/05/2016 2:49 PM, bert hubert wrote:
> Can you check if it helps? The master packages are better than the alpha
> packages in all ways we know about.

Hi Bert,

Thanks for that I have upgraded the packages and so far it looks good,
with the config I sent earlier I am seeing it flat line at about 800mb
of RAM where as it never flat lined before, it was always on a constant
upwards trend.

I'll wait for a couple of days to make sure, if so then I should be able
to increase the number of cache entries again (I had it set to 4M before
but reduced it due to the memory issue).

Thanks
_______________________________________________
Pdns-users mailing list
[hidden email]
https://mailman.powerdns.com/mailman/listinfo/pdns-users