Easily updating IP lists for server & firewall configurations

Image: ©2017 volkan.xyz

If you are using Cloudflare (or any other CDN), Nginx, Apache or any other web server configuration should be updated with the IP range of CDN to ensure server logs and rate limiting configurations work correctly. CDN’s IP range must also be whitelisted on local firewall to bypass firewall limitations. Updating the blacklist of local firewall with a public bogon IP list is also a good idea for many servers, even without a CDN.

I have written a simple script to automate the whole process for end user such as, downloading IP list, validating IP addresses, removing duplicates (for multiple source bogon lists), creating server configurations for Nginx or Apache, updating IP sets for whitelisting or blacklisting with iptables and more…

ip-list-updater is a PHP-CLI script, simply updates CDN/trusted proxy/reverse proxy and bogon IP lists for firewall and server configurations.

This single line just downloads Cloudflare IP range, updates Nginx configuration and reloads the server.

$ ip-list-updater.php --update --mode="nginx" --ipv=4 --output="/etc/nginx-cloudflare.conf" --sources="cloudflare" --success="/usr/bin/nginx -s reload"

And the following lines demonstrate how to whitelist current Cloudflare IP range.

$ ipset create whitelist hash:net family inet hashsize 1024 maxelem 131072
$ iptables -I INPUT -p tcp -m multiport --dports 80,443 -m set --match-set whitelist src -j ACCEPT
$ ip-list-updater.php --update --mode="ipset" --setname="whitelist" --ipv=4 --output="/etc/whitelist.txt" --sources="cloudflare"

This one demonstrates how to blacklist a bogon IP range.

$ ipset create bogonlist hash:net family inet hashsize 1024 maxelem 131072
$ iptables -I INPUT -m set --match-set bogonlist src -j DROP
$ ip-list-updater.php --update --mode="ipset" --setname="bogonlist" --ipv=4 --output="/etc/bogonlist.txt" --sources="spamhaus"

ip-list-updater supports Iptables/Ipset, Nginx, Apache and any server or firewall with raw mode.

Have built-in download sources for CDN services (Cloudflare, Cloudfront, Fastly, Maxcdn) and Bogon IP update lists (Spamhaus, Cymru).

There is also a ready-to-use Docker image for ip-list-updater.

Please take a look at the following example to see how easy it is in Docker Compose:

version: '2'
        image: vkucukcakar/ip-list-updater
        container_name: ip-list-updater
            SCHEDULE: "15 3 * * *"
            MODE: nginx
            IPV: 4
            SOURCES: cloudflare
            RELOAD_SERVER: server-proxy
            EXTRA_PARAMETERS: '--timeout==60'
            - /var/run/docker.sock:/var/run/docker.sock
            - ./configurations:/configurations

The Compose snippet above will download Cloudflare IP address ranges, create or update /configurations/ip-list-updater.lst and finally reload Nginx without restarting the container if a change is detected. All you need to do is including /configurations/ip-list-updater.lst from your Nginx configuration by mounting or using a named volume. Please click here to read more about the Docker image.

Actually, the Docker image version is not tested that much. I prefer running ip-list-updater script on host machine rather than running it in a container.

And here is a real world example:

In the following crontab entries, the first line downloads Spamhaus bogon IPv4 list daily at 03:15 AM, updates Ipset named “bogonlist”, which is used by sptables (my firewall script), and only logs error output.

The second line downloads the Cloudflare IPv4 range, updates Ipset named “whitelist”, which is used by the firewall. (There should be another line if we had IPv6 set support as IPv4 sets are not compatible with IPv6 sets.)

The third line downloads the Cloudflare IP range, updates the server configuration and reloads Nginx with zero downtime by sending a HUP signal to the container by Docker.

15 3 * * * root /usr/local/bin/ip-list-updater.php --update --mode="ipset" --setname="bogonlist" --ipv=4 --output="/etc/bogonlist.txt" --sources="spamhaus" --success="ipset save bogonlist -f /etc/sptables/data/bogonlist.save" >/dev/null 2>/var/log/ip-list-updater.log
45 3 * * * root /usr/local/bin/ip-list-updater.php --update --mode="ipset" --setname="proxylist" --ipv=4 --output="/etc/proxylist.txt" --sources="cloudflare" --success="ipset save proxylist -f /etc/sptables/data/proxylist.save" >/dev/null 2>/var/log/ip-list-updater.log
30 3 * * * root /usr/local/bin/ip-list-updater.php --update --mode="nginx" --ipv=all --output="/lemp/configurations/server-proxy/cdn.conf" --sources="cloudflare" --success="docker kill --signal=HUP server-proxy" >/dev/null 2>/var/log/ip-list-updater.log

In my honest opinion, this one is the simplest usage of the script.

Source: https://volkan.xyz


You can share this blog post only by giving appropriate credit as described at Terms & Conditions.


Leave a Reply

Your email address will not be published. Required fields are marked *