Useful commands to analyse access.log

Include and exclude strings in search term:

grep -F "google"  /var/log/nginx/access.log | grep -v "33.44.55.66"  > google.txt

Get unique IPs:

awk '{print $1}' google.txt|sort -u

Or in a single line:

grep -F "google"  /var/log/nginx/access.log > googl.txt && awk '{print $1}' google.txt|sort -u

 ---------------------------

Convert list of IPs to unique IP blocks for nginx deny:

sed -i -e   's/$/;/' file.txt && sed -i -e   's/[0-9]\?[0-9]\?[0-9];$/0\/24;/' file.txt &&  sed -i -e   's/^/deny /' file.txt | sort -u


Add 'deny' to the begining of each line of a file:

 sed -i -e   's/^/deny /' file.txt

Add ';' at the end of each line of a file:

sed -i -e   's/$/;/' file.txt

Now convert IPs to 24 blocks:

 sed -i -e   's/\.*;/.0\/24;/' file.txt

Perpare deny list in one line:

sed -i -e   's/$/;/' file.txt && sed -i -e   's/[0-9]\?[0-9]\?[0-9];$/0\/24;/' file.txt &&  sed -i -e   's/^/deny /' file.txt