Introduction

Sometimes we need to know the country of the user using the application, for different purposes:

Automatically select the most appropriate language
Send a 302 to redirect the user to the closest POP from his location
Allow only a single country to browse the site, for legal reason
Block some countries we don’t do business with and which are the source of most web attacks

IP based location

To achieve this purpose, the most “reliable” information we have from a user is his IP address.
Well, not as reliable as we could hope:

not “reliable” because it’s easy to use a proxy installed in a foreign country to fake your IP address…
not “reliable” because GeoIP databases are not accurate.
not “reliable” because GeoIP databases rely on information provided by the ISP.
not “reliable” because any subnets can be routed from anywhere on earth.

When an ISP does a request for a new subnet to its local RIR, it has to tell the country where it will be used.
This country is supplied as a code composed by two letters, normalized by ISO 3166.

You can use whois tool can be used to know the country code of a IP address:

whois 1.1.1.1
[...]
country:        AU
[...]

geolocation definition

Well, it’s quite easy to understand: Geolocation is the process to link a third party to a geographicl location. In easier words: know the country of a client IP address.
On Internet, such base is called GeoIP.

geolocation databases

There are a few GeoIP databases available on the Internet, most of them uses IP ranges to link an IP address to its country code.
An IP range is simply a couple of IP addresses representing the first and the last IP address of a range.
NOTE: It might correspond to a real subnet, but in most cases, it doesn’t ;).

In example:

"1.1.2.0","1.1.63.255","16843264","16859135","CN","China"

What’s the issue with HAProxy then???

HAProxy can only use CIDR notation, with real subnets.
It means we’ll have to turn the IP ranges into CIDR notation. This is not a easy task since you must split the ip range in multiple subnets…
Once done, we’ll be able to configure HAProxy to use them in ACLs and do anything an ACL can do.

For example, the range above should be translated to the following subnets:

1.1.2.0/23 "CN"
1.1.4.0/22 "CN"
1.1.8.0/21 "CN"
1.1.16.0/20 "CN"
1.1.32.0/19 "CN"

Now, you can understand why GeoIP databases uses IP ranges: it takes fewer lines 🙂

iprange tool

To ease this job, Willy released a tool called iprange in HAProxy sources contrib directory.
You can see it here, in HAProxy’s git: http://www.haproxy.org/git/?p=haproxy.git;a=tree;f=contrib/iprange
It can be used to extract CIDR subnets from an IP range.

iprange installation

Just downlaod both Makefile and iprange.c then run make:

make
gcc -s -O3 -o iprange iprange.c

too complicated 🙂

iprange usage

iprange take a single incoming format, composed by 3 columns separated by commas:

first IP
Last IP
Country code

For example:

"1.1.2.0","1.1.63.255","CN"

NOTE: in the example below, we’ll work with Maxmind Country code lite database.

The database looks like:

$ head GeoIPCountryWhois.csv
"1.0.0.0","1.0.0.255","16777216","16777471","AU","Australia"
"1.0.1.0","1.0.3.255","16777472","16778239","CN","China"
"1.0.4.0","1.0.7.255","16778240","16779263","AU","Australia"
"1.0.8.0","1.0.15.255","16779264","16781311","CN","China"
"1.0.16.0","1.0.31.255","16781312","16785407","JP","Japan"
"1.0.32.0","1.0.63.255","16785408","16793599","CN","China"
"1.0.64.0","1.0.127.255","16793600","16809983","JP","Japan"
"1.0.128.0","1.0.255.255","16809984","16842751","TH","Thailand"
"1.1.0.0","1.1.0.255","16842752","16843007","CN","China"
"1.1.1.0","1.1.1.255","16843008","16843263","AU","Australia"

In order to make it compatible with iprange tool, use cut:

$ cut -d, -f1,2,5 GeoIPCountryWhois.csv | head
"1.0.0.0","1.0.0.255","AU"
"1.0.1.0","1.0.3.255","CN"
"1.0.4.0","1.0.7.255","AU"
"1.0.8.0","1.0.15.255","CN"
"1.0.16.0","1.0.31.255","JP"
"1.0.32.0","1.0.63.255","CN"
"1.0.64.0","1.0.127.255","JP"
"1.0.128.0","1.0.255.255","TH"
"1.1.0.0","1.1.0.255","CN"
"1.1.1.0","1.1.1.255","AU"

Now, you can use it with iprange:

$ cut -d, -f1,2,5 GeoIPCountryWhois.csv | head | ./iprange 
1.0.0.0/24 "AU"
1.0.1.0/24 "CN"
1.0.2.0/23 "CN"
1.0.4.0/22 "AU"
1.0.8.0/21 "CN"
1.0.16.0/20 "JP"
1.0.32.0/19 "CN"
1.0.64.0/18 "JP"
1.0.128.0/17 "TH"
1.1.0.0/24 "CN"
1.1.1.0/24 "AU"

Country codes and HAProxy ACLs

Now we’re ready to turn IP ranges into subnets associated to a country code.
Still need to be able to use it in HAProxy.
The easiest is to write all the subnets concerning a country code in a single file.

$ cut -d, -f1,2,5 GeoIPCountryWhois.csv | ./iprange | sed 's/"//g' 
| awk -F' ' '{ print $1 >> $2".subnets" }'

And the result is nice:

$ ls *.subnets
A1.subnets  AX.subnets  BW.subnets  CX.subnets  FJ.subnets  GR.subnets  IR.subnets  LA.subnets  ML.subnets  NF.subnets  PR.subnets  SI.subnets  TK.subnets  VE.subnets
A2.subnets  AZ.subnets  BY.subnets  CY.subnets  FK.subnets  GS.subnets  IS.subnets  LB.subnets  MM.subnets  NG.subnets  PS.subnets  SJ.subnets  TL.subnets  VG.subnets
AD.subnets  BA.subnets  BZ.subnets  CZ.subnets  FM.subnets  GT.subnets  IT.subnets  LC.subnets  MN.subnets  NI.subnets  PT.subnets  SK.subnets  TM.subnets  VI.subnets
AE.subnets  BB.subnets  CA.subnets  DE.subnets  FO.subnets  GU.subnets  JE.subnets  LI.subnets  MO.subnets  NL.subnets  PW.subnets  SL.subnets  TN.subnets  VN.subnets
AF.subnets  BD.subnets  CC.subnets  DJ.subnets  FR.subnets  GW.subnets  JM.subnets  LK.subnets  MP.subnets  NO.subnets  PY.subnets  SM.subnets  TO.subnets  VU.subnets
AG.subnets  BE.subnets  CD.subnets  DK.subnets  GA.subnets  GY.subnets  JO.subnets  LR.subnets  MQ.subnets  NP.subnets  QA.subnets  SN.subnets  TR.subnets  WF.subnets
AI.subnets  BF.subnets  CF.subnets  DM.subnets  GB.subnets  HK.subnets  JP.subnets  LS.subnets  MR.subnets  NR.subnets  RE.subnets  SO.subnets  TT.subnets  WS.subnets
AL.subnets  BG.subnets  CG.subnets  DO.subnets  GD.subnets  HN.subnets  KE.subnets  LT.subnets  MS.subnets  NU.subnets  RO.subnets  SR.subnets  TV.subnets  YE.subnets
AM.subnets  BH.subnets  CH.subnets  DZ.subnets  GE.subnets  HR.subnets  KG.subnets  LU.subnets  MT.subnets  NZ.subnets  RS.subnets  ST.subnets  TW.subnets  YT.subnets
AN.subnets  BI.subnets  CI.subnets  EC.subnets  GF.subnets  HT.subnets  KH.subnets  LV.subnets  MU.subnets  OM.subnets  RU.subnets  SV.subnets  TZ.subnets  ZA.subnets
AO.subnets  BJ.subnets  CK.subnets  EE.subnets  GG.subnets  HU.subnets  KI.subnets  LY.subnets  MV.subnets  PA.subnets  RW.subnets  SY.subnets  UA.subnets  ZM.subnets
AP.subnets  BM.subnets  CL.subnets  EG.subnets  GH.subnets  ID.subnets  KM.subnets  MA.subnets  MW.subnets  PE.subnets  SA.subnets  SZ.subnets  UG.subnets  ZW.subnets
AQ.subnets  BN.subnets  CM.subnets  EH.subnets  GI.subnets  IE.subnets  KN.subnets  MC.subnets  MX.subnets  PF.subnets  SB.subnets  TC.subnets  UM.subnets
AR.subnets  BO.subnets  CN.subnets  ER.subnets  GL.subnets  IL.subnets  KP.subnets  MD.subnets  MY.subnets  PG.subnets  SC.subnets  TD.subnets  US.subnets
AS.subnets  BR.subnets  CO.subnets  ES.subnets  GM.subnets  IM.subnets  KR.subnets  ME.subnets  MZ.subnets  PH.subnets  SD.subnets  TF.subnets  UY.subnets
AT.subnets  BS.subnets  CR.subnets  ET.subnets  GN.subnets  IN.subnets  KW.subnets  MG.subnets  NA.subnets  PK.subnets  SE.subnets  TG.subnets  UZ.subnets
AU.subnets  BT.subnets  CU.subnets  EU.subnets  GP.subnets  IO.subnets  KY.subnets  MH.subnets  NC.subnets  PL.subnets  SG.subnets  TH.subnets  VA.subnets
AW.subnets  BV.subnets  CV.subnets  FI.subnets  GQ.subnets  IQ.subnets  KZ.subnets  MK.subnets  NE.subnets  PM.subnets  SH.subnets  TJ.subnets  VC.subnets

Which makes subnets available for 246 countries !!!!!

In example, the subnets associated to AUstralia are:

$ cat AU.subnets 
1.0.0.0/24
1.0.4.0/22
1.1.1.0/24
[...]

The bash loop below prepares the ACLs in a file called haproxy.cfg:

$ for f in `ls *.subnets` ; do echo $f | 
awk -F'.' '{ print "acl "$1" src -f "$0 >> "haproxy.cfg" }' ; done

$ head haproxy.cfg 
acl src A1 -f A1.subnets
acl src A2 -f A2.subnets
acl src AD -f AD.subnets
acl src AE -f AE.subnets
acl src AF -f AF.subnets
acl src AG -f AG.subnets
acl src AI -f AI.subnets
acl src AL -f AL.subnets
acl src AM -f AM.subnets
acl src AN -f AN.subnets

That makes a lot of countries 🙂

Continent codes and HAProxy ACLs

Fortunately, we can summarizes it to continent. Copy and paste into a file the country code and continent relation from maxmind website: http://www.maxmind.com/app/country_continent.

The script below will create files with named with the continent name and containing the country codes related to it:

$ for c in `fgrep -v '-' country_continents.txt | sort -t',' -k 2` ; 
do echo $c | awk -F',' '{ print $1 >> $2".continent" }' ; done

We have now 7 new files:

$ ls *.continent
AF.continent  AN.continent  AS.continent  EU.continent  
NA.continent  OC.continent  SA.continent

Let’s have a look for countries in South America:

$ cat SA.continent 
AR
BO
BR
CL
CO
EC
FK
GF
GY
PE
PY
SR
UY
VE

Let’s aggregate subnets for each country in a continent into a single file:

$ for f in `ls *.continent` ; do for c in $(cat $f) ; 
do cat ${c}.subnets >> ${f%%.*}.subnets ; done ;  done

Now we can generate the HAProxy configuration file to use them:

$ for c in AF AN AS EU NA OC SA ; do 
echo acl $c src -f $c.subnets >> "haproxy.conf" ; done

Usage in HAProxy

Coming soon, an article giving some examples on how to use the files generated below to improve performance and security of your platforms.

9 Comments

Cyril Bonté on September 27, 2012 at 22:03

Hi Baptiste, great article ! never too late to say that 😉

I noticed some typos :
1. “iprage usage” should be “iprange usage”
2. the acls shown in the example after “$ head haproxy.cfg” are not valid : “acl A1 -f A1.subnets” should be “acl A1 src -f A1.subnets” (“src” is missing).

cheers
- Baptiste Assmann on October 8, 2012 at 08:52
  
  Hi Cyril,
  
  Thanks a lot for notifying.
  Article updated.
  
  cheers
gruena on October 16, 2012 at 19:15

Hi, thanks for the article! Could you publish the list for chinese subnets? They are spamming my forum and i want to block them.
Eric on November 11, 2014 at 21:58

The URL no longer works
http://haproxy.1wt.eu/git?p=haproxy.git;a=tree;f=contrib/iprange
- Baptiste Assmann on November 12, 2014 at 10:15
  
  Hi Eric
  
  Thanks for notifying.
  Replaced by the new one:
  http://www.haproxy.org/git/?p=haproxy.git;a=tree;f=contrib/iprange
  
  Baptiste
Surya on May 9, 2017 at 04:52

Hi Baptiste,

Can we please have few examples of configurations, where you can show the geoip Configuration being used.

Thanks in Advance !!!
- wtarreau on June 14, 2017 at 16:37
  
  Just an example in use on the haproxy.org servers :
  
  log-format %ci:%cp_%[src,map_ip(/etc/haproxy/ip-country.lst)] ...
  
  This leads to this in the logs :
  
  192.168.1.2:12345_FR
Ryan Naylor on May 30, 2017 at 20:36

GIT URL doesn’t work and would be great updated to know if this worked with the newer HAPROXY 1.7
- Chad Lavoie on May 31, 2017 at 18:09
  
  The GIT URL for IPRange (http://www.haproxy.org/git/?p=haproxy.git;a=tree;f=contrib/iprange) seems to work okay to me; is there another one that isn’t working?
  It works with 1.7 as well; map files are as alive as ever.

Use GeoIP database within HAProxy

Introduction

IP based location

geolocation definition

geolocation databases

What’s the issue with HAProxy then???

iprange tool

iprange installation

iprange usage

Country codes and HAProxy ACLs

Continent codes and HAProxy ACLs

Usage in HAProxy

Links

9 Comments

Submit a Comment Cancel reply

Categories

Follow us on Twitter

Use GeoIP database within HAProxy

Introduction

IP based location

geolocation definition

geolocation databases

What’s the issue with HAProxy then???

iprange tool

iprange installation

iprange usage

Country codes and HAProxy ACLs

Continent codes and HAProxy ACLs

Usage in HAProxy

Links

More posts:

9 Comments

Submit a Comment Cancel reply

Categories

Follow us on Twitter