New HTTP Analyzer Plug-in: CEM User Groups by Geographical Location

[Legacy User]

Have you ever wanted to automatically create User Groups in CEM based on geographic location or CIDR? Do you currently group users by IP Subnet, but are tired of ambigous names like "UserGroup-63.117.33.0" and "UserGroupTim-10.1.178.0"? Have you been forced to manually put users into geographic groups to support regional SLAs?

Now, with the recently uploaded GeoIPCemPlugin, you can use MaxMind's IP geolocation data to group users in a more meaningful way! When fully implemented, this plugin allows CEM to group detected users by customizable geographic data, such as city, country, or time-zone, among others. This HTTP Analyzer plugin for CEM works by parsing an HTTP request parameter that contains the client IP address, such as X-Forwarded-For. It then performs a IP geolocation query for this IP Address and outputs an additional request parameter that contains a customizable string of identifying geographic data for the user. Additionally, IP address ranges or CIDR Notation can be used to specify custom geographies or groups instead of geographic data or when no geolocation data is available for the user's IP address. CEM can then be configured to use this Plug-in request parameter to define the User Groups for Business Applications. For Enterprise applications in CEM, this is a more sustainable and meaningful mechanism than grouping by IP Subnet.

Please note: IP Geolocation is not an exact science; not all IP Addresses have known or strict geographies, and if a request is coming from a proxy server rather than a user’s IP Address, the geolocation data will be for the proxy, not the user. Nevertheless, this data can be extremely useful in logically grouping users and allowing for more impactful SLA definitions without requiring manual user maintenance in CEM.

As always, feedback is appreciated, either as part of this thread or as part of the Community Document comments.

[Haruhiko Davis]

Very cool! I'll put it through some stuff on my test VMs.

Thanks again for all your effort!

[Legacy User]

Hiko,
Even if it's to point out a typo in the documentation, I look forward to hearing your thoughts on it! :grin:

- Jack

[Anon Anon]

Hello --

We are trying to implement this plugin for a Enterprise Business Application in our Nonprod CEM environment but we are unable to get the plugin to work. We have followed the word doc and configured the env as described, however, we are not seeing user groups populate as we test hitting our app from an External IP address.

We did notice however for a different APM env we have setup, we see the plugin working as expected for a CEM defined E-Commerence application.

We are able to see the external ip when looking at the logs after the URL is hit.

Do you recommend any troubleshooting steps or traces to take as to why the plugin is not creating the user groups for an Enterprise Application inside CEM?

Thank you!

[Legacy User]

Have you confirmed that the extention is not having a problem? The first place to do this is from the Tim Plugin Status page, which is accesible here (where <TIM_IP> is the IP Address of your TIM appliance): http://<TIM_IP>/cgi-bin/wily/packages/cem/tim/index. You should see the name "GeoIP CEM Plugin" listed with an enabled value of Yes. If you do not, then the plugin was not able to start correctly. If so, the TIM Log should contain an error message as to why. Via the command line, the TIM Log is located here: /usr/local/wily/cem/tim/logs/timlog.txt, but can also be viewed on the view via the View Tim Log page (same URL as before).

If it appears to be installed and running successfully, then my guess would be a configuration problem. Let me know what you find out.

[Anon Anon]

We are seeing "GeoIP CEM Plugin" with a value of Yes when we navigated to the url you mentioned.

WIll take a look again at the timlog.txt however initially we did not see anything. Will let you know if we see any issues in timlog.

[Anon Anon]

Wow Jack! Thanks for sharing this with the APM Community. Has anyone checked it out yet?

Mary

[Anon Anon]

Slick! Nice work, cant wait to try it

[Anon Anon]

Have there been any recent updates to this plugin?  I would really like to see things like extending the lookups beyond just maxmind, something like a map for internal IP's that would override the masking of "internal address".

[Legacy User]

We've not made any updates, as we've not received much feedback. Can you write up a brief summary of what you'd like to see and how you envision it functioning? 

[Anon Anon]

It would work very much like it does today.  Since it looks like the address spaces are already being filtered out for internal addresses, there could be an option to declare a lookup against a CSV or other database.  The value returned would be the group name on those internal addresses.

-Ramiro

[Legacy User]

When specifying the internal "geographies", would you want it to support IP Address ranges, specific IP Adresses, subnet masks, or a combination thereof? Additionally, would you want it to support IPv4, IPv6, or both? 

[Anon Anon]

My usecase would be for it to support an incoming IP to match to an IP range in cidr.  Such as 10.100.0.0/22

Only for IPv4 for now.

 

 

 

[Anon Anon]

Awesome! Thanks, I'll give it a go!

[Legacy User]

I'm getting a "Documents and Media is temporarily unavailable" message when I try to update this Plugin, so I've attached the newest version here. It should now provide you with the ability you want.

EDIT: Now also supports CIDR notation for specifying IP Address ranges.  

EDIT #2: The Community Document has been re-added and the links in this thread have been updated. 

[Anon Anon]

Seem to be having some issues with CIDR addresses.

I have downloaded the maxmind DB because it seems that the plugin fails without one and have loaded the properties file with many ranges all of which have mutliple CIDR entries (comma separated) but they dont want to match.

Example I have an IP of "10.62.80.60" that should match a range specified with "10.0.0.0/8" but the plugin returns "Unknown Region".

I've tried removing all of the other IP ranges/names as well as made sure there is only one CIDR in the ranges for that location but with no success.

Any help is appreciated.

 

Thanks!

[Legacy User]

Thanks for the feedback! I'll look into this and hopefully post an update by end-of-day. 

[Legacy User]

Okay. Here's the latest version [EDIT: now back in Community Documents, thanks to Hiko] - it passed the expanded unit tests. If you have problems, set the log level to DEBUG, disable and re-enable the plugin (or restart the TIM), and send me the log file after it runs for a while. 

[Anon Anon]

So far so good Jack, Thanks!  I just loaded the updated version, I'll let it run for a while and post back.  I already see many of my ranges matching with a few that arent, so I'll take a closer look later to see if its just my configuration.

Thanks again for this, it is extremely helpful!

-Ramiro

[Legacy User]

Ramiro, 

Is the plug-in still working as you'd hoped? 

[Anon Anon]

Yeah it's going pretty well so far.  I am noticing though that I have a lot of groups getting created with "null" wherever it can't find the field value.  Not sure if this is because we're using the free version of maxmind or if there was an error in the plugin.  

I was thinking just now, if it is cause by the value not being found that rather than printing "null", it would be cleaner to only include what it can find.  Say the format is "City, Region, CountryName", if only Region, CountryName are found, that it print those and leave out City.  Or any combination thereof.

What do you think?

[Legacy User]

Hmm. I'd like to see a debug log. I agree, you should not be seing "null" in the values, and I'd like to investigate why you are. Can you set the log level to DEBUG for the plugin and either restart the TIM or disable / enable the plug-in via CEM? Once a decent log file is produced, please attach it to this thread or send to me via private message. 

[Anon Anon]

We have installed the Geo pluin following the word document thoroughly. I see plugin is enable and active in both TIM and CEM.

We tried to test it by creating some slow transactions, we got defects but dont see the user location we were expecting.  what would have went wrong?

FYI.. we are not filtering traffic to the plugin yet.

Tue Mar  4 14:15:27 2014 28438 ! Warning: MTPComm: MTP socket "/var/run/nqcapd_sock" does not exist
Tue Mar  4 14:15:27 2014 28438   AdapterManager: "eth1": thread starting
Tue Mar  4 14:15:27 2014 28438   DeviceCtl: running: /usr/sbin/ethtool --offload eth1 gro off 2>&1
Tue Mar  4 14:15:27 2014 28438   JavaHttpPlugin: analysis thread 0x2aaab403d550 attached to JVM
Tue Mar  4 14:15:27 2014 28438   JavaHttpPlugin: analysis thread 0x2aaab403d550 initialized for JVM
Tue Mar  4 14:15:27 2014 28438   AdapterManager: "eth1": network device opened
Tue Mar  4 14:15:27 2014 28438   AdapterManager: "eth1": initializing to reject all packets
Tue Mar  4 14:15:27 2014 28438   AdapterManager: "eth1": receive buffer size for is 125829120
Tue Mar  4 14:15:27 2014 28438   AdapterManager: "eth1": packet filter set
Tue Mar  4 14:25:26 2014 28438   Service: running: tim-ageout -data /etc/wily/cem/tim/data -minFreePercent 25 -reclaimPercent 10
Tue Mar  4 14:25:26 2014 28438   Service: tim-ageout exited with status 0
Tue Mar  4 14:26:13 2014 28438   WebServer: POST request for /tess/setpluginconfig from 127.0.0.1
Tue Mar  4 14:26:13 2014 28438   WebServer: request forwarded from 156.81.248.191
Tue Mar  4 14:26:13 2014 28438   WebServer: loading javapluginconfig
Tue Mar  4 14:26:13 2014 28438   ConfigFile: writing configuration file /etc/wily/cem/tim/config/javapluginconfigs/1.xml
Tue Mar  4 14:26:13 2014 28438   JavaPluginManager: received config file
Tue Mar  4 14:26:13 2014 28438   JavaPluginConfig: id=1, enabled=0, version=1
Tue Mar  4 14:26:13 2014 28438   JavaHttpPlugin: stopping plugin 1 ("GeoIP CEM Plugin")
Tue Mar  4 14:26:16 2014 28438   WebServer: POST request for /tess/setpluginconfig from 127.0.0.1
Tue Mar  4 14:26:16 2014 28438   WebServer: request forwarded from 156.81.248.191
Tue Mar  4 14:26:16 2014 28438   WebServer: loading javapluginconfig
Tue Mar  4 14:26:16 2014 28438   ConfigFile: writing configuration file /etc/wily/cem/tim/config/javapluginconfigs/1.xml
Tue Mar  4 14:26:16 2014 28438   JavaPluginManager: received config file
Tue Mar  4 14:26:16 2014 28438   JavaPluginConfig: id=1, enabled=1, version=1, name="GeoIP CEM Plugin"
Tue Mar  4 14:26:16 2014 28438   JavaPluginConfig: URL filter is not defined
Tue Mar  4 14:26:16 2014 28438   JavaPluginConfig: no server filter
Tue Mar  4 14:26:16 2014 28438   JavaHttpPlugin: starting plugin 1 ("GeoIP CEM Plugin") with:
Tue Mar  4 14:26:16 2014 28438   JavaHttpPlugin:   /etc/wily/cem/tim/config/javaplugins/1.jar
Tue Mar  4 14:26:16 2014 28438   JavaHttpPlugin: plugin "GeoIP CEM Plugin" started

 

 

[Anon Anon]

And here is what is configured in plugin property

geoip.city.file=/usr/local/wily/cem/tim/data/GeoLiteCity.dat
geoip.log.level=DEBUG
geoip.cem.incoming.parameter.name=X-Forwarded-For
geoip.cem.outgoing.parameter.name=X-GeoIPLocation
geoip.internal.networks=A,B,C,D
geoip.internal.networks.A.name=A
geoip.internal.networks.A.ranges=11.0.0.0-255.0.0.0
geoip.internal.networks.B.name=B
geoip.internal.networks.B.ranges=157.146.0.0-255.255.0.0
geoip.internal.networks.C.name=C
geoip.internal.networks.C.ranges=156.82.0.0-255.255.0.0
geoip.internal.networks.D.name=D
geoip.internal.networks.D.ranges=169.225.0.0-255.255.0.0
geoip.cem.parameter.format={0}, {1}
geoip.cem.parameter.value.0=countryName
geoip.cem.parameter.value.1=timeZone

[Legacy User]

It sounds like everything is working as expected. The plug-in has started up normally without errors and you've configured some IP ranges that will override the GeoIP lookups. The reason you are not seeing any results is that you have not directed any traffic to be handled by the GeoIPCemPlugin: 
 

Tue Mar  4 14:26:16 2014 28438   JavaPluginConfig: URL filter is not defined
Tue Mar  4 14:26:16 2014 28438   JavaPluginConfig: no server filter

 

This means that you've likely not set the IP Address range and URL filter for the plugin. I recommend using 0.0.0.0 - 255.255.255.255 for the IP Address Range and /* for the URL match pattern. 

[Anon Anon]

We change the url pattern to /* as you have suggested and we see the user groups created as “international” and “United States, America/New_York”.

 

We have the following in our properties file

geoip.city.file=/usr/local/wily/cem/tim/data/GeoLiteCity.dat

geoip.log.level=DEBUG

geoip.cem.incoming.parameter.name=client_ip

geoip.cem.outgoing.parameter.name=X-GeoIPLocation

geoip.internal.networks=International,test1,test2,test3

geoip.internal.networks.International.name=International

geoip.internal.networks.International.ranges=11.0.0.0-255.0.0.0

geoip.internal.networks.test1.name=test1

geoip.internal.networks.test1.ranges=157.146.0.0-255.255.0.0

geoip.internal.networks.test2.name=test2

geoip.internal.networks.test2.ranges=156.82.0.0-255.255.0.0

geoip.internal.networks.test3.name=test3

geoip.internal.networks.test3.ranges=169.225.0.0-255.255.0.0

geoip.cem.parameter.format={0}, {1}

geoip.cem.parameter.value.0=countryName

geoip.cem.parameter.value.1=timeZone

 

 

We are expecting to see 156.82.244.165 in test2 user group, but it shows currently under international user group. So, we are wondering why it is not mapping properly.

Please help

[Legacy User]

It is mapping properly, it's just that your IP Address ranges are huge and overlapping.  You seem to be using a subnet mask as the last part of your IP range, which might be part of the confusion. I recommend reading up on IP range blocks to get a better understanding of how you can specify these. 

Also, the IP range overrides are first-match; since the International range is specified first in the geoip.internal.networks property, it's the first match for the 156.82.244.165 address, even though it could also be contained by the test2 range. Once you redefine your ranges so that they do not overlap, you should get your expected results. 

[Anon Anon]

Thanks much for the response
We figured it out last evening that we were providing subnet mask rather than the range. We corrected yesterday and it shows properly now.
Except that – for most of them it shows the correct location in the http trace for that defect (for example as -

X-GeoIPLocation:

International

)

However it does not put the user in that group. More details below:

Now:

User group : new users

[cid:[e-mail]413D20]

What we were expecting:

User group: International

[cid:[e-mail]413D20]



From: CA APM/Wily Global User CommunityMessage Boards [mailto:[e-mail]]
Sent: Thursday, March 06, 2014 10:46 AM
To: [e-mail]
Subject: [CA APM General Discussion] RE: New HTTP Analyzer Plug-in: CEM User Groups by Geographical Location


It is mapping properly, it's just that your IP Address ranges are huge and overlapping. You seem to be using a subnet mask as the last part of your IP range, which might be part of the confusion. I recommend reading up on IP range blocks<http://www.mediawiki.org/wiki/Help:Range_blocks> to get a better understanding of how you can specify these.

Also, the IP range overrides are first-match; since the International range is specified first in the geoip.internal.networks property, it's the first match for the 156.82.244.165 address, even though it could also be contained by the test2 range. Once you redefine your ranges so that they do not overlap, you should get your expected results.
Posted by:jakbutler
--
CA Communities Message Boards
110615353
[e-mail]<mailto:[e-mail]>
https://communities.ca.com

[Legacy User]

If you're seeing the output parameter, the plug-in is working correctly. I would double-check the Business Application definition and make sure that, for each business application (including Default), the X-GeoIPLocation Plug-in Parameter is defined for the User Group Identification settings. 

[Anon Anon]

Thanks for the response!

We found that the user group info is not accurate due to users previously in new users group. We delete all previous user groups and cleaned up users and now we see plugin is working perfectly. We intend to do more testing though. Thanks much for the plugin







From: CA APM/Wily Global User CommunityMessage Boards [mailto:[e-mail]]
Sent: Thursday, March 06, 2014 12:59 PM
To: [e-mail]
Subject: [CA APM General Discussion] RE: [CA APM General Discussion] RE: New HTTP Analyzer Plug-in: CEM User Gro


If you're seeing the output parameter, the plug-in is working correctly. I would double-check the Business Application definition and make sure that, for each business application (including Default), the X-GeoIPLocation Plug-in Parameter is defined for the User Group Identification settings.
Posted by:jakbutler
--
CA Communities Message Boards
110623925
[e-mail]<mailto:[e-mail]>
https://communities.ca.com

[Legacy User]

I'm glad to hear that! Let me know if you run into any difficulty or if you have enhancement requests. 

[Anon Anon]

We wanted to disable geo plugin as for some reason its not giving the accurate location probably based on geolite data. I disabled it in CEM console and i see NOT enabled in TIM.

But for some we are still seeing the usergroups from Geoplugin. Can you help provide the proper process to disable the plugin?

Thanks for your help

[Legacy User]

Are you seeing new users get put into the geographic groups, or are you referring to the existing users still being assigned to the user group from the GeoIPCemPlugin? Even when disabled, the existing groups will not be removed or modified unless you manually re-group the users yourself (which can be done group by group via the CEM UI).

 

It sounds like you've followed all the necessary steps to disable, and you can confirm this by logging on to the TIM appliance and looking at the GeoIPCemPlugin.log (/usr/local/wily/cem/tim/logs/GeoIPCemPlugin.log). Additionally, upon restart of the TIM, it will usually log which extensions have been turned on and should not list the GeoIPCem plugin (/usr/local/wily/cem/tim/logs/timlog.txt); the TIM log is also visible through the CEM UI.

[lycohen]

I really appreciate this feature.

Thanks for your effort.

I have some questions regarding the installation since it seems that it's not working and I assume that I missing something.

 

I'm looking for geolocalize my internet users but even when it seems that is working nor groups are discovered and the last line in the log is "Tue May 24 17:37:09 ART 2016 [INFO] Plugin succesfully Initialized". Even when the propertie file sets the log in DEBUG mode.  Also I can say that the plugin is correctly enabled in the TIM.

 

I assume that I have X-Forwarded-For but using autodiscovery to check how the CEM captures the IP I could not find that field. I do find the field URL ClientIP.

How do I know if I setting correctly the incoming and outgoing parameters.? I saw an example posted where they are using a different field : client_ip. Should be another one? How do I know if its working ?

 

I send you the properties file just in case.

 

Best regards.

 

 

geoip.city.file=data/GeoLiteCity.dat

geoip.log.level=DEBUG

geoip.log.max.byte.size=2000000

geoip.cem.incoming.parameter.name=X-Forwarded-For

geoip.cem.outgoing.parameter.name=X-GeoIPLocation

geoip.internal.networks=A,B,C

geoip.internal.networks.A.name=Internal Network, 24-bit block

geoip.internal.networks.A.ranges=10.0.0.0/8

geoip.internal.networks.B.name=Internal Network, 20-bit block

geoip.internal.networks.B.ranges=172.17.0.0/12

geoip.internal.networks.C.name=Internal Network, 16-bit block

geoip.internal.networks.C.ranges=192.168.0.0/16

geoip.cem.parameter.format={0}, {4}

geoip.cem.parameter.value.0=countryName

geoip.cem.parameter.value.1=city

[Hallett German]

Hi Lenardo:

Check your TIM Collector Logs for errors related to this feature

 

Thanks

Hal German