Mobile device IP not necessarily a geolocator

Following on from my post yesterday 'Calculate distance between web site visitor IP and list of locations' I've come across an interesting revelation.

The client was off playing with my new page that establishes the web site visitor's geolocation from their IP number. And he complained because nothing was right.

Turns out he was browsing the page via iPhone. So I set up a test with my own iPhone, and hey presto, the IP geolocator system database thinks my phone is in Sydney. Which is definitely incorrect because it's sitting on my desk in front of me here in Melbourne. I proved the point by switching the iPhone's wifi back on, and now it correctly reports its location as Melbourne.

What I presume is that mobile carriers assign IP numbers to mobile devices from a central place – eg the IP range is probably designated to their head office or datacentre. And thus, like me, your phone can think it's in Sydney when in reality it's in Melbourne.

Of course iPhone Apps can access a phone's coordinates in a different way via the cellular network – hence some applications popping up a message asking you to agree to them accessing your location. That's how cool gadgets like Four Square work.

But it you rely on IP/coordinate lookups then the answer will be wrong.

For the moment I think we'll disable the fancy stuff on the web site for people browsing with mobile devices, but now I'm wondering how we can deal with this revelation.

 

Calculate distance between web site visitor IP and list of locations

I’ve been having fun this morning writing code to calculate the distance between a web site visitor’s location, and a number of fixed locations, using the visitor’s IP number.

I have a list of events in different geographic locations. The idea is that when someone arrives at the web site we show them information about the event that is closest to them.

I’ve used a couple of tools to make this work.

First up I found the GeoLiteCity database from MaxMind. This is a free database that can calculate the longitude and latitude of an IP number. They say they cover the whole of the IPv4 system (and a fair chunk of the IPv6).

MaxMind does have a web service available (paid) but you can download the GeoLiteCity database and query it directly – they have a good set of example APIs. I’m using the COM object on a Windows server.

The database is a binary file that you reference via file path. They do supply a CSV to import into SQL but they make the point the binary is seriously faster, so unless you need additional access to the data, it’s not worth the loss of speed. If you also use the COM just take note of their instruction to rename the database file from GeoLiteCity.dat to GeoIPCity.dat. I have the file sitting just above the www root folder so it’s not visible to the world.

Finding the site visitor’s latitude and longitude is a doddle:

Dim geoip,visitorLat,visitorLong
set geoip = Server.CreateObject("GeoIPCOMEx.GeoIPEx")
geoip.set_db_path("D:\inetpub\GeoIPDb\")
geoip.find_by_addr(request.ServerVariables("REMOTE_ADDR"))
visitorLat=geoip.latitude
visitorLong=geoip.longitude


I then hunted around online and found the longitute and latitude of the cities where my events are occuring – we’re not looking for ‘to the metre’ resolution here, just a quick guesstimate of which event and city is closest. When I get some more time I’ll sit down and figure out the coordinates of the actual venue for each event.

So now I need a way to calculate the distance between the visitor’s IP, and all of the events. Along the way I found this really useful page from Moveable Type  ‘Calculate distance, bearing and more between Latitude/Longitude points‘. It’s a great introduction to all the maths needed.

But another Google search found a forum post with a SQL stored procedure that calculates the distance between two coordinates, and it works well.

CREATE FUNCTION dbo.UDF_Distance
(
@FromLat float, @FromLong float, @ToLat float, @ToLong float
)
RETURNS FLOAT
AS
BEGIN
-------------------------------------------------------------------------------------------------
-- Function Name	: UDF_Distance
-- Description		: Returns distance in KM between two points on earth
-- Created by		: David Eedle
-- Created on		: 24 Jan 2011
-- Source:			: http://www.dbnewsgroups.net/sqlprogramming/t19131-sql-distance-between-latitude-longitude.aspx
-------------------------------------------------------------------------------------------------
declare @X float
SET @X =
Sin(Radians(@FromLat))
* Sin(Radians(@ToLat))
+ Cos(Radians(@FromLat))
* Cos(Radians(@ToLat))
* Cos(Radians(@ToLong)-Radians(@FromLong))
SET @X = Acos(@X)
RETURN 1.852 * 60.0 * Degrees(@X)
END
GO
/*
select dbo.uf_Distance (41.63,-87.73,41.7,-88.07)
go
*/

I used the Moveable Type page to double check the results and the procedure yields numbers within a few kilometres of MT’s answers over distances of 12,000km. If I was a maths whiz I’d probably now sit down for a day and figure out the inconsistency but I’m not trying to send a rocket to the moon so we’ll leave that for the experts.

Finally it was just a question of writing a quick query that pulls the coordinates of the events out, along with a calculation of the distance between the visitor and each event along the lines of:

SELECT cityId,cityName,cityState,cityDate,
CAST((select dbo.UDF_Distance ("&visitorLat&","&visitorLong&",V_ListCities.cityLat,V_ListCities.cityLong)) AS int) distanceKM
FROM V_ListCities ORDER BY (select dbo.UDF_Distance ("&visitorLat&","&visitorLong&",V_ListCities.cityLat,V_ListCities.cityLong))


 

 

 

 

 

 

 

 

ISAPI ReWrite Rules on Individual Web Sites

ISAPI ReWrite has been driving me up the wall for the last day or two. I have Version 3 (fully registered) installed on a web server, which has half a dozen individual web sites up and running. I wanted to implement some rewrite rules for one of the web sites, so I used the ISAPI ReWrite Manager interface to edit the .htaccess file in the root of that specific web site and add a rule along the lines of:

RewriteRule /register/([^/]+)/ /register/register.asp\?eventId=$1 [I,L]

I already have this web site running on my development server, and I know this rule works.

The rules didn't work. I moved them up to the top level of the Manager, which edits the httpd.conf file in the installation directory for ISAPI ReWrite. And the rules worked.

I turned on logging by adding this to the httpd.conf file:

#enabling rewrite.log
RewriteLogLevel 9
#enabling error.log
LogLevel debug

The logging clearly showed the .htaccess file being accessed and the rules in that file being run through:

Htaccess process request d:\inetpub\wwwroot\mydomain.com\.htaccess
applying pattern '/register/([^/]+)/' to uri 'register/My-Event-Name/'

But no dice in the browser, just file not found errors. So the rule is being processed, but the resulting URL clearly is not right.

I read through the troubleshooting information on the Helicon web site. Which is when this phrase caught my eye:

Also remember that rewriting base inside .htaccess files by default is different from httpd.conf. ISAPI_Rewrite automatically strips the local directory prefix from the path when in .htaccess or <Directory> section and applies rules only to the remainder. So if your rules start with a '/' character, this should be removed in .htaccess file in the root folder of web site.

This is the first time I've used the registered version of ISAPI ReWrite, in the past I've only used the Lite version, mostly because I didn't need to bring rules down to the level of the individual web site (on this project I will need to because multiple sites will need the same URL/folder combinations).

So the idea that there is a subtle but key difference in the way you construct the rewrite rules between .htaccess at the individual site level, and the top level httpd.conf file has never come across my radar before.

I amended the rule to:

RewriteRule register/([^/]+)/ /register/register.asp\?eventId=$1 [I,L]

Notice there is no forward slash in front of the register/ name. And the redirect worked completely fine.

 

 

 

 

I love descriptive, intuitive error messages

If you see an error like the following in your web code and SQL Server, what do you naturally think is the problem:

Microsoft Cursor Engine error '80040e21'
Multiple-step operation generated errors. Check each status value.

Well, looks like a database error, given the 'cursor engine' phrase. But a 'multi-step operation'?

I've now discovered that it appears almost always to be you trying to insert a string that's too long for the field.

I love descriptive, intuitive error messages.

Solving ‘Login timeout expired’ Problem on Windows Server 2008

I’ve been helping set up a new web site, we have two servers, all very off the shelf, both running Windows Server 2008, one with SQL Server 2008 as the database sever, the other acting as the web server.

The two boxes sit behind a firewall with only the necessary ports open (eg 80, 21, 443 etc). Windows FIrewall is turned on for both boxes, and on the SQL box we have port 1433 open but restricted to only the internal IP of the web server, so the web application can connect to the database.

The application is talking to the database using a standard connection string along the lines of:

Provider=sqloledb; Network Library=DBMSSOCN; Data Source=[sql server ip]; Initial Catalog=[db name]; User ID=[db user]; Password=[password];

We loaded up the site without major problems, but then started to notice this error occuring at seemingly random moments:

Microsoft OLE DB Provider for SQL Server error ‘80004005’
Login timeout expired

There seemed no rhyme nor reason, one minute you are browsing pages, the next up pops this error, then you refresh the page and all was well.

Clearly it’s a connection error between the web application and the SQL server. I dug into the Windows and SQL logs everywhere trying to correlate entries with the time the error occured, but to no avail.

There’s a fair amount online about this error, although it’s often tied to simple connectivity issues, such as incorrect IP numbers, server address, firewall settings and the like.

Finally I stumbled over this page ‘Troubleshooting: Timeout Expired‘ which gave the clue, and particularly:

‘Either start the SQL Server Browser service, or connect specifying the TCP/IP port number.’

We don’t have the SQL Server Browser service running – it’s not started by default. So I simply added the port number to the connection string and, fingers crossed, everything’s been fine since, eg:

Provider=sqloledb; Network Library=DBMSSOCN; Data Source=[sql server ip],[port]; Initial Catalog=[db name]; User ID=[db user]; Password=[password];

Actually I normally DO have the port number in the connection string – because I fell into the habit for a long time of running SQL on a non-standard port to avoid nasties with 1433 worms and the like in situations where I wasn’t so confident about the firewalling, or where we needed to leave the SQL Server exposed with IP or tunnel protection.

 

 

Scheduled Tasks Failing to Fire VBS Scripts

I have a Windows Server 2003 box with a bunch of scheduled tasks that fire VBS scripts through the day, handling various maintenance, data import and other jobs for a client's web site.

Over the past few days I noticed the jobs were not firing, and after a fair bit of research I finally found a solution.

The first clue came when I tried to call the VBS scripts directly in the command prompt, and received the following error message:

cscript error loading your settings failed access is denied

Which seemed a strange error given I was logged in as a user with Administrator permissions.

After a little more poking around I noticed two more oddities:

1. The Start Menu didn't look right – for example, the Accessories folder didn't contain many of the shortcuts I'd grown used to

2. The Documents and Settings folder contained duplicate user profile folders, for example:

– [username]
– [username].[machinename]
– [username].[machinename].000
– [username].[machinename].001
– [username].[machinename].002

all the way through to

– [username].[machinename].012

I also noticed in my command prompt window that I appeared to be logged in under the '012' profile.

I found this article that pointed the way:

http://support.microsoft.com/?kbid=314045

I searched the Registry for the ProfileImagePath key and found it was set to:

%SystemDrive%\Documents and Settings\[username].[machinename].002

I changed it to:

%SystemDrive%\Documents and Settings\[username]

I logged off, and logged back on again, and this time I was logged in with the [username].[machinename] profile – confirmed by opening the command prompt:

C:\Douments and Settings \[username].[machinename]>

I then tried to call the VBS scripts in the command prompt and they worked fine. Then via Scheduled Jobs window. Again all ok. And my Start Menu is back to normal as well.

As a sidebar I've noticed that the duplicate user profile folders were all created over the space of a single day – Christmas Day. Which is strange because I know that on Xmas Day I wasn't hunched over a computer RDCing into web servers, rather I was enjoying a day off with my family. The duplicate folders were all empty (eg they didn't have the normal complement of folders such as My Documents, Desktop, Favourites and so forth). All except the last one, 012. But that was the one I was logged into. So somehow Windows reacted to something and created the duplicates – but presumably whatever the 'something' was didn't constitute a login otherwise it would have populated the folders with the subfolders needed for the profile.

I've Google a little but can't immediately find anything that explains a big bunch of duplicate user profile folders turning up that are all empty.

All a bit strange, and I wish I had a more concrete explanation for how this all came about.