Forum Discussion
Converting Latitude & Longitude to Nautical Miles
mtarler Thanks for trying to help. I don't think you're answer is very clear or actionable. Either way, I succeeded in converting my coordinates into radians. From there I found two formulas to convert those coordinates into miles.
The first formula I found was from https://blog.batchgeo.com/manipulating-coordinates-in-excel/ and had the following sequence =3443.8985*(ACOS(SIN(N2)*SIN(R2)+COS(N2)*COS(R2)*COS(S2-O2))). I believe this formula solves for nautical miles which is longer than the traditional mile.
The second formula I found http://www.cpearson.com/excel/latlong.aspx and could be retrieved by downloading the spreadsheet. I believe the second formula (utilized the 'P-Theorum' you discussed in your answer which) in this case solves for the radius of the earth in miles =3958.82*((2*ASIN(SQRT((SIN((N2-R2)/2)^2)+COS(N2)*COS(R2)*(SIN((O2-S2)/2)^2))))).
As you can see, there is a slight difference between the two formulas.
To be clear, if you are looking for the distance in populated areas with buildings and roads to navigate, this calculation will be even further from accurate. For example:
The image above shows the distance between two stations, displaying both the direct distance (roughly 570.14m or 0.354 mi) and navigatable distance (0.5mi). Considering it would be impossible to navigate your bicycle through buildings, the practical route along navigatable roads is longer.
Even if we would compare the direct distance (0.354mi) results given by Google Maps and then compare them to the results received by the output of our two formulas there is also a slight difference. With the nautical miles formula output of 0.327mi and formal miles output of 0.376mi. As you can see, there are still slight differences between both results. However, when comparing the actual navigatable distance (0.5mi), there is an even larger discrepancy which could definitely skew results.
With (hundreds of) thousands of rows of data, I don't know if it would be possible to 'add-in' some sort of Google API that could measure the practical navigatable distance between the two points by using, for example, street names. Furthermore, with missing data (i.e. no street names) and only having the latitudinal and longitudinal coordinates, I don't know how it would be possible to have an API that would recognize the practical navigatable distance between two coordinates (i.e. navigating using available roads, not passing through buildings). Understandably, it would also depend on how accurate the coordinates are and whether those coordinates could attach to the most approximate address.
I did see a product by https://www.cdxtech.com/cdxzipstream/free-trial/ that seemed interesting and perhaps could solve this issue. Either way, I would like to find out what other options are out there. Let the hunt begin!
To convert latitudinal and longitudinal coordinates into miles first you must convert those coordinates into radians for excel. That can be done in two ways. Input the =RADIANS() function to your latitudinal and longitudinal values. Or by using the formula =(Value)/180)*PI(). Ensure your cells are in the correct data type format to prevent error messages and incorrect outputs.