<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7651.14">
<TITLE>Geocoding with PAGC Primer (long)</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P ALIGN=CENTER><B><FONT FACE="Times New Roman">Geocoding with PAGC</FONT></B></P>
<P><B><FONT FACE="Times New Roman">Backgrounder</FONT></B>
<UL>
<UL>
<LI><FONT FACE="Times New Roman">The moment we started to leak word out about PAGC we quickly realized that data issues swamped software issues.</FONT></LI>
<LI><FONT FACE="Times New Roman">Like everything in this area, software and data issues go hand-in-hand</FONT></LI>
<LI><FONT FACE="Times New Roman">There is a lot of art in geocoding. </FONT></LI>
<LI><FONT FACE="Times New Roman">It turns out that a 90% success rate is sort of the industry standard in terms of geocoding addresses. </FONT></LI>
<LI><FONT FACE="Times New Roman">Commercial companies really boast about doing better than this, and they typically do it by resorting to the use of FSA centroids (which can be wildly off)</FONT></LI>
<BR>
<BR>
</UL></UL>
<P><B><FONT FACE="Times New Roman">Priority: Augmented RNF</FONT></B>
<UL>
<UL>
<LI><FONT FACE="Times New Roman">PAGC tools work with augmented data that is currently locked up under copyright. Effort to produce augmented data free of copyright restrictions</FONT></LI>
<LI><FONT FACE="Times New Roman">Key to the kingdom</FONT></LI>
<LI><FONT FACE="Times New Roman">data issues are huge.</FONT></LI>
<LI><FONT FACE="Times New Roman">with FSAs and Cities/towns</FONT></LI>
<LI><FONT FACE="Times New Roman">Once we get enough information into the StatsCan RNF, it could be the basis for developing an open source standardizer</FONT></LI>
<BR>
</UL></UL>
<P><B><FONT FACE="Times New Roman">Labour intensive aspects: </FONT></B>
<UL>
<OL TYPE=1>
<LI><FONT FACE="Times New Roman">determining what road segments to keep in an RNF that contains only the FSA boundary roads for multi-FSA cities and creating the finished layers for those cities</FONT></LI>
<LI><FONT FACE="Times New Roman">Determining what other linear features to use that are not roads. Ie railroads, rivers, etc</FONT></LI>
<LI><FONT FACE="Times New Roman">augmenting the Atlas of Canada populated place data to include FSAs.</FONT></LI>
<BR>
<BR>
</OL></UL>
<P><B><FONT FACE="Times New Roman">Populated place names with FSA</FONT></B>
<BR><U><FONT FACE="Times New Roman">Urban</FONT></U><FONT FACE="Times New Roman">:</FONT>
<UL>
<OL TYPE=1>
<LI><FONT FACE="Times New Roman">populated place names like Ottawa have multiple FSA’s per populated place</FONT></LI>
<LI><FONT FACE="Times New Roman">urban areas with multiple FSAs relying only on the RNF is nearly sufficient for creating FSA polygons. </FONT></LI>
<LI><FONT FACE="Times New Roman">The other piece of information that is needed are the names of the roads that typically act as FSA boundaries in these areas </FONT></LI>
<LI><FONT FACE="Times New Roman">although railways and bike paths are also used as FSA boundaries in a few instances </FONT></LI>
<LI><FONT FACE="Times New Roman">This information is provided by Canada Post in their document entitled "Canada.pdf,"</FONT></LI>
<LI><FONT FACE="Times New Roman">we can get most of the boundaries by removing all but the FSA boundary street segments from the RNF via ogr2ogr</FONT></LI>
</OL></UL>
<P><U><FONT FACE="Times New Roman">Rural</FONT></U>
<UL>
<OL TYPE=1>
<LI><FONT FACE="Times New Roman">Rural locations may have multiple populated areas per FSA </FONT></LI>
<LI><FONT FACE="Times New Roman">What will work pretty well is using the populated places point file from the Atlas of Canada and then adding a field to this file that gives the FSA of each populated place </FONT></LI>
<LI><FONT FACE="Times New Roman">do this for populated places that have a population category value of 1, places in population category 2 and above are likely to have multiple postal codes). </FONT></LI>
<LI><FONT FACE="Times New Roman">Once this is done, the Census CSD (or maybe even the Census CD) polygon layer can be used to determine the rural FSA a CSD or CD falls into, and then doing a dissolve to merge the polygons that have the same assigned FSA.</FONT></LI>
<BR>
</OL></UL>
<P><B><FONT FACE="Times New Roman">Milestone Issues</FONT></B>
<UL>
<UL>
<LI><FONT FACE="Times New Roman">we aren't in a position to create the FSA data until StatsCan releases the 2006 RNF and CSD layers, which should be in the next few weeks 9as of September 19, 2006). </FONT></LI>
<LI><FONT FACE="Times New Roman">In the mean time, we have enough data (for Ottawa) to figure out if I'm right on how to proceed </FONT></LI>
<LI><FONT FACE="Times New Roman">With the side benefit of creating a fully augmented RNF for Ottawa</FONT></LI>
<BR>
</UL></UL>
<P><B><FONT FACE="Times New Roman">FAQ’s</FONT></B>
<UL>
<OL TYPE=1>
<LI><FONT FACE="Times New Roman">Lists without postal codes or city info require manual intervention: As for the POSTAL issue, some sort of local identifier is needed. A CITY should suffice, and I will talk to Walter about the possibility of requiring either one of the two. </FONT></LI>
<LI><FONT FACE="Times New Roman">Completely blowing away this postal check would be unwise, however, since you know someone is going to try to geocode things based on the RNF for the entire country. In that instance, only a POSTAL field would solve the problem, hence why it is really important to create an augmented RNF for Canada.</FONT></LI>
<LI><FONT FACE="Times New Roman">You should not have to touch either the the rules or the gazetteer files. PAGC was built with StatsCan (and US TIGER/Line) in mind. </FONT></LI>
<LI><FONT FACE="Times New Roman">Don't get too nervous about the errors in the road index build error file either.</FONT></LI>
<LI><FONT FACE="Times New Roman">Creating smaller RNF road files that don’t have multi-part lines: In terms of the road segment problem, I think a safer approach is to use the polygon to attach a flag variable to the attribute table of the road layer, and then use ogr2ogr, with a -where "flag=1" like option to select only those road segments that are in (or are on the border of) the Ottawa polygon. This approach is extremely unlikely to "damage" a road segment since ogr will extract the full segment intact. It does mean that the Ottawa polygon will have a few "whiskers" (road segments that partially lie outside the Ottawa polygon). Using the whisker analogy, it appears that Open Jump snipped an "ingrown hair," and that causes problems. I've been working on attaching the needed flag to the RNF attribute table over the past few days (working on it for slightly less than an hour a day), and should have something tonight or tomorrow.</FONT></LI>
<LI><FONT FACE="Times New Roman">PAGC chokes on line 960: It turns out the error is caused by the 961st road segment (which is given number 960 since counting starts at 0 in C in the error file). </FONT></LI>
<LI><FONT FACE="Times New Roman">Polylines are topologicaly ok but chokes PAGC: Polylines shouldn't have bounding boxes, ring direction, and so on. This road segment has these extra attributes (which means it is actually being written as a polygon or multi part polyline, rather than a single polyline), hence PAGC's complaining about it having too many parts . </FONT></LI>
<LI><FONT FACE="Times New Roman">what about FSA centroids from Geoconnections: The resulting FSA centroid file is more of a dog's breakfast than one would think it would be. In the case of BC, data is unavailable for two FSAs (V1K in Merritt and V8B in Squamish), at least one FSA is put in the wrong community (the one I know about is V1H which shows up in Prince George, but should be in Vernon, several hundred km away), and the province field has the wrong data in three cases. In addition, the rural FSA centroids are problematic since there is a single centroid for areas that are enormous (multiple hundreds if not thousands of square kilometres in size). Finally, I get a few centroids that wind up in the Georgia Strait. This can happen since the centroid of a polygon may fall outside the polygon itself. All in all the FSA centroid file is useful, but it has a lot of kinks.</FONT></LI>
<BR>
</OL></UL>
<P><B><FONT FACE="Times New Roman">Supplemental Products</FONT></B>
<UL>
<OL TYPE=1>
<LI><FONT FACE="Times New Roman">FSA Centroids</FONT></LI>
<BR>
<BR>
</OL></UL>
<P><B><FONT FACE="Times New Roman">Derived Data Products:</FONT></B>
<UL>
<OL TYPE=1>
<LI><FONT FACE="Times New Roman">intersection and cross roads</FONT></LI>
<LI><FONT FACE="Times New Roman">fully augmented RNF for Ottawa</FONT></LI>
<LI><FONT FACE="Times New Roman">an address standardizer database</FONT></LI>
<LI><FONT FACE="Times New Roman">a street intersection layer</FONT></LI>
<BR>
</OL></UL>
<P><B><FONT FACE="Times New Roman">Future Software Development</FONT></B>
<UL>
<OL TYPE=1>
<LI><FONT FACE="Times New Roman">Address standardization: developed for direct marketing mail campaigns to clean-up dirty addresses</FONT></LI>
<LI><FONT FACE="Times New Roman">Already have written R scripts to extract road intersections from an RNF and attach attributes (like the names of streets that form the intersection) which forms the input data of intersection geocoding</FONT><B></B></LI>
<LI><FONT FACE="Times New Roman">Walter already has code in PAGC for doing point matching,</FONT><B></B></LI>
<LI><FONT FACE="Times New Roman">Recode PAGC from command prompt program to a library, and build command line program to call library. Allows for inclusion into other GIS software packages.</FONT><B></B></LI>
<BR>
<BR>
</OL></UL>
<P><B><FONT FACE="Times New Roman">Resource Links:</FONT></B>
<BR><B><FONT FACE="Times New Roman">Canada post FSA Maps</FONT></B>
<BR><B></B><A HREF="http://www.canadapost.ca/common/tools/pg/fsamaps/pdf/Canada.pdf"><B></B><B><U><FONT COLOR="#0000FF" FACE="Times New Roman">http://www.canadapost.ca/common/tools/pg/fsamaps/pdf/Canada.pdf</FONT></U></B><B></B></A><B></B>
</P>
<P><B><FONT FACE="Times New Roman">Wikipedia Canadian FSA Lists:</FONT></B>
<BR><B></B><A HREF="http://en.wikipedia.org/wiki/List_of_A_Postal_Codes_of_Canada"><B></B><B><U><FONT COLOR="#0000FF" FACE="Times New Roman">http://en.wikipedia.org/wiki/List_of_A_Postal_Codes_of_Canada</FONT></U></B><B></B></A><B></B>
</P>
<P><B><FONT FACE="Times New Roman">Free or no licence issues</FONT></B>
<BR><A HREF="http://www.postalcodelookup.ca/"><U><FONT COLOR="#0000FF" FACE="Times New Roman">http://www.postalcodelookup.ca/</FONT></U></A><B></B>
</P>
<P><B><FONT FACE="Times New Roman">Unknown Licence issues</FONT></B>
<BR><B></B><A HREF="http://www.postescanada.ca/cpc2/addrm/hh/current/indexp/tpALL-e.asp"><B></B><B><U><FONT COLOR="#0000FF" FACE="Times New Roman">http://www.postescanada.ca/cpc2/addrm/hh/current/indexp/tpALL-e.asp</FONT></U></B><B></B></A><B></B>
</P>
<P><B><FONT FACE="Times New Roman">Other geocoders: (web )</FONT></B>
<BR><B></B><A HREF="http://geocoder.ca/"><B></B><B><U><FONT COLOR="#0000FF" FACE="Times New Roman">http://geocoder.ca/</FONT></U></B><B></B></A><B></B>
<BR><B></B><A HREF="http://www.batchgeocode.com/"><B></B><B><U><FONT COLOR="#0000FF" FACE="Times New Roman">http://www.batchgeocode.com/</FONT></U></B><B></B></A><B></B>
<BR><B></B><A HREF="http://geoservices.cgdi.ca/postalcode/sample.html"><B></B><B><U><FONT COLOR="#0000FF" FACE="Times New Roman">http://geoservices.cgdi.ca/postalcode/sample.html</FONT></U></B><B></B></A><B></B>
</P>
<BR>
</BODY>
</HTML>