[postgis-tickets] [SCM] PostGIS branch master updated. 79e999360209cbe4a98ad26f39819fd1440db4f8
git at osgeo.org
git at osgeo.org
Wed Jan 15 01:51:46 PST 2020
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "PostGIS".
The branch, master has been updated
via 79e999360209cbe4a98ad26f39819fd1440db4f8 (commit)
via de6f53284f12eee226b226b954ec5ccdfce94957 (commit)
from 5cf0d27f40e9ed1c1e7454aefc1757c6c7d03faa (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 79e999360209cbe4a98ad26f39819fd1440db4f8
Author: Sandro Santilli <strk at kbt.io>
Date: Wed Jan 15 10:50:50 2020 +0100
Reorganize install section
Put "from template", "via extension" and "via script" sections
under a common "Creating spatial databases" section.
diff --git a/doc/installation.xml b/doc/installation.xml
index 86b9a1e..c3a9f29 100644
--- a/doc/installation.xml
+++ b/doc/installation.xml
@@ -1797,7 +1797,10 @@ All 2 tests passed.
</sect2>
</sect1>
- <sect1 id="create_new_db_extensions">
+ <sect1 id="create_spatial_db">
+ <title>Creating spatial databases</title>
+
+ <sect2 id="create_new_db_extensions">
<title>Creating a spatial database using EXTENSIONS</title>
<para>
@@ -1840,9 +1843,9 @@ All 2 tests passed.
E.g. if you upgraded from 2.4.3 to 2.5.0, then you need to reinstall the legacy.sql packaged with 2.5.0. This is because some of the functions make reference to the library
and the library is named with the minor in it.</para></note>
<para>You can later run <filename>uninstall_legacy.sql</filename> to get rid of the deprecated functions after you are done with restoring and cleanup.</para>
- </sect1>
+ </sect2>
- <sect1 id="create_new_db">
+ <sect2 id="create_new_db">
<title>Create a spatially-enabled database without using extensions</title>
<note><para>This is generally only needed if you cannot or don't
@@ -1940,262 +1943,9 @@ environment).</para></note>
and GeoServer. If you have views that use things like distance / length etc, you'll need the full blown <filename>legacy.sql</filename></para></note>
<para>You can later run <filename>uninstall_legacy.sql</filename> to get rid of the deprecated functions after you are done with restoring and cleanup.</para>
- </sect1>
-
- <sect1 id="installing_pagc_address_standardizer"><title>Installing and Using the address standardizer</title>
- <para>The <code>address_standardizer</code> extension used to be a separate package that required separate download. From PostGIS 2.2 on, it is now bundled in.
- For more information about the address_standardize, what it does, and how to configure it for your needs, refer to <xref linkend="Address_Standardizer" />.</para>
- <para>This standardizer can be used in conjunction with the PostGIS packaged tiger geocoder extension as a replacement for the <xref linkend="Normalize_Address" /> discussed.
- To use as replacement refer to <xref linkend="tiger_pagc_address_standardizing" />.
- You can also use it as a building block for your own geocoder or use it to standardize your addresses for easier compare of addresses.</para>
-
- <para>The address standardizer relies on PCRE which is usually already installed on many Nix systems,
-but you can download the latest at: <ulink url="http://www.pcre.org">http://www.pcre.org</ulink>. If during <xref linkend="installation_configuration" />, PCRE is found, then the address standardizer extension will automatically be built. If you have a custom pcre install you want to use instead, pass to configure <code>--with-pcredir=/path/to/pcre</code> where <filename>/path/to/pcre</filename> is the root folder for your pcre include and lib directories.</para>
-
- <para>For Windows users, the PostGIS 2.1+ bundle is packaged with the address_standardizer already so no need to compile and can move straight to <code>CREATE EXTENSION</code> step.</para>
-
-
- <para>Once you have installed, you can connect to your database and run the SQL:</para>
- <programlisting>CREATE EXTENSION address_standardizer;</programlisting>
-
- <para>The following test requires no rules, gaz, or lex tables</para>
- <programlisting>SELECT num, street, city, state, zip
- FROM parse_address('1 Devonshire Place PH301, Boston, MA 02109');</programlisting>
- <para>Output should be</para>
- <screen> num | street | city | state | zip
------+------------------------+--------+-------+-------
- 1 | Devonshire Place PH301 | Boston | MA | 02109</screen>
-
- <sect2><title>Installing Regex::Assemble</title>
- <para>Perl Regex:Assemble is no longer needed for compiling address_standardizer extension since the files it generates are part of the source tree. However if you need to edit the <filename>usps-st-city-orig.txt</filename> or <filename>usps-st-city-orig.txt usps-st-city-adds.tx</filename>, you need to rebuild <filename>parseaddress-stcities.h</filename> which does require Regex:Assemble.</para>
- <programlisting>cpan Regexp::Assemble</programlisting>
- <para>or if you are on Ubuntu / Debian you might need to do</para>
- <programlisting>sudo perl -MCPAN -e "install Regexp::Assemble"</programlisting>
- </sect2>
- </sect1>
-
- <sect1 id="loading_extras_tiger_geocoder">
- <title>Installing, Upgrading Tiger Geocoder and loading data</title>
-
- <para>Extras like Tiger geocoder may not be packaged in your PostGIS distribution. If you are missing the tiger geocoder extension or want a newer version than what your install comes with, then use
- the <filename>share/extension/postgis_tiger_geocoder.*</filename> files from the packages in <ulink url="http://postgis.net/windows_downloads/">Windows Unreleased Versions</ulink> section for your version of PostgreSQL.
- Although these packages are for windows, the postgis_tiger_geocoder extension files will work on any OS since the extension is an SQL/plpgsql only extension.</para>
- <sect2 id="install_tiger_geocoder_extension">
- <title>Tiger Geocoder Enabling your PostGIS database: Using Extension</title>
- <para>If you are using PostgreSQL 9.1+ and PostGIS 2.1+, you can take advantage of the new extension model for installing tiger geocoder. To do so:</para>
- <orderedlist>
- <listitem><para>First get binaries for PostGIS 2.1+ or compile and install as usual. This should install the necessary extension files as well for tiger geocoder.</para></listitem>
- <listitem><para>Connect to your database via psql or pgAdmin or some other tool and run the following SQL commands. Note that if you are installing in a database that already has postgis, you don't need to do the first step. If you have <varname>fuzzystrmatch</varname> extension already installed, you don't need to do the second step either.</para>
- <para><programlisting>CREATE EXTENSION postgis;
-CREATE EXTENSION fuzzystrmatch;
-CREATE EXTENSION postgis_tiger_geocoder;
---this one is optional if you want to use the rules based standardizer (pagc_normalize_address)
-CREATE EXTENSION address_standardizer;</programlisting></para>
-
-<para>If you already have postgis_tiger_geocoder extension installed, and just want to update to the latest run:</para>
-<programlisting>ALTER EXTENSION postgis UPDATE;
-ALTER EXTENSION postgis_tiger_geocoder UPDATE;</programlisting>
-<para>If you made custom entries or changes to <varname>tiger.loader_platform</varname> and <varname>tiger.loader_variables</varname> you may need to update these.</para>
-</listitem>
- <listitem><para>To confirm your install is working correctly, run this sql in your database:</para>
- <programlisting>SELECT na.address, na.streetname,na.streettypeabbrev, na.zip
- FROM normalize_address('1 Devonshire Place, Boston, MA 02109') AS na;</programlisting>
- <para>Which should output</para>
- <para><screen> address | streetname | streettypeabbrev | zip
----------+------------+------------------+-------
- 1 | Devonshire | Pl | 02109</screen></para>
- </listitem>
- <listitem><para>Create a new record in <varname>tiger.loader_platform</varname> table with the paths of your executables and server. </para>
- <para>So for example to create a profile called debbie that follows <code>sh</code> convention. You would do:</para>
- <programlisting>INSERT INTO tiger.loader_platform(os, declare_sect, pgbin, wget, unzip_command, psql, path_sep,
- loader, environ_set_command, county_process_command)
-SELECT 'debbie', declare_sect, pgbin, wget, unzip_command, psql, path_sep,
- loader, environ_set_command, county_process_command
- FROM tiger.loader_platform
- WHERE os = 'sh';</programlisting>
- <para>And then edit the paths in the <emphasis>declare_sect</emphasis> column to those that fit Debbie's pg, unzip,shp2pgsql, psql, etc path locations.</para>
-
- <para>If you don't edit this <varname>loader_platform</varname> table, it will just contain common case locations of items and you'll have to edit the generated script after the script is generated.</para>
- </listitem>
- <listitem><para>As of PostGIS 2.4.1 the Zip code-5 digit tabulation area <varname>zcta5</varname> load step was revised to load current zcta5 data and is part of the <xref linkend="Loader_Generate_Nation_Script" /> when enabled.
-It is turned off by default because it takes quite a bit of time to load (20 to 60 minutes), takes up quite a bit of disk space, and is not used that often.</para>
-<para>To enable it, do the following:</para>
-<programlisting>UPDATE tiger.loader_lookuptables SET load = true WHERE table_name = 'zcta510';</programlisting>
-<para>
-If present the <xref linkend="Geocode" /> function can use it if a boundary filter is added to limit to just zips in that boundary.
-The <xref linkend="Reverse_Geocode" /> function uses it if the returned address is missing a zip, which often happens with highway reverse geocoding.</para></listitem>
- <listitem><para>Create a folder called <filename>gisdata</filename> on root of server or your local pc if you have a fast network connection to the server. This folder is
-where the tiger files will be downloaded to and processed. If you are not happy with having the folder on the root of the server, or simply want to change to a different folder for staging, then edit the field <varname>staging_fold</varname> in the <varname>tiger.loader_variables</varname> table.</para></listitem>
- <listitem><para>Create a folder called temp in the <filename>gisdata</filename> folder or whereever you designated the <varname>staging_fold</varname> to be. This will be
-the folder where the loader extracts the downloaded tiger data.</para></listitem>
- <listitem><para>Then run the <xref linkend="Loader_Generate_Nation_Script" /> SQL function make sure to use the name of your custom profile and copy the script to a .sh or .bat file. So for example to build the nation load:</para>
- <programlisting>psql -c "SELECT Loader_Generate_Nation_Script('debbie')" -d geocoder -tA > /gisdata/nation_script_load.sh</programlisting>
- </listitem>
-
- <listitem><para>Run the generated nation load commandline scripts.</para>
- <programlisting>cd /gisdata
-sh nation_script_load.sh</programlisting>
- </listitem>
-
- <listitem><para>After you are done running the nation script, you should have three tables in your <code>tiger_data</code> schema and they should be filled with data. Confirm you do by doing the following queries from psql or pgAdmin</para>
- <programlisting>SELECT count(*) FROM tiger_data.county_all;</programlisting>
-<screen> count
--------
- 3233
-(1 row)</screen>
- <programlisting>SELECT count(*) FROM tiger_data.state_all;</programlisting>
-<screen>
- count
--------
- 56
-(1 row)
-</screen>
- </listitem>
-
- <listitem><para>By default the tables corresponding to <varname>bg</varname>, <varname>tract</varname>, <varname>tabblock</varname> are not loaded. These tables are not used by the geocoder but are used by folks for population statistics.
- If you wish to load them as part of your state loads, run the following statement to enable them.</para>
- <programlisting>UPDATE tiger.loader_lookuptables SET load = true WHERE load = false AND lookup_name IN('tract', 'bg', 'tabblock');</programlisting>
- <para>Alternatively you can load just these tables after loading state data using the <xref linkend="Loader_Generate_Census_Script" /></para></listitem>
-
- <listitem><para>For each state you want to load data for, generate a state script <xref linkend="Loader_Generate_Script" />.</para><warning><para>DO NOT Generate the state script until you have already loaded the nation data, because the state script utilizes county list loaded by nation script.</para></warning></listitem>
-
- <listitem><programlisting>psql -c "SELECT Loader_Generate_Script(ARRAY['MA'], 'debbie')" -d geocoder -tA > /gisdata/ma_load.sh</programlisting></listitem>
-
- <listitem><para>Run the generated commandline scripts.</para>
- <programlisting>cd /gisdata
-sh ma_load.sh</programlisting>
- </listitem>
- <listitem><para>After you are done loading all data or at a stopping point, it's a good idea to analyze all the tiger tables to update the stats (include inherited stats)</para>
- <programlisting>SELECT install_missing_indexes();
-vacuum (analyze, verbose) tiger.addr;
-vacuum (analyze, verbose) tiger.edges;
-vacuum (analyze, verbose) tiger.faces;
-vacuum (analyze, verbose) tiger.featnames;
-vacuum (analyze, verbose) tiger.place;
-vacuum (analyze, verbose) tiger.cousub;
-vacuum (analyze, verbose) tiger.county;
-vacuum (analyze, verbose) tiger.state;
-vacuum (analyze, verbose) tiger.zip_lookup_base;
-vacuum (analyze, verbose) tiger.zip_state;
-vacuum (analyze, verbose) tiger.zip_state_loc;</programlisting>
- </listitem>
- </orderedlist>
- <sect3 id="convert_tiger_geocoder_extension"><title>Converting a Tiger Geocoder Regular Install to Extension Model</title>
- <para>If you installed the tiger geocoder without using the extension model, you can convert to the extension model as follows:</para>
- <orderedlist>
- <listitem><para>Follow instructions in <xref linkend="upgrade_tiger_geocoder" /> for the non-extension model upgrade.</para></listitem>
- <listitem><para>Connect to your database with psql or pgAdmin and run the following command:</para>
- <programlisting>CREATE EXTENSION postgis_tiger_geocoder FROM unpackaged;</programlisting>
- </listitem>
- </orderedlist>
- </sect3>
-
- </sect2>
- <sect2 id="install_tiger_geocoder">
- <title>Tiger Geocoder Enabling your PostGIS database: Not Using Extensions</title>
- <para>
- First install PostGIS using the prior instructions.
- </para>
-
- <para>
- If you don't have an extras folder, download <ulink url="&postgis_download_url;">&postgis_download_url;</ulink>
- </para>
-
- <para>
- <command>tar xvfz postgis-&last_release_version;.tar.gz</command>
- </para>
-
- <para>
- <command>cd postgis-&last_release_version;/extras/tiger_geocoder</command>
- </para>
-
- <para>Edit the <filename>tiger_loader_2015.sql</filename> (or latest loader file you find, unless you want to load different year) to the paths of your executables server etc or alternatively you can update the <varname>loader_platform</varname> table once installed. If you don't edit this file or the <varname>loader_platform</varname> table, it will just contain common case locations of items and you'll have to edit the generated script after the fact when you run the <xref linkend="Loader_Generate_Nation_Script" /> and <xref linkend="Loader_Generate_Script" /> SQL functions.
- </para>
- <para>If you are installing Tiger geocoder for the first time edit either the <filename>create_geocode.bat</filename> script If you are on windows
- or the <filename>create_geocode.sh</filename> if you are on Linux/Unix/Mac OSX with your PostgreSQL specific settings and run the corresponding script from the commandline. </para>
-
-
- <para>Verify that you now have a <varname>tiger</varname> schema in your database and that it is part of your database search_path. If it is not, add it with a command something along the line of: <programlisting>ALTER DATABASE geocoder SET search_path=public, tiger;</programlisting></para>
- <para>The normalizing address functionality works more or less without any data except for tricky addresses. Run this test and verify things look like this:
- <programlisting>SELECT pprint_addy(normalize_address('202 East Fremont Street, Las Vegas, Nevada 89101')) As pretty_address;
-pretty_address
----------------------------------------
-202 E Fremont St, Las Vegas, NV 89101
- </programlisting>
- </para>
- </sect2>
- <sect2 id="tiger_pagc_address_standardizing"><title>Using Address Standardizer Extension with Tiger geocoder</title>
- <para>One of the many complaints of folks is the address normalizer function <xref linkend="Normalize_Address" /> function that normalizes an address for prepping before geocoding. The normalizer is far from perfect and trying to patch its imperfectness takes a vast amount of resources. As such we have integrated with another
- project that has a much better address standardizer engine. To use this new address_standardizer, you compile the extension as described in <xref linkend="installing_pagc_address_standardizer" /> and install as an extension in your database.</para>
-
- <para>Once you install this extension in the same database as you have installed <code>postgis_tiger_geocoder</code>, then the <xref linkend="Pagc_Normalize_Address" /> can be used instead of <xref linkend="Normalize_Address" />. This extension is tiger agnostic, so can be used with other data sources such as international addresses. The tiger geocoder extension does come packaged with its own custom versions of <xref linkend="rulestab" /> ( <code>tiger.pagc_rules</code>) , <xref linkend="gaztab" /> (<code>tiger.pagc_gaz</code>), and <xref linkend="lextab" /> (<code>tiger.pagc_lex</code>). These you can add and update to improve your standardizing experience for your own needs.</para>
- </sect2>
- <sect2 id="tiger_geocoder_loading_data">
- <title>Loading Tiger Data</title>
- <para>The instructions for loading data are available in a more detailed form in the <filename>extras/tiger_geocoder/tiger_2011/README</filename>. This just includes the general steps.</para>
- <para>The load process downloads data from the census website for the respective nation files, states requested, extracts the files, and then loads each state into its own separate
- set of state tables. Each state table inherits from the tables defined in <varname>tiger</varname> schema so that its sufficient to just query those tables to access all the data and drop a set of state tables at any time using the <xref linkend="Drop_State_Tables_Generate_Script" /> if you need to reload a state or just don't need a state anymore.</para>
- <para>In order to be able to load data you'll need the following tools:</para>
- <itemizedlist>
- <listitem><para>A tool to unzip the zip files from census website.</para>
- <para>For Unix like systems: <varname>unzip</varname> executable which is usually already installed on most Unix like platforms.</para>
- <para>For Windows, 7-zip which is a free compress/uncompress tool you can download from <ulink url="http://www.7-zip.org/">http://www.7-zip.org/</ulink> </para>
- </listitem>
- <listitem><para><filename>shp2pgsql</filename> commandline which is installed by default when you install PostGIS.</para></listitem>
- <listitem><para><filename>wget</filename> which is a web grabber tool usually installed on most Unix/Linux systems.</para>
- <para>If you are on windows, you can get pre-compiled binaries from <ulink url="http://gnuwin32.sourceforge.net/packages/wget.htm">http://gnuwin32.sourceforge.net/packages/wget.htm</ulink> </para>
- </listitem>
- </itemizedlist>
- <para>If you are upgrading from tiger_2010, you'll need to first generate and run <xref linkend="Drop_Nation_Tables_Generate_Script" />. Before you load any state data, you need to load the nation wide data which you do with <xref linkend="Loader_Generate_Nation_Script" />. Which will
- generate a loader script for you. <xref linkend="Loader_Generate_Nation_Script" /> is a one-time step that should be done for upgrading (from 2010) and for new installs.</para>
- <para>To load state data refer to <xref linkend="Loader_Generate_Script" /> to generate a data load script for your platform for the states you desire.
- Note that you can install these piecemeal. You don't have to load all the states you want all at once. You can load them as you need them.</para>
-
- <para>After the states you desire have been loaded, make sure to run the:
- <programlisting>SELECT install_missing_indexes();</programlisting> as described in <xref linkend="Install_Missing_Indexes" />.</para>
- <para>To test that things are working as they should, try to run a geocode on an address in your state using <xref linkend="Geocode" /> </para>
- </sect2>
- <sect2 id="upgrade_tiger_geocoder">
- <title>Upgrading your Tiger Geocoder Install</title>
- <para>
- If you have Tiger Geocoder packaged with 2.0+ already installed, you can upgrade the functions at any time even from an interim tar ball if there are fixes you badly need. This will only work for Tiger geocoder not installed with extensions.
- </para>
-
- <para>
- If you don't have an extras folder, download <ulink url="&postgis_download_url;">&postgis_download_url;</ulink>
- </para>
-
- <para>
- <command>tar xvfz postgis-&last_release_version;.tar.gz</command>
- </para>
-
- <para>
- <command>cd postgis-&last_release_version;/extras/tiger_geocoder/tiger_2011</command>
- </para>
-
- <para>Locate the <filename>upgrade_geocoder.bat</filename> script If you are on windows
- or the <filename>upgrade_geocoder.sh</filename> if you are on Linux/Unix/Mac OSX. Edit the file to have your postgis database credentials.</para>
-
- <para>If you are upgrading from 2010 or 2011, make sure to unremark out the loader script line so you get the latest script for loading 2012 data.</para>
- <para>
- Then run th corresponding script from the commandline.
- </para>
-
- <para>Next drop all nation tables and load up the new ones. Generate a drop script with this SQL statement as detailed in <xref linkend="Drop_Nation_Tables_Generate_Script" /></para>
- <programlisting>SELECT drop_nation_tables_generate_script();</programlisting>
- <para>Run the generated drop SQL statements.</para>
- <para>Generate a nation load script with this SELECT statement as detailed in <xref linkend="Loader_Generate_Nation_Script" /></para>
- <para><emphasis role="bold">For windows</emphasis></para>
- <programlisting>SELECT loader_generate_nation_script('windows'); </programlisting>
- <para><emphasis role="bold">For unix/linux</emphasis></para>
- <programlisting>SELECT loader_generate_nation_script('sh');</programlisting>
- <para>Refer to <xref linkend="tiger_geocoder_loading_data" /> for instructions on how to run the generate script. This only needs to be done once.</para>
- <note><para>You can have a mix of 2010/2011 state tables and can upgrade each state separately. Before you upgrade a state to 2011, you first need to drop the 2010 tables for that state using <xref linkend="Drop_State_Tables_Generate_Script" />.</para></note>
- </sect2>
-
- </sect1>
+ </sect2>
- <sect1 id="templatepostgis">
+ <sect2 id="templatepostgis">
<title>Create a spatially-enabled database from a template</title>
<para>
@@ -2220,10 +1970,12 @@ pretty_address
</para>
<programlisting>postgres=# CREATE DATABASE my_spatial_db TEMPLATE=template_postgis</programlisting>
+ </sect2>
+
</sect1>
<sect1 id="upgrading">
- <title>Upgrading</title>
+ <title>Upgrading spatial databases</title>
<para>
Upgrading existing spatial databases can be tricky as it requires
@@ -2528,6 +2280,261 @@ ALTER EXTENSION postgis_topology UPDATE TO "&last_release_version;next";</progra
</sect2>
</sect1>
+
+ <sect1 id="installing_pagc_address_standardizer"><title>Installing and Using the address standardizer</title>
+ <para>The <code>address_standardizer</code> extension used to be a separate package that required separate download. From PostGIS 2.2 on, it is now bundled in.
+ For more information about the address_standardize, what it does, and how to configure it for your needs, refer to <xref linkend="Address_Standardizer" />.</para>
+ <para>This standardizer can be used in conjunction with the PostGIS packaged tiger geocoder extension as a replacement for the <xref linkend="Normalize_Address" /> discussed.
+ To use as replacement refer to <xref linkend="tiger_pagc_address_standardizing" />.
+ You can also use it as a building block for your own geocoder or use it to standardize your addresses for easier compare of addresses.</para>
+
+ <para>The address standardizer relies on PCRE which is usually already installed on many Nix systems,
+but you can download the latest at: <ulink url="http://www.pcre.org">http://www.pcre.org</ulink>. If during <xref linkend="installation_configuration" />, PCRE is found, then the address standardizer extension will automatically be built. If you have a custom pcre install you want to use instead, pass to configure <code>--with-pcredir=/path/to/pcre</code> where <filename>/path/to/pcre</filename> is the root folder for your pcre include and lib directories.</para>
+
+ <para>For Windows users, the PostGIS 2.1+ bundle is packaged with the address_standardizer already so no need to compile and can move straight to <code>CREATE EXTENSION</code> step.</para>
+
+
+ <para>Once you have installed, you can connect to your database and run the SQL:</para>
+ <programlisting>CREATE EXTENSION address_standardizer;</programlisting>
+
+ <para>The following test requires no rules, gaz, or lex tables</para>
+ <programlisting>SELECT num, street, city, state, zip
+ FROM parse_address('1 Devonshire Place PH301, Boston, MA 02109');</programlisting>
+ <para>Output should be</para>
+ <screen> num | street | city | state | zip
+-----+------------------------+--------+-------+-------
+ 1 | Devonshire Place PH301 | Boston | MA | 02109</screen>
+
+ <sect2><title>Installing Regex::Assemble</title>
+ <para>Perl Regex:Assemble is no longer needed for compiling address_standardizer extension since the files it generates are part of the source tree. However if you need to edit the <filename>usps-st-city-orig.txt</filename> or <filename>usps-st-city-orig.txt usps-st-city-adds.tx</filename>, you need to rebuild <filename>parseaddress-stcities.h</filename> which does require Regex:Assemble.</para>
+ <programlisting>cpan Regexp::Assemble</programlisting>
+ <para>or if you are on Ubuntu / Debian you might need to do</para>
+ <programlisting>sudo perl -MCPAN -e "install Regexp::Assemble"</programlisting>
+ </sect2>
+ </sect1>
+
+ <sect1 id="loading_extras_tiger_geocoder">
+ <title>Installing, Upgrading Tiger Geocoder and loading data</title>
+
+ <para>Extras like Tiger geocoder may not be packaged in your PostGIS distribution. If you are missing the tiger geocoder extension or want a newer version than what your install comes with, then use
+ the <filename>share/extension/postgis_tiger_geocoder.*</filename> files from the packages in <ulink url="http://postgis.net/windows_downloads/">Windows Unreleased Versions</ulink> section for your version of PostgreSQL.
+ Although these packages are for windows, the postgis_tiger_geocoder extension files will work on any OS since the extension is an SQL/plpgsql only extension.</para>
+ <sect2 id="install_tiger_geocoder_extension">
+ <title>Tiger Geocoder Enabling your PostGIS database: Using Extension</title>
+ <para>If you are using PostgreSQL 9.1+ and PostGIS 2.1+, you can take advantage of the new extension model for installing tiger geocoder. To do so:</para>
+ <orderedlist>
+ <listitem><para>First get binaries for PostGIS 2.1+ or compile and install as usual. This should install the necessary extension files as well for tiger geocoder.</para></listitem>
+ <listitem><para>Connect to your database via psql or pgAdmin or some other tool and run the following SQL commands. Note that if you are installing in a database that already has postgis, you don't need to do the first step. If you have <varname>fuzzystrmatch</varname> extension already installed, you don't need to do the second step either.</para>
+ <para><programlisting>CREATE EXTENSION postgis;
+CREATE EXTENSION fuzzystrmatch;
+CREATE EXTENSION postgis_tiger_geocoder;
+--this one is optional if you want to use the rules based standardizer (pagc_normalize_address)
+CREATE EXTENSION address_standardizer;</programlisting></para>
+
+<para>If you already have postgis_tiger_geocoder extension installed, and just want to update to the latest run:</para>
+<programlisting>ALTER EXTENSION postgis UPDATE;
+ALTER EXTENSION postgis_tiger_geocoder UPDATE;</programlisting>
+<para>If you made custom entries or changes to <varname>tiger.loader_platform</varname> and <varname>tiger.loader_variables</varname> you may need to update these.</para>
+</listitem>
+ <listitem><para>To confirm your install is working correctly, run this sql in your database:</para>
+ <programlisting>SELECT na.address, na.streetname,na.streettypeabbrev, na.zip
+ FROM normalize_address('1 Devonshire Place, Boston, MA 02109') AS na;</programlisting>
+ <para>Which should output</para>
+ <para><screen> address | streetname | streettypeabbrev | zip
+---------+------------+------------------+-------
+ 1 | Devonshire | Pl | 02109</screen></para>
+ </listitem>
+ <listitem><para>Create a new record in <varname>tiger.loader_platform</varname> table with the paths of your executables and server. </para>
+ <para>So for example to create a profile called debbie that follows <code>sh</code> convention. You would do:</para>
+ <programlisting>INSERT INTO tiger.loader_platform(os, declare_sect, pgbin, wget, unzip_command, psql, path_sep,
+ loader, environ_set_command, county_process_command)
+SELECT 'debbie', declare_sect, pgbin, wget, unzip_command, psql, path_sep,
+ loader, environ_set_command, county_process_command
+ FROM tiger.loader_platform
+ WHERE os = 'sh';</programlisting>
+ <para>And then edit the paths in the <emphasis>declare_sect</emphasis> column to those that fit Debbie's pg, unzip,shp2pgsql, psql, etc path locations.</para>
+
+ <para>If you don't edit this <varname>loader_platform</varname> table, it will just contain common case locations of items and you'll have to edit the generated script after the script is generated.</para>
+ </listitem>
+ <listitem><para>As of PostGIS 2.4.1 the Zip code-5 digit tabulation area <varname>zcta5</varname> load step was revised to load current zcta5 data and is part of the <xref linkend="Loader_Generate_Nation_Script" /> when enabled.
+It is turned off by default because it takes quite a bit of time to load (20 to 60 minutes), takes up quite a bit of disk space, and is not used that often.</para>
+<para>To enable it, do the following:</para>
+<programlisting>UPDATE tiger.loader_lookuptables SET load = true WHERE table_name = 'zcta510';</programlisting>
+<para>
+If present the <xref linkend="Geocode" /> function can use it if a boundary filter is added to limit to just zips in that boundary.
+The <xref linkend="Reverse_Geocode" /> function uses it if the returned address is missing a zip, which often happens with highway reverse geocoding.</para></listitem>
+ <listitem><para>Create a folder called <filename>gisdata</filename> on root of server or your local pc if you have a fast network connection to the server. This folder is
+where the tiger files will be downloaded to and processed. If you are not happy with having the folder on the root of the server, or simply want to change to a different folder for staging, then edit the field <varname>staging_fold</varname> in the <varname>tiger.loader_variables</varname> table.</para></listitem>
+ <listitem><para>Create a folder called temp in the <filename>gisdata</filename> folder or whereever you designated the <varname>staging_fold</varname> to be. This will be
+the folder where the loader extracts the downloaded tiger data.</para></listitem>
+ <listitem><para>Then run the <xref linkend="Loader_Generate_Nation_Script" /> SQL function make sure to use the name of your custom profile and copy the script to a .sh or .bat file. So for example to build the nation load:</para>
+ <programlisting>psql -c "SELECT Loader_Generate_Nation_Script('debbie')" -d geocoder -tA > /gisdata/nation_script_load.sh</programlisting>
+ </listitem>
+
+ <listitem><para>Run the generated nation load commandline scripts.</para>
+ <programlisting>cd /gisdata
+sh nation_script_load.sh</programlisting>
+ </listitem>
+
+ <listitem><para>After you are done running the nation script, you should have three tables in your <code>tiger_data</code> schema and they should be filled with data. Confirm you do by doing the following queries from psql or pgAdmin</para>
+ <programlisting>SELECT count(*) FROM tiger_data.county_all;</programlisting>
+<screen> count
+-------
+ 3233
+(1 row)</screen>
+ <programlisting>SELECT count(*) FROM tiger_data.state_all;</programlisting>
+<screen>
+ count
+-------
+ 56
+(1 row)
+</screen>
+ </listitem>
+
+ <listitem><para>By default the tables corresponding to <varname>bg</varname>, <varname>tract</varname>, <varname>tabblock</varname> are not loaded. These tables are not used by the geocoder but are used by folks for population statistics.
+ If you wish to load them as part of your state loads, run the following statement to enable them.</para>
+ <programlisting>UPDATE tiger.loader_lookuptables SET load = true WHERE load = false AND lookup_name IN('tract', 'bg', 'tabblock');</programlisting>
+ <para>Alternatively you can load just these tables after loading state data using the <xref linkend="Loader_Generate_Census_Script" /></para></listitem>
+
+ <listitem><para>For each state you want to load data for, generate a state script <xref linkend="Loader_Generate_Script" />.</para><warning><para>DO NOT Generate the state script until you have already loaded the nation data, because the state script utilizes county list loaded by nation script.</para></warning></listitem>
+
+ <listitem><programlisting>psql -c "SELECT Loader_Generate_Script(ARRAY['MA'], 'debbie')" -d geocoder -tA > /gisdata/ma_load.sh</programlisting></listitem>
+
+ <listitem><para>Run the generated commandline scripts.</para>
+ <programlisting>cd /gisdata
+sh ma_load.sh</programlisting>
+ </listitem>
+ <listitem><para>After you are done loading all data or at a stopping point, it's a good idea to analyze all the tiger tables to update the stats (include inherited stats)</para>
+ <programlisting>SELECT install_missing_indexes();
+vacuum (analyze, verbose) tiger.addr;
+vacuum (analyze, verbose) tiger.edges;
+vacuum (analyze, verbose) tiger.faces;
+vacuum (analyze, verbose) tiger.featnames;
+vacuum (analyze, verbose) tiger.place;
+vacuum (analyze, verbose) tiger.cousub;
+vacuum (analyze, verbose) tiger.county;
+vacuum (analyze, verbose) tiger.state;
+vacuum (analyze, verbose) tiger.zip_lookup_base;
+vacuum (analyze, verbose) tiger.zip_state;
+vacuum (analyze, verbose) tiger.zip_state_loc;</programlisting>
+ </listitem>
+ </orderedlist>
+ <sect3 id="convert_tiger_geocoder_extension"><title>Converting a Tiger Geocoder Regular Install to Extension Model</title>
+ <para>If you installed the tiger geocoder without using the extension model, you can convert to the extension model as follows:</para>
+ <orderedlist>
+ <listitem><para>Follow instructions in <xref linkend="upgrade_tiger_geocoder" /> for the non-extension model upgrade.</para></listitem>
+ <listitem><para>Connect to your database with psql or pgAdmin and run the following command:</para>
+ <programlisting>CREATE EXTENSION postgis_tiger_geocoder FROM unpackaged;</programlisting>
+ </listitem>
+ </orderedlist>
+ </sect3>
+
+ </sect2>
+ <sect2 id="install_tiger_geocoder">
+ <title>Tiger Geocoder Enabling your PostGIS database: Not Using Extensions</title>
+ <para>
+ First install PostGIS using the prior instructions.
+ </para>
+
+ <para>
+ If you don't have an extras folder, download <ulink url="&postgis_download_url;">&postgis_download_url;</ulink>
+ </para>
+
+ <para>
+ <command>tar xvfz postgis-&last_release_version;.tar.gz</command>
+ </para>
+
+ <para>
+ <command>cd postgis-&last_release_version;/extras/tiger_geocoder</command>
+ </para>
+
+ <para>Edit the <filename>tiger_loader_2015.sql</filename> (or latest loader file you find, unless you want to load different year) to the paths of your executables server etc or alternatively you can update the <varname>loader_platform</varname> table once installed. If you don't edit this file or the <varname>loader_platform</varname> table, it will just contain common case locations of items and you'll have to edit the generated script after the fact when you run the <xref linkend="Loader_Generate_Nation_Script" /> and <xref linkend="Loader_Generate_Script" /> SQL functions.
+ </para>
+ <para>If you are installing Tiger geocoder for the first time edit either the <filename>create_geocode.bat</filename> script If you are on windows
+ or the <filename>create_geocode.sh</filename> if you are on Linux/Unix/Mac OSX with your PostgreSQL specific settings and run the corresponding script from the commandline. </para>
+
+
+ <para>Verify that you now have a <varname>tiger</varname> schema in your database and that it is part of your database search_path. If it is not, add it with a command something along the line of: <programlisting>ALTER DATABASE geocoder SET search_path=public, tiger;</programlisting></para>
+ <para>The normalizing address functionality works more or less without any data except for tricky addresses. Run this test and verify things look like this:
+ <programlisting>SELECT pprint_addy(normalize_address('202 East Fremont Street, Las Vegas, Nevada 89101')) As pretty_address;
+pretty_address
+---------------------------------------
+202 E Fremont St, Las Vegas, NV 89101
+ </programlisting>
+ </para>
+ </sect2>
+ <sect2 id="tiger_pagc_address_standardizing"><title>Using Address Standardizer Extension with Tiger geocoder</title>
+ <para>One of the many complaints of folks is the address normalizer function <xref linkend="Normalize_Address" /> function that normalizes an address for prepping before geocoding. The normalizer is far from perfect and trying to patch its imperfectness takes a vast amount of resources. As such we have integrated with another
+ project that has a much better address standardizer engine. To use this new address_standardizer, you compile the extension as described in <xref linkend="installing_pagc_address_standardizer" /> and install as an extension in your database.</para>
+
+ <para>Once you install this extension in the same database as you have installed <code>postgis_tiger_geocoder</code>, then the <xref linkend="Pagc_Normalize_Address" /> can be used instead of <xref linkend="Normalize_Address" />. This extension is tiger agnostic, so can be used with other data sources such as international addresses. The tiger geocoder extension does come packaged with its own custom versions of <xref linkend="rulestab" /> ( <code>tiger.pagc_rules</code>) , <xref linkend="gaztab" /> (<code>tiger.pagc_gaz</code>), and <xref linkend="lextab" /> (<code>tiger.pagc_lex</code>). These you can add and update to improve your standardizing experience for your own needs.</para>
+ </sect2>
+ <sect2 id="tiger_geocoder_loading_data">
+ <title>Loading Tiger Data</title>
+ <para>The instructions for loading data are available in a more detailed form in the <filename>extras/tiger_geocoder/tiger_2011/README</filename>. This just includes the general steps.</para>
+ <para>The load process downloads data from the census website for the respective nation files, states requested, extracts the files, and then loads each state into its own separate
+ set of state tables. Each state table inherits from the tables defined in <varname>tiger</varname> schema so that its sufficient to just query those tables to access all the data and drop a set of state tables at any time using the <xref linkend="Drop_State_Tables_Generate_Script" /> if you need to reload a state or just don't need a state anymore.</para>
+ <para>In order to be able to load data you'll need the following tools:</para>
+ <itemizedlist>
+ <listitem><para>A tool to unzip the zip files from census website.</para>
+ <para>For Unix like systems: <varname>unzip</varname> executable which is usually already installed on most Unix like platforms.</para>
+ <para>For Windows, 7-zip which is a free compress/uncompress tool you can download from <ulink url="http://www.7-zip.org/">http://www.7-zip.org/</ulink> </para>
+ </listitem>
+ <listitem><para><filename>shp2pgsql</filename> commandline which is installed by default when you install PostGIS.</para></listitem>
+ <listitem><para><filename>wget</filename> which is a web grabber tool usually installed on most Unix/Linux systems.</para>
+ <para>If you are on windows, you can get pre-compiled binaries from <ulink url="http://gnuwin32.sourceforge.net/packages/wget.htm">http://gnuwin32.sourceforge.net/packages/wget.htm</ulink> </para>
+ </listitem>
+ </itemizedlist>
+ <para>If you are upgrading from tiger_2010, you'll need to first generate and run <xref linkend="Drop_Nation_Tables_Generate_Script" />. Before you load any state data, you need to load the nation wide data which you do with <xref linkend="Loader_Generate_Nation_Script" />. Which will
+ generate a loader script for you. <xref linkend="Loader_Generate_Nation_Script" /> is a one-time step that should be done for upgrading (from 2010) and for new installs.</para>
+ <para>To load state data refer to <xref linkend="Loader_Generate_Script" /> to generate a data load script for your platform for the states you desire.
+ Note that you can install these piecemeal. You don't have to load all the states you want all at once. You can load them as you need them.</para>
+
+ <para>After the states you desire have been loaded, make sure to run the:
+ <programlisting>SELECT install_missing_indexes();</programlisting> as described in <xref linkend="Install_Missing_Indexes" />.</para>
+ <para>To test that things are working as they should, try to run a geocode on an address in your state using <xref linkend="Geocode" /> </para>
+ </sect2>
+ <sect2 id="upgrade_tiger_geocoder">
+ <title>Upgrading your Tiger Geocoder Install</title>
+ <para>
+ If you have Tiger Geocoder packaged with 2.0+ already installed, you can upgrade the functions at any time even from an interim tar ball if there are fixes you badly need. This will only work for Tiger geocoder not installed with extensions.
+ </para>
+
+ <para>
+ If you don't have an extras folder, download <ulink url="&postgis_download_url;">&postgis_download_url;</ulink>
+ </para>
+
+ <para>
+ <command>tar xvfz postgis-&last_release_version;.tar.gz</command>
+ </para>
+
+ <para>
+ <command>cd postgis-&last_release_version;/extras/tiger_geocoder/tiger_2011</command>
+ </para>
+
+ <para>Locate the <filename>upgrade_geocoder.bat</filename> script If you are on windows
+ or the <filename>upgrade_geocoder.sh</filename> if you are on Linux/Unix/Mac OSX. Edit the file to have your postgis database credentials.</para>
+
+ <para>If you are upgrading from 2010 or 2011, make sure to unremark out the loader script line so you get the latest script for loading 2012 data.</para>
+ <para>
+ Then run th corresponding script from the commandline.
+ </para>
+
+ <para>Next drop all nation tables and load up the new ones. Generate a drop script with this SQL statement as detailed in <xref linkend="Drop_Nation_Tables_Generate_Script" /></para>
+ <programlisting>SELECT drop_nation_tables_generate_script();</programlisting>
+ <para>Run the generated drop SQL statements.</para>
+ <para>Generate a nation load script with this SELECT statement as detailed in <xref linkend="Loader_Generate_Nation_Script" /></para>
+ <para><emphasis role="bold">For windows</emphasis></para>
+ <programlisting>SELECT loader_generate_nation_script('windows'); </programlisting>
+ <para><emphasis role="bold">For unix/linux</emphasis></para>
+ <programlisting>SELECT loader_generate_nation_script('sh');</programlisting>
+ <para>Refer to <xref linkend="tiger_geocoder_loading_data" /> for instructions on how to run the generate script. This only needs to be done once.</para>
+ <note><para>You can have a mix of 2010/2011 state tables and can upgrade each state separately. Before you upgrade a state to 2011, you first need to drop the 2010 tables for that state using <xref linkend="Drop_State_Tables_Generate_Script" />.</para></note>
+ </sect2>
+
+ </sect1>
+
+
<sect1>
<title>Common Problems during installation</title>
<para>
diff --git a/doc/reference_raster.xml b/doc/reference_raster.xml
index b66a4f2..7cf5587 100644
--- a/doc/reference_raster.xml
+++ b/doc/reference_raster.xml
@@ -204,7 +204,7 @@ VALUES (1,
Each raster has 1 or more bands each having a set of pixel values. Rasters can be georeferenced.</para>
<note>
<para>
- Requires PostGIS Raster support to be enabled in the database. See <xref linkend="postgis_installation" /> for more details.
+ Requires PostGIS Raster support to be enabled in the database. See <xref linkend="create_spatial_db" /> for more details.
</para>
</note>
<note>
commit de6f53284f12eee226b226b954ec5ccdfce94957
Author: Sandro Santilli <strk at kbt.io>
Date: Wed Jan 15 10:41:22 2020 +0100
Add link from raster section to install section
diff --git a/doc/reference_raster.xml b/doc/reference_raster.xml
index 5d42fd1..b66a4f2 100644
--- a/doc/reference_raster.xml
+++ b/doc/reference_raster.xml
@@ -203,6 +203,11 @@ VALUES (1,
<para>raster is a spatial data type used to represent raster data such as those imported from JPEGs, TIFFs, PNGs, digital elevation models.
Each raster has 1 or more bands each having a set of pixel values. Rasters can be georeferenced.</para>
<note>
+ <para>
+ Requires PostGIS Raster support to be enabled in the database. See <xref linkend="postgis_installation" /> for more details.
+ </para>
+ </note>
+ <note>
<para>Requires PostGIS be compiled with GDAL support. Currently rasters can be implicitly converted to geometry type, but the conversion returns the
<xref linkend="RT_ST_ConvexHull" /> of the raster. This auto casting may be removed in the near future so don't rely on it.</para>
</note>
-----------------------------------------------------------------------
Summary of changes:
doc/installation.xml | 525 ++++++++++++++++++++++++-----------------------
doc/reference_raster.xml | 5 +
2 files changed, 271 insertions(+), 259 deletions(-)
hooks/post-receive
--
PostGIS
More information about the postgis-tickets
mailing list