[postgis-users] tiger data pain

[Stephen Woodbridge]

Hi Andy,

Sweat! very nice job. I took a brief look at and have clone it for a 
more detailed look. Looks like it does a lot of stuff and its nice that 
you used threads.

Thanks,
   -Steve

On 2/7/2013 5:18 PM, Andy Colson wrote:
> On 12/13/2011 10:35 PM, Stephen Woodbridge wrote:
>> On 12/13/2011 10:16 PM, Andy Colson wrote:
>>> I downloaded PostGis 2 to play around with, and tried out the tiger
>>> stuff. I have to say, that's kinda painful.
>>>
>>> The doc says: add "set -e -u", which causes as many problems as it might
>>> catch.
>>> Editing the .sql was a little confusing, and the resulting .sh file is
>>> not really re-startable.
>>>
>>> Would anyone be interested in a perl script to simplify things? I'm
>>> thinking of replacing the stored proc that generates a script, all of
>>> which you have to edit, with a perl script (which would have a config to
>>> edit).
>>>
>>> Or... maybe other's have not had the problems I had? I don't really have
>>> a use for this right now, just playing around. If anyone had some
>>> requests, I might be interested in working on it.
>>
>> Andy,
>>
>> I think this would be a good idea. One trick I use a lot with Perl is
>> find2perl which I use to walk a directory structure like the Tiger
>> source tree to find the files I want to load.
>>
>> Also Census just released the Tiger2011 files, unfortunately, they have
>> only released them in the type directory structure and not in the state
>> directory structure. Each year there are file name changes and column
>> name changes within the files. I think using Perl would make it easier
>> to remap these column names.
>>
>> -Steve
>> _______________________________________________
>> postgis-users mailing list
>> postgis-users at postgis.refractions.net
>> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
> Its only been a year... not to bad for a turn around time :-)
>
> I started on a perl program to import data, but then got side tracked
> and forgot about it.
>
> Few weeks ago I needed to pull more data, so I dusted off my script,
> cleaned it up, ran it, and published it:
>
> https://github.com/AndyColson/tigerLoader
>
> It only does 2010 data, and I might have gotten carried away with the
> threads.  I'm not sure how easy it'll be for others to use.
>
> But there it is.
>
> -Andy
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users