Application Data

[Stephen Woodbridge]

I also might add to Brent's good suggestions that you prototype your
application with a small amount of data before you try loading the whole
thing. Load 1, 10 or 100 MB, see what issues you run into, get it
working with mapserver, and work out the issues on a small set, then
load progressively more data, and work out indexing and performance
issues, etc. This allows you to easily change the schema, or indexing,
etc early on if there are performance problems.

If you load everything and performance is bad, it is hard to know where
the problem is. Is, the indexes, is it the query, is it mapserver, is it
your applications codes, is it your schema, etc.

-Steve Woodbridge

Brent Wood wrote:
>>I would like to know, if it is possible to convert data produced from
>>a third party application to a shapefile or PostGIS table.
>
>
> In a nutshell, yes it is possible. Basic shell scripts, perl or awk can all be
> used to reformat such data to a format which can be loaded into PostGIS.
>
>
>>Hypothetically speaking, the data from the third party application
>>will be in the following format.
>>
>>SSN, Lat, Long, Age etc
>
>
> eg, along the lines of:
>
> cat <your_file> | awk "(<write out the reformatted data>)" | psql <database> -c
> "copy <table> from STDIN [with delimiter];"
>
>
> There are many ways of getting the data into PostGIS. This is a shell based
> Linux/UNIX approaches, but it can be done under Windows if you use that, with
> something like Cygwin.
>
>
>>Imagine that the above data is huge (as large as 100GB), so I need to
>>dump this information to a database.
>
>
> Using that many records in a single table may not be optimal for performance,
> so breaking it up into subsets based on some sort of sensible categories is
> probably worth looking into. Even for the loading process it may be worth
> processing subsets, one at a time. But performance issues aside, if you have
> the disk space it should work.
>
>
>>I would like to know, how to use this data from a table to Map it onto
>>a MAPServer.
>
>
> I suggest you focus on one issue at a time, starting with how to load the data.
> Then how to query it with SQL, & then via other applications such as Mapserver.
> Taking things in a logical sequence, instead of everything at once, may make it
> easier to understand.
>
> Brent Wood
>