[Qgis-user] Batch query web database

[Alex Mandel]

You could write a python script which hits the webpage with urllib (I
think that's the name) and then takes the return page and extracts out
the data you want saving down as a csv that can be joined to the table
or if a QGIS plugin put directly into the table.

This is known as screen scraping and is a time honored way of getting
around the fact that a website doesn't give you what you need. If you're
worried about them noticing you can put a sleep timer to only make
request x times per second (assuming you want to do it all in batch).

I have done this before with the Google/Yahoo geocoders which just limit
how many request per day.

Enjoy,
Alex

On 07/12/2011 05:42 PM, Lee wrote:
> I'm not sure I follow. I've never created an attribute with a link.
> 
> The information on the website can only be obtained one at a time. I need to
> link ALL of the parcel ID's with ALL of the records, so any time I use the
> information tool, those records will show up.
> 
> On Tue, Jul 12, 2011 at 2:43 PM, Brad Nesom <gisbradokla at live.com> wrote:
> 
>> Perhaps just build an attribute with the link in it? In most cases the
>> atttribute table would be displayed in a seperate window anyway. so if the
>> browser is the seperate window so what?
>>
>>
>>
>> On Tue, Jul 12, 2011 at 12:45 PM, Lee <muellerl at gmail.com> wrote:
>>
>>> Not necessarily a qgis question, but one which is in our realm.
>>>
>>> I have a vector file which contains parcels across a city. In the
>>> attribute file of that layer, is the parcel id number. I have a website
>>> where I can look up the parcel id number and obtain a host of additional
>>> information. I want to append this additional information to my vector
>>> attribute table.
>>>
>>> I imagine there is a way to bulk query all of my parcel ids. I also
>>> imagine most websites would prevent people from doing such, but I rather ask
>>> than assume.
>>>
>>> Any solutions come to mind?
>>>
>>> --
>>> all the best,
>>> Lee
>>