[GRASS-dev] [EXTERNAL] vector patching frustration
Michael Barton
Michael.Barton at asu.edu
Fri Mar 15 17:37:05 PDT 2024
I got this to work *finally*. But there are still complications. Here is an example work flow:
1. Create a map of vector features
2. Create a basic attribute table (e.g., with v.db.addtable).
3. Export that table into a csv file using db.out.ogr
4. Add some attribute fields to that csv file
5. Try to import that csv file using db.in.ogr
6. Error: it won't import the cat field
7. Edit the csv file to rename cat to mycat
8. Try to import the csv file again using db.in.ogr
9. If you have not changed the sort order of the csv file, the new cat numbers that db.in.ogr automatically assigns (no option about this) will match the original cat numbers.
10. If cat does not match mycat, no problem because when you connect the attribute table to the vector with v.db.connect, you can specify mycat as the key field.
11. Try to patch this vector with another vector that has the same structure of attribute table. *Fail* Because patching creates new cat numbers in some order that is not necessarily related to the old linkage. The attribute table of the 2nd map is properly added to the first attribute table, but the second map is not linked properly to the attribute records.
In my case, patching updated my non-cat keyfield to the max cat value of the original table (154) + the max cat value of the patched table (155) = 309. Very odd. This did not match any record. Fortunately, because I only have only one vector feature in the 2nd patched map, I could edit the cat field in the table (because it was not the key field) to match the new cat of the 2nd vector feature, and rejoin the table to the vector using cat. If I would have had multiple features in the 2nd file, it would have been difficult to impossible.
I'm not sure how to fix this. It may need to be a combination of allowing various vector modification modules to specify a cat value for a feature, assuming no duplication. It may need some change to db.in.ogr to allow for importing a field named cat. Maybe more changes to patch (and other features that have legacy underlying assumptions from the old pre-version6 vector format).
Michael
_____________________________
C. Michael Barton
Associate Director, School of Complex Adaptive Systems (https://scas.asu.edu<https://scas.asu.edu/>)
Professor, School of Human Evolution & Social Change (https://shesc.asu.edu<https://shesc.asu.edu/>)
Director, Center for Social Dynamics & Complexity (https://complexity.asu.edu<https://complexity.asu.edu/>)
Arizona State University
Tempe, AZ 85287-2701
USA
Executive Director, Open Modeling Foundation (https://openmodelingfoundation.github.io<https://openmodelingfoundation.github.io/>)
Director, Network for Computational Modeling in Social & Ecological Sciences (https://comses.net<https://comses.net/>)
personal website: http://www.public.asu.edu/~cmbarton
On Mar 15, 2024, at 5:05 PM, Michael Barton <michael.barton at asu.edu> wrote:
Never mind. I figured it out when I saw the post. Now for the next step.
Michael
_____________________________
C. Michael Barton
Associate Director, School of Complex Adaptive Systems (https://scas.asu.edu<https://scas.asu.edu/>)
Professor, School of Human Evolution & Social Change (https://shesc.asu.edu<https://shesc.asu.edu/>)
Director, Center for Social Dynamics & Complexity (https://complexity.asu.edu<https://complexity.asu.edu/>)
Arizona State University
Tempe, AZ 85287-2701
USA
Executive Director, Open Modeling Foundation (https://openmodelingfoundation.github.io<https://openmodelingfoundation.github.io/>)
Director, Network for Computational Modeling in Social & Ecological Sciences (https://comses.net<https://comses.net/>)
personal website: http://www.public.asu.edu/~cmbarton
On Mar 15, 2024, at 3:29 PM, Michael Barton <Michael.Barton at asu.edu> wrote:
Something is just not working here.
I can't try the new PR until I can get the original cats fixed. v.category does not seem to be working as expected. I have a map with a single area. I clipped it out of another map (where it had cat=18) using v.select. When I check the categories, it looks like this:
v.category input=SA5 at SAA2024 type=centroid option=report
Layer/table: 1/SA5
type count min max
point 0 0 0
line 0 0 0
boundary 0 0 0
centroid 1 28 28
area 0 0 0
face 0 0 0
kernel 0 0 0
all 1 28 28
So I want to follow your example to first delete categories and then add them back in starting at 1.
v.category --overwrite input=SA5 at SAA2024 output=NA5a option=del cat=-1
But delete does not seem to work.
GRASS : ~ > v.category input=SA5a at SAA2024 option=report
Layer/table: 1/SA5a
type count min max
point 0 0 0
line 0 0 0
boundary 0 0 0
centroid 1 28 28
area 0 0 0
face 0 0 0
kernel 0 0 0
all 1 28 28
It stays at cat=28. Oddly, when I display the map and query it, it has no cat number visible in the query output in the GUI. But v.category shows that it still has a category somewhere. And if I then use v.category to try to add a new cat, starting at 1, the result is a map with cat=28 again. And when I patch, using your PR, I get the same results I had before.
Michael
_____________________________
C. Michael Barton
Associate Director, School of Complex Adaptive Systems (https://scas.asu.edu<https://scas.asu.edu/>)
Professor, School of Human Evolution & Social Change (https://shesc.asu.edu<https://shesc.asu.edu/>)
Director, Center for Social Dynamics & Complexity (https://complexity.asu.edu<https://complexity.asu.edu/>)
Arizona State University
Tempe, AZ 85287-2701
USA
Executive Director, Open Modeling Foundation (https://openmodelingfoundation.github.io<https://openmodelingfoundation.github.io/>)
Director, Network for Computational Modeling in Social & Ecological Sciences (https://comses.net<https://comses.net/>)
personal website: http://www.public.asu.edu/~cmbarton
On Mar 15, 2024, at 8:51 AM, Huidae Cho <grass4u at gmail.com> wrote:
Michael,
Just confirmed your issue.
v.random tmp100 npoints=100 seed=100
v.db.addtable tmp100 col="id varchar(20)"
v.db.update tmp100 col=id qcol="'tmp100_' || cat"
v.random tmp10 npoints=10 seed=10
v.db.addtable tmp10 col="id varchar(20)"
v.db.update tmp10 col=id qcol="'tmp10_' || cat"
v.patch tmp100,tmp10 out=tmp110 -e
v.category tmp110 op=report
Layer/table: 1/tmp110
type count min max
point 110 2 112
line 0 0 0
boundary 0 0 0
centroid 0 0 0
area 0 0 0
face 0 0 0
kernel 0 0 0
all 110 2 112
In my case, all features in tmp10 are linked in tmp110.
You can recategorize it.
v.category tmp110 out=tmp110_nocats op=del cat=-1
v.category tmp110_nocats out=tmp110_recats op=add
npnts100=$(v.info<https://urldefense.com/v3/__http://v.info__;!!IKRxdwAv5BmarQ!eDFcy3DMNQS8CMpiCzhp6trLuDrQNXlfIg-UEXoodTPi_yE5QtH-fHSHLEzHn3h0gXJrHPn76a9pFhgeoBImgQ$> tmp100 -t | sed '/points=/!d; s/points=//')
npnts10=$(v.info<https://urldefense.com/v3/__http://v.info__;!!IKRxdwAv5BmarQ!eDFcy3DMNQS8CMpiCzhp6trLuDrQNXlfIg-UEXoodTPi_yE5QtH-fHSHLEzHn3h0gXJrHPn76a9pFhgeoBImgQ$> tmp10 -t | sed '/points=/!d; s/points=//')
v.db.update tmp110_recats col=cat qcol=cat-1 where="cat<=$npnts100+1"
v.db.update tmp110_recats col=cat qcol=cat-2 where="cat>$npnts100+1"
Yeah... I know what you may think... Please create an issue on GitHub.
Regards,
Huidae
On Thu, Mar 14, 2024 at 4:42 PM Michael Barton via grass-dev <grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>> wrote:
Thanks Doug and Vincent.
None of this works. I vaguely remember having this same kind of road block several years back.
Nothing in v.category lets you change an existing cat value to a different value
v.in.ogr MUST add a column that MUST be named cat (all else causes an error--this is a bug) that is a series of consecutive integers from 1-n. So I cannot create a file with a cat field that matches my vector areas.
v.edit won't let you change an existing cat number AFAICT. No error but nothing changes.
Once I've imported a table using v.in.ogr, I cannot use db.drop.column to delete the cat column--even if it is not being used as a key field. This raises an error.
No way to rename a column from cat to something else (or something else to cat) unless you've already connected it to a vector map even if cat is not the key field. So I can't create an integer column to link up my lost vector area with a record in my csv file.
This simple task is just not possible in GRASS AFAICT. Or if it is possible, it can only be done by such non-obvious and convoluted means that I've yet to find a method that works in spite of asking a large number of very skilled GRASS users.
Michael
_____________________________
C. Michael Barton
Associate Director, School of Complex Adaptive Systems (https://scas.asu.edu<https://scas.asu.edu/>)
Professor, School of Human Evolution & Social Change (https://shesc.asu.edu<https://shesc.asu.edu/>)
Director, Center for Social Dynamics & Complexity (https://complexity.asu.edu<https://complexity.asu.edu/>)
Arizona State University
Tempe, AZ 85287-2701
USA
Executive Director, Open Modeling Foundation (https://openmodelingfoundation.github.io<https://urldefense.com/v3/__https://openmodelingfoundation.github.io/__;!!IKRxdwAv5BmarQ!eDFcy3DMNQS8CMpiCzhp6trLuDrQNXlfIg-UEXoodTPi_yE5QtH-fHSHLEzHn3h0gXJrHPn76a9pFhi94Wl0mw$>)
Director, Network for Computational Modeling in Social & Ecological Sciences (https://comses.net<https://urldefense.com/v3/__https://comses.net__;!!IKRxdwAv5BmarQ!eDFcy3DMNQS8CMpiCzhp6trLuDrQNXlfIg-UEXoodTPi_yE5QtH-fHSHLEzHn3h0gXJrHPn76a9pFhgAFoliVQ$>)
personal website: http://www.public.asu.edu/~cmbarton
On Mar 14, 2024, at 7:10 AM, Newcomb, Doug <doug_newcomb at fws.gov<mailto:doug_newcomb at fws.gov>> wrote:
Have you tried a .csvt file for your .csv file? https://gdal.org/drivers/vector/csv.html<https://urldefense.com/v3/__https://gdal.org/drivers/vector/csv.html__;!!IKRxdwAv5BmarQ!dv7XjZX6-VnNpFbpXP6R1XYWkuaA4Y-gDR4RvL3bWazWUkLfURuKDMWqiBFqBS6jlNSHDKZCo02GJKjauaAJ-pCVIKD5JDc$>
________________________________
From: grass-dev <grass-dev-bounces at lists.osgeo.org<mailto:grass-dev-bounces at lists.osgeo.org>> on behalf of Michael Barton via grass-dev <grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>>
Sent: Wednesday, March 13, 2024 7:02 PM
To: GRASS developers <grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>>; GRASS user list <grass-user at lists.osgeo.org<mailto:grass-user at lists.osgeo.org>>
Subject: [EXTERNAL] [GRASS-dev] vector patching frustration
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.
I am completely stymied in my attempt to do what should be a simple task. I have a map of vector areas, linked with an attribute table. I would like to patch in one more vector area, for which I have equivalent attribute information. I have tried this multiple ways and I cannot make this patch happen so that the added area has attribute info.
The closest I've come is to create and link a one line table to the new area that has exactly the same fields as the larger vector area map. The first map has 154 areas (i.e., cat=1-155). When I patch the maps and look at the resulting attribute table, I indeed see line and cat 155. But it is not linked with the patched area--which has been assigned a cat=183 for reasons I cannot fathom. The patch also renumbers my cat field to cat=2-155 from the original 1-154.
This has been made more complicated by the fact that v.in.ogr imports all columns of a *.csv as text, regardless of what is in them and assigns cat numbers starting at 1. So I can't specify an integer key field of 155 to try the linking. Nor can I change the assigned cat of the single area I am trying to patch from 1 to 155 using v.category (or anything else I can find).
I'm hoping that someone has a clever solution that I've not seen or I'll just have to do this fairly simple and straightforward vector operation in QGIS.
Michael
_____________________________
C. Michael Barton
Associate Director, School of Complex Adaptive Systems (https://scas.asu.edu<https://scas.asu.edu/>)
Professor, School of Human Evolution & Social Change (https://shesc.asu.edu<https://shesc.asu.edu/>)
Director, Center for Social Dynamics & Complexity (https://complexity.asu.edu<https://complexity.asu.edu/>)
Arizona State University
Tempe, AZ 85287-2701
USA
Executive Director, Open Modeling Foundation (https://openmodelingfoundation.github.io<https://urldefense.com/v3/__https://openmodelingfoundation.github.io/__;!!IKRxdwAv5BmarQ!dv7XjZX6-VnNpFbpXP6R1XYWkuaA4Y-gDR4RvL3bWazWUkLfURuKDMWqiBFqBS6jlNSHDKZCo02GJKjauaAJ-pCVvuAToIA$>)
Director, Network for Computational Modeling in Social & Ecological Sciences (https://comses.net<https://urldefense.com/v3/__https://comses.net/__;!!IKRxdwAv5BmarQ!dv7XjZX6-VnNpFbpXP6R1XYWkuaA4Y-gDR4RvL3bWazWUkLfURuKDMWqiBFqBS6jlNSHDKZCo02GJKjauaAJ-pCVdW6V-z8$>)
personal website: http://www.public.asu.edu/~cmbarton
_______________________________________________
grass-dev mailing list
grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/grass-dev<https://urldefense.com/v3/__https://lists.osgeo.org/mailman/listinfo/grass-dev__;!!IKRxdwAv5BmarQ!eDFcy3DMNQS8CMpiCzhp6trLuDrQNXlfIg-UEXoodTPi_yE5QtH-fHSHLEzHn3h0gXJrHPn76a9pFhgJetvH4Q$>
--
Huidae Cho, Ph.D., GISP, /hidɛ<https://urldefense.com/v3/__http://ipa-reader.xyz/?text=hid**A__;yZs!!IKRxdwAv5BmarQ!eDFcy3DMNQS8CMpiCzhp6trLuDrQNXlfIg-UEXoodTPi_yE5QtH-fHSHLEzHn3h0gXJrHPn76a9pFhg78TNF6Q$> t͡ɕo/, 조희대, 曺喜大
GRASS GIS Developer
https://idea.isnew.info/<https://urldefense.com/v3/__https://idea.isnew.info/__;!!IKRxdwAv5BmarQ!eDFcy3DMNQS8CMpiCzhp6trLuDrQNXlfIg-UEXoodTPi_yE5QtH-fHSHLEzHn3h0gXJrHPn76a9pFhi_b6t8JA$>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20240316/f406aa7e/attachment-0001.htm>
More information about the grass-dev
mailing list