[Qgis-user] Working with irregular magnetometry data points - converting them to an image

Sat Apr 21 03:27:52 PDT 2018

Hi Calvin,
I am working with magnetometry data for archaeology purposes since over 
20 years(http://en.ggh-online.de/geophysik): starting with geoscan FM36, 
then geometrics G858 with four caesium probes and FEREX from Foerster. 
usually we are processing the raw data with gis software: first started 
with ArcView 3, then ArcGis and since a couple of years we switched to 
QGIS3 which is a good choice to do it with. I have wrote a bunch of 
python scripts to
  * bring the raw data exported from the different magnetometers into a 
gis format (mainly shape)
  * then make some correction within the shapefile due to walking 
zick-zack-mode
  * interpolate a regular grid from the measured values (the are more or 
less spaced 0,1 x 0,5 meters). for that I use IDW (inverse distance 
weighted) with the following parameters: cell size 0.25 and search 
radius 0.5 and power 1). The results are good (see attached image with a 
roman fort). Just take the inbuild processing tools like SAGA inverse 
distance interpolation.
   * the last processing step is to make a line correction (heading 
effect) where I use a line media filter. That is a bit tricky, the best 
is to export your grid to numpy and use numpys power to do it.

Which magnetometer do you use to measure the data? If you send me a data 
file I can have a look at it. I have attached an example which is 
processed with qgis.

Best regards
stefan

Mit freundlichen Grüßen
Stefan Giese
Projektleiter/Consultant
-----------------------------------
Aufwind durch Wissen!
Qualifizierte Open-Source-Schulungen
bei der www.foss-academy.com
-----------------------------------
WhereGroup GmbH & Co. KG
Schwimmbadstr. 2
79100 Freiburg
Germany

Fon: +49 (0)761 / 519 102 - 61
Fax: +49 (0)761 / 519 102 - 11

stefan.giese at wheregroup.com
www.wheregroup.com
Amtsgericht Bonn, HRA 6788
-------------------------------
Komplementärin:
WhereGroup Verwaltungs GmbH
vertreten durch:
Olaf Knopp, Peter Stamm
-------------------------------

Am 2018-04-20 20:13, schrieb C Hamilton:
> Thanks Ben,
> 
> I will give that a try. I am finding that the Interpolation algorithm
> is taking forever on a small data set.
> 
> Doesn't the Postgres PDAL extension handle point data like this? Would
> there be a benefit of putting the data in PostGIS?
> 
> Thanks,
> 
> Calvin
> 
> On Fri, Apr 20, 2018 at 1:10 PM, Benjamin Ducke <benducke at fastmail.fm>
> wrote:
> 
>> Hi Calvin,
>> 
>> On 20/04/18 18:46, Richard McDonnell wrote:
>>> Calvin,
>>> 
>>> Under Processing Toolbox, Type interpolation into the search bar,
>> the
>>> tool you want can be found there.
>> 
>> None of the typical GIS interpolation tools will be
>> of much help here, since they have all been optimized
>> to work with relatively sparse data. They will be
>> too inefficient for high-resolution geomagnetic data.
>> 
>> You data is already dense, with some gaps. So you need
>> something that is fast and efficient for small
>> interpolation radii, and will not produce too many
>> artefacts.
>> 
>> If you can use the GRASS7 plug-in from within QGIS,
>> than this is a good processing path:
>> 
>> 1. Rasterize your data points, using a cell size
>> small enough so that you don't have too much information
>> loss (i.e. not too many occurrences of more than one data
>> point per cell). This might take a little trial and error
>> to find good settings. You should end up with a raster
>> where you have gaps of max. 1-3 empty cells between cells
>> with measurement data.
>> 
>> Now you need to fill these relatively small gaps.
>> 
>> 2. Use r.fill.stats from current GRASS 7.4:
>> 
>> https://grass.osgeo.org/grass74/manuals/r.fill.stats.html [1]
>> 
>> ... because that has been designed exactly for the
>> purpose. Set the interpolation radius as small as
>> possible, so that it just manages to fill the gaps.
>> With a small radius, r.fill.stats is very fast and
>> memory efficient, even on very large raster datasets.
>> 
>> Note that r.fill.stats is part of GRASS 7 only since
>> version 7.4. If your QGIS ships with an earlier
>> version, then you might have to update your GRASS 7
>> binaries manually.
>> 
>> Best,
>> 
>> Ben
>> 
>>> Regards,
>>> 
>>> Richard.
>>> 
>>> On 20/04/2018 16:42, C Hamilton wrote:
>>>> I don't know how many of you have worked with archaeological
>>>> magnetormetry data, but I have been asked to help with
>> magnetometry
>>>> data of the ancient Mound Builders in North America. I am looking
>> for
>>>> some suggestion on how to best turn the irregular magnetic data
>> points
>>>> into an image.
>>>> 
>>>> I see an Interpolation plugin that perhaps is the solution, but I
>> also
>>>> see that it is not part of QGIS 3. Is this because it has not
>> been
>>>> ported yet, or is not going to be porter, or is there a better
>> solution?
>>>> 
>>>> I don't know if any of the Lidar tools would work for this or
>> not.
>>>> LASTools is a commercial product and the locked version only
>> works
>>>> with small data sets and adds noise unless you purchase a
>> license. At
>>>> least that is how it was the last time I used it.
>>>> 
>>>> Thanks for any suggestions.
>>>> 
>>>> Calvin
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Qgis-user mailing list
>>>> Qgis-user at lists.osgeo.org
>>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user [2]
>>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> [2]
>>> 
>>> --
>>> *_Richard McDonnell_*
>>> *GIS Specialist PgD GIS AssocSCSI*
>>> *OPW FRM Data Management*
>>> *52 Stephens Green, Dublin 2.*
>>> TEL: 01 6476543
>>> 
>>> <http://www.opw.ie/>
>>> 
>>> 
>>> 
>>> Email Disclaimer: http://www.opw.ie/en/disclaimer/ [3]
>>> 
>>> 
>>> _______________________________________________
>>> Qgis-user mailing list
>>> Qgis-user at lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user [2]
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> [2]
>>> 
>> 
>> --
>> Dr. Benjamin Ducke
>> Deutsches Archäologisches Institut (DAI)
>> Zentrale Berlin, IT-Referat
>> * Projekt "Stunde Null" *
>> 
>> _______________________________________________
>> Qgis-user mailing list
>> Qgis-user at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user [2]
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user [2]
> 
> 
> 
> Links:
> ------
> [1] https://grass.osgeo.org/grass74/manuals/r.fill.stats.html
> [2] https://lists.osgeo.org/mailman/listinfo/qgis-user
> [3] http://www.opw.ie/en/disclaimer/
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Image1.jpg
Type: image/jpeg
Size: 48414 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20180421/345c3064/attachment.jpg>