[Qgis-developer] shapefile editing memory issue

[Tim Sutton]

Hi

On Thu, Jul 12, 2012 at 8:00 PM, Thorp, Kelly <Kelly.Thorp at ars.usda.gov> wrote:
> Hello List:
>
>
>
> I am developing a QGIS plug-in for the purpose of running simulation models
> for unique land units, as represented by polygons in a polygon shapefile.
> Essentially, the plug-in uses spatial information in the polygon layer to
> write a simulation model input file.  It then uses Python's 'subprocess'
> module to call the model executable and run the simulation.  It then reads
> an output file generated by the model, sets the polygon shapefile to be
> editable, changes some attributes based on the model output, and then
> commits those changes.  Then repeating this in a loop for each feature in
> the polygon shapefile.
>
>
>
> I've also included an optimization routine for the purpose of model
> calibration.  This essentially means doing the above procedures repeatedly
> tens or hundreds of thousands of times.   After each model simulation, the
> updated 'simulated' attribute values are compared with other attributes that
> contain 'observed' data.  The optimization then repeatedly adjusts the model
> parameters until simulated and observed data are in best agreement.
>
>
>
> This process can take many hours or days, while I am repeatedly editing the
> shapefile and commiting the edits.  However, I am not adding or removing any
> features or attributes.  Problem is the memory slowly creeps up and if the
> optimization takes too long, I'll run out of memory and QGIS will crash. I
> notice that if my optimization routine does finish, I can completely shut
> down my plugin's dialog window, but the memory usage remains high.  This
> makes me think the problem is not in my plug-in.  Any ideas for me?
>
>

You would probably need to run your plugin under a memory profiler
like valgrind to see where the leaks are occurring.

My apologies if that is too low level an answer.

Regards

Tim


>
> Thanks,
>
> Kelly
>
>
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Kelly Robert Thorp, Ph.D., P.E.
>
> Research Agricultural Engineer
>
> USDA-ARS
>
> U.S. Arid Land Agricultural Research Center
>
> 21881 N Cardon Lane
>
> Maricopa, AZ  85138
>
> 520-316-6375 (Phone)
>
> 520-316-6330 (Fax)
>
> kelly.thorp at ars.usda.gov
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
>
>
>
>
>
> This electronic message contains information generated by the USDA solely
> for the intended recipients. Any unauthorized interception of this message
> or the use or disclosure of the information it contains may violate the law
> and subject the violator to civil or criminal penalties. If you believe you
> have received this message in error, please notify the sender and delete the
> email immediately.
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>



-- 
Tim Sutton - QGIS Project Steering Committee Member (Release  Manager)
==============================================
Please do not email me off-list with technical
support questions. Using the lists will gain
more exposure for your issues and the knowledge
surrounding your issue will be shared with all.

Visit http://linfiniti.com to find out about:
 * QGIS programming and support services
 * Mapserver and PostGIS based hosting plans
 * FOSS Consulting Services
Skype: timlinux
Irc: timlinux on #qgis at freenode.net
==============================================