<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Tim,<br>
<br>
On 17/10/2010 21:36, Tim Sutton wrote:
<blockquote
cite="mid:AANLkTi=fUaKBZZA9oWkxFf41O+G-q=m7V2ZikXF1Ho81@mail.gmail.com"
type="cite">
<pre wrap="">Hi
On Sun, Oct 17, 2010 at 7:15 PM, Benoit de Cabissole <a class="moz-txt-link-rfc2396E" href="mailto:benoit@exigesa.com"><benoit@exigesa.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap=""> Hi Tim,
Many thanks for your reply.
I see that the API doc classifies HistogramVector as a QVector but I
cannot find any reference of QVector in the Qt doc (there are QVector2D,
QVector3D and QVector4D). Could it be the problem?
</pre>
</blockquote>
<pre wrap="">
I found QVector in my (Qt 4.7) docs using Qt Assistant - not sure why
its not showing up for you. Perhaps it is not ported to the python SIP
lib.
</pre>
</blockquote>
Well it is not in the PyQt (4.6.2) API doc I'm using!<br>
<blockquote
cite="mid:AANLkTi=fUaKBZZA9oWkxFf41O+G-q=m7V2ZikXF1Ho81@mail.gmail.com"
type="cite">
<pre wrap=""></pre>
<blockquote type="cite">
<pre wrap="">
What I'm trying to do is to improve the speed of my plugin '1-band
raster colour table' that allows to create colour table from a colour
palette and a colouring method.
</pre>
</blockquote>
<pre wrap="">
Yup I use your plugin regularly - its great!
</pre>
</blockquote>
Thanks!<br>
<blockquote
cite="mid:AANLkTi=fUaKBZZA9oWkxFf41O+G-q=m7V2ZikXF1Ho81@mail.gmail.com"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">As I'm doing it in Python, the
computation of the stats and histogram takes for ever on large rasters.
I was hoping to piggyback on the stats and histo function built in QGIS
(they themselves are (very) slow on large rasters).
</pre>
</blockquote>
<pre wrap="">
Right this is an issue. Part of the reason why QGIS is so slow is that
it actually does two complete scans of each pixel to calculate the
stats. This is needed for computing stddev - although since writing
that code I have seen a way to compute stddev in a single pass - I
just need to find the formula and the time to implement that.
</pre>
</blockquote>
Oh no! I'm also using two passes, because I need to compute the
min/max in the first pass to compute the classes for the histogram
in the second pass.<br>
The STDev that I use is computed in a single pass with the following
formula:<br>
<br>
STD = SQRT( ( N * Sum(X^2) - (Sum(X))^2 ) / ( N * (N-1) ) )<br>
<br>
Note that Sum(X^2) is the 'sum of uncentered squares', not the one
presently computed by QGIS/GDAL.<br>
Manuel Massing has said that QGIS/GDAL computes the 'empirical
variance of the data' (i.e. sum (value-mean)<sup class="moz-txt-sup">2</sup>
) and calls it the 'sum of the squared values'.<br>
<br>
So between us two we could reduce the time by having only one loop!
How do you compute the histogram in one pass? Do you, a priori, know
the max and min values? Are they available when the raster is
loaded? I know an estimate of them is available.<br>
<br>
<blockquote
cite="mid:AANLkTi=fUaKBZZA9oWkxFf41O+G-q=m7V2ZikXF1Ho81@mail.gmail.com"
type="cite">
<pre wrap="">
GDAL computes stats more quickly as it a) doesnt calculate stddev and
b) will estimate things like min/max, histograms etc if you like based
on a sampling of the data.
GDAL can also store gathered stats (you may have noticed aux.xml files
lying around in your raster dirs).
</pre>
</blockquote>
Yes, I've noticed that one. Is there a function in QGIS to read its
content in one go? Or do I have to parse it myself?<br>
<br>
For version 1.x of the plugin, I've eliminated the "imagette" (as
Agus calls it) and replaced it by a new temporary layer in the
mapcanvas because I'm now using QGIS functions to display the raster
instead of my very slow matrix based raster used in the published
plugin.<br>
Problem now is that QGIS compute the stats for that temp raster each
time it is launched, and it can be very very slow. Is there a way to
tell QGIS to use an (arbitrary) aux.xml file instead of computing
the stats the first time the colour table changes? In my experience
rasters files rarely change so keeping the stats file for that
raster somewhere close to it would be a good idea, isn't it?<br>
<br>
<blockquote
cite="mid:AANLkTi=fUaKBZZA9oWkxFf41O+G-q=m7V2ZikXF1Ho81@mail.gmail.com"
type="cite">
<pre wrap="">
In the longer term I would like to a) persist QGIS gathered stats, b)
implement faster stddev calcs as mentioned above and c) only compute
IO intensive stats like stddev if they are actually needed (e.g. if
the user elects to use stddev in the raster symbology options).
</pre>
<blockquote type="cite">
<pre wrap="">
I agree that this work should be patched to the current code, but I'm
not able to program in C++/C...
So once the python plugin works as expected, it would be a good idea to
hand it over to a C++ guru for patching!
</pre>
</blockquote>
<pre wrap="">
Yeah if I have time I may take a look - your plugin is really needed
in the core of QGIS.
</pre>
</blockquote>
<br>
Please before doing that wait for version 1.x it could be better to
do the translation on it rather than on the current version.<br>
<br>
Cheers,<br>
Benoit<br>
<br>
<blockquote
cite="mid:AANLkTi=fUaKBZZA9oWkxFf41O+G-q=m7V2ZikXF1Ho81@mail.gmail.com"
type="cite">
<pre wrap="">
Regards
Tim
</pre>
</blockquote>
</body>
</html>