[postgis-devel] Something wrong with build

[Paragon Corporation]

Have any idea how long its been going on.  I could take out some of the
sections I added to  postgis_aggs_mm.xml.xsl  to see if that improves
things, but the documentation however you look at it is significantly larger
in 2.0.  Just the Raster section alone adds about 50 pages.

Though like I mentioned on our local doesn't take that long (probably about
2 minutes at most), but we don't have the doc book dtd etc included so
perhaps the extra dtd validation is eating up resources.

-----Original Message-----
From: Kevin Neufeld [mailto:kneufeld at refractions.net] 
Sent: Wednesday, October 13, 2010 4:51 PM
To: Paragon Corporation
Cc: 'PostGIS Development Discussion'
Subject: Re: Something wrong with build

  In PostGIS 1.5, the whole thing builds in 10 minutes.

The time differences are in the xsltproc command.  IE., the "make
chunked-html" command does this:

/usr/bin/xsltproc ./xsl/postgis_aggs_mm.xml.xsl postgis.xml>
postgis_aggs_mm.xml

This runs practically instantaneously in 1.5, but takes 10 minutes in trunk.
Since building the docs makes several calls to xsltproc, it all adds up to
about an hour in trunk.

Either the postgis.xml file is significantly larger in trunk and it's
exceeding some memory threshold in xsltproc, or the xml file is malformed
and xsltproc is confused (though no errors/warnings are issued).

Separate volumes may not be such a bad idea.

-- Kevin

On 10/13/2010 11:59 AM, Paragon Corporation wrote:
> Kevin,
>
> For what its worth, doing the
>
> Make comments
>
> On our Xp box takes about 2 minutes.  We are running a newer version 
> of xsltproc than you.
> -- ours is --
> Using libxml 20703, libxslt 10124 and libexslt 813 xsltproc was 
> compiled against libxml 20701, libxslt 10124 and libexslt 813 libxslt 
> 10124 was compiled against libxml 20701 libexslt 813 was compiled 
> against libxml 20701
> --- end --
>
> When I cheat by copying in a blank  postgis_aggs_mm.xml   -- it takes
about
> 15-30 seconds to build both the postgis_comments.sql and 
> postgis_raster.sql
>
> --
>
> I suppose this may be my fault :( -- the postgis_aggs_mm.xml.xsl is 
> doing a tad bit more than it used to in PostGIS 1.5.
>
> 2 more columns for the function matrix (so much for my dreams of 
> having a raster function matrix - though that wouldn't be quite as 
> useful anyway) Another 50 someodd functions and an additional what's 
> new section (hmm a what's new broken out into two because we have too 
> many new functions and enhanced functions)
>
> Our manual for PostGIS 2.0 is already about 100 pages longer than the 
> PostGIS 1.5.  At this rate, we might have to start breaking it out 
> into volumes :)
>
>
>   How does the speed compare to building PostGIS 1.5 by the way?  I 
> imagine its much longer.
>
> Thanks,
> Regina
>
> -----Original Message-----
> From: Kevin Neufeld [mailto:kneufeld at refractions.net]
> Sent: Wednesday, October 13, 2010 1:32 PM
> To: Paragon Corporation
> Cc: PostGIS Development Discussion
> Subject: Re: Something wrong with build
>
>    Right, so the current autobuild process is taking about 60 minutes 
> to complete.
>
> The build itself happens within minutes.  Building the documentation 
> on the other hand is what is dragging the process out.
>
> # takes about 20 minutes
> make -e chunked-html 2>  doc-errors.log
>
> # takes about 15 minutes
> make comments 2>>  doc-errors.log
>
> # takes about 22 minutes
> make pdf
>
>
> During chuck-html task, it looks like we're stalling on this command 
> (stalls for about 10 minutes, idle cpu, i/o):
> /usr/bin/xsltproc ./xsl/postgis_aggs_mm.xml.xsl postgis.xml> 
> postgis_aggs_mm.xml
>
> and this (stalls here too for about 10 minutes):
> /usr/bin/xsltproc \
>     --param section.autolabel 1 \
>     --param section.label.includes.component.label 1 \
>     --param chunk.section.depth 0 \
>     --param generate.section.toc.level 1 \
>     --param funcsynopsis.style ansi \
>     --param admon.graphics 1 \
>     --param admon.textlabel 0 \
>     --param simplesect.in.toc 0 \
>     --param use.id.as.filename 1 \
>     --param chunk.quietly 1 \
>     --stringparam html.stylesheet style.css  \
>     --output html/ \
>     --stringparam saxon.character.representation decimal \
>     /usr/share/sgml/docbook/xsl-stylesheets/html/chunk.xsl \
>     postgis-out.xml
>
> Then, on the comments task (taking about 15 minutes):
> /usr/bin/xsltproc ./xsl/postgis_comments.sql.xsl postgis.xml> 
> postgis_comments.sql /usr/bin/xsltproc ./xsl/raster_comments.sql.xsl 
> postgis.xml>  raster_comments.sql
>
> The pdf task used to take only 5-10 minutes, now we're up to 22 
> minutes, stalling on the dblatex command.
>
> Anyone have ideas why xsltproc is getting slower and slower?  The 
> build machine is a standalone PostGIS build box ...  load average is
minimal.
>
> Is it just the configuration / software versions on this box?  Are 
> others able to build the docs in a reasonable amount of time?
>
> For reference:
> $ xsltproc --version
> Using libxml 20626, libxslt 10117 and libexslt 813 xsltproc was 
> compiled against libxml 20626, libxslt 10117 and libexslt 813 libxslt 
> 10117 was compiled against libxml 20626 libexslt 813 was compiled 
> against libxml 20626
>
>
> Cheers,
> Kevin
>
> On 10/12/2010 2:06 PM, Paragon Corporation wrote:
>> Kevin,
>>
>> Can you take a look and see what is taking the build so long.
>>
>> http://office.refractions.net:1500/view/PostGIS%20Trunk/job/PostGIS-t
>> r
>> unk-bu
>> ild/
>>
>> Last  change was trying to include raster_comments build in the
>>
>> make comments
>>
>> I think it broke last time I tried that too so had to roll back the
> change.
>> Not giving an error just stalling as far as I can tell.
>>
>>
>