[pdal] PDAL decrease performance after hardware change

[Oscar Martinez Rubi]

Hi guys,

I have two problems that are driving me quite crazy. The first one is 
related to compilation of PDAL:

I can not manage to compile the current version in the repository. It 
crashes during compilation. The error I get is (I used embed_boost):

[ 48%] Building CXX object src/CMakeFiles/pdalcpp.dir/StageFactory.cpp.o
In file included from 
/home/oscar/sw/PDAL/src/../include/pdal/drivers/bpf/BpfReader.hpp:37,
from /home/oscar/sw/PDAL/src/../include/pdal/Drivers.hpp:46,
from /home/oscar/sw/PDAL/src/StageFactory.cpp:43:
/home/oscar/sw/PDAL/src/../include/pdal/IStream.hpp: In member function 
‘std::istream* pdal::IStream::popStream()’:
/home/oscar/sw/PDAL/src/../include/pdal/IStream.hpp:78: error: ‘nullptr’ 
was not declared in this scope
/home/oscar/sw/PDAL/src/../include/pdal/IStream.hpp: In member function 
‘pdal::ILeStream& pdal::ILeStream::operator>>(float&)’:
/home/oscar/sw/PDAL/src/../include/pdal/IStream.hpp:173: warning: 
dereferencing type-punned pointer will break strict-aliasing rules
/home/oscar/sw/PDAL/src/../include/pdal/IStream.hpp: In member function 
‘pdal::ILeStream& pdal::ILeStream::operator>>(double&)’:
/home/oscar/sw/PDAL/src/../include/pdal/IStream.hpp:181: warning: 
dereferencing type-punned pointer will break strict-aliasing rules
In file included from 
/home/oscar/sw/PDAL/boost/boost/interprocess/detail/managed_open_or_create_impl.hpp:15,
from 
/home/oscar/sw/PDAL/boost/boost/interprocess/managed_shared_memory.hpp:22,
from /home/oscar/sw/PDAL/src/../include/pdal/PointBuffer.hpp:44,
from /home/oscar/sw/PDAL/src/../include/pdal/drivers/sbet/Reader.hpp:38,
from /home/oscar/sw/PDAL/src/../include/pdal/Drivers.hpp:48,
from /home/oscar/sw/PDAL/src/StageFactory.cpp:43:
/home/oscar/sw/PDAL/boost/boost/interprocess/detail/os_thread_functions.hpp: 
In function ‘int 
pdalboost::interprocess::ipcdetail::thread_launch(pdalboost::interprocess::ipcdetail::OS_thread_t&, 
F)’:
/home/oscar/sw/PDAL/boost/boost/interprocess/detail/os_thread_functions.hpp:554: 
warning: ‘auto_ptr’ is deprecated (declared at 
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/backward/auto_ptr.h:86)
make[2]: *** [src/CMakeFiles/pdalcpp.dir/StageFactory.cpp.o] Error 1
make[1]: *** [src/CMakeFiles/pdalcpp.dir/all] Error 2
make: *** [all] Error 2

I also tried without embed boost but also crashed.

It compiles fine if I use the tag 0.9.8 though. I do not know if anybody 
else is having this issue or whether you are already aware of this (if 
that is the case I am sorry!)

I also tried the PDAL in https://github.com/pramsey/PDAL/ since I want 
to use PDAL for pointcloud in postgres (actually what is the difference 
between the two repositories?). In this one I manage to compile the 
latest but I always get the error Caught PDAL exception: 
drivers.pgpointcloud.writer num_points > m_patch_capacity! when loading 
files.

If I use a previous version of the code in the pramsey repository, for 
example revision 50ed6d4 (git reset --hard 50ed6d4) then it works fine.

Any help?

The second problem I have is a decrease of PDAL performance after a 
hardware/software change on my main testing server.

This RedHat 6.5 server had originally a RAID 6 with 80TB. We have this 
test to load 20M points in postgres pointcloud from a LAZ file. It took 
80 seconds. Another test is to load them in a flat table (using las2txt 
piped with psql -c 'copy from') which takes 600 seconds (PDAL rocks!).

Then, we changed the hardware. Now we have two RAID systems of 40TB 
each. The idea was to use one for reading the LAZ files and the other to 
write to the DB. We also upgraded Postgres and Postgis and PDAL with 
latest versions (well PDAL I had to use a previous revision). After the 
upgrade the time of the PDAL loading was 130 seconds (instead of 80)! 
But the time of the flat table loading is still the same (600 secs). So, 
something have changed that affects only PDAL.

I am also testing using Oracle (not with PDAL) and MonetDB to load point 
clouds and these other systems are not affected by hardware change 
(actually they improve)

I have some virtual machines (centos 6.4) in another server. I tried 
different combinations of Postgres, PostGIS, PDAL, GDAL to see if I saw 
some combination that was worse than the rest but they are more or less 
the same (which is expected). So, essentially I could not reproduce the 
same performance decrease in a different system which is very annoying.

Anyway, I tried to install back in the main testing server the same 
Postgres, PostGIS, PDAL software that I originally had with no luck. The 
time is still around 130 seconds (for something that should take 80 
seconds).

Any idea why could this happen? Excuses because I know that something 
that is nor reproducible is very tricky to correct but I am quite 
desperate here! Any idea/hint would be very appreciated.

Thanks a lot in advance and sorry for long mail!

Regards,

Oscar