[Liblas-devel] Re: LASzip versus LT compressor
Michael Rosen
mrosen at lizardtech.com
Wed Apr 27 18:17:02 EDT 2011
One thing that needs to be made clear is what we mean when we say "decode time." Specifically, are we decoding the whole thing, or small piece of the whole thing? Those are both important questions but they occur in different workflows. When you extract from a static archive, you care about how long it takes to recover your original file. When you render a scene from a working format, you care about how long it takes extract that scene. MrSID compression was designed to be a working format, allowing reasonable performance without decoding the entire dataset. If I understand Martin's note, the decode timing being compared is exactly the full dataset. Those are two separate workflows.
Here are my original remarks on this:
> WRT extraction time, for smaller files, the MG4?s computational overhead (*) favors LAZ for all but the smallest extractions
> For larger files, the "break even" point is much further to the right.
> For larger files, with very small extractions, the built-in index of MG4 allows faster extractions than raw (unindexed) LAS.
Here's a link to a presentation I gave on this last February: http://prezi.com/y90nhhyucxra/lidar-distribution-challenges-and-solutions/
If you zoom into the "Compression" part you can see the graphs on this.
msr
-----Original Message-----
From: liblas-devel-bounces at lists.osgeo.org [mailto:liblas-devel-bounces at lists.osgeo.org] On Behalf Of liblas-devel-request at lists.osgeo.org
Sent: Wednesday, April 27, 2011 8:55 AM
To: liblas-devel at lists.osgeo.org
Subject: Liblas-devel Digest, Vol 40, Issue 21
Send Liblas-devel mailing list submissions to
liblas-devel at lists.osgeo.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.osgeo.org/mailman/listinfo/liblas-devel
or, via email, send a message with subject or body 'help' to
liblas-devel-request at lists.osgeo.org
You can reach the person managing the list at
liblas-devel-owner at lists.osgeo.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Liblas-devel digest..."
Today's Topics:
1. Re: LASzip versus LT compressor (Martin Isenburg)
----------------------------------------------------------------------
Message: 1
Date: Wed, 27 Apr 2011 08:54:53 -0700
From: Martin Isenburg <martin.isenburg at gmail.com>
Subject: Re: [Liblas-devel] LASzip versus LT compressor
To: mpg at flaxen.com
Cc: liblas-devel at lists.osgeo.org
Message-ID: <BANLkTimkWBiUNDtVwPf=AzNNn2BBcb4opg at mail.gmail.com>
Content-Type: text/plain; charset="windows-1252"
hi,
On Wed, Apr 27, 2011 at 8:28 AM, Michael P. Gerlek <mpg at flaxen.com> wrote:
> Martin, can you give the data below in terms of compression ratio
> against the original file sizes?
>
>
sure.
SID LAZ file_name original_file_size (MB)
3.1 5.9 autzen-colorized-1.2-3.las 345
2.8 7.0 Dallas.las 100
6.9 7.2 GrassLakeSmall.las 118
3.3 8.3 IowaDNR-CloudPeakSoft-1.0-UTM15N.las 156
6.5 7.4 LAS12_Sample_withRGB_QT_Modeler.las 95
4.3 4.6 LASFile_1.las 46
4.5 4.8 LASFile_2.las 42
4.2 4.6 LASFile_3.las 16
4.5 4.9 LASFile_4.las 46
4.4 4.7 LDR030828_212242_0.las 57
4.5 4.8 LDR030828_213023_0.las 56
4.3 4.6 LDR030828_213450_0.las 51
2.8 5.2 LDR091111_181233_1.las 52
2.8 5.3 LDR091111_182803_1.las 52
2.8 5.1 Ldr100402_220229_1.las 1781
6.1 6.5 Lincoln.las 177
4.0 3.8 line_27007_dd.las 103
6.3 8.3 MARS_Sample_Filtered_LiDAR.las 156
2.8 5.2 merrick_vertical_1.2.las 52
12.9 12.2 MountStHelensNov202004.las 110
6.4 6.6 MountStHelensOct42004.las 129
3.1 3.3 ncwc000008.las 60
6.5 6.8 PalmBeachPreHurricane.las 49
8.0 8.6 radiohead_data1.las 397
8.0 8.7 radiohead_data2.las 433
3.4 7.9 S1C1_strip021.las 75
3.3 9.1 SerpentMoundModelLASData.las 87
2.8 5.8 Tetons.las 100
2.9 5.3 USACE_Merrick_lots_of_VLRs.las 96
9.7 10.5 xyzrgb_manuscript.las 53
Also, are you willing to report the timing data?
>
for the 1.7 GB file it took LASzip 1:33 min to encode and 1:35 min to decode. it took the LT compressor 18:26 min to encode and 4:46 min to decode. your own mpg will vary depending on disk and compressor speeds ...
cheers,
martin @lastools
> *From:* liblas-devel-bounces at lists.osgeo.org [mailto:
> liblas-devel-bounces at lists.osgeo.org] *On Behalf Of *Martin Isenburg
> *Sent:* Wednesday, April 27, 2011 8:12 AM
> *To:* liblas-devel at lists.osgeo.org
> *Subject:* [Liblas-devel] LASzip versus LT compressor
>
>
>
> Hello,
>
> People sometimes ask me how LASzip compares to the LIDAR compressor
> from Lizard Tech and I usually refer them to Michael's email (see
> below). Because his benchmarking was wrong on one model, namely MG4
> does not outperform LASzip on 2398_400.las, i did my own experiments
> that suggest that LASzip compresses about 35% better and is faster.
>
> A set of 27 LAS files (see below for a listing) compresses to 403 MB
> with LASzip and to 610 MB with the LIDAR compressor from Lizard Tech.
>
> A 1.7GB LAS file of a flight swath (see below for the details)
> compresses to 352 MB with LASzip and to 648 MB with the LIDAR
> compressor from Lizard Tech. LASzip encoding is about 10 times faster.
> LASzip decoding is about 3 times faster. timings measurements included
> all disk I/O from compressed file to uncompressed file (and vice-versa) using two separate drives.
> disclaimer: the LASzip compressor is tuned for LAS files that contain
> LIDAR in acquisition order.
>
> Cheers,
>
> martin @lastools
>
> the list of 27 LAS files. the first number is the compressed file size
> in bytes for the LIDAR Compressor of Lizardtech. the second number is
> the compressed file size in bytes for LASzip. all files can be found
> here http://liblas.org/samples except "Dallas.las" and "Tetons.las"
> which are
> here: http://bin.us.lizardtech.com/lidar/LT_LiDAR_Sample_Data.zip
>
> 115,857,121 61,809,700 autzen-colorized-1.2-3.las
> 37,462,947 14,881,473 Dallas.las
> 18,035,893 17,128,065 GrassLakeSmall.las
> 48,947,417 19,621,507 IowaDNR-CloudPeakSoft-1.0-UTM15N.las
> 15,248,628 13,382,538 LAS12_Sample_withRGB_QT_Modeler.las
> 11,222,840 10,444,300 LASFile_1.las
> 9,805,767 9,154,780 LASFile_2.las
> 3,966,869 3,665,433 LASFile_3.las
> 10,691,410 9,940,731 LASFile_4.las
> 13,522,405 12,672,774 LDR030828_212242_0.las
> 13,058,811 12,157,072 LDR030828_213023_0.las
> 12,244,190 11,502,895 LDR030828_213450_0.las
> 19,820,246 10,414,626 LDR091111_181233_1.las
> 19,424,894 10,193,907 LDR091111_182803_1.las
> 30,451,604 28,680,682 Lincoln.las
> 27,076,520 28,593,056 line_27007_dd.las
> 25,753,118 19,594,207 MARS_Sample_Filtered_LiDAR.las
> 19,820,246 10,414,626 merrick_vertical_1.2.las
> 8,943,713 9,493,209 MountStHelensNov202004.las
> 20,937,807 20,337,536 MountStHelensOct42004.las
> 20,386,038 19,036,134 ncwc000008.las
> 7,900,248 7,539,341 PalmBeachPreHurricane.las
> 22,831,603 9,920,385 S1C1_strip021.las
> 28,006,302 10,036,738 SerpentMoundModelLASData.las
> 37,216,167 18,169,153 Tetons.las
> 34,980,891 18,961,597 USACE_Merrick_lots_of_VLRs.las
> 5,756,508 5,351,794 xyzrgb_manuscript.las
>
> the lasinfo details of the 1.7GB LAS file containing one swath
>
> lasinfo Ldr100402_220229_1.laz
> reporting all LAS header entries:
> file signature: 'LASF'
> file source ID: 0
> reserved (global_encoding):0
> project ID GUID data 1-4: 0 0 0 ''
> version major.minor: 1.0
> system identifier: 'ALSXX'
> generating software: 'ALSXX_PP V2.69 BUILD#7 BETA'
> file creation day/year: 92/2010
> header size 227
> offset to point data 5697
> number var. length records 4
> point data format 1
> point data record length 28
> number of point records 66705904
> number of points by return 58445315 6743224 1404140 113225 0
> scale factor x y z 0.001 0.001 0.001
> offset x y z 13000000 0 0
> min x y z 12991192.425 588397.501 611.122
> max x y z 13142242.349 594146.283 3032.417
> variable length header record 1 of 4:
> reserved 43707
> user ID 'LeicaGeo'
> record ID 1001
> length after header 5120
> description ''
> variable length header record 2 of 4:
> reserved 43707
> user ID 'LeicaGeo'
> record ID 1002
> length after header 22
> description 'MissionInfo'
> variable length header record 3 of 4:
> reserved 43707
> user ID 'LeicaGeo'
> record ID 1003
> length after header 54
> description 'UserInputs'
> variable length header record 4 of 4:
> reserved 43707
> user ID 'LASF_Projection'
> record ID 34735
> length after header 56
> description 'Projection Info'
> GeoKeyDirectoryTag version 1.1.0 number of keys 6
> key 1024 tiff_tag_location 0 count 1 value_offset 1 -
> GTModelTypeGeoKey: ModelTypeProjected
> key 1025 tiff_tag_location 0 count 1 value_offset 2 -
> GTRasterTypeGeoKey: RasterPixelIsPoint
> key 3076 tiff_tag_location 0 count 1 value_offset 26990 -
> ProjLinearUnitsGeoKey: look-up for 26990 not implemented
> key 2052 tiff_tag_location 0 count 1 value_offset 9002 -
> GeogLinearUnitsGeoKey: Linear_Foot
> key 4096 tiff_tag_location 0 count 1 value_offset 5103 -
> VerticalCSTypeGeoKey: VertCS_North_American_Vertical_Datum_1988
> key 4099 tiff_tag_location 0 count 1 value_offset 9002 -
> VerticalUnitsGeoKey: Linear_Foot
> the header is followed by 2 user-defined bytes LASzip compression
> (version 1.0r0 c1): POINT10 1 GPSTIME11 1 reporting minimum and
> maximum for all LAS point record entries ...
> x -8807574 142242349
> y 588397501 594146283
> z 611122 3032417
> intensity 0 255
> edge_of_flight_line 0 0
> scan_direction_flag 0 1
> number_of_returns_of_given_pulse 1 4
> return_number 1 4
> classification 1 1
> scan_angle_rank -26 31
> user_data 161 255
> point_source_ID 161 511
> gps_time 511349.016753 512063.402540 overview over number of returns
> of given pulse: 51686151 10678566 3886669
> 454518 0 0 0
> histogram of classification of points:
> 66705904 Unclassified (1)
>
>
> ----------------------------------------------------------------------
> ----------------
>
> michael's email (the graphs he mentions can be found in the archive)
>
> http://lists.osgeo.org/pipermail/liblas-devel/2011-February/001199.htm
> l
>
> On Fri, Feb 4, 2011 at 3:21 PM, Michael Rosen <mrosen at lizardtech.com>
> wrote:
>
> Here?s the summary of some LT-internal (I guess not so internal now?)
> benchmarking. Highlights:
>
> - I can?t really draw any conclusions about relative compression
> sizes: 2398_400 favors MG4 2:1, HGAC_Extract and AutZen favor LAZ
> 2:1, MtStHelens is a wash,
>
> - WRT extraction time, for smaller files, the MG4?s computational
> overhead (*) favors LAZ for all but the smallest extractions
>
> - For larger files, the ?break even? point is much further to the
> right.
>
> - For larger files, with very small extractions, the built-in
> index of MG4 allows faster extractions than raw (unindexed) LAS.
>
>
>
> The methodology here was to run ?las2las? as shown before cropping out
> increasingly large rectangles (at full resolution)
>
> I compared this with the same extraction from MG4 using a command line
> tool
> (internal) but this time, writing the output to a las file.
>
>
>
> I spot checked that the number of points written in all three cases
> was the same.
>
>
>
> (*) Note that the title on the graphs is not quite right. It?s not
> ?Decode Time? but ?Decode Time plus LAS Write Time? vs Scene Size.
> There is some speculation (based on what we were observing when
> omitting the output) that LT?s LAS Writer is unusually slow. It?s
> using the liblas v1.2 writer ? so some here may have well-informed opinions on this.
>
>
>
> Here is some raw data and some graphs:
>
>
>
> 01/28/2011 03:50 PM 61,301,311 2398_400.las
>
> 01/28/2011 04:28 PM 8,906,275 2398_400.laz
>
> 01/28/2011 04:25 PM 4,650,992 2398_400.sid
>
>
>
> 01/28/2011 04:03 PM 362,213,959 autzen-colorized-1.2-3.las
>
> 01/28/2011 04:28 PM 61,809,700 autzen-colorized-1.2-3.laz
>
> 01/28/2011 04:27 PM 115,857,121 autzen-colorized-1.2-3.sid
>
>
>
> 01/28/2011 03:59 PM 123,876,781 Grass Lake Small.las
>
> 01/28/2011 04:29 PM 17,128,065 Grass Lake Small.laz
>
> 01/28/2011 04:25 PM 18,035,893 Grass Lake Small.sid
>
>
>
> 02/02/2011 08:18 AM 711,065,603 HGAC_Extract.las
>
> 02/02/2011 08:23 AM 151,159,393 HGAC_Extract.laz
>
> 02/02/2011 08:29 AM 269,491,108 HGAC_Extract.sid
>
>
>
> 01/28/2011 03:50 PM 34,065,751 hobu.las
>
> 01/28/2011 04:29 PM 7,732,878 hobu.laz
>
> 01/28/2011 04:24 PM 9,301,431 hobu.sid
>
>
>
> 01/28/2011 04:00 PM 185,565,975 Lincoln.las
>
> 01/28/2011 04:29 PM 28,680,682 Lincoln.laz
>
> 01/28/2011 04:25 PM 30,451,604 Lincoln.sid
>
>
>
> 01/28/2011 03:58 PM 107,603,879 line_27007.las
>
> 01/28/2011 04:30 PM 22,269,252 line_27007.laz
>
> 01/28/2011 04:25 PM 24,588,596 line_27007.sid
>
>
>
> 01/28/2011 03:58 PM 115,737,877 MtStHelens.las
>
> 01/28/2011 04:30 PM 9,493,209 MtStHelens.laz
>
> 01/28/2011 04:24 PM 8,943,713 MtStHelens.sid
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/liblas-devel/attachments/20110427/6147bba2/attachment.html
------------------------------
_______________________________________________
Liblas-devel mailing list
Liblas-devel at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/liblas-devel
End of Liblas-devel Digest, Vol 40, Issue 21
********************************************
More information about the Liblas-devel
mailing list