<html><head></head><body><div>
<meta charset="utf-8">
<div id="compose-body-wrapper" dir="auto"><div dir="auto"><div dir="auto">Hello Laurentiu,</div><div dir="auto">The color palette is working fine. The issues seem to be caused by using UInt16 because I changed to GDT_Byte per your suggestion and everything is so much smoother (this was an oversight of mine as I recycled a snippet from somewhere and should have paid much closer attention to this detail). </div><div dir="auto">As for compressing and using overviews (for future use), I'm having some difficulty implementing them but it's less urgent and will keep working at it.</div><div dir="auto">Thanks,</div><div dir="auto">Arturo</div><div><br></div></div><div dir="auto">Laurențiu Nicola via gdal-dev wrote:<br></div></div><br><br><div><blockquote cite="mid:ccff3aad-d290-4a41-a950-207855e485e9@betaapp.fastmail.com" type="cite" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style><div style="font-family:Arial;">Hi,<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">There are a couple of things you can try:<br></div><ul><li style="font-family:Arial;">you seem to be scaling the data to 8-bit, but saving it as UInt16; GDT_Byte should work, yielding a 50% reduction. I didn't quite understand the comment about the color, does the palette not work with 8-bit images?<br></li><li style="font-family:Arial;">you can enable and tune compression: check out COMPRESS, PREDICTOR and the level options in <a href="https://gdal.org/drivers/raster/gtiff.html">https://gdal.org/drivers/raster/gtiff.html</a>. ZSTD and DEFLATE are pretty good.<br></li><li style="font-family:Arial;">you can enable the tiled mode (TILED on the same page as above), which sometimes yields smaller files<br></li><li style="font-family:Arial;">if you find large files difficult to use, you can enable overview creation or use the COG driver (it's a subset of GTiff)<br></li><li style="font-family:Arial;">finally, you can use a lossy compression format like JPEG or LERC, but it's probably not what you want<br></li></ul><div style="font-family:Arial;">Regards,<br></div><div style="font-family:Arial;">Laurentiu<br></div><div style="font-family:Arial;"><br></div><div>On Thu, Jun 1, 2023, at 17:35, afernandez wrote:<br></div><blockquote type="cite" id="qt" style=""><div><div id="qt-compose-body-wrapper" dir="auto"><div dir="auto"><div dir="auto">Hello,<br></div><div dir="auto">I'm generating a raster file with
GDAL. The pseudo-code (where the raster is loaded as 'var') for the colored
version reads:<br></div><div dir="auto"># Initial manipulations<br></div><div dir="auto"> dims =
var.dimensions<br></div><div dir="auto">
shape = var.shape<br></div><div dir="auto">
driver_name = 'GTIFF'<br></div><div dir="auto">
driver =
gdal.GetDriverByName(driver_name) <br></div><div dir="auto">
np_dtype = var.dtype<br></div><div dir="auto"> type_code =
gdal_array.NumericTypeCodeToGDALTypeCode(np_dtype)<br></div><div dir="auto"> gdal_ds =
driver.Create(_my_path_, cols, rows, 1, gdal.GDT_UInt16) <br></div><div dir="auto">
gdal_ds.SetProjection(_my_projection_)<br></div><div dir="auto">
gdal_ds.SetGeoTransform(_my_transformation_)<br></div><div dir="auto">#
Creation of the bands and scaled matrix <br></div><div dir="auto">
band =
gdal_ds.GetRasterBand(1) <br></div><div dir="auto">
data = var[_chosen_index_]<br></div><div dir="auto">
data = ma.getdata(data)<br></div><div dir="auto"> data_scaled =
np.interp(data, (data.min(), data.max()), (0, 255))<br></div><div dir="auto"> data_scaled2 =
data_scaled.astype(int) # This is to rescale into integers so that it can
color the layer<br></div><div dir="auto"># *** Lines to set up the color
palette ***<br></div><div dir="auto"># Write the array to band once everything
has been rescaled<br></div><div dir="auto">
band.WriteArray(data_scaled2)<br></div><div dir="auto">
gdal_ds.FlushCache() <br></div><div dir="auto">This works well but the problem is that the generated file
becomes too large and difficult to work with. If I change to a black and
white representation by simply changing the 7th line to: <br></div><div dir="auto"> gdal_ds =
driver.Create(out_path, cols, rows, 1, type_code) <br></div><div dir="auto">The new file has a size smaller than 1% of the colored one. I
was wondering if there is anything intermediate (with colors but generating
a smaller size) or if I would need a more radical approach such as using a
driver different from GTIFF or something else.<br></div><div dir="auto">Thanks.<br></div></div></div></div><div>_______________________________________________<br></div><div>gdal-dev mailing list<br></div><div><a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br></div><div><a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br></div><div><br></div></blockquote><div style="font-family:Arial;"><br></div></blockquote></div><div><blockquote cite="mid:ccff3aad-d290-4a41-a950-207855e485e9@betaapp.fastmail.com" type="cite" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><title></title><style type="text/css">p.MsoNormal,p.MsoNoSpacing{margin:0}</style><div style="font-family:Arial;">Hi,<br></div><div style="font-family:Arial;"><br></div><div style="font-family:Arial;">There are a couple of things you can try:<br></div><ul><li style="font-family:Arial;">you seem to be scaling the data to 8-bit, but saving it as UInt16; GDT_Byte should work, yielding a 50% reduction. I didn't quite understand the comment about the color, does the palette not work with 8-bit images?<br></li><li style="font-family:Arial;">you can enable and tune compression: check out COMPRESS, PREDICTOR and the level options in <a href="https://gdal.org/drivers/raster/gtiff.html">https://gdal.org/drivers/raster/gtiff.html</a>. ZSTD and DEFLATE are pretty good.<br></li><li style="font-family:Arial;">you can enable the tiled mode (TILED on the same page as above), which sometimes yields smaller files<br></li><li style="font-family:Arial;">if you find large files difficult to use, you can enable overview creation or use the COG driver (it's a subset of GTiff)<br></li><li style="font-family:Arial;">finally, you can use a lossy compression format like JPEG or LERC, but it's probably not what you want<br></li></ul><div style="font-family:Arial;">Regards,<br></div><div style="font-family:Arial;">Laurentiu<br></div><div style="font-family:Arial;"><br></div><div>On Thu, Jun 1, 2023, at 17:35, afernandez wrote:<br></div><blockquote type="cite" id="qt" style=""><div><div id="qt-compose-body-wrapper" dir="auto"><div dir="auto"><div dir="auto">Hello,<br></div><div dir="auto">I'm generating a raster file with
GDAL. The pseudo-code (where the raster is loaded as 'var') for the colored
version reads:<br></div><div dir="auto"># Initial manipulations<br></div><div dir="auto"> dims =
var.dimensions<br></div><div dir="auto">
shape = var.shape<br></div><div dir="auto">
driver_name = 'GTIFF'<br></div><div dir="auto">
driver =
gdal.GetDriverByName(driver_name) <br></div><div dir="auto">
np_dtype = var.dtype<br></div><div dir="auto"> type_code =
gdal_array.NumericTypeCodeToGDALTypeCode(np_dtype)<br></div><div dir="auto"> gdal_ds =
driver.Create(_my_path_, cols, rows, 1, gdal.GDT_UInt16) <br></div><div dir="auto">
gdal_ds.SetProjection(_my_projection_)<br></div><div dir="auto">
gdal_ds.SetGeoTransform(_my_transformation_)<br></div><div dir="auto">#
Creation of the bands and scaled matrix <br></div><div dir="auto">
band =
gdal_ds.GetRasterBand(1) <br></div><div dir="auto">
data = var[_chosen_index_]<br></div><div dir="auto">
data = ma.getdata(data)<br></div><div dir="auto"> data_scaled =
np.interp(data, (data.min(), data.max()), (0, 255))<br></div><div dir="auto"> data_scaled2 =
data_scaled.astype(int) # This is to rescale into integers so that it can
color the layer<br></div><div dir="auto"># *** Lines to set up the color
palette ***<br></div><div dir="auto"># Write the array to band once everything
has been rescaled<br></div><div dir="auto">
band.WriteArray(data_scaled2)<br></div><div dir="auto">
gdal_ds.FlushCache() <br></div><div dir="auto">This works well but the problem is that the generated file
becomes too large and difficult to work with. If I change to a black and
white representation by simply changing the 7th line to: <br></div><div dir="auto"> gdal_ds =
driver.Create(out_path, cols, rows, 1, type_code) <br></div><div dir="auto">The new file has a size smaller than 1% of the colored one. I
was wondering if there is anything intermediate (with colors but generating
a smaller size) or if I would need a more radical approach such as using a
driver different from GTIFF or something else.<br></div><div dir="auto">Thanks.<br></div></div></div></div><div>_______________________________________________<br></div><div>gdal-dev mailing list<br></div><div><a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br></div><div><a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br></div><div><br></div></blockquote><div style="font-family:Arial;"><br></div></blockquote></div>
</div></body></html>