[QGIS-Developer] Unit tests: compare images is failing in my local setup
Jorge Gustavo Rocha
jgr at di.uminho.pt
Wed Feb 5 07:40:43 PST 2020
Hi devs,
In my local development setup (Ubuntu 18.04, Qt 5.11.1), my unit tests
are failing when comparing images. (I'm running these tests on master,
to make sure everything is ok. The tests are working properly on CI.)
The images are equal (according to imagemagick compare), when compared
manually:
compare -verbose -metric AE
/home/jgr/dev/cpp/QGIS/tests/testdata/control_images/qgis_server/WMS_GetPrint_TwoMaps/WMS_GetPrint_TwoMaps.png
/tmp/WMS_GetPrint_TwoMaps_result.png result_diff.png
/home/jgr/dev/cpp/QGIS/tests/testdata/control_images/qgis_server/WMS_GetPrint_TwoMaps/WMS_GetPrint_TwoMaps.png
PNG 3507x2480 3507x2480+0+0 8-bit sRGB 175786B 0.130u 0:00.130
/tmp/WMS_GetPrint_TwoMaps_result.png PNG 3507x2480 3507x2480+0+0 8-bit
sRGB 176003B 0.130u 0:00.129
Image:
/home/jgr/dev/cpp/QGIS/tests/testdata/control_images/qgis_server/WMS_GetPrint_TwoMaps/WMS_GetPrint_TwoMaps.png
Channel distortion: AE
red: 0
green: 0
blue: 0
alpha: 0
all: 0
/home/jgr/dev/cpp/QGIS/tests/testdata/control_images/qgis_server/WMS_GetPrint_TwoMaps/WMS_GetPrint_TwoMaps.png=>result_diff.png
PNG 3507x2480 3507x2480+0+0 8-bit sRGB 142733B 1.370u 0:00.719
But when I run the tests, the tests are failing.
For example:
ctest -V -R PyQgsServerWMSGetPrint
(...)
553: src/server/qgsserver.cpp:165 : (configPath) [0ms]
MAP:/home/jgr/dev/cpp/QGIS/tests/testdata/qgis_server_accesscontrol/project.qgs
553: src/server/qgsserver.cpp:165 : (configPath) [0ms]
MAP:/home/jgr/dev/cpp/QGIS/tests/testdata/qgis_server_accesscontrol/project.qgs
553: src/core/qgsmessagelog.cpp:29 : (logMessage) [0ms]
2020-02-05T14:30:41 [1] Service WMS 1.1.1 not found, returning default
553: src/core/qgsmessagelog.cpp:29 : (logMessage) [0ms]
2020-02-05T14:30:41 Server[0] WMS Request parameters:
553: src/core/qgsmessagelog.cpp:29 : (logMessage) [0ms]
2020-02-05T14:30:41 Server[0] - CRS : EPSG:3857
553: src/core/qgsmessagelog.cpp:29 : (logMessage) [0ms]
2020-02-05T14:30:41 Server[0] - FORMAT : png
553: src/core/qgsmessagelog.cpp:29 : (logMessage) [0ms]
2020-02-05T14:30:41 Server[0] - TEMPLATE : layoutA4twoMaps
553: src/core/qgsmessagelog.cpp:29 : (logMessage) [0ms]
2020-02-05T14:30:41 Server[0] - VERSION : 1.1.1
553: ERROR 6: The PNG driver does not support update access to existing
datasets.
553:
553: <DartMeasurementFile name="Rendered Image WMS_GetPrint_TwoMaps
(prefix qgis_server/)"
type="image/png">/tmp/WMS_GetPrint_TwoMaps_result.png</DartMeasurementFile>
553: <DartMeasurementFile name="Expected Image WMS_GetPrint_TwoMaps
(prefix qgis_server/)"
type="image/png">/home/jgr/dev/cpp/QGIS/tests/testdata/control_images/qgis_server/WMS_GetPrint_TwoMaps/WMS_GetPrint_TwoMaps.png</DartMeasurementFile>
553: Expected size: 3507w x 2480h
553: Actual size: 3507w x 2480h
553: <DartMeasurementFile name="Difference Image WMS_GetPrint_TwoMaps
(prefix qgis_server/)"
type="image/png">/tmp/WMS_GetPrint_TwoMaps_result_diff.png</DartMeasurementFile>
553: <DartMeasurement name="Mismatch Count"
type="numeric/integer">0/8697360</DartMeasurement>
553: 0/8697360 pixels mismatched (100 allowed)
553: .exportToPdf call: /usr/bin/pdftocairo -png -singlefile -r 300 -x 0
-y 0 -f 1 -l 1 /tmp/WMS_GetPrint_Basic_Pdf_result.pdf
/tmp/WMS_GetPrint_Basic_Pdf_result
553: exportToPdf call: /usr/bin/pdftocairo -png -singlefile -r 300 -x 0
-y 0 -f 1 -l 1 /tmp/WMS_GetPrint_Selection_Pdf_result.pdf
/tmp/WMS_GetPrint_Selection_Pdf_result
553:
553: ======================================================================
553: FAIL: test_wms_getprint_atlas (__main__.TestQgsServerWMSGetPrint)
553: ----------------------------------------------------------------------
553: Traceback (most recent call last):
553: File
"/home/jgr/dev/cpp/QGIS/tests/src/python/test_qgsserver_wms_getprint.py", line
607, in test_wms_getprint_atlas
553: self._img_diff_error(r, h, "WMS_GetPrint_Atlas")
553: File "/home/jgr/dev/cpp/QGIS/tests/src/python/test_qgsserver.py",
line 236, in _img_diff_error
553: self.assertTrue(test, message)
553: AssertionError: False is not true : Image is wrong\: diff file
/tmp/WMS_GetPrint_Atlas_result_diff.png
553:
553: ======================================================================
553: FAIL: test_wms_getprint_scale (__main__.TestQgsServerWMSGetPrint)
553: ----------------------------------------------------------------------
553: Traceback (most recent call last):
553: File
"/home/jgr/dev/cpp/QGIS/tests/src/python/test_qgsserver_wms_getprint.py", line
405, in test_wms_getprint_scale
553: self._img_diff_error(r, h, "WMS_GetPrint_Scale")
553: File "/home/jgr/dev/cpp/QGIS/tests/src/python/test_qgsserver.py",
line 236, in _img_diff_error
553: self.assertTrue(test, message)
553: AssertionError: False is not true : Image is wrong\: diff file
/tmp/WMS_GetPrint_Scale_result_diff.png
553:
553: ----------------------------------------------------------------------
553: Ran 16 tests in 32.438s
553:
553: FAILED (failures=2)
553: CMake Error at PyQgsServerWMSGetPrint.cmake:15 (MESSAGE):
553: Test failed: 1
553:
553:
1/1 Test #553: PyQgsServerWMSGetPrint ...........***Failed 33.16 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 33.28 sec
The following tests FAILED:
553 - PyQgsServerWMSGetPrint (Failed)
Errors while running CTest
----------------------------------------------------------------------
1) As far as I can undesstand, <DartMeasurement name="Mismatch Count"
type="numeric/integer">0/8697360</DartMeasurement> means that the pixels
are equal
2) The error is due to the fact that the images have different sizes on
the file system.
The images are available at: http://webgis.di.uminho.pt/~jgr/QGIS/images.zip
In fact, the images have different sizes on the file system. I don't
know why.
-rw-r--r-- 1 jgr jgr 175786 mai 29 2019 WMS_GetPrint_TwoMaps.png
-rw-r--r-- 1 jgr jgr 176003 fev 5 15:19 WMS_GetPrint_TwoMaps_result.png
-rw-r--r-- 1 jgr jgr 31288 fev 5 15:19
WMS_GetPrint_TwoMaps_result_diff.png
My libpng-dev version is 1.6.34-2ubuntu0.1.
Does any one else have experienced the same problem?
Regards,
J. Gustavo
--
Jorge Gustavo Rocha
Departamento de Informática
Universidade do Minho
4710-057 Braga
Gabinete 3.29 (Piso 3)
Tel: +351 253604480
Fax: +351 253604471
Móvel: +351 910333888
skype: nabocudnosor
More information about the QGIS-Developer
mailing list