[SoC] OSM cgimap GSOC 2013 report #5

[Paul Norman]

Note: A day late, due to odd schedules.

What did I get done this week?

This week I switched focus from cgimap coding to integration testing. To do
this, I had to develop a way to test API responses. I spent awhile looking
at different frameworks for testing that would work on a RESTful API and
allow me to check the contents of XML responses. Because multiple XML
documents may parse the same but be different on a byte-by-byte level I
couldn't do exact comparisons with files, and because there are multiple
ways to represent the same data in OSM XML (e.g. reordering the tag elements
for an object) I could not simply compare XML trees.

I settled on Gatling, a tool originally designed for performance testing. It
has a DSL which is rich enough to test API responses and includes Xpath 1.0
support. Xpath allows me to test conditions like the <osm/> element has one
child, and that child is a node element with id=1234 and that node has two
tags which are a=foo and b=bar.

I am also writing an OSM XML file which contains the test cases which gets
loaded into the API software of choice, and an OSC XML file which then gets
applied. These allow me to check stuff like deleted nodes, moved nodes, etc.


The repository I am using is
https://github.com/pnorman/openstreetmap-api-testsuite

I believe I have completed an adequate test suite for node and nodes + diffs

Next week:

More API tests!

I also need to follow-up on some issues found with apidb, rails port,
existing cgimap code, osmosis, and gatling.

Still outstanding: pgsnapshot changes, raising a plea for a backport of 9.3
postgres index fudge factor changes

Problems

I'm not really sure how much to test in each test. As an example, I have the
"one tag", "multiple tags" and "different tag order" tests at
https://github.com/pnorman/openstreetmap-api-testsuite/blob/master/Node.scal
a#L152-207 which are checking the visible, uid, user, lon, lat, etc of the
nodes the API is emitting. On the other hand, I've already tested that the
API emits those attributes successfully in the earlier "attributes" and
"different attributes" tests.

Feedback from someone more experienced at specifying tests would be nice.

I'm also not sure how closely to define the API responses. For example, the
API wiki page specifies a 410 response for a deleted node. The rails port
will always emit an empty response body with Content-Length: 0 and
Cache-Control: no-cache. Should I be testing these other headers?