[Tilecache] make dir error

[Fredrik Lundh]

Christopher Schmidt <crschmidt at metacarta.com> wrote:

>  And they both get called at the same time, you could get a condition
>  where:
>
>   1: /0 is created
>   2: tries to make /0, bails
>   1. Successfully makes /0/0/0/0/0/0 whatever
>   2. Has no /1 directory, writing file fails
>
>  Am I wrong here, or is this a possibility?

makedirs starts by attempting to create the full directory, and then
works backwards if that doesn't work -- so the one that gets to the
topmost directory first will create it, and the other one will get an
EEXIST error, which is handled internally for intermediate
directories.

but maybe I'm missing something.

there's another potential collision here as well, since you're using
pid numbers to distinguish multiple processes, which isn't reliable in
Stefan scenario (multiple machines sharing a cache over NFS).
intuitively, PID collisions should be pretty unlikely, but I've
learned not to trust my intuition when it comes to race problems ;-)

Stefan, does the attached patch (untested) change the behaviour in any way?

</F>

--- Disk.py.bak Thu Apr 10 17:27:09 2008
+++ Disk.py     Thu Apr 10 18:05:32 2008
@@ -1,7 +1,9 @@
 # BSD Licensed, Copyright (c) 2006-2007 MetaCarta, Inc.

 from TileCache.Cache import Cache
-import sys, os, time
+import sys, os, time, errno, socket
+
+hostname = socket.gethostname()

 class Disk (Cache):
     def __init__ (self, base = None, umask = '002', **kwargs):

@@ -64,7 +70,7 @@
         dirname  = os.path.dirname(filename)
         if not self.access(dirname, 'write'):
             self.makedirs(dirname)
-        tmpfile = filename + ".%d.tmp" % os.getpid()
+        tmpfile = filename + ".%s.%d.tmp" % (hostname, os.getpid())
         old_umask = os.umask(self.umask)
         output = file(tmpfile, "wb")
         output.write(data)