Since cloud is the buzzword of the month I decided to have a look at TileCache’s support for Amazons cloud storage S3. Since TileCache version 2 the whole architecture is much more modular. It is easy to write new backends for different kind of caches. One of those backends is AWSS3 (Amazons S3 storage service).
For EduGIS we figured that it might be easier and more scalable to serve all tiles from S3, using the bandwidth and storage space of amazon, instead of buying more servers and bandwidth ourselves. Already we use tilecache in a test setup and the initial idea was to replace the diskcache for an AWSS3 cache. This was we could tap into the vast storage space of amazon, however, since every request had to be forwarded to S3 and back, the bandwidth-load was doubled. So instead of hosting the TileCache on our server we moved the TileCache instance to a virtual computer on Amazons EC2 platform.
To do this, I picked a sample machine-image of a small, headless, ubuntu 8.10 setup. Installed tilecache (apt-get install tilecache 
) and changed it to use mod_python instead of the default cgi in apache. Since this means that the machine-image is changed, you need to bundle it into a new machine image and register it with amazon. this way you can easily create multiple instances, also if you powerdown your machine it gets destroyed so you loose all your changes.
So once I mastered the art of cloud-herding I pointed our test environment to TileCache in the cloud to see if it would work at all and how quick the bill would increase. After a few days and some seedings I noticed that the AWSS3 backend uses a simple dump in a single folder backend. This quickly produces insane amounts of files and it becomes impossible to have a quick look at the status of your cache with tools like S3fox. So I wrote a patch for the AWSS3 cache module to store the tiles in a TMS style folder structure.
Now all we need to do is to create a tilecache.cfg with all 200+ layers we have and start seeding.
This post is tagged