Click here to view the embedded video.

However modern browsers have the <canvas> element. This element allows for the manipulation of single pixels within this element. So I figured if I could encode the building-dates in a PNG and use canvas to display only those pixels which represent a building older than the given date it should be possible to time-slide through the buildings. Fortunately the fancy new mapping library Leaflet.js has a canvas tile layer build in. The BAG data is already available through EduGIS, so I only needed it to encode the data differently in the PNGs.

The encoding is very simple: per pixel there are 4 values available: red, green, blue and alpha. Since I only needed to encode 200-odd values I used just the red value. The years before 1850 are encoded as groups, since the data is so sparse, after 1850 each year is individually encoded. This means that from 1850 onwards the red value increases with 1. The client retrieves the encoded PNGs as normal tiles and look like this:

The image appears grey because I kept the green and blue values of the PNG the same as the red. This image is loaded into canvas and the imageData is retrieved using ctx.getImageData(0, 0, 256, 256) and stored as a jQuery data object on the canvas. This is important, since for visual effect we will manipulate the imageData on the canvas and we want to keep track of the original values. Once the imageData is attached to the canvas the colors are being calculated. It will take the original values, compares them to the current year and will decide whether or not to show the pixel and in which color.

Since only the grey tile is needed, the actual sliding through time is really fast because it doesn’t need to retrieve more data. With 4 bands of 255 values each you can encode an insane amount of data into a PNG, readily available through canvas for direct manipulation. Apart from time sliding, is detailed representation of DEM data an obvious use case.

However a map consists of more than points. So I wrote a simple Perl script to transform the coordinates of a GeoJSON into Dymaxion. It should find any coordinate pairs in Point, LineString, Polygon, MultiPoint, MultiLineString and MultiPolygon features and replace the pairs with dymaxion ones. These new pairs do not make a lot of sense in normal GIS software, but exporting them to SVG will give you a nice Dymaxion map. GeoJSON is not a common format (yet!?) but it is easy to create form a shapefile using ogr2ogr.

So the steps to follow are:
1 convert your input shapefile to json:
ogr2ogr -f GeoJSON processed.json input.shp

2 dymaxionify your json:

./dymyx.pl

3 load the json in qgis

qgis output.json

4 export the map as SVG

unfortunately I don’t know a comandline for that, but if you use the print composer tool, and compose a map you can save that as SVG. There might be easier ways, but haven’t found them yet.

For example I ran the 50m-coastlines and created a new dymaxion coastline SVG as seen here

Note: it doesn’t handle encoded characters as it should aka a bug

OSM coverage of Berlin over time

I was asked to design a cover for the thesis of our intern working on crowd quality in OSM. This image is a modification of the nice animations GeoFabrik produce on demand, in this case Berlin. Since it needs to be printed I couldn’t use the animation itself. Time moves within the image from left to right. You see that early in OSM-history there was just one highway and over time the data becomes more complete and hopefully more accurate.

Click here to view the embedded video.

The Surface has been hanging around in our office for a while and we’ve done lots of interesting small and big projects for it. However they’ve been pretty fragmented and to show what we could do with the Surface you had to use several applications which had many similarities, which was confusing. So we decided that it was time to create an application which would show all our new ideas and developments. Looking for a nice name we came across ‘kõik‘ which both means everyone and every(thing), which is pretty much sums up what we want to achieve: an application which can be used by everyone and shows everything we do.

It was originally designed for landuse planning. Planners need to be able to move and zoom a map, but also annotate on it. The trouble with touch devices is that there is no difference between drawing and dragging, unlike keyboard/mouse (drawing is left mousebutton and dragging middle for instance). Having to press a button somewhere on the Surface to enable/disable drawing was too confusing, people tend to forget that they pressed them and in what modus they are now.

Having a physical object on the table turned out to be much easier to use and remember. If there was ‘the’ object on the table, you could draw and if you want to move the map, you just lift the object and move the map. Having established the object as the drawing-toggle, I then moved all drawing related interaction to a menu activated and related to the object. Currently you can choose the color and line thickness and whether you want to draw a line or polygon or you want to edit existing features, undo/redo actions and add new drawing layers.

(Image taken from the presentation I’m giving at foss4g 2010)

I’m involved in EduGIS and one of the nice things about this project is that we have hundreds of map layers. However that begins to pose a problem for us; to produce a decent speed for classes full of schoolkids using it at the same time we are using tiling.

We currently use TileCache; a very nice and quick open source tilecaching program. However we keep adding layers and we need those to be quick when students are using them, but we don’t have the capacity to prerender (seed) everything, nor to do this on the fly. Since we mostly have dutch data, served at dutch students, we figured that we only need to seed a certain set of tiles; the entire country for the higher levels and the rural areas a bit deeper and the urban areas even deeper.

So I came up with the idea to create a desktop application to make it easier to create a complex seed that does more zoomlevels for certain areas and less for others.

Thanks to generous development time invested by Christian there is a first alpha available around foss4g at googlecode

At least that is the theory Martijn tries to work with.

The age in days for OpenStreetMap nodes

You can see a clear pattern emerging here, apparently there are areas which get mapped in a short period of time and never touched again. The white squares just didn’t have any nodes in it, so they contain no data.

The version of OSM nodes

It is more difficult to see patterns here. Whereas with age in days the number gives you an instant grasp of the meaning, the version number is less obvious. We probably need to do a statistical analysis of version numbers in OSM. This way we can attach some (relative) meaning to the average version number.

The red circle is the final location of the node and the path is the order of changes. You can see that the first changes were much more rough than the last ones.

I also did the same for an entire way, where the way itself is shown in blue and the versions of the nodes are shown next to the nodes.

The trouble is that it is currently quite difficult to get the full (spatial) history of an area in OpenStreetMap. Once we find a way to do so I hope to generate more and play a bit with different visualization styles and methods.

Click here to view the embedded video.

But the minimum interval is 30 seconds, making it useless for timelapses of people moving about. Also the quality of the sensor and the lens is not as good as my dSLR. With gTimelapse I can control the interval to anything I like. gTimelapse is rather new so there’s a bit of fiddling involved, to be able to build it under ubuntu 9.10 you need these packages:

sudo apt-get install build-essential libgphoto2-2-dev libwxbase2.8-dev libwxgtk2.8-dev

From then on it is pretty straight forward, connect your camera (make sure it is in PTP mode) also ubuntu nowadays mounts it automagically, gTimelapse doesn’t like that. So unmount the camera using the filemanager and start gTimelapse. Depending on your camera you get all kinds of settings you can fiddle with. (side note I discovered that if you first start gTimelapse and than attach the camera none of the settings appear, but the application still works). Once you’re happy you can press start and sit back.

Creating a movie is still a lot more complicated I’m afraid. I still haven’t found a decent movie editor for linux. Kino is nice but only supports DV camera’s as far as I can tell, dragging something non-DV gives an eternal Kino Import progress bar kdenlive looks promising, but it kept freezing so I gave up on it for the moment. To just get a short movie I threw mencoder at them:

mencoder -nosound -ovc lavc -lavcopts vcodec=mpeg4:mbd=2:mv0:trell:v4mv:cbp:last_pred=3:predia=2:dia=2:vmax_b_frames=2:vb_strategy=1:precmp=2:cmp=2:subcmp=2:preme=2:qns=2 -o test5.avi -mf type=jpeg:fps=5 mf://@files.txt

Where files.txt was a list of the filenames of all the images I wanted to include. This resulted in this movie:

Click here to view the embedded video.

This means it possible to create timelapse movies with linux, but postprocessing is still difficult. The next timelapse project I’ll probably shoot with linux and edit on my mac

Click here to view the embedded video.

To visualize these tracks I used Processing, ‘an open source programming language and environment for people who want to program images, animation, and interactions.’  There are lots of plugins to read different data formats, an interesting one for this project is the gpx library. The plan was to render the tracks of each person in a different color, tracks which happened at the same time should be drawn at the same time. This way you can see which people are at the same time at the same location. Each track would fade out in time, leaving a thin line for past tracks.

To do this on a track-basis however turned out to be quite difficult for my limited knowledge of Processing and the time I had available. So instead of using the gpx module I created a flat table with all locations sorted by time. Each row of the table contains lon,lat,color-id,time. The program iterates through all 42000+ rows and draws the locations as a dot. The size of the dot depends on how ‘old’ the point is. The last 200 points are drawn with a diminishing size, creating the worm-like effect in the animation. Note that it doesn’t take into account the actual time between points, just if it is next in line.

A few tweaks were added: Holland in unprojected lat-lon looks odd to dutch people, so I implemented a simple projection library to reproject the points to a semi-RD (dutch projection). Also I wanted to dynamically zoom to the action (not everything was happening in Amsterdam). I did this by calculating the minimum and maximum X and Y for the last 200 points. I wanted the location of our office building to be in view all the time, to provide a point of reference.

The source code can be found here if you are really interested. There are three files: Table.pde as written by Ben Fry for his book Visualizing Data, Soprojection.pde, written by Steven Fruijtier at my request and Drawing.pde is hacked together by me. This is a very crude and brute-force approach to render a series of tracks, so please use it with care. Since it needs to recalculate each point for each frame it tends to become slower as it advances through the table.

• Geoserver 2.0 (SNAPSHOT downloaded 8 december 10.19UTC)
• Mapserver 5.6.0 (MS4W Base installer v3.0 Beta 7 + MapServer 5.6.0 release Upgrade)
• Deegree 2.3rc1 (Apache Tomcat 5.5.28)

To find out what the problem exactly was I’ve done some testing. I’ve taken a GeoTiff and configured all three the WCS applications to serve it. Gdalinfo gives us the following information, basically it is a GeoTIFF in epsg:3035 with a native resolution of 100m/pixel.

My test sequence was:

1. request the full extent in native resolution
2. request the full extent in a 10th of the resolution
3. request the full extent with the same size as the original data (256×256)
4. request part of the data in native resolution
5. request part of the data in non-native, non-multiple resolution

I’m not going to show all the results here, because that would result in an unhealthy long document

To verify the results I loaded the original GeoTiff in Quantum GIS and highlighted a specific feature to show the possible offset of the different results.

Corine GeoTiff

1  request the full extent in native resolution:

deegree: http://localhost:8080/deegree-wcs1/services?service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&resx=100&resy=100&format=GeoTIFF gives a three band(!) GeoTiff of 262,674 bytes with a shift of around 40m:

Deegree result

MapServer: http://localhost/cgi-bin/mapserver56/mapserv.exe?map=maps/corine.map&service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&resx=100&resy=100&format=GTiff gives a three band(!) GeoTiff of 197,477 bytes without a shift:

Mapserver result

GeoServer: http://localhsot:8081/geoserver/ows?service=WCS&version=1.0.0&request=GetCoverage&coverage=corine_merge_gtiff_proj_121_108&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&resx=100&resy=100&format=GeoTiff gives a single band GeoTiff of 8,848 bytes with a shift of  around 50m:

Geoserver result

Note:

Deegree both shifts the data and returns a multi-band image; Mapserver doesn’t shift the data, but returns a multi-band image; Geoserver shifts the data 1/2 a pixel (50m), but does give a single band image.

2. request the full extent in a 10th of the resolution

deegree: http://localhost:8080/deegree-wcs1/services?service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&resx=10&resy=10&format=GeoTIFF produces a 26,237,726 multi band GeoTiff which shifts the data the other way than the first request:

Deegree result on a tenth of the resolution

Mapserver: http://localhost/cgi-bin/mapserver56/mapserv.exe?map=maps/corine.map&service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&resx=10&resy=10&format=GTiff produces a multi-band GeoTiff of 19,681,941 bytes with no shift.

Geoserver: http://localhost:8081/geoserver/ows?service=WCS&version=1.0.0&request=GetCoverage&coverage=corine_merge_gtiff_proj_121_108&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&resx=10&resy=10&format=GeoTiff gives a single band GeoTiff of 158,802 bytes with a shift of around 50m, though there is a small difference with the previous result.

Note:

Mapserver and Geoserver both produce a bigger result (as expected) with the same location as the previous request. Deegree shifts the data differently than previously and also produces a bigger result.

3. request the full extent with the same size as the original data (256×256)

geoserver: http://localhost:8081/geoserver/ows?service=WCS&version=1.0.0&request=GetCoverage&coverage=corine_merge_gtiff_proj_121_108&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&width=256&height=256&format=GeoTiff produces exactly the same image as in the first request.

mapserver: http://localhost/cgi-bin/mapserver56/mapserv.exe?map=maps/corine.map&service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&width=256&height=256&format=GTiff produces exactly the same image as in the first request.

deegree: http://localhost:8080/deegree-wcs1/services?service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&crs=EPSG:3035&BBOX=3539200,3402400,3564800,3428000&width=256&height=256&format=GeoTIFF produces a slightly smaller image than the first request (262,674 bytes) with no shift!.

Deegree result with width en height

Note:

Since the request is a different way of describing the same data one would expect that you get the same results. Mapserver and Geoserver do so, Deegree does not, somehow using resx and resy in the request results in shifted data.

4. request part of the data in native resolution

deegree: http://localhost:8080/deegree-wcs1/services?service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&resx=100&resy=100&crs=EPSG:3035&BBOX=3550000,3413200,3554000,3417200&format=GeoTIFF has the correct left-top location but the size of the pixels is wrong, producing a shift of the data which increases as you go right/down:

Deegree result for a part request

geoserver: http://localhost:8081/geoserver/ows?service=WCS&request=GetCoverage&format=GeoTiff&version=1.0.0&coverage=geodan:Corine&resx=100&resy=100&crs=EPSG:3035&BBOX=3550000,3413200,3554000,3417200 produces an image with almost exactly the same shift as the first request.
mapserver: http://localhost/cgi-bin/mapserver56/mapserv.exe?map=maps/corine.map&service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&resx=100&resy=100&crs=EPSG:3035&BBOX=3550000,3413200,3554000,3417200&format=GTiff produces an image with almost exactly the same location as the first request.

Note:

Mapserver and Geoserver shift (or not) the data similar to the first request, Deegree has once again a different shift than any of the previous ones.

5. request part of the data in non-native, non-multiple resolution

deegree: http://localhost:8080/deegree-wcs1/services?service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&resx=30.0&resy=30.0&crs=EPSG:3035&BBOX=3550000,3413200,3554000,3417200&format=GeoTIFF shifts data:

geoserver: http://localhost:8081/geoserver/ows?service=WCS&request=GetCoverage&format=GeoTiff&version=1.0.0&coverage=geodan:Corine&resx=30.0&resy=30.0&crs=EPSG:3035&BBOX=3550000,3413200,3554000,3417200 shifts data and adds a black border to the left and top of the result:

Geoserver result for non-native, non-multiple resolution

mapserver: http://localhost/cgi-bin/mapserver56/mapserv.exe?map=maps/corine.map&service=WCS&version=1.0.0&request=GetCoverage&coverage=Corine&resx=30.0&resy=30.0&crs=EPSG:3035&BBOX=3550000,3413200,3554000,3417200&format=GTiff shifts data:

Mapserver result for non-native, non-multiple resolution

Note:

This request was my initial prompt to find out why all WCS applications shift the data in the results. However this is a false conclusion. What’s happening here is that the service has to extract 30m wide pixels from 100m wide source data, since 100 divided by 30 isn’t a whole number it will add the remaining 10m in an adjacent pixel. If you overlay the result and the source data you get a pixel problem, data of source pixel 2 will most likely end up in result pixel b, which is partly over source pixel 1, resulting in a data shift:

Overlapping 100m with 30m

Conclusion:

Geoserver has a bug feature which offsets all the results by half a pixel, this is a known issue with the definition of the location of a pixel. Added to this there’s the no-data border which appears with non-native, non-multiple requests. I presume that will be gone once the pixel issue is resolved. (update: apparently I’ve misunderstood the Geoserver developers; there’s an issue between pixel placement according to standards and real world implementations, where Geoserver adheres to the standard).

Mapserver doesn’t offset the data unless it is physically impossible (non-native, non-multiple resolutions, extents which don’t snap to source data) but produces a multi-band (this was a configuration issue, thanks to FrankW for spotting this – proper config is IMAGEMODE BYTE) geotiff where the source data is single band.

Deegree has a bug which offsets some of the results, but I don’t know what causes it and although it is resolution-related I don’t see a pattern. It also produces a multi-band geotiff instead of a single band.

Configuration files and source data can be found here.