Greg Klein's Blog

Archive for the ‘python’ Category

Wikipedia, Python, and Maps

leave a comment »

Wikipedia is very cool. As of this post (March 19th), the English language wikipedia has about 4.2 million articles written by volunteers. Of these, a little under 1.5 million have geographical information associated with them in the form of geographical coordinates. For example, on the article about UCSC┬áif you look in the upper right hand corner, you’ll see a small bit of text displaying the geographical coordinates. There’s a whole group of people on wikipedia dedicated to adding this information to articles. And they’ve done a great job.

I happened to stumble upon a very useful data dump created by someone with the account name “dispenser” on the wikimedia toolserver. It is a .sql file of all of the geotagged articles in the english language wikipedia (find it here).

So I dumped this file into a mysql database on my computer, and sure enough, I’m able to make queries and get geographical information out!

mysql> select gc_lat, gc_lon, gc_name from coord_enwiki where gc_name = "University of California, Santa Cruz";
| gc_lat      | gc_lon        | gc_name                              |
| 37.00000000 | -122.06000000 | University of California, Santa Cruz |
1 row in set (0.43 sec)

So the next step I thought would be neat was to display all of the geotagged wikipedia articles near where I live (Santa Cruz, California). I chose to use Python because it’s pretty well suited for simple hacks like this, and I can always use the practice. Interfacing Python with MySQL is easy enough using the MySQLdb package, but we have another problem: how to create maps easily in Python. After a very short amount of time searching, I found the answer: basemap.

Basemap makes drawing all sorts of maps in all sorts of projections super simple. So with these tools in hand, I was able to quickly create the following map:

All geotagged articles near Santa Cruz, CA.

All geotagged articles near Santa Cruz, CA.

Neat! So the next step for me was to try to produce a global map of wikipedia articles. After some experimentation, I found that using a heatmap would work much better than placing points, as the denser parts of the map just end up black, which isn’t very useful.

So after a bit more work I’m left with this:

Global view of English Wikipedia articles.

Global view of English Wikipedia articles.

I’ve posted all of the sourcecode to make the heatmap above, you can find it here.

Written by gregklein

March 19, 2013 at 4:06 pm

Posted in python

Chess (and why data is fun)

leave a comment »

I’ve been playing chess (and with chess) quite a lot recently. I started up playing online chess again (on, and I’ve been having quite a lot of fun with it. This quickly moved into playing with chess in Python, and I’ve had quite a lot of fun with that, especially thanks to a nifty chess library I found for python. One of the things I realized is that makes quite a lot of data available for each user. For example, I can download a .pgn file of the last few weeks of games that I have played (I ended up downloading about 40 chess games that I’ve played — I could get more games but unfortunately I need to pay for a real account if I want this).

This is a lot of data, so I decided to try to do something with it. I ended up producing this:

All of the chess moves that I've made in the last 4 weeks.

All of the chess moves that I’ve made in the last 4 weeks.

This represents all of the chess moves made in about 40 chess games I’ve played, coloured by piece. Rooks are gold, kings are purple, pawns are blue, bishops are green, knights are red, and queens are black. I also made a few images representing only certain pieces, here’s one with only the knights:

All of the moves made by knights in my recent games.

All of the moves made by knights in my recent games.

Most of the work that I did on this project actually went into producing the graphics rather than interpreting the files. I got the best results (that you see here) by using PYX curves. Each arc that you see in the images represents a move. The degree of the arcs (how wide the are, etc) is randomly generated so that multiple “moves” don’t overlap. This way we can see, that, for example, I like to move my knights to c3 and g3 quite a lot.

Oh, and if you’d like to play chess with me, feel free to challenge me online. I can’t guarantee that I’m any good at chess, though, so it might be a bit embarrassing for me.

Written by gregklein

January 22, 2013 at 4:50 pm

Posted in chess, python

T-Square Fractal Cubes

with one comment

If you don’t already know, I like fractals a lot.

A few months ago I was playing around with the T-Square fractal. Recently I decided to revisit this fractal and try to add another dimension. So instead of using squares, I decided that I’d use cubes and make the rules the same. For each free corner on a cube, create another cube and repeat.

So I wrote a small program in Python that creates a scene for Povray (speaking of which, I wasn’t able to find any good libraries for talking to Povray in Python! If anyone knows of such a library, I’d love to know.)

Anyway, here are the results:

T-Cube Fractal

T-Cube fractal iterated 9 deep

I’d love to make a video out of this but I don’t think my budget can afford it right now. The frame you see above you took about an hour or so to render on my laptop. Unless anyone has a cluster they’re willing to lend me, I’ll have to stick to stills for now.

Update 3/26/12:

I decided to 3d print it. Printed on a 3d powder printer at Makers Factory in Santa Cruz.

In order to get it made into a proper 3d file, I actually ended up writing a Python script for Blender, you can find that here. Turns out that scripting Blender is way easier than writing .STL files — just to write a cube you need to define a bunch of vertices’s and get them all in the right order or the normals are screwed up.

And a 3d print of the fractal with only 6 iterations this time.

Update (again) 4/16/12:

I sent away to Shapeways to get it printed in black plastic.

3d printed fractal from Shapeways

Written by gregklein

February 17, 2012 at 5:03 pm

Posted in fractal, math, python

Tagged with

Fun with an HP 7475 Plotter

with one comment

So I got an HP 7475 Plotter used. I’ve been playing with it using a Python library called Chiplotle.

Recently I’ve been making fractals on it. Here’s a Hilbert curve I made:

Hilbert curve on a plotter

Hilbert curves are space filling fractal curves. This means that when iterated to infinity, at no point is there a straight line; thus it is a curve. Very neat!

Iterated deeper this time

In order to get this to work, I wrote a turtle graphics class for the Chiplotle library. If anyone finds this useful, you can grab the code here.

Oh, and here’s a Sierpinski triangle:

A Sierpinski Triangle

Written by gregklein

January 16, 2012 at 9:05 pm

Posted in fractal, math, python