Wikipedia, Python, and Maps

Wikipedia is very cool. As of this post (March 19th), the English language wikipedia has about 4.2 million articles written by volunteers. Of these, a little under 1.5 million have geographical information associated with them in the form of geographical coordinates. For example, on the article about UCSC if you look in the upper right hand corner, you’ll see a small bit of text displaying the geographical coordinates. There’s a whole group of people on wikipedia dedicated to adding this information to articles. And they’ve done a great job.

I happened to stumble upon a very useful data dump created by someone with the account name “dispenser” on the wikimedia toolserver. It is a .sql file of all of the geotagged articles in the english language wikipedia (find it here).

So I dumped this file into a mysql database on my computer, and sure enough, I’m able to make queries and get geographical information out!

mysql> select gc_lat, gc_lon, gc_name from coord_enwiki where gc_name = "University of California, Santa Cruz";
+-------------+---------------+--------------------------------------+
| gc_lat      | gc_lon        | gc_name                              |
+-------------+---------------+--------------------------------------+
| 37.00000000 | -122.06000000 | University of California, Santa Cruz |
+-------------+---------------+--------------------------------------+
1 row in set (0.43 sec)

So the next step I thought would be neat was to display all of the geotagged wikipedia articles near where I live (Santa Cruz, California). I chose to use Python because it’s pretty well suited for simple hacks like this, and I can always use the practice. Interfacing Python with MySQL is easy enough using the MySQLdb package, but we have another problem: how to create maps easily in Python. After a very short amount of time searching, I found the answer: basemap.

Basemap makes drawing all sorts of maps in all sorts of projections super simple. So with these tools in hand, I was able to quickly create the following map:

All geotagged articles near Santa Cruz, CA.

Neat! So the next step for me was to try to produce a global map of wikipedia articles. After some experimentation, I found that using a heatmap would work much better than placing points, as the denser parts of the map just end up black, which isn’t very useful.

So after a bit more work I’m left with this:

Global view of English Wikipedia articles.

I’ve posted all of the sourcecode to make the heatmap above, you can find it here.

Chess (and why data is fun)

I’ve been playing chess (and with chess) quite a lot recently. I started up playing online chess again (on chess.com), and I’ve been having quite a lot of fun with it. This quickly moved into playing with chess in Python, and I’ve had quite a lot of fun with that, especially thanks to a nifty chess library I found for python. One of the things I realized is that chess.com makes quite a lot of data available for each user. For example, I can download a .pgn file of the last few weeks of games that I have played (I ended up downloading about 40 chess games that I’ve played — I could get more games but unfortunately I need to pay for a real account if I want this).

This is a lot of data, so I decided to try to do something with it. I ended up producing this:

All of the chess moves that I’ve made in the last 4 weeks.

This represents all of the chess moves made in about 40 chess games I’ve played, coloured by piece. Rooks are gold, kings are purple, pawns are blue, bishops are green, knights are red, and queens are black. I also made a few images representing only certain pieces, here’s one with only the knights:

All of the moves made by knights in my recent games.

Most of the work that I did on this project actually went into producing the graphics rather than interpreting the files. I got the best results (that you see here) by using PYX curves. Each arc that you see in the images represents a move. The degree of the arcs (how wide the are, etc) is randomly generated so that multiple “moves” don’t overlap. This way we can see, that, for example, I like to move my knights to c3 and g3 quite a lot.

Oh, and if you’d like to play chess with me, feel free to challenge me online. I can’t guarantee that I’m any good at chess, though, so it might be a bit embarrassing for me.

Playing with Surface Mount Shift Registers

I’ve been playing for a while with making my own circuit boards with OSH Park. Usually these are reasonably large two sided boards and they typically cost me $8-$25 each. But OSH Park doesn’t have a setup fee or minimum board size. So I decided to see how small of a circuit board I could make.

A bunch of surface mount LEDs sitting on a tiny PCB.

The result is a really simple board that’s pretty fun to solder. The boards came out to something like $2 each, and the parts, not including connectors, were about as cheap (connectors are pricey! This was a surprise for me).

If you couldn’t guess, each PCB is just a breakout board for a 595 surface mount shift register mounted on the backside of the board. Each output pin of the shift register is tied to an LED with an inline resistor.

Here’s a video of these boards being driven by an arduino:

All in all, a really cheap and easy project.

 

Programming External Chips with the MSP Launchpad

This is more of a note to myself than anything else, but here goes.

I’ve been playing around a lot with the MSP430 line of microcontrollers and I’ve been using the MSP Launchpad as a programmer. It’s an exceedingly cheap programmer ($4.30), so it’s pretty fun to play around with. Basically, there’s a socket for a 14 or 20 pin DIP MSP430 with the pins broken out. Pop a chip in, program it, and play. Neat.

But sometimes I develop something where the chip is soldered in and then there’s no good way to pop out the chip to reprogram. Like the pocketwatch I made (actually the chip is socketed, but I’d like to avoid that):

My pocketwatch.

So, to program the MSP430 microcontroller without socketing it into the launchpad, all that needs to be done is the following: remove all of the jumpers from the MSP launchpad. Then connect TEST and RST from the uppermost section of the launchpad to the chip that you’d like to program. Also connect GND to the ground of your design.

That should be it, you’re ready to program. And don’t forget to put a 47kΩ resistor from Vcc to RST on your chip in your design or it won’t boot!

All connected and ready to program

T-Square Fractal Cubes

If you don’t already know, I like fractals a lot.

A few months ago I was playing around with the T-Square fractal. Recently I decided to revisit this fractal and try to add another dimension. So instead of using squares, I decided that I’d use cubes and make the rules the same. For each free corner on a cube, create another cube and repeat.

So I wrote a small program in Python that creates a scene for Povray (speaking of which, I wasn’t able to find any good libraries for talking to Povray in Python! If anyone knows of such a library, I’d love to know.)

Anyway, here are the results:

T-Cube fractal iterated 9 deep

I’d love to make a video out of this but I don’t think my budget can afford it right now. The frame you see above you took about an hour or so to render on my laptop. Unless anyone has a cluster they’re willing to lend me, I’ll have to stick to stills for now.

Update 3/26/12:

I decided to 3d print it. Printed on a 3d powder printer at Makers Factory in Santa Cruz.

In order to get it made into a proper 3d file, I actually ended up writing a Python script for Blender, you can find that here. Turns out that scripting Blender is way easier than writing .STL files — just to write a cube you need to define a bunch of vertices’s and get them all in the right order or the normals are screwed up.

And a 3d print of the fractal with only 6 iterations this time.

Update (again) 4/16/12:

I sent away to Shapeways to get it printed in black plastic.

3d printed fractal from Shapeways

Lasers, Smoke, and Ice

Smoke is really interesting. It’s a gas, so it ends up being very chaotic, especially when turbulent.

I was playing with lasers the other day and had the idea to make two-dimensional a plane of laser light and project it through smoke, effectively visually ‘cutting’ the laser smoke into a slice that I could see. So I attached a mirror with a 45 degree tilt to a computer fan, pointed a 5mW green laser at it, and spun it up.

After searching for the appropriate medium to create smoke (incense didn’t make enough to be very visual), my friends and I settled on using dry ice.

Here’s a picture with the lights on.

Pouring dry ice vapor through a plane of laser light.

And another with the lights out, this time of just a cube of dry ice held above the laser:

Dry ice cube held in a plane of laser light

The whole device was really easy to build (most computer case fans take 12 volts DC and you can get small mirrors at art supply stores). If you do end up making one of these devices, just please remember to be safe around lasers and dry ice. Never look a laser in the eye and never keep dry ice in an airtight container (it will explode).

And I’ll leave you with a video:

Oh, and thanks to Julia for helping and Jon for the photography.

Fun with an HP 7475 Plotter

So I got an HP 7475 Plotter used. I’ve been playing with it using a Python library called Chiplotle.

Recently I’ve been making fractals on it. Here’s a Hilbert curve I made:

Hilbert curve on a plotter

Hilbert curves are space filling fractal curves. This means that when iterated to infinity, at no point is there a straight line; thus it is a curve. Very neat!

Iterated deeper this time

In order to get this to work, I wrote a turtle graphics class for the Chiplotle library. If anyone finds this useful, you can grab the code here.

Oh, and here’s a Sierpinski triangle:

A Sierpinski Triangle

LEDs on a String

Okay, so a friend and I got bored and decided to play with this idea. We attached a few leds to some thin string (we used dental floss), creating a pendulum. Then we turned off the lights, pointed a camera up, opened the shutter for about 25 seconds and released the led:

A 25 second exposure of a single red led.

And then we tried making a double pendulum by adding a second, green, led.

And again, but with two leds to create a double pendulum.

Double pendulums are neat! They’re incredibly complex despite their simple construction, but can be correctly modelled with math.

Here’s our setup for the double pendulum. And remember, if you decide to try this, use a resistor in series with each led.

The double pendulum with the lights on.

Quadcopter Crash

So I’m building a quadcopter. Here’s the project webpage.

A big part of the whole process is learning to fly. As I’ve never flown an rc aircraft before, it’s pretty tricky to learn on something you just designed and built. So, more often than not, I crash.

 

(Don’t worry, no permanent damage was done.)

Now a lot of these crashes are because I lost control or because I just can’t fly. But I think this one was very interesting. If you take a look at the video again, you’ll notice that I was tilting the aircraft back towards me when the rear motor seems to cut out, levelling the craft and eventually crashing it. I’m fairly sure that I had nothing to do with that.

What I think happened is that my quadcopter went into something similar to a deep stall. When the quadcopter is leaning forward too much, the rear rotor is sucking in turbulent air, and unable to provide enough lift.

As the quadcopter moves forward, the rear propeller gets turbulent air and can't provide lift any more.

I’ll be playing around with the thing soon and I hope to reproduce the conditions under which this happened.

Buddhabrot 4-Dimensional Rotation

I’ve been playing with fractals recently. More specifically, the Buddhabrot fractal.

The Buddhabrot set is closely related to the Mandelbrot set:

The mandelbrot set is displayed on the complex plane, where one axis is the real component of (we’ll call it ) and the other is the imaginary component of (a.k.a ).

With the Buddhabrot set, we can add a few more axes. Instead of just using and , we also use the real and imaginary components of as axes. Whereas in the Mandelbrot set, is assumed to be , in our 4D Buddhabrot set, we make these values variable as well.

So instead of the two dimensional image you’re used to seeing of the Mandelbrot set, we’ve instead got 4 axes: and .

Which leads us to our next problem — how do you visualise a 4 dimensional object on a computer? Well, it’s difficult. You obviously can’t just draw an image of it, or even just show a 3 dimensional picture of it. Instead, we show it rotation in 4 dimensional space.

So, here’s the result:

I’ve made all of this code open source — here’s the project page.