Greg Klein's Blog

Wikipedia, Python, and Maps

leave a comment »

Wikipedia is very cool. As of this post (March 19th), the English language wikipedia has about 4.2 million articles written by volunteers. Of these, a little under 1.5 million have geographical information associated with them in the form of geographical coordinates. For example, on the article about UCSC if you look in the upper right hand corner, you’ll see a small bit of text displaying the geographical coordinates. There’s a whole group of people on wikipedia dedicated to adding this information to articles. And they’ve done a great job.

I happened to stumble upon a very useful data dump created by someone with the account name “dispenser” on the wikimedia toolserver. It is a .sql file of all of the geotagged articles in the english language wikipedia (find it here).

So I dumped this file into a mysql database on my computer, and sure enough, I’m able to make queries and get geographical information out!


mysql> select gc_lat, gc_lon, gc_name from coord_enwiki where gc_name = "University of California, Santa Cruz";
+-------------+---------------+--------------------------------------+
| gc_lat      | gc_lon        | gc_name                              |
+-------------+---------------+--------------------------------------+
| 37.00000000 | -122.06000000 | University of California, Santa Cruz |
+-------------+---------------+--------------------------------------+
1 row in set (0.43 sec)

So the next step I thought would be neat was to display all of the geotagged wikipedia articles near where I live (Santa Cruz, California). I chose to use Python because it’s pretty well suited for simple hacks like this, and I can always use the practice. Interfacing Python with MySQL is easy enough using the MySQLdb package, but we have another problem: how to create maps easily in Python. After a very short amount of time searching, I found the answer: basemap.

Basemap makes drawing all sorts of maps in all sorts of projections super simple. So with these tools in hand, I was able to quickly create the following map:

All geotagged articles near Santa Cruz, CA.

All geotagged articles near Santa Cruz, CA.

Neat! So the next step for me was to try to produce a global map of wikipedia articles. After some experimentation, I found that using a heatmap would work much better than placing points, as the denser parts of the map just end up black, which isn’t very useful.

So after a bit more work I’m left with this:

Global view of English Wikipedia articles.

Global view of English Wikipedia articles.

I’ve posted all of the sourcecode to make the heatmap above, you can find it here.

Written by gregklein

March 19, 2013 at 4:06 pm

Posted in python

Chess (and why data is fun)

leave a comment »

I’ve been playing chess (and with chess) quite a lot recently. I started up playing online chess again (on chess.com), and I’ve been having quite a lot of fun with it. This quickly moved into playing with chess in Python, and I’ve had quite a lot of fun with that, especially thanks to a nifty chess library I found for python. One of the things I realized is that chess.com makes quite a lot of data available for each user. For example, I can download a .pgn file of the last few weeks of games that I have played (I ended up downloading about 40 chess games that I’ve played — I could get more games but unfortunately I need to pay for a real account if I want this).

This is a lot of data, so I decided to try to do something with it. I ended up producing this:

All of the chess moves that I've made in the last 4 weeks.

All of the chess moves that I’ve made in the last 4 weeks.

This represents all of the chess moves made in about 40 chess games I’ve played, coloured by piece. Rooks are gold, kings are purple, pawns are blue, bishops are green, knights are red, and queens are black. I also made a few images representing only certain pieces, here’s one with only the knights:

All of the moves made by knights in my recent games.

All of the moves made by knights in my recent games.

Most of the work that I did on this project actually went into producing the graphics rather than interpreting the files. I got the best results (that you see here) by using PYX curves. Each arc that you see in the images represents a move. The degree of the arcs (how wide the are, etc) is randomly generated so that multiple “moves” don’t overlap. This way we can see, that, for example, I like to move my knights to c3 and g3 quite a lot.

Oh, and if you’d like to play chess with me, feel free to challenge me online. I can’t guarantee that I’m any good at chess, though, so it might be a bit embarrassing for me.

Written by gregklein

January 22, 2013 at 4:50 pm

Posted in chess, python

Playing with Surface Mount Shift Registers

leave a comment »

I’ve been playing for a while with making my own circuit boards with OSH Park. Usually these are reasonably large two sided boards and they typically cost me $8-$25 each. But OSH Park doesn’t have a setup fee or minimum board size. So I decided to see how small of a circuit board I could make.

A bunch of surface mount LEDs sitting on a tiny PCB.

The result is a really simple board that’s pretty fun to solder. The boards came out to something like $2 each, and the parts, not including connectors, were about as cheap (connectors are pricey! This was a surprise for me).

If you couldn’t guess, each PCB is just a breakout board for a 595 surface mount shift register mounted on the backside of the board. Each output pin of the shift register is tied to an LED with an inline resistor.

Here’s a video of these boards being driven by an arduino:

All in all, a really cheap and easy project.

 

Written by gregklein

October 16, 2012 at 11:41 pm

Posted in arduino

Programming External Chips with the MSP Launchpad

leave a comment »

This is more of a note to myself than anything else, but here goes.

I’ve been playing around a lot with the MSP430 line of microcontrollers and I’ve been using the MSP Launchpad as a programmer. It’s an exceedingly cheap programmer ($4.30), so it’s pretty fun to play around with. Basically, there’s a socket for a 14 or 20 pin DIP MSP430 with the pins broken out. Pop a chip in, program it, and play. Neat.

But sometimes I develop something where the chip is soldered in and then there’s no good way to pop out the chip to reprogram. Like the pocketwatch I made (actually the chip is socketed, but I’d like to avoid that):

My pocketwatch.

So, to program the MSP430 microcontroller without socketing it into the launchpad, all that needs to be done is the following: remove all of the jumpers from the MSP launchpad. Then connect TEST and RST from the uppermost section of the launchpad to the chip that you’d like to program. Also connect GND to the ground of your design.

That should be it, you’re ready to program. And don’t forget to put a 47kΩ resistor from Vcc to RST on your chip in your design or it won’t boot!

All connected and ready to program

Written by gregklein

March 4, 2012 at 5:58 pm

Posted in msp430

Tagged with ,

T-Square Fractal Cubes

with one comment

If you don’t already know, I like fractals a lot.

A few months ago I was playing around with the T-Square fractal. Recently I decided to revisit this fractal and try to add another dimension. So instead of using squares, I decided that I’d use cubes and make the rules the same. For each free corner on a cube, create another cube and repeat.

So I wrote a small program in Python that creates a scene for Povray (speaking of which, I wasn’t able to find any good libraries for talking to Povray in Python! If anyone knows of such a library, I’d love to know.)

Anyway, here are the results:

T-Cube Fractal

T-Cube fractal iterated 9 deep

I’d love to make a video out of this but I don’t think my budget can afford it right now. The frame you see above you took about an hour or so to render on my laptop. Unless anyone has a cluster they’re willing to lend me, I’ll have to stick to stills for now.

Update 3/26/12:

I decided to 3d print it. Printed on a 3d powder printer at Makers Factory in Santa Cruz.

In order to get it made into a proper 3d file, I actually ended up writing a Python script for Blender, you can find that here. Turns out that scripting Blender is way easier than writing .STL files — just to write a cube you need to define a bunch of vertices’s and get them all in the right order or the normals are screwed up.

And a 3d print of the fractal with only 6 iterations this time.

Update (again) 4/16/12:

I sent away to Shapeways to get it printed in black plastic.

3d printed fractal from Shapeways

Written by gregklein

February 17, 2012 at 5:03 pm

Posted in fractal, math, python

Tagged with

Lasers, Smoke, and Ice

with 6 comments

Smoke is really interesting. It’s a gas, so it ends up being very chaotic, especially when turbulent.

I was playing with lasers the other day and had the idea to make two-dimensional a plane of laser light and project it through smoke, effectively visually ‘cutting’ the laser smoke into a slice that I could see. So I attached a mirror with a 45 degree tilt to a computer fan, pointed a 5mW green laser at it, and spun it up.

After searching for the appropriate medium to create smoke (incense didn’t make enough to be very visual), my friends and I settled on using dry ice.

Here’s a picture with the lights on.

Pouring dry ice vapor through a plane of laser light.

And another with the lights out, this time of just a cube of dry ice held above the laser:

Dry ice cube held in a plane of laser light

The whole device was really easy to build (most computer case fans take 12 volts DC and you can get small mirrors at art supply stores). If you do end up making one of these devices, just please remember to be safe around lasers and dry ice. Never look a laser in the eye and never keep dry ice in an airtight container (it will explode).

And I’ll leave you with a video:

Oh, and thanks to Julia for helping and Jon for the photography.

Written by gregklein

January 26, 2012 at 1:53 pm

Posted in math

Fun with an HP 7475 Plotter

with one comment

So I got an HP 7475 Plotter used. I’ve been playing with it using a Python library called Chiplotle.

Recently I’ve been making fractals on it. Here’s a Hilbert curve I made:

Hilbert curve on a plotter

Hilbert curves are space filling fractal curves. This means that when iterated to infinity, at no point is there a straight line; thus it is a curve. Very neat!

Iterated deeper this time

In order to get this to work, I wrote a turtle graphics class for the Chiplotle library. If anyone finds this useful, you can grab the code here.

Oh, and here’s a Sierpinski triangle:

A Sierpinski Triangle

Written by gregklein

January 16, 2012 at 9:05 pm

Posted in fractal, math, python

Follow

Get every new post delivered to your Inbox.