Beginning the day

There were a couple of blog articles I had wanted to read over the last couple of days that I just hadn’t had time to get to. First up, Redis: Data Cheeseburgers. I’m really intrigued by the idea of serializing data in non-relational formats (this from the guy in the office that writes mathematical relational algebra out on the white board) to store data. I’ve been playing around in my spare time with MongoDB and the mongomapper wrapper in rails, but have been hearing a lot of good things about Redis.

One of the most interesting aspect of these key-value store approaches, especially from a digital humanities perspective, is it allows you to concentrate more of the object model without worrying so much about the gotchas of how an object gets translated into SQL. If you change your model, it just keeps working, you don’t need to go back and disassociate any foreign-key constraints, run a migration, and remember to put the key constraints back (because that’s always done perfectly with no orphaned children, right?).

The ThoughtBot article doesn’t go too deeply into what Redis can do (or why people would want to use it, other than to say that Craigslist uses it), but it does provide some useful examples of what the platform is capable of accomplishing. If you want to try Redis out (without installing), check out Try Redis.

Murphy Strikes!

Dave Lester was down visiting the Scholars’ Lab the last few days to work on his HistoryPlot web application. After a few discussions, I thought this would be a really cool (and relatively painless) to do with Rails + LocalSolr since he’d already pulled the coordinate locations for all the places in his data. The first issue we ran into this morning was getting his system upgraded (Rails, Ruby, gems, mysql, etc.). We worked through most of those problems, but still couldn’t get the mysql gem to compile properly. As a work around, we just used sqlite (just so we could keep working).

Next came getting LocalSolr up and running. That took a few tweaks to actually get working (their instructions were a bit off, and the stack trace a bit misleading), but we got that up and running after a bit of tinkering.

The big project is actually getting the data out of Omeka in to this different format. Omeka has a very robust metadata infrastructure and we needed to convert that to our Place object in Rails. Because of the way Omeka structures its fields, an item record will have as many row associated with it as it has metadata fields; basically we needed to shift the axis of the data so each object has n columns and not n rows. Dave was working on some PHP to convert this before he left…