The Aboutbox

We build Web, Search and Video applications.

RubyConf Wrap-Up

Since RubyConf was in Denver this year, it would have been crazy not to go. Last time I went to RubyConf, it was 2009, and it’s amazing how much the community has changed since then.

I remember listening to a presentation on some exotic Erlang-based message-passing system, and the speaker, from Github, referred to Java programmers as ‘Drones’. Perhaps the Ruby community was still in phase where most of the members where ‘enterprise refugees’ and we were determined to burn down everything Enterprise, and remake it in a more Agile, more Flexible image.

Three years passes. There have been a few major Rails releases. There are a lot of people now who have these giant ‘Monorail’ applications, and now they are looking for solutions for some of these scaling problems, and if there is one thing the Enterprise knows, it’s how to operate at scale. It’s as if all of us hippies cut our hair, went to college, started families and are getting some real jobs. We need to look to the Enterprise for wisdom, while hopefully avoiding some of the mistakes the previous generation made. Almost every talk gushed on the awesomeness of JRuby and the JVM, about which I totally agree. Many talks talked about breaking apart large apps into services, and ways of structuring multi-machine architectures, so it’s still possible to test quickly and deploy quickly.

There was also a focus on monitoring, gathering data and using it to track usage patterns and find problems.

Here is a round-up of some of the awesome things that I discovered.

  • T, the command-line twitter client - A twitter client with truly deep karate searching through twitter streams.

  • Gitnesse, running Cucumber Features from a Git Wiki Part of the “sell” for Cucumber is that you write feature descriptions English, so stakeholders can understand or write the stories. If that’s the goal, why are we keeping the stories in the same place as the source code? Wouldn’t a wiki be better?

  • Activesupport notifications The most important feature of Rails that I didn’t know anything about. A really powerful way to log lots of events, which can then be captured, aggregated and presented by other tools.

  • Metriks - A lot of people sung the praises of Metriks as a means of capturing metrics from an application for publication to another piece of software, such as graphite or statsd.

  • Metriksd - A server that can capture and aggregate metrics

  • Graphite - Front-end and back-end system to store time-series data

  • d3.js - Javascript library to visualize data

  • Cube Cubism - Projects from Square that collect and present time-series data.

  • System-metrics - A lo-fi system for collecting metrics

  • Issue Triage - A system Heroku put together to help triage bugs and issues in open source projects

  • mRuby - Minimalistic subset of ruby designed to run on a very low size and memory footprint

  • MobiRuby - Runtime for iOS devices based on MobiRuby

  • @iDoHaiku - Neural network that creates a new haiku every day.

  • metro - Taking the Rails approach to developing 2d games with Gosu

Migrating Part of a Subversion Repository

Recently I needed to take part of a monolithic Subversion repository, and create a new, smaller repository, while keeping all of the history intact. For all intents and purposes, I needed to move:

https://svn.example.com/lots_of_projects/project_a

to its own repository rooted at

https://svn.example.com/project_a

Oh, and to make matters more fun, I don’t have admin access to any of these repos.

Import/Export

The easy way to copy things from one repository to another is do an svn export followed by an svn import to the new repository. This works, but you lose all of the history, which is a bummer.

Git-Svn

Another possibility to keep all of the revisions intact is to use git-svn to clone the subversion repository with history and then use it as a basis to populate a new subversion repository. I haven’t tried it, but something like this would probably work:

git svn clone [args] [subversion repo] temp-repo

Follow the instructions here on Google Code but when you fetch, fetch from the temp-repo you just created.

One nice thing about this approach is that it requires nothing on the subversion server side to work. The downside is that you will probably lose dates and user information in the history.

Sync + Filter

Probably the most robust way to do this is without physical access to the svn server is to create a local repository, mirror the source repository, filter out everything that is NOT what you want to copy, and then sync that with the destination repo. There are a few steps involved and it’s better to actually get access to both repositories (then you can just use Subversion’s admin tools) to get everything, but in the absence of access to the servers, then svnsync is the next best thing to get a repository’s contents, with history.

svnadmin create temp-repo
cd temp-repo/hooks
echo "exit 0" > pre-revprop-hook
chmod a+x pre-revprop-hook
cd ../..
svnsync initialize file://[path-to-temp-repo] https://svn.example.com/lots_of_projects
svnsync synchronize file://[path-to-temp-repo]
(Time passes)
svnadmin dump ./temp-repo > everything.dump
cat everything-dump | svndumpfilter include project_a --drop-empty-revs --renumber-revs > project-a-dump
svnadmin create project_a; svnadmin load project_a < project_a_dump
svnsync initialize https://svn.example.com/project_a file://[path-to-project_a-repo]
svnsync synchronize https://svn.example.com/project_a

In essence, what we are doing is creating a local mirror copy of the big repo that we can play with. Once we have a mirror that we can manipulate, we can dump its history and filter out only the project that we are interested in. Once we have that, we create a new local repo with only those commits in it, and synchronize it with its new home.

The only problem with this approach is that there needs to be a pre-revprop-hook installed on the target repository, which will require some cooperation from the administrator of that repository.

If anyone else knows of a way to copy part of a subversion repository with all of its history, I would love to hear it.

Newly Discovered Things That Rock

In the past few days I’ve run across a few tools that have been very, very handy.

First is virtualhost.sh. This is a shell script that sets up virtual hosts using the Mac OS X built-in Web server. I’ve been working on a Wordpress project for one of my clients, where they have a bunch of plugins that implements part of their Web site, but they do not provide a seamless user experience. Basically they want their customers, when they sign up for a membership, they enter all of their information once and it creates a user with the right access, it puts the user’s information in their back-end database system, and it charges their credit card. The current experience has the user entering information on a bunch of different pages and a lot of stuff needs to be typed in over and over again. Not the best impression.

One of the problems developing sites locally is that a lot of sites really like to be at the root of their own domain/subdomain. Some parts of this wordpress instance’s configuration assume that. So I needed to define a virtual host for this site, and thanks to this script, it took about five minutes to download, install the script and create the virtual host.

For Rails development, that brings me to the second awesome tool: Passenger + PassengerPane.

I cannot believe it has taken me so long to discover passenger for development on the mac. It’s nice not to have to run script/server all the time and worry about more then one app being on the same port. With Passenger on the mac, you can just set up hostfile aliases for all the apps that are under development and just hit them in the browser. It’s especially nice for some of the Flex+Rails work that I’ve been doing because I can set a domain that is allowed under the production servers’ crossdomain.xml file.

PassengerPane makes it super easy to define vhosts and hostfile entries for as many rails applications as you want.

I love it when the things that seem to be hard turn out easy!

Guacamole

I am really good at making guacamole. It’s one of my favorite foods in the world, so I consider myself a little bit of a connoisseur. So I am going to let you in on a little secret to making really awesome guacamole: most people over-complicate things. In fact, I am going to share my recipe with you right now:

  • Two ripe avacados
  • Juice from one lime
  • About one teaspoon salt

Start by mushing the avacados with a little lime juice and salt. Add lime juice until you can taste a hint of lime. Add salt to taste. It’s that simple. When people say that it’s really good and I tell them what’s in it, they suggest lots of ingredients that I could put in it. I could, but sometimes perfection is acheived when there is nothing left to take away.

I am sure there are some lessons to be learned here with writing software. One thing that I’ve learned is to think very carefully before adding a feature to a product. Every feature is a promise and a commitment, and not to be entered into lightly. When cooking, when in doubt, omitting ingredients is better than random substitutions.

I heard somewhere something along the lines of: if you’re not ashamed of your first release, then you’re shipping too late.

On Wireframes

I reached my goal yesterday for my iPhone app: have a set of wireframes complete. I am a fan of using wireframes or mockups to determine scope and requirements when working with clients. It is a good way to talk about functionality in a way that is easy to visualize.

In this case, I was able to figure out what screens there should be, and what should be on them. I was also able to show them to my brilliant wife and she was able to give me some feedback as a ‘normal user’.

So what are wireframes? Wireframes are mockups of how an application should look and feel. There is really not a right or wrong way of doing wireframes. A lot of people like graph paper and a ruler. Lots of people like PowerPoint, and there are many other tools that people use. Here are a few off the top of my head:

Needless to say there are lots of tools that one can use to create wireframes, the only advice I can give you on how to choose:

The best wireframe tool is the one that you have, you know well, and you can use to create wireframes quickly

I am not an expert on design or wireframes, but as someone who finds them useful, there are some tips that I’ve found useful:

  • Don’t be a perfectionist The goal is not to create a realistic mockup. That comes later. The point is to figure out what information you need to display and what functionality needs to be there.
  • Don’t worry about every little button If your app needs a video player, it’s ok to put a big box labeled “Video Player” on the screen
  • Get feedback quickly As soon as you have something to show, show it to some people to get some feedback and suggestions. See if people can tell what the heck you’re talking about.

This is just the tip of the iceberg. One UX Designer offers some good tools and tips on uxbooth.com.

Wisdom of the Crowds

We’ve been trying to figure out what the girls want for Christmas, which is not an easy task, mainly because when you ask them what they would want, they point to every single item in the toy store. The challenge is to find something that makes it past the first 24 hours and still gets some play.

I have to admit that my typical strategy is to go to Amazon, or FatBrain and find the highest rated toys for three year old girls or whatever other age group, and get something there.

Perhaps I’m lazy but sometimes technology does make life easier.

Ten Things About Kids’ iPhone Games

Ten things about children’s games on the iPhone, learned from watching the usage patterns of a two year old and four year old girl.

  1. Forget the ads - they just confuse the kids and cause the phone to get thrown across the room or car. 99 cents is the ideal price point.

  2. Keep the intro focused and to the point. You get about 30 seconds to make an impression that will determine whether they will actually play the game.

  3. Too many options at the beginning causes frustration - keep the first parts of the game ridiculously simple.

  4. Once the game is underway and the player is engaged then layer in some challenge and complexity to keep the player stimulated.

  5. Invest in a good artwork for the icon. First impressions matter a lot.

  6. Things that make a good icon: pictures of dessert foods, cartoon animals, jewelery, sparkles.

  7. For coloring games, keep the drawings simple. Little fingers get frustrated trying to point in tiny spaces.

  8. If the game involves preparing food, you had better be able to ‘eat’ your virtual concoction.

  9. When eating your virtual goodies, there should be loud, garish sound effects. Nom Nom Nom!

  10. Cheering when the player completes a level is a good way to encourage playing the next level.

Rabbit Starvation

Years ago I stumbled across a random blog post that has stuck with me to this day about Rabbit Starvation. Rabbit Starvation is the notion that if you’re lost in the woods in the middle of winter, it’s tempting to go out hunting rabbits because there are lots of them. However, once you consider that it takes energy to catch rabbits, and they have practically no fat, you burn off precious fat reserves to eat a food low in calories. According to the Arctic explorer Vilhjalmur Stefansson:

This trouble is worst, so far as North America is concerned, among those forest Indians who depend at times on rabbits, the leanest animal in the North, and who develop the extreme fat-hunger known as rabbit-starvation. Rabbit eaters, if they have no fat from another source—beaver, moose, fish—will develop diarrhoea in about a week, with headache, lassitude and vague discomfort. If there are enough rabbits, the people eat till their stomachs are distended; but no matter how much they eat they feel unsatisfied. Some think a man will die sooner if he eats continually of fat-free meat than if he eats nothing…

It’s stated much more succinctly in How to Stay Alive in the Woods:

One would probably be better off on just water, then on rabbit and water

Some believe that Rabbit Starvation is apocryphal and others believe it’s what ultimately killed Christopher McCandless, but it makes for a wonderful parable for the business person. Even though I read this post and thought I had absorbed its message, my business nearly succumbed to it. When things get slow, and trust me, at some point they will, it’s tempting to just start grabbing every piece of business you can. There are lots of small jobs out there and for a short period of time, it can be fun to work on so many different things, but most people can’t create a sustainable, growing business out of small projects.

Don’t get me wrong, I love small projects and small clients. There is nothing like being able to do a few hours of work and really move the needle for someone’s business. They are also a great opportunity to think creatively, be agile and minimalist, and perhaps even learn something new. People skills, communication skills and creative problem solving are a must in that environment. But as the wilderness survival folklore teaches, if you are going to survive the winter, it’s better to go hunting for bigger game. Rabbits are delicious and as part of a balanced diet can be very good for you, but you can’t live off of rabbits alone, and they’re not the best choice when you’re starving, tempting as they may be.

Code Is the Best Documentation

Early in my career, we considered it a Best Practice to produce reams of documentation along with the software we created. It was part of the waterfall process model. The first step is that all the stake holders fight it out to determine what the requirements of a product should be. Next, we developers, or at least the ones who were chosen to participate in the design process would draw out UML Diagrams to flesh out the major subsystems and components. Then those pieces, once their connections and interfaces were defined, would get farmed out to different developers who would write each component, and part of the deliverable were sets of design documentation that detailed interfaces, contracts and implementation details.

If it sounds like a painful and inefficient way to produce software that is late, over-complicated, bloated and doesn’t do what the end user wants, you’re right. Almost nobody does waterfall anymore. Most new projects run with some form of agile process, whether it be Scrum, Extreme Programming or some other hybrid approach.

One unintended consequence of this switch is the focus away from documentation that is not part of the software itself. I used to be an avid commenter - when I wrote a module or piece of software, I would write all the comments first and then go in and add the code between the statements. Now, for better or worse, I consider excessive commenting to be a code smell. If you need to write a novel about why you did something some way, maybe that’s a sign that it’s too complicated or the wrong approach. Sometimes comments are necessary and good: for any published API, the best way to document it is by commenting the public API methods and using an automated tool to generate documentation. I also don’t want to malign Literate Programming too much - some incredible software has been produced with it, and it may be the right tool for pedagogical applications.

Where the shift has been incredibly good and useful is that documentation is starting to take the form of executable code that defines the specification of a program in human-readable language and can be run be an automated tool to verify that the program actually adheres to that specification. My favorite tool for doing that is called Cucumber. What is nice about this tool is it provides a way to communicate requirements that stakeholders can understand, but it can be executed whenever the application is changed to verify that the product really does meet those requirements.

This is a simple example of a cucumber feature

Feature: Adding numbers
    Because I am really bad at math, I need a computer program to add two numbers together
    Scenario: Simple addition
        Given a calculator
        And I press 1
        And I press +
        And I press 1
        And I press =
        Then I should see 2

Behind the scenes, you implement ‘definitions’ for all the steps, and even if you’re not a programmer, you know what this means. The other thing that is nice, is if the implementation becomes out of sync with the specifications, then you know, and you can fix one or the other.

Clickable Links With Spark RichEditableText Control

I just ran into a problem adding clickable HTML links to a Spark RichEditableText control, and none of the books that I read or sites that I found in Google had a completely workable solution, so I figured I’d post what I came up with here:

The component declaration

<s:RichEditableText id="description" width="{summaryContainer.width}" textFlow="{descriptionTextFlow(clip)}" 
    editable="false" styleName="description"/>

Event Handlers


private function descriptionTextFlow(clip:Object):TextFlow {
    //  Wrap existing HTML in a TextFlow element
    var markup:String = "" +
        clip.htmlDescription +  ""; 
    //  Converts html text into a TextFlow
    var flow:TextFlow = TextConverter.importToFlow(markup, TextConverter.TEXT_FIELD_HTML_FORMAT);
    // Gets dispatched whenever ANY link is clicked
    flow.addEventListener(FlowElementMouseEvent.CLICK,onLinkClickedSpark,false,0,true); 
    return flow;
}
            
private function onLinkClickedSpark(e:FlowElementMouseEvent):void
{
    // Short circuit any further processing
    e.preventDefault();
    e.stopImmediatePropagation();
    // Fire an event for our controller to handle the link - send the href text for processing
    dispatchEvent(new LinkClickedEvent(LinkClickedEvent.LINK_CLICKED,LinkElement(e.flowElement).href));
}

Pay special attention to the flow.addEventListener() call. We disable capturing and enable weak references to cancel the existing control’s behavior. Also, we use weak references because this application will replace the TextFlow frequently with new text, and if we have old TextFlows laying around, they cause problems, and they do not get garbage collected.