Simple prescriptions and making choices

Shanti Bradford wrote a response to my rant. (Yes, it was a rant.)

I read the article and he makes some valid points. It’s a good pivot to summarize some of the things that came up in the discussions.

But then he wrote:

“[Terry’s article] makes the classic mistake of assuming (or implying) most apps will ever need to remotely scale to hundreds of thousands, if not millions, of requests a day (where Twitter currently stands).”

Time to pull out the can of whup ass! 😉

[Clarifying Ruby on Rails’s place in the world.]

A rant

At first I was angry. Then I realized that his list of “Respect” included Joel on Software and Paul Graham. The first guy makes his living selling a shitty version of Bugzilla. The second guy’s claim to fame is a “plan for spam” that actually increases spam by a factor of a hundred: which is obvious to any kid who’s taken a pattern recognition course in college (Ahh, what a misspent youth!) Market defects: the big blind spot in every libertarian.

(Given some of the reactions to Mark Pilgrim’s post and mine: “I’m joking, guys!”)

If you still thing this is a PHP vs. Rails thing, then please go to RailsEnvy. As someone points out, the only problem is the name of the website—the sort of thinking that goes into that title demonstrates the primary point of my rant. I could argue with the videos they have there, but I prefer to just enjoy them. You’ve got to give them points for both style and humor.

Getting serious

I’m actually not making that mistake he mentions: it’s too easy to say “Use PHP for highly scalable websites, use Rails for everything else” since neither is exactly true. If you read very closely at the rhetorical questions I mention about twitter, I feel that a combination of hype, time to market and low barrier to entry (Shanti’s Plan A) makes Twitter’s use of Rails justified. (The jury is still out until they’re profitable. On one hand a fellow developer from Yahoo! pointed out: “They can’t even keep IM notifications up!”; on the other hand, it’s still the best microblogging app going.)

I should have been more clear on some of my points:

  1. The PHP world doesn’t need to take advice on how to run their community from the Rails world. As crazy as PHP is (and it is pretty crazy), it’s very clear which is the more mature language and community.
  2. The twitter folks know more about scaling Rails than 37signals. Lecturing twitter from that position might provoke a “PHP terrorist” pointing out that you are standing in a glass house.
  3. It’s not that Rails can’t scale, it’s that it doesn’t practically scale at this point. Investing in Rails for a viral application is asking to be the crash test dummy for others.
  4. Writing code is about making choices. Using any framework? Some choices have been made for you. When you pick up one end of a framework, you pick up the others.
  5. A la Pablo Picasso’s maxim: there are a lot of interesting ideas that are worth stealing from others. In fact most of the good things in a language were stolen from somewhere else. Yes, including PHP learning from Ruby. Stealing requires respect and understanding. That’s why I respect Rails.
  6. I have my doubts about PHP stealing the core philosophy from Rails—the two approaches seem antithetical. There are a number of PHP developers trying to prove me wrong, which is a healthy thing if I turn out to be wrong!

Beyond scalability

Besides scalability, your future web application may map onto WordPress (like this blog and Shanti’s), Wikimedia, or various forums. It may be a support app like Trac, Bugzilla, SugarCRM, etc. The application download is always an option.

Or, finally, it could be just a quick fun hack like some of the apps being created to mashup some other sites REST APIs and run on various social networks. PHP excels at these things.

The Rails framework sits as optimal somewhere outside these area. I agree there are a lot of apps there. I don’t know what they are exactly, but there are a number of small projects (including 37signals’s applications) that fit in that area.

I notice that a number of Rails sites seem to be modeled around the one site per application/feature where that feature is too big for one-offs (where PHP is very good at and Perl is undisputed champion) and too small to really scale (when shared architectures engender a huge cost).

The siren’s song of Ruby

A number of non-programmers ask me, “What’s the deal with Ruby?” The answer is very dangerous. On one hand, “It’s all hype.” is wrong; on the other hand, the number of sites making the jump to Rails is just going to generate a lot of unnecessary business to Thoughtworks and OmniTI. (Yeah, I like both companies, but I like the people there more and, quite frankly, I know it’s no fun fixing a mess someone else created.)

Put simply, I think that Ruby on Rails is very seductive. I can relate when I read this experience. Maybe you can. Programmers like the clean design of Rails and hate the messiness of PHP. Rails is like a rounded rectangle and PHP is like a ball of nails. All I’m asking is for a programmer begin with the end in mind and survey the entire terrain before making a choice.

Here is a thought experiment: Most people ask: “Can you build WordPress in Rails? Can you build Wikipedia in Rails?” The answer to both questions is “Yes, and it’ll be damn quicker than PHP to boot.” I ask “Do you think Matt Mullenweg could have built WordPress in Rails? Do you think Jimmy Wales could have built Wikipedia in Rails?” There is a subtle but important difference. It takes a programmer of a certain ability to “get” rails. Even then, the programmer has to spend a lot of work at it—the ATK experience is normative for Ruby on Rails conversions. I think Matt would prefer to spend his time taking photographs and Jimmy would prefer to spend his time dressing in east asian clothing and ranting about Google.

I’m always proud of when I can explain a complex software architecture in a manner simple for a non-programmer to understand. That is the height of understanding and the basis for creativity. I feel for the web problem, PHP is it’s simplest answer—so simple that even a non-programmer can express their answers creativity.

That’s why I said: “There is an opportunity in the listener’s institutions where PHP is a very powerful tool, very easy to understand, waiting for the listeners to use to build these amazing complex systems by stringing these simple things that PHP gives us.” When you choose another language you close yourself off to that potential.

You can really create a slow flawed spaghetti architecture. You may make a site that is insecure and easy to hack. You really do have to work hard to winnow the wheat from the chaff when hiring developers. There are consequences to a choice. If PHP was so perfect, then it would have a supermajority of the top websites instead of just a plurality.

The danger of simple prescription

The more general problem is that people are going to read this article or the previous and want a simple prescription. Yesterday, Kit Seeborg, heard my rant and started to worry for a project written in Rails. I’m not a fear monger. I don’t want her to worry.

My answer to her was this anecdote: Did you know the #1 social network out there, MySpace, was originally written in ColdFusion? At nine million users, they used a tool to translate ColdFusion into ASP.NET and I imagine some of their stuff is written and reoptimized in C# today.

There is a path. A programmer just has to find it. Maybe it’s JRuby. (I doubt it will be that simple, a performance gain is nice, but certain large sites scale exponentially—you’re going to have to do something about a framework that wasn’t built with scalability in mind.)

Twitter is trying to find it. In the meantime, I suffer the cats that have taken over their servers. I’m a consumer, if I’m pissed enough, I’ll switch to Jaiku. That’s what they mean when they say, “the market has spoken.”

On scaling Rails

On an anecdotal level scaling issues hit Ruby on Rails sooner than you think. I’ve heard about a number of mongrel scaling issues at much smaller size than Twitter. (Twitter’s problem is passed mongrel scaling into the area of database federation.)

Mongrel issues show the point where a shared-none “architecture” starts paying off. Basically Ruby on Rails (and other shared/threaded models including some from the Python world) require meticulous coding to avoid these problems. PHP doesn’t.

One of the most quoted things in the office recently was when I said: “PHP is a good language for the web because it doesn’t give you anything to shoot yourself in the foot with.” It’s a strength and it’s a weakness.

Personally, this leads to my feeling that the best area for Ruby on Rails is internal enterprise applications, where the upper bound of scale is well determined (company or team size). Some friends of mine feel differently, and that Rails can be the right choice for many customer facing applications. That’s okay. The fact that I root for the Pittsburgh Steelers doesn’t mean that I refuse to talk with a Cleveland Browns fan about football. Of course, we don’t argue about the fundamentals, like a good offense or a good defense, either. If you read the articles I linked you’ll see the differences, but more importantly, you’ll see commonality.

Making choices

I can’t emphasize this enough:

Writing software is about making choices.

One of my favorite interview questions for a senior developer is asking them to explain a programming design pattern, what the consequences are, and when to use and not use the pattern they chose. It’s actually shocking how many programmers can’t. Choices have consequences. Not realizing that makes you a design pattern bigot.

Be honest about the consequences of your actions: whether it is the programming language you choose, a framework/no-framework you adopt, a design pattern you apply, or the choice to simply download someone else’s application and install it. Choose wisely. A lack of self-reflection leads to a poor choice and spells disaster.

I am definitely not saying that the PHP community is so great or the language is so perfect. Ever hear of Friendster?

26 thoughts on “Simple prescriptions and making choices

  1. This is a great follow up post, and I think it articulates your point more-so than the first example. The overall point I received from this article is – ‘Be a smart programmer.’ This means selecting the right tools for the job, and for seeing the big picture and planning to meat goals. I stated this in Chris Shiflett’s recent post (and this is the big pull to RoT) – it is about getting things done – getting things out the door. Too many programmers can overthink applications and never get anything done (the just keep thinking of different scenarios). As your example of myspace points out, they had to refactor down the line and make some different decisions. The same can be true with any project.

    I also agree that the speed issues of Twitter seem to be more geared to their DB, which in turn *still* has somewhat to do with the application and its processes. ActiveRecord is fine for smaller sites, but larger ones seem to have some issues. I think it would be smart for any programmer to see how the database is getting hit behind the scenes. It is nice to abstract the SQL layer, but sometimes this means in some really poor queries (or multiple queries).

    PHP has its fair share of slow sites too, that is for sure. Again, it is about the programmer making the right decisions. I also pointed out the new site by Guy Kawasaki, Truemors. That site was slow as dirt from the very beginning (still is pretty slow from what I hear). It’s running on WordPress. However, I don’t feel this is a WordPress issue – I think there are many other factors involved.

    The short of it: Be smart as a programmer. Choose the right tools for the job. Know how to use the tools (this means understanding what is going on under the hood of your framework, to). Know your goals and your environment.

  2. Another great article. I was searching for the proper wording, and “mistake” was probably improperly used.

    Also was (and still am) a little unclear on the tone of the original article. Sarcasm / humor can be hard to pick up on the web. I assumed some of it was in jest, other parts were true jabs at Ruby/Rails/DHH, which is all good in my book! (time for RoR to start eating humble pie)

    You make a great point about MySpace. I guess it wouldn’t end up being the end of the world if Twitter (or some other high-profile Rails site) ended up switching to PHP once it reached critical mass.

    I just know that if that happened, I’d probably end up on more PHP / Python / etc. projects and be sitting there 6-10 months after launch, rolling my eyes at the scaling decision when the app could very easily (and likely) still be sitting at a measly 10k daily uniques.

  3. @Shanti: I understand what you’re saying. The point of this article is that you’ll find among your fellow programmers more commonality than difference.

    Funny but true was what I’m always shooting for. 😀 When I watch the RailsEnvy commercials on PHP, I laugh but it hurts. 😉

    I’m actually thinking that Java might be a better migration for a site like twitter. twitter -> JRuby -> pull out parts of ActiveRecord that need to be federated and rewrite in Java. There is still the issue of the mongrels, etc. so I won’t say it’s easy. Something along those lines.

  4. @Nate: Yes. We may disagree in the details, but I think we’ll agree on the larger things. When I bag on someone (like you), I just want you to remember that we’re talking about details.

  5. @tychay – I agree with your sentiments on the RailsEnvy commercials. They’re amusing, but they did seem to violate an unspoken nicety the Ruby community has tried to engender over the years.

    It only takes a few rotten apples to give an entire community a bad name.

  6. “I also pointed out the new site by Guy Kawasaki, Truemors. That site was slow as dirt from the very beginning (still is pretty slow from what I hear). It’s running on WordPress. However, I don’t feel this is a WordPress issue – I think there are many other factors involved.”

    Actually, WordPress *is* inherently slow, in my experience. I ran a site based on it that wasn’t particularly popular (top 150,000 in alexa) and it was unbearably slow and made for huge CPU hits without extremely aggressive caching via wp-cache + apache mod_cache stuff for images + mysql cache tweaking. Out of the box, WordPress is not optimized well, and could use a lot of tuning up in the DB access area. Wp-cache is a hack to get around something the developers are ignoring, or don’t have enough experience to fix.

  7. Haha, I just noticed your comment about my posting this on my del.icio.us linkblog! Better late than never…

    I pulled that quote because I have a good friend who is (1) a PHP coder, (2) a libertarian, and (3) a subscriber to my linkblog. So I knew choosing that as the pull quote would get him going 😉

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.