Original article posted to PHP Advent 2009, Click to jump to discussion. Happy Holidays to you and yours!

Even the best of us can only write 1500 lines of code a day, so we need to make those lines count.

There were so many great articles in PHP Advent this year, that I couldn’t think of a good topic—I like to believe my peers stole all the good ideas this year… 🙂

A decade of PHP

This advent marks the day I’ve been coding PHP professionally for a decade. At the time, I was consulting for a startup consisting mostly of Asians, which meant a lot of smoke breaks. During one of those particular breaks, a programmer asked me, “Hey, Terry, Have you heard of PHP?”

“No, what’s that? Some new designer drug?”

“Haha. No. It’s a web language like ASP, only it’s free. You should check it out, you might like it more than Perl.”

Instead of smoking a pack of This, in honor of that day, let’s begin with some startup stories spanning that decade…

The John Henry of C++

A few startups ago, I worked with a guy who was a better programmer than you.

One day, we got into an argument over a piece of open-source import code, written in the Python language, that he had ported to C++. He had just got done telling me how much faster he had made it, when I asked, “What’s the point in that? Now that you’ve rewritten it, you own the maintenance of it (Stoyan would disagree 😉). There is no evidence this code is even the bottleneck)?”

The point?!? The code was crap and he had fixed it—the massive multiple improvement in efficiency he had obtained was an added bonus.

“Look you’re right: it’s true I prefer to use a crappy, ugly, unperformant language like PHP and you crank out C++ like John Henry drives steel. But while you’re busting code with a hammer in each hand, I’m the guy with the steam-powered jackhammer. Sure, you win, but your heart will burst and you’ll die.”

…

“And here’s the thing,” I added with a devilish grin, “There’s only one you; there are a lot of people like me out there.”

1500 lines of code a day

That philosophy started out with an observation I made while being the director of engineering at an earlier startup. It was the sort of startup that every programmer has to cut their teeth on sometime—the one where you, like Laura did, put in 170 hour weeks and had to code uphill both ways.

While I did not actually put in 170 hour weeks, I did sleep under my desk. While I did not code uphill both ways, I did spend my days coding, and my nights integrating all my engineers daily check-ins. And while doing that integration, I came across a startling observation:

While quite often less, no matter what the programming language, no engineer (including me) wrote more than about 1500 lines of code a day.

This observation affected my engineering judgements and decisions for the next decade. It is a maxim I live by today. If we can only write 1500 lines a day, then a quarter of a million line codebase means around a year of pure programming. If we can only write 1500 lines, we better choose the right language and use it in the best way to maximize the expression of our creativity.

(When I told this to a friend, he added, “1500 lines a day…except in Java. J2EE programmers only write five lines of code a day.” When I laughed he added, “But they are five really good lines.”)

Java: It only takes three years to put in a hard days work!

The Death of PHP

Writing software is about making choices, and it helps to ask what language we choose to develop in. That is a choice.

Obviously, I think the right choice is very often PHP. The reason I choose PHP is that it is a web-based templating language that is simple, scalable and pragmatic.

With every choice there are consequences. If not, there’d be a One Language to rule them all. And we’re not Java developers. 😉

One consequence of PHP is that it is now stuck between a rock and a hard place.

Unfortunately, the rock and hard place are not a bottle of Jameson and a pint of Guiness. Given the drinking habits of most PHP developers at major conferences, it is crazy not to be very, very worried for the future of the language.

On one end, the ubiquity of Ajax-driven rich web sites means the inherent advantage of a templating language is no longer there, having been replaced by a much larger demand for design, as Helgi mentioned. If your view is entirely in HTML/CSS, your controller in Javascript, and all your web end is doing is feeding the API data as JSON, as Christian discussed, then being able to embed stuff in a template is no longer a plus. So many real-world applications are now written this way that discussions of MVC and templating systems sound like they are so last advent.

On the other end, social networks have sped up demands of data so that they live more in RAM in the form of memcached instead of in disk in the form a relational database. When making a web page was tied to a database, and the database is disk bound, performance discussions are pushed to database performance discussions, which really is a discussion of disk performance. Discussing application server performance is as pointless as porting some ancillary import code from Python to C++ since the PHP part was always waiting on the database. In those cases it makes sense to trade off performance for simplicity, scalability, and directness.

As Ilia has already noted, web performance is now a complicated beast. With the advent of memcached and a highly connected social graph which necessarily must be partitioned, more computations have been pushed from the database to the application server. Now the inherent performance of PHP may become the rate-determining step and PHP has never been a speed demon.

Good programming is a practice. It is reasonable to ask ourselves if PHP is still a good language to be applying our practice on.

Perhaps we should move on to Ruby and Rails?

Rails Fails

For years I worked at a C/C++-based web startup and kept quiet. But when I returned to a startup based on PHP a few years ago, it was time to unleash a few years of pent-up Ruby on Rails fury in the form of an article. As Matt has mentioned earlier, this article created a firestorm of defense from the Rails community and provided some easy jokes in my talks for the next couple years.

“First they ignore you, then they laugh at you, then they fight you, then you win.”
—Mahatma Ghandi

The ideas behind the article were simple:

1. Rails does not represent a threat to PHP;
2. If the PHP world had Twitter, we’d be trumpetting it, not pillorying it;
3. And, nobody cares about this because your language or framework choice isn’t going to get you a date.

In my days before startups (college years), the most “different” dorms were the ones where their residents tended to walk around campus barefoot. It always caught me as a bit strange that all these people were being unique by doing exactly the same thing—walking around barefoot. The fad that was Ruby on Rails reminds me of that: it always seemed to me that Ruby adoption was chasing something new simply because it was new—being different without Thinking Different.

A couple years back, a couple ex-Ruby developers told me that the Ruby world had even made a Rails version of MapReduce. That’s akin to trying to write a web server in PHP, and indeed they mentioned that the software was incapable of implementing any sort of reduce function—it was just a MapNoReduce. We had a good laugh about that over a couple beers and pizza. Championing such pointless stupidity while pillorying Twitter—that’s the Rails community of the time.

Two and a half years later, I see that Twitter has broken into the top-20 websites—being one of only two Rails-based sites in the top 100. I can gloat about how the Rails community shouldn’t have been such assholes about a showcase application like Twitter.

The creator of Ruby on Rails has two words for us.

But instead, I’ll point out that I was wrong.

Rails and PHP

Don’t fear. Rails is still no threat to PHP (and I don’t think using Rails is going to get you many more women, either).

My mistake was implying that because Rails can’t be run easily in a shared hosted environment, Ruby might never find a niche as a web development language:

Ruby is really good at what it does. The problem is that for what Ruby [on Rails] does really well, I can download Matt Mullenweg’s WordPress… It is really good at building those apps that have been solved before. PHP is good at finding out what the next WordPress is.

For Rails and Rails applications to have anywhere close to the uptake of PHP and PHP applications such as WordPress, SugarCRM, Joomla, Drupal, Magento, and MediaWiki, it’d have to run in such an environment. Without shared hosting, open-sourced Rails applications were doomed, the entire thesis of what Matt said explains this symbiosis for PHP: PHP is really almost always mod_php and integrates so well with the preferred web-server for shared hosting, Apache, there is an acronym for that—LAMP. All mod_ruby instances will share the same Ruby interpreter in Apache. Besides performance not scaling the way Apache was designed to scale, it means that someone else’s Rails code can trample your Rails code on the same machine.

Whenever spoons gives me a batch of these to give away, it never lasts very long.

Ick!

Strength from weakness

I can trace the exact moment, I found out that was wrong. It was seven months before I wrote that article and the day after I became a PHP Terrorist.

If the PHP world puts your face on a deck of cards, you must be a PHP terrorist.

In the Fall of 2006 and was being interviewed by Cal Evans. He asked me, “What was the most interesting thing you saw today at ZendCon?”

“The S3 data storage that Amazon is SmugMug showed is impressive, but the EC2 cloud computing stuff was the most interesting because it is disruptive.”

In the book, The Innovator’s Dilemma, Clayton M. Christensen introduces the concept of disruptive technology, which is an innovation that breaks the way things have been done previously.

Virtual hosting is disruptive.

Yes, it’ll never be as cost-efficient as shared hosting, managed hosting, or colocation, but last I checked film cameras are still cheaper than digital, notebook hard drives store less and cost more than their desktop companions. All disruptive technologies start out inferior to that which they eventually supplant. And whether you were sitting there in that talk in 2006, or looking at all the Web 2.0 startups built on top of slicehost or EC2 today, virtual hosting was a rare instance where the disruption it would create is obvious.

And if you are a Ruby on Rails developer who can’t choose shared hosting, what do you do? You do virtual hosting. And what do you need to do that? You need tools to manage it. And what language do you write those tools in? You write it in the language you know—Ruby.

The best documentation on doing virtual hosting are written by Ruby developers. The best tools for managing and setting virtual hosts are in Ruby. Very often the best web services built around virtual hosting is written in Rails.

Ruby found a niche in the web world.

The flaws in the design of Ruby on Rails forced Ruby to adopt technologies that work around them. Ruby is not a particularly good language, and Rails is not a particularly good web framework. Ruby on Rails never had a chance treading the same path as PHP via symbiosis with MySQL, Apache and Linux and ubiquity on shared-hosting. But to extent that was a limitation, they were able to turn that into a strength for it forced them to adopt things solutions like virtual hosting and that Ruby attitude spread adoption of New New Things like Git that actually turned out to be actually useful—You get the MapNoReduce, but you also get the GitHub.

PHP was never about the language

What lessons can we learn from that?

If we want to find PHP’s strengths we have to look at PHP’s weaknesses. Just as Rails didn’t tread PHP’s path, PHP won’t tread Rails. I once said: Rails is like a rounded rectangle, and PHP is like a ball of nails. But when you throw PHP at something, it sticks to shit.

In fact, that became a conference t-shirt.

PHP has always been a web templating language that has never been especially performant. This weakness, which is causing its share to be eroded by Javascript UI libraries on one end and the demands of application server performance on the other, points to the salient fact of the language: PHP is glue code.

PHP is simple, direct, and scalable. Simplicity made it a dynamically-typed scripting language—performance would never be the best. The directness meant it is a templating language first—as went the web, so went PHP. And scalability meant it’d be dependent on other software by pushing the difficult problems to them and outside PHP.

As Esser noted, PHP was never Ruby or Java. You’d never see us make a MapNoReduce when a real MapReduce is available called Hadoop and is around written in Java. Even though CakePHP is based on Ruby on Rails, assumes the existence of the Apache web server, Rails’s “convention over configuration” eliminates even that.

It was never about the language, and we always knew it. We invented the term LAMP in homage to other parts. It’s just a matter of finding more parts to bind to, to overcome those weaknesses.

There is no “Not Invented Here” here because almost all of the PHP world was not.

Without PHP, it’d just be LAM(e)

Now some of you might be saying, “I can see where you are headed here, but, this is the PHP advent, shouldn’t you be advocating PHP? Aren’t you a PHP terrorist?”

As Ben said, PHP’s not just a language. PHP, the language, only supposed to do one thing, solve the web problem, and do it well.

A Java person says, “Look at it this way, Java is like a knife. A knife can do many things: a good one is essential for cooking, it can cut open packages, cut a cord, and it really comes in handy in a fist-fight. Java can do a lot of things and do them very well.”

A PHP person, if they had to fight, would rather have something that is designed to solve that one problem, and solve it well.

A PHP person says, “That knife is nice, but I’d rather have a gun.”

PHP is not about the language, it’s about the attitude.

“Next time I meet this really short girl who notices my scar, I’ll tell her I got it in a knife fight with a Java developer.”

Be a force multiplier

Let’s go back to the John Henry story. My basic rant centered around two points:

1. If it’s not the bottleneck, it doesn’t need to be performant
2. Writing in a more performant language means writing more lines of code to do the same amount of work.

The first tells us that the solution to our problems is the same one it always has been: don’t solve things in PHP, move the tough stuff outside the language and bind to it. In the early days it was the database, now it has to be something else. I’ll get to that later.

The second requires more explanation.

It is easy to forget that coding is not construction. We are not construction workers; we are engineers. We do not build things; our compiler does that. We design design documents that our compilers compile to machine code (or virtual machine code in the case of scripting languages). Our design documents may be written in a variety of ways called programming languages.

Now some languages like C generate about 3-5 lines of machine code for every line of C. If your design document is that detailed, the builder (compiler) is bound to get it exactly the way you specified it) and you can make things very efficient in terms of performance.

But higher level languages work differently. Something like BASIC generates about 15-20 lines of assembly code for every line of C. It’s less efficient in terms of performance, but far more efficient in terms of time—it does 5x more. If you can only write 1500 lines of code, your code does more work this way.

It’s hard to estimate how many lines of work a line of PHP does, because PHP is not simply a high level language, it is a scripting language, a dynamically-typed one at that, and one that is built on so many libraries. Just as Linux means more than the kernel but the entire GNU stack on top of it, “PHP” is more than the Zend interpreter, but all the extensions and practices that we’ve stacked on top of it. In other words, a line of PHP leverages between 30 and thousands of lines of machine code.

Even if you are the John Henry of C++, it’s hard to compete with that, and your heart is going to burst trying.

The attitude of PHP means adopting tools that act as force multipliers for the lines of code you do write—we all seek that steam-powered jackhammer to make our coding lives easier.

Sure, we write only 1500 lines of code a day—nobody said those lines had to be PHP.

Be the Borg

As I mentioned earlier, over half a decade ago, memcached changed the web. It was originally written in Perl (later C) for a Perl-based web site. Instead of writing their own version of this like the Java world, the PHP world adopted it. Those that did thrived. You can see on the homepage of memcached where the plurality of sites are PHP by a large margin: Wikipedia, Flickr, YouTube, Digg, WordPress, Craigslist… If you look today, you see most of the recent contributors to it and libmemcached hail from the LAMP world or work on PHP-based websites—Facebook being the most notable website driving development.

If it’s better, why fight it? Why not join it. In other words, If it makes code management easier, who cares if GitHub is written in Rails. If it automates your deployment, as akrabat mentioned, who cares Capistrano is written in Ruby. And if Twitter get’s you noticed for a job, like @snookca mentioned, or located like in the case of @a, who cares what language it’s in?

It’s not like you are leaving PHP. The pragmatic attitude stays the same, it just adding the biological and technological distinctiveness of another language or project into your own.

Resolution for a new year (and a new decade)

I know it is the advent, but I’d like to look ahead to the beginning of the new year—the beginning of a new decade.

A decade ago, someone on their smoke break pointed me in a direction that would consume the next 10 years of my life.

Five years ago, a good New Year’s resolution might have been to learn memcached, but what about now? How will we level up in the World of PHPCraft, as Sara asks? What was true years ago is no longer true today, for software is infinitely malleable and change will happen as Luke mentioned.

They say, it takes a decade of diligent practice to achieve mastery. I’ve put in my ten years, but I don’t feel like a master of anything. I can’t give you something specific with any certainty. I can, as my peers have already, hopefully pointed you in a direction.

If you believe that virtual hosting is disruptive, then it couldn’t hurt to adopt some tools written for managing your AWS or slicehost, even if they’re Ruby-based. The same can be said for mastering of other tools to manage and automated deployments, testing a la Greg, and other operational considerations. As design takes over development, similarly development means more than just programming.

Furthermore, there are a lot of projects out there to overcome PHP’s weaknesses. Just one example is Gearman, which is a distributed API infrastructure. Getting PHP to work with Gearman has already been explained indirectly by Sean Coates earlier. Other areas are Hadoop, MogileFS, CouchDB, and Tokyo Cabinet.

The Pragmatic Programmer recommends you learn a new language every year. I don’t necessarily agree because it is very important what language you choose to learn. From the weaknesses of PHP, two languages seem to be stand out choices.

Heading in the direction of the user-interface, in the past you would have studied design and gotten more than a passing understanding of CSS and HTML logical markup that Marco points out is essential today. Now with Ajax-based UI, the development language to learn is definitely Javascript as Ed has already mentioned before. If I could add anything, I’d add a good UI framework like: YUI or JQuery UI to that, or you could go into a multi-domain problems like using server-side code to improve client-side bottlenecks as Ilia mentioned.

Of course, that’s Javascript for the web, but Ed noted how Javascript is more prevalent than that and Derick showed that PHP can also be found in unusual places.

Heading in the direction to find solutions that offset performance, al3x, a twitter developer, recommends Scala, but the most exciting language here for the PHP developer is Erlang. While being a functional programming language focused on high concurrency, it may not seem like a language at all like PHP. It is, however, a very high level scripting language that is strong where PHP is weak.

You can even go in all those directions at once as a PHP developer, Jan, did: to CouchDB, a distributed MapReduce-based data-store written in Erlang and with a Javascript-based Reduce library.

Relax, Jan is on the job

Summary

I used to joke that Matt didn’t want to create WordPress, he just wanted to post his crummy photos on his blog, PHP was the shortest distance for him to that solution, and WordPress was the result. He’s my boss now, so I can’t get away with saying that, but the spirit still holds true, as does the reason I made a mistake about Ruby on Rails: it’s not about writing a piece of blog software, it’s about making the next WordPress.

Matt accidentally was late to this USAToday photoshoot because I talked his ear off at lunch that day.

Maybe one of you will do that. That may seem and insurmountable goal now, but, as Lorna said, you get there one step at a time, but eventually you do. So when, a decade hence, you ring in the New Year sipping on a glass of Château d’chatalé, you can recall this article being that part of the moment that pointed you in the right direction.

It’s not about PHP, it’s about the attitude.

As Paul said, code is our conversation—we can only express ourselves in 1500 lines a day so we had better choose them wisely. To do so, look to integrate the strengths that overcome your weaknesses. Writing good code is about force multiplication—make every line you write do the most.

(And maybe, one day, we can aspire to getting away with only 5 lines like those enterprise Java/J2EE developers do. 😀 )

Coda

What ever happened to that John Henry of C++? He’s now at Facebook as the technical lead of HPHP: the switchover of their PHP codebase to C/C++. See, even after a decade of practice, we continue to fight the same battles testing our differing philosophies. 🙂

They are not different for the sake of being different, but because we really are. The distinctiveness we forge with each choice we make and every one of those 1500 lines we choose to write to express that difference.

And that difference is one I can really celebrate this holiday season.

Please join the discussion below.