[This post is in progress. There may be a number of errors. Please comment below with corrections and I’ll update this article.]
Facebook announced the release of HipHop for PHP today.
Facebook, Palo Alto, California
Sony DSC-WX1
1/80sec @ ƒ2.4, ISO400, 4.2mm (24mm)
When I left Plaxo, Haiping was the server architect there. Today, he works at Facebook and announced HipHop for PHP. It’s nice to see the world recognizing his talents.
Since apparently, I was the first person to ever publicly spill the beans on HipHop, I need to do penance by clarifying what HipHop is and what it means for PHP.
For those of you who don’t have time to read the rest, here is what HipHop for PHP is:
- HipHop is a PHP-to-C++ cross-compiler. What this means is it takes PHP code and translates it into C++ code for further compiling. It is not another language. It is not a just-in-time compiler (JIT).
- HipHop will be open-sourced by Facebook under the same licensing as the respective PHP codebase it sets to mirror. Facebook hopes that the community will improve HipHop and add extensions to HipHop that mirror PHP functionality. Also, it hopes that future PHP core development will code features that would be more amenable to optimizations that HipHop does.
- HipHop was the current survivor of a number of projects over the years at Facebook to improve the performance of the site. Facebook, as the second largest trafficked website in the world, is built mostly on PHP. HipHop is currently running side-by-side next to many LAMP PHP servers at Facebook and they are claiming an average of 2x increase in performance on those machines.
- HipHop accomplishes this by surveying the entire codebase of your PHP-based application with a parser and then building out a C++ project based on it. The C++ project then compiles and runs as its own web server. Because of this, Apache and the PHP Zend engine are completely bypassed.
- In order for this to work, some features of the PHP language are no longer supported. Also, C-specific PHP extensions will need to be translated to HipHop C++ extensions in order for them to run.
- The benefit in speed is mostly due HipHop’s static analyzer which parses your PHP code looking for ways to optimize dynamic parts into static maps. Because of this, your performance gain may vary — more structured code is rewarded with larger performance boosts.
What HipHop means
If you use some open-source PHP applications on your hosted website, the answer is nothing. You don’t have the ability to compile HipHop, you don’t have access to server restricted ports, etc.
If you are developing a PHP application that currently can be run on two servers or less (or virtual servers in the cloud) the answer is nothing. You don’t have the scale for this to be worth your time.
If you do not have a separate development and deployment environment, don’t have a developer who knows C/C++, or use any PHP libraries where the source is not available (thankfully the encoded scripting market is small to non-existent in the PHP world), then the answer is nothing. You don’t have the development model that can support HipHop. Also note, HipHop has bugs, and—given the state of APC development as a model—will never have true compatibility with PHP. You’ll need some resources to either recode around those bugs or fix HipHop.
If you are a developer of an open-source PHP application, then the answer is not much. Most PHP applications will be deployed in a shared-hosted environment. They won’t be using HipHop.
If you are a shared hosting company, the answer is not much. This is because the HipHop parser needs access to all the PHP in an application in order for it to create a working project. The exception is if you provide software as a service that you maintain (say a static build of WordPress, or a custom site tool written in PHP). You can have HipHop optimize this and get the performance increase.
If PHP is not the operational bottleneck of your web application (your app spends a lot of time waiting on the database, disk, a 3rd party Web API call, etc.), the answer is not yet. At this time, there’s no point in getting a performance gain in PHP. If you don’t know what I’m talking about, your bottleneck is the database. 😉
If you have an application already scaled across many machines, a significant number of them running PHP in processor-intensive tasks, have separate development/deployment, have your entire PHP source code, have modest C/C++ resources, then the answer is possibly. It wouldn’t hurt for a developer there to try a hand at cross-compiling the PHP into HipHop and seeing if it runs. An operational deployment will return about 50% of those machines to a pool for other uses or future growth—or, put differently HipHop will basically double that processing on the same hardware/power.
If you make a turnkey application based on PHP, the answer is somewhat. These are rare, but now you can shrink-wrap PHP into a binary. This isn’t the intended use of HipHop, so some development might have to be done to get this fully supported. Also this is a true binary, not an op-code compile—it cannot run across platforms.
If you are developing a PHP framework, the answer is some. If your framework can compile and run successfully in HipHop, then it should be a good selling point to enterprises in case their application becomes bottlenecked on performance.
If you have highly-cohesive parts of your architecture that fall into above requirements and those parts are weakly-coupled (via API?) to the rest of the system, then the answer is a lot. Those parts can probably benefit from HipHop, and it should be relatively easy to try it.
If you are making a decision on which web language to build your site in, the answer is a heck of a lot. Arguing against PHP for performance reasons no longer holds water. PHP under HipHop will probably now out-benchmark Perl, Python, Ruby and possibly even Java and C#. In practice, you can get the advantages of having a scripting language without operational costs. Moreover, because the target is C++ which is more easy to integrate as a library, if you have a multi-language support, you can now provide C++, Python, and other languages with access to components that have before only been written in PHP (without resorting to a web API).
If you are making an argument to recode your entire site from PHP to some other language, the answer is you just lost that argument. (I never bought the argument of recoding an entire site from another language to PHP.)
There are language features, for better or for worse, that PHP must support and HipHop must not. Because of it’s unique approach to compiling, HipHop will never replace the Zend Engine. Because it doesn’t supersede PHP user-space syntax, HipHop does not and will never change PHP development (much).
HipHop is a showcase. With it the PHP world can point to Facebook as being the busiest site built in a scripting language in the world.
Visiting Facebook
The rumors have been flying over what’s going on over at Facebook headquarters. The word has been that a PHP team was brought in and made to sign non-disclosure agreements before discussing a PHP project that has been in development for the past two years.
—ReadWriteWeb
Since I have known about HipHop since before it was a project, I was one of those people invited to a Facebook PHP Summit on January 21st. (I note a tad bit offense that I didn’t receive an invite to it until after my advent article—making me the one of the last invitees. I guess I’m still persona-non-grata in the PHP world—or something.) And ReadWriteWeb is wrong, we were specifically not under an NDA.
Facebook, Palo Alto, California
Sony DSC-WX1
1/30sec @ ƒ2.4, ISO200, 4.2mm (24mm)
At Facebook, you return your badges by plastering it on a big ball of previously returned badges. I noticed they use the same labeler as me—Rock on!
We were just politely asked to not talk about it until Facebook’s official release today and not record a small section of the presentation where Facebook revealed internal performance numbers which they hadn’t decided to release yet—these numbers, anyway, were subject to change since HipHop was still under development.
I’ll be covering the first day of the summit in the order it was presented:
- Scaling Facebook
- PHP at Facebook (before HipHop)
- HipHop for PHP
- How HipHop works
- Roadmap
Facebook, Palo Alto, California
Sony DSC-WX1
1/80sec @ ƒ4.5, ISO200, 13mm (73mm)
David Recordon recently joined Facebook as their Senior Open Programs Manager. Facebook felt that some of their open-source releases were blowing up in their faces and needed someone to do better outreach.
But before that, I want to address some myths about PHP and clarify some misunderstandings about HipHop.
Hmmm I keep hearing the "only need if you're big". All problems / migration issues aside, if switching to HipHop takes your typical response time down from say 130ms to 90ms, you're winning – http://www.useit.com/papers/responsetime.html . Google have shown this to be true in practice – http://mem.to/t/1c9E
My blog post isn’t finished but I haven’t claimed “only if you’re big.” (yet). My claim is that you need over 2 machines and a bottleneck in the application server (which is rare).
To your point specifically, Harry, you are correct. If your latency gets degreased than this is good. But taking a real world example, before I joined one startup, "Hello World" took 240msec, a rearchitecture (without something as drastic as HipHop) dropped it to 15msec. I don’t think latency would be a win at that point alone. My guess is similar improvements can be found at other companies that are >90msec. However that is not always the case: Rasmus feels this may be a win for frameworks—their bloat usually destroys response time.
In Facebook‘s case, all the "big latency" hurdles were eliminated when they moved to lazy-loading APC, they are clearly thinking about eliminating even more with the ability to snapshot the core and restart from there (that that approach is highly complex). And their big issue is sheer cost, not latency. So to them total time matters and latency is simply a matter of running the servers sparsely.
Unfortunately I cannot quote Facebook’s numbers on CPU time. You will have to ask them, or figure it out yourself by the copyright trick.
My recent post Faster PHP fo shizzle—HipHop for PHP
Maybe I came off wrong in my comment but it was more a reflection of the hype and not the technology.
I shouldn't have acted so quickly to say that WordPress used eval all over the place because as you say that isn't correct. I was trying to find the least complicated reason to give for why people using WordPress shouldn't bother.
My last resort comment was based on there not being much more you can do to speed up PHP than compiling it to binary. If there were then Facebook would have done that instead. That isn't a bad thing but it is a reflection of what people will be using the tool for.
The hype around HipHop makes it out to be something everyone that uses PHP will be using (that is why you get comments like Patric's about how great it will be for WordPress users). You need to be committed to using PHP compiled with HipHop. I'm sure the hype will die down but what is worrisome is that people who don't understand what to use it for will fail in their attempts and complain about it.
On another subject, does anyone know when it is it going to be released? I want to actually use it so my comments are based on some reality.
My recent post Developing Adobe Air Apps with Linux
@Carson. Don’t take it personally. Right now your comment falls into the unfinished part of my article. This contains assorted splurgings that lack both tact and facts. Hopefully it won’t sound as nasty when it’s integrated in the final article.
My recent post Faster PHP fo shizzle—HipHop for PHP
I'd be careful about the benchmarking claims. Java's starting pretty far ahead of standard PHP: http://shootout.alioth.debian.org/u64/benchmark.p…
Hmm, what part of “possibly” did you miss?
In any case, it depends on the benchmark, but if the benchmark is artificial enough (like many of those in the Alioth shootout), then the static analyzer can replace nearly everything with native C++ calls. At that point, you’re basically benchmarking Java vs. C++, not PHP.
If you look at the same tables you reference, you’ll note that C++ does better in CPU usage than Java. Both C++ and PHP (native) already do miles better than Java in total memory usage (because of automatic garbage collection).
In practice, I’d say it puts them in the same class in terms of CPU—mostly slightly slower, but a few times much faster. This should come as no surprise because Java has a JIT and HipHop is a cross-compile to C++ which is a straight compile.
My recent post Faster PHP fo shizzle—HipHop for PHP
HipHop is interesting but, I'll definitely argue about it being PHP. By picking and choosing what language features they'll support they're building a language kinda sorta like PHP but not quite PHP.
Considering how many OSS apps and frameworks use eval() I also think it's disingenuous of them to characterize it as a rarely used feature. Now, maybe it's one that _should_ be rarely used but, that's a different argument.
My recent post HipHop for PHP is not PHP
I understand what you’re saying but it’s a losing argument. The frameworks you mention that can’t implement HipHop almost all because they depend on dynamic scripting of template pages for performance. That would no longer be needed in HipHop.
I’m not saying you are wrong right now, I’m just saying that it’s a lot easier to port frameworks than you think. They simply have to add a flag to allow you to turn off any dependency on dynamic scripting components like Smarty. They shouldn’t be necessary to run the base framework.
Before HipHop, there was no reason to not do dynamic scripting and a whole host of reasons why performance improves when you do. Now, HipHop changes that cost-benefit. To not expect framework developers (who I feel have as a failing their alacrity in which they adopt anything new), to change due to that is short-sighted.
I argue in the article why OSS apps probably won’t change.
My recent post Faster PHP fo shizzle—HipHop for PHP
Oh, I think that much of the OSS world will adapt and quickly. Supporting HipHop will likely become a checklist feature and looking at the usage of eval() in some projects it would be trivial to remove. I mainly cited them as part of taking issue with their "rarely used feature" characterization.
My larger point is that instead of actually supporting the PHP language, they're moving the goal posts to a position more convenient for them and calling it PHP. For better or for worse, eval() is part of PHP.
My recent post HipHop for PHP is not PHP
Oh, I think that much of the OSS world will adapt and quickly. Supporting HipHop will likely become a checklist feature and looking at the usage of eval() in some projects it would be trivial to remove. I mainly cited them as part of taking issue with their "rarely used feature" characterization.
My larger point is that instead of actually supporting the PHP language, they're moving the goal posts to a position more convenient for them and calling it PHP. For better or for worse, eval() is part of PHP.
My recent post HipHop for PHP is not PHP
I buy that argument.
My recent post LH-3 Lens hood for the 35mm F1.2 NOKTON
I'm interested in what language features besides eval() are not supported. They give eval() as an example but imply there are others. Seems kind of important to be able to consider what will and will not be available before getting TOO excited…
I have a list of some which I’ll get to when I finish the article but here is a quick rundown off the top of my head.
– eval() not supported
– dynamic scripting is not allowed (That's where you use PHP to create a PHP file. Like when you use Smarty to compile a file).
– create_function() is not supported
– preg_replace when using e (execute PHP code on match)
– some functions are not implemented yet/were overlooked (An example was that was php_version() was not returning anything which was crashing HPHPi when it was running against the WordPress codebase. These bugs should be reported and fixed though.)
…and there was something to do with ordering where it works in PHP but won't when the static analyser hits it. Meaning in some of your scripts you may have to move things around for it to work.
My recent post Faster PHP fo shizzle—HipHop for PHP
hiphop won’t change the fact that php is a language most people grow to hate. i don’t know anyone who likes it more after a year than they did on day 1. so hiphop doesn’t make the rewrite argument go away. it might delay it, but inevitably the pain of actually writing and maintaining php remains.
You dance with the one who brung you.
I didn’t advocate when Friendster decided to switch from Java to PHP. I didn’t advocate when Del.icio.us decided to switch from Perl/Mason to PHP/symfony. I don't advocate anyone switch to PHP because of HipHop on PHP. Why would I start arguing that a company leave PHP because apparently according to your limited experience nobody "likes it after more than one year"?
Architecture changes are hard because they are inherently waterfall. They are especially hard since the web development cycle is tight (if the company is any good). If you want to shoot yourself in the head, (or if you are a consultant, cause your clients to shoot themselves) be my guest.
My recent post Faster PHP fo shizzle—HipHop for PHP
"I guarantee those engineers who failed were a lot smarter than you."
? you don't even know who the hell i am. i single-handedly created the most popular news website in the world, which has been #1 for a decade. you can piss on that too or you can admit maybe you and your cabal are by no means the last word in who knows how to build websites.
No, I don’t know who you are. The fact that you hide behind a curtain of anonymity while I don’t speaks volumes as to your authority.
You’re afraid to put up, therefore you get shut down.
Unlike you, I am never secret in my affiliations. Thus, to my knowledge, I’m not part of a cabal. But, I would like to know the number of your crack dealer, since you are obviously on it. 😀
I started using PHP for side projects about 8–9 years ago. I started using it as one of my primary responsibilites at work about 4 years ago. While I don't recall exactly how much I liked it 9 years ago, I'm quite fond of it now.
Poorly designed code is painful, regardless of its language, and "the pain of actually writing and maintaining [code]" is part of software. I don't see how that's unique to PHP.
I started using PHP for side projects about 8–9 years ago. I started using it as one of my primary responsibilites at work about 4 years ago. While I don't recall exactly how much I liked it 9 years ago, I'm quite fond of it now.
Poorly designed code is painful, regardless of its language, and "the pain of actually writing and maintaining " is part of software. I don't see how that's unique to PHP.
It's a shame that so much time has been wasted creating a PHP to C++ cross compiler. Sure, it will help Facebook, and some other large websites in speeding up their systems, but it encourages more PHP usage, which is a downright awful language. PHP needs to die.
Also, your arguments about PHP being a more universally supported language than some other scripting languages is archaic. Shared hosting is approaching the end of its lifetime, and anyone who wants to create a Python/Ruby/Scala/etc website will be able to do so thanks to on-demand cloud computing.
TLDR: PHP is dead. Get over it.
Ahh, another anonymous comment with a blanket “PHP is bad” statement and no evidence to back it up. Did I get slashdotted and no one tell me?
Shared hosting may be EOL for those people doing Web 2.0 startups, but for the SME market it is not only alive and well, but thriving. The SME market is many orders of magnitude larger than Web 2.0 startups—talk to GoDaddy sometime before you make that claim. In fact, I’ve noticed that 3 of the top 3 open source CMSs (which pretty much own about 90% of the open-source CMS market) are written in PHP. Shared hosting was instrumental, and no amount of slicehosting will eliminate that, since slicehosting is not used by non web-based SMEs.
My recent post Faster PHP fo shizzle—HipHop for PHP
"phalanger? – MS bought the team, they’ve disappeared"
Are you sure about that? Phalanger is alive and well and being developed to a new version (3.0) by a UK Software company and a team at CHarles University. It's being deployed in the Enterprise and in Government.
My recent post Drumma Boy discography
"phalanger? – MS bought the team, they’ve disappeared"
Are you sure about that? Phalanger is alive and well and being developed to a new version (3.0) by a UK Software company and a team at CHarles University. It's being deployed in the Enterprise and in Government.
My recent post Drumma Boy discography
No I am not sure. That’s a straight copy of my notes and has not yet been integrated into the article proper. Because of this, no fact-checking was done.
But thanks for the catch. Odds are I wouldn’t have found the error.
My recent post Faster PHP fo shizzle—HipHop for PHP
I also think that many developers will not use it, as many people say it only makes sense if you know what you are doing, if the problem is cpu/memory AFTER profiling the code, and if you have at least 3 servers and you can perhaps save one of them.
I'm really interested in the source code, examples, "compatibility lists", translated extensions and so on, this will take a while until we are able to use it I think.
My recent post HipHop für PHP
For those who hate PHP, you need to admit it, PHP is the No.1 web/scripting languages: http://www.tiobe.com/index.php/content/paperinfo/…
Get used to it, get your shit done by using xxx language, who fuckin care your business?
That is like the assertion that until 1994, COBOL was the language of choice, because it took that long for the new code to outstrip the legacy. And if you count ABAP/4 as COBOL, you could claim it came even later.
Wow, passing C++ and Visual Basic. That’s phenomenal (and unexpected).
Still most problems can be solved with any language. I feel even Facebook’s could have been. The issue is that language would have come with its own baggage.
This informtiaon is off the hizool!
As seen on a bumper sticker: Eat shit, one billion flies can't be wrong.
Howdy, we don't really run Percona MySQL builds – you can follow our own branch developments at http://launchpad.net/mysqlatfacebook
@Domas Thanks for the correction. Obviously my notes were in error. I’ll put in a correction in the original article.
I too have a question about the 'only worthwhile if you're big' sentiment: wouldn't improved memory efficiency be very important to a site that's running on a tiny VPS? Or are the memory gains not really that substantial?
You are too pessimistic, HipHop PHP is going to change everything!!!! Be more excited and happy my friend, this enable developers to streamline other development concerns by increasing performance.
For instance, our development cluster in-house is larger than our production because we do massive testing (i.e. download the entire site and analyze it). This will be a great win for many development teams if harnessed correctly.
My recent post HipHop PHP is going to save the world
that seem to be a great tool. thanks for this article
My recent post Tuborg Green Beat Festival 2010
Terry, can you , please, notify people somehow (twitter probably) once you finish this article? I don't want to check every week if you updated it (or not). That would be nice.
And, it seems like there is some issue with displaying/formatting posts' dates on your blog.
Hip Hop is Great Ive used it on a few sites seen here http://devonthedesigner.com/websitegraphicdesignp…