Manage your expenses via Email, SMS, Twitter, Voice (Jott: Call and say your expense), IM (Yahoo, AIM, MSN), or Web.
Say What?
Responding to this warspire post about the Twitter-Rails-Scaling broo-ha-ha here since the warspire site throws a 500 error when I try to comment there:
…
What are you trying to say here? You seem to not like statistics, ok. You mention 3 categories of perceived performance (though you list 4), ok. Then you spend some time on how performance, scaling, and load can mean any variety of things.
What is your point? In the case of twitter, they’re talking about real performance impact - as in usage of the site is increasing (load), and the user response times are getting impacted in real ways that can be measured (performance). For example, you can call one of their APIs and measure how long it takes, and see that it takes longer now than it did 2 months ago.
The caching comment is totally random. Yes, caching is good. Twitter is using caching in several places.
“only metric that matters at all: what does the person using the site think?” - how is this not measurable? If your site gets slow, people don’t like it. There are plenty of studies on this. Are you saying a slow site can’t be distinguished from a fast one? That impact of css and javascript load times can’t be measured? Since you’re a front-end developer surely you’re familiar with Firebug?
“we can’t possibly measure the single metric that matters for performance” - this just doesn’t make sense. Nobody’s talking about a single metric translating to performance. Let’s go with your definition of the single metric - perceived performance of the site. Twitter is trying to fix this very problem.
If your argument is that there are multiple definitions of performance and therefore we can’t really comment on performance, then you’re wrong - we can comment on each definition of performance and pick which definition is most important to us (to use your analogy, is it zero to 60 or top speed).
If you’re saying the true definition of performance (user perceived performance) can’t be measured, then again that’s not correct. You can get fairly decent measures of performance as perceived by users.
So help me out here: what exactly is your point?
Parand,
I believe you misinterpreted my post in quite a number of ways. I do not dislike statistics; I love them. Remember, I’m a Engineer and deal with statistics on a day-to-day basis.
There are a ton of meaningful statistics you can measure about your application, but what I’m saying is that we cannot provide any meaningful results out of them. They are are all relative measures of performance are great ways to measure gains (i.e. 10x faster than version 1.0), and great ways to measure bottlenecks. However, due to the number of changing variables, comparing one entire site to another is a bit ridiculous.
The same goes for comparing two frameworks, or languages.
Do you often see statements like: “Suburus are faster than VWs”? You don’t because of the number of variables involved. Which cars? What kind of fast? Etc. However, you see almost every day statements like “Django is faster than Rails” Which versions? What kind of fast? Those questions are never answered, making their statistics non-meaningful.
“due to the number of changing variables, comparing one entire site to another is a bit ridiculous.”
This is the fundamental point we disagree on - you can compare the performance of an entire site to another, and you absolutely should. Speed is a feature. Altavista got its early gains precisely because it was lighting fast. Google not only prided itself on being relevant, but also very fast.
At every occasion possible and with every meaningful measurement possible you should compare performance. I think it’s a cop-out to say it’s complicated and there are many variables, therefore we can’t make any statements.
You’re right - “X is faster than Y” is a gross approximation, but sometimes gross approximations are good. “Ferraris are faster than Kias”. That would be a pretty accurate statement.
Parand, that was exactly my point. We can compare two sites — but not by the metrics most people use. We can compare them based on the metrics (subjective metrics) I was describing. Google is “fast” because people perceive it to be fast. Not because of the technology it is built on.
I’m not copping out, I’m simply throwing out meaningless data. To me, any kind of request/second measurement based on technology is meaningless. Because it’s irrelevant. It doesn’t describe the data we’re trying to compare (perceptive speed).
It’s like saying “ferraris are faster than kias” then racing the two with the ferrari hauling a trailer, and the Kia stripped down to bare mechanics. While the numbers might suggest the ferrari is faster, the truth is quite different.