The Great Debate: Performance is Not Secondary

At DevFacto, we utilize the enterprise social networking tool, Yammer, on a daily basis. It is a great means to share information and to socialize with everyone in the company, regardless if they are working at a client site, in another city, or just down the hall. While we often post random but relevant (or sometimes irrelevant but highly entertaining) articles or links, every now and then we get into a great company-wide debate that deserves to be shared:

Mark: “Performance is not secondary. One of the mistakes people make when thinking about performance is thinking that it is about doing the same thing faster. When you have performance, people start using things differently.” – Linus Torvalds

Curtis: Just heard about an excellent example of this. I was listening to an interview with the guys who build ScreenHero. Their goal was to create a blazing fast screenshare/collaboration tool. They’ve gotten the screen refresh speeds so fast that they have had reports from users that ScreenHero is being used to allow remote people to watch movies together! Not an intended use-case but their performance opened doors.

Mark: I think it’s worth thinking about whenever we build things. IMO, designers and developers *both* need to be part of the UX conversation. The results of disconnected conversations probably don’t work that well. The technical limitations (or the opposite – things that not everyone knows are possible) can contribute to the limitations or possibilities of a thing.

Chris: I disagree. Performance is almost always secondary to considerations such as correctness, usability, maintainability, etc. ScreenHero is an example of when performance is primary, but it’s an exception, not the rule. From what I’ve been able to find by Googling, Torvalds was referring to source control systems when he said this, not software in general, and particularly not business software.

Mark: He was giving a talk about git, but referring to software in general.

Curtis: As with most things, ‘it depends’. I think the point is that if you can make a highly performing system, clever people will find interesting things to do with your product that you didn’t think of.

Chris: Then I’ll have to go ahead and disagree with him, in general. I think having it in your head that “performance is not secondary” would lead most people to prematurely optimize their code and designs when usually the effort would be better spent on other considerations.

Alvaro: I was watching some videos on MVC’s Bundles. The guy went in, used fiddler, and mimicked a slow connection and no cached data. He loaded a simple page with “hello world” however, it had a bunch of scripts, jquery, modernizer,bootstrap etc. He shows that in the end, you download almost a 1mb, taking about 56 seconds to display a simple “hello world” app. No user will wait 56 seconds anymore. Performance is now very important and should be a first class component to solve in any software. To solve it may be different in every application but developers should think about how to optimize the code as part of the refactoring process, the unit tests will make sure your logic meets the business requirements. I agree that in the end, people find different ways to use the application if speed is not an issue, because people are left to explore the software and not wait for it.

Sean: I think often times people think about performance in the wrong way. In going for performance they try and prematurely optimize specific parts of the application and forget about the big picture. Thinking about the design/structure of your application properly can have just as much, if not more of an effect on performance than optimizing specific parts of the application. Using the correct data structures and object structures (keeping related data next to one another), which is important for building an understandable maintainable solution, can be more important for performance than optimizing that one section of database queries for example. The slow queries/calculations you can optimize once you know where the bottlenecks are (based on ACTUAL performance testing) if your application is segregated properly. I think the problem with thinking about performance too early is you get all kinds of weird abstractions and optimizations all over the place that solve problems you don’t even know exist yet. Too many of the solutions we create to be “more performing” aren’t actually based on data, but on “I feel this is faster”, which creates all kinds of little odd bubbles in the codebase. To summarize, I agree with both sides =)

Chris: What Sean’s describing is the right way to think about performance for business software. Build it first, then measure and determine if performance improvement is at the top of your stack of things to do based on cost vs. benefit. Then find and address the bottlenecks. Don’t try to paint your whole system with a broad brush of optimization.

Mark: “Performance” may be an overly general term. Performance can be a usability issue. No one wants to use a feature that takes 30 seconds to return data. Performance can be a correctness or scalability issue too. If your system (or feature) can’t handle a reasonable number of users… I agree that premature optimization is a real problem (and time waster), but it’s also important to understand the performance implications of your code. To Sean’s point, being oblivious to performance and not having a basic understanding of algorithms and data structures can result in even simple code that behaves in O(n^2) where it should really be O(n) or O(logn), etc. Since we often don’t maintain the systems we build, we shouldn’t be delivering things that perform poorly, either. I think that Torvald’s main point is that if you consider performance as part of the design and think that you just want to make the same thing go faster, you may be building the wrong thing to begin with. If you can think about how people would use it if it performed well, then you can think about what you really should be building.

Chris: Mark, I still think you’re using too broad a brush. I appreciate what you’re saying at the macro level, but I still disagree at the micro level. A feature that takes 30 seconds to return data is fine, if it’s intended for one person to use once a week. It’s probably not worth spending any amount of time optimizing that feature. If a function only ever operates on a collection of a dozen items, then it probably doesn’t matter if it runs in O(n^2) or O(n), unless it runs many times a second, i.e. is a bottleneck. You should write that function for maintainability instead of thinking about the algorithm.

Mark: Like Curtis is saying, it depends. Does your feature take 30 seconds because it’s a reasonable implementation and you don’t want to prematurely optimize, or does it take 30 seconds because you don’t know how to write code that performs reasonably well the first time without spending a lot of time on it? It’s our professional responsibility to do a number of things, including:

1. Write quality code and do the best thing we know how.

2. Not waste a bunch of time on things that aren’t providing value.

Chris: I’m arguing that BY DEFAULT, performance should be secondary. You should not be continually thinking about algorithm, or things like whether String.Format() is faster than concatenating strings with the “+” operator, when you write a method. The primary concern should be maintainability, with optimization coming later when you’ve determined that the performance of that method is a problem. At the feature level, then yes, you should CONSIDER whether performance is important, but again, for most features, it’s going to be a secondary consideration. This is going to sound even more controversial than I’ve already been on this thread, but I am, in fact, arguing that the great majority of the time, when writing code, it is _more responsible_ to be _oblivious_ to performance considerations.

Mark: Sure, I don’t disagree with the fact that an obsession on squeezing performance is not productive. We shouldn’t be obscuring our code to get a tiny fractional improvement in speed. I don’t think it’s responsible to be oblivious to anything. Understanding the tradeoffs and balancing them with our work is the important thing. In .NET, should we concern ourselves with disposing of managed resources? I’ve seen lots of code that doesn’t dispose of resources properly, including the system I am currently auditing. Is calling Dispose() on objects that implement IDisposable a premature optimization, or part of our professional responsibility to do things properly?

What about when we use ORMs? Should we remain oblivious to how they work and trust that they just make things work, or should we also be conscious when we write code to avoid the N+1 query problem?

Should we learn about the data structures available to us in the language we are using and use each one when appropriate or should we just use List<T> for everything because we don’t want to take the time to learn?

Lots of times we get experience getting bitten by certain things and then avoiding them become 2nd nature to us, but we are often thinking about performance. As we gain experience, we rarely write the most naive code possible. And in this case, thinking about performance doesn’t necessarily take more time. The more you understand and are aware of language features and their implications, the more you are able to write things fairly well the first time.

Chris: Yeah, I don’t really think being oblivious to anything is responsible, and I suppose I put it that way to emphasize the point. I’d amend my statement to, “…it is less responsible to PRIORITIZE performance considerations.” I feel like my side of the argument has veered toward the idea that I don’t think performance is at all important, which isn’t the case. But I stand by the idea that it is secondary.

Mark: I don’t think we really disagree at the core. I think we’re just talking about this from slightly different angles. We’re always fighting to balance what should be important and where best to spend our time and effort to build quality systems.

What are your thoughts on performance? Share them with us on Twitter using the hashtag #greatdebate.