Defining “Better”

I hereby award the “Tweet of INFORMS 2012” prize to Marc-Andre Carle:

#orms is definitely the science of better: each software presented at #INFORMS2012 is better than its competitors 🙂

(Thanks to Michael Trick for the tip!)

Okay, so what is “better” anyway? I get the sense that for many operations research insiders, “better” is another word for “faster”, but that is wrong, wrong, wrong. “Better” means different things to different people. For example:

  • More accurate.
  • Less prone to failure.
  • Easier to use by a broader set of people.
  • Faster to develop a solution.
  • Easier to integrate with other systems.
  • Better supported.
  • Cheaper.
  • Easier to customize and modify.
  • Easier to share.
  • Uses less memory.
  • Unencumbered by intellectual property concerns.

as well as…

  • Faster.

“Better” is a multiobjective problem: most of us actually want many of the things on the list. How we weight the various factors depends in part on what the software is being used for:

  • For academic research,
  • For rapid prototyping,
  • To create a model for a consulting engagement,
  • For a production system.
  • Some of these factors can be measured (and are, thanks to the tireless efforts of Hans Mittelmann and others) while others are more subjective. Even if we are focusing exclusively on “faster”, the picture remains complicated. In a production system what matters is how quickly users get the results they want. So we care about not only the time that the solver takes, but also:

  • How long it takes to retrieve the data and assemble the model to be solved,
  • The predictability of response time over different user requests,
  • How the solver performs in the face of many simultaneous requests.

Solver runtime differences of 5-10% don’t matter that much, generally speaking. I like to categorize how long an operation takes in real-world terms (I stole this idea from somebody else, but I don’t remember who):

  • instantaneous (subsecond)
  • the time it takes to check and/or twitter (5-10 seconds)
  • get coffee (a few minutes)
  • have lunch (30 minutes or so)
  • overnight
  • a weekend
  • It’s usually not worth the effort making an engineering decision based solely on performance if you can’t move to a different bucket. Otherwise you probably have better things to do.


Author: natebrix

Follow me on twitter at @natebrix.

6 thoughts on “Defining “Better””

  1. Hi,

    I pretty much like your categorization of run times. It seems to fit just well how people experience decision support tools. I had started to write a blog post about that but I’m going to link to yours instead and add my personal view on the matter.

    1. Hi Marc-Andre,

      Glad to hear it! I am looking forward to reading your view on things as well. Let me know when it is posted and I will link to it here.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: