Home > Analytics, Optimization, Software Engineering > Defining “Better”

Defining “Better”

I hereby award the “Tweet of INFORMS 2012” prize to Marc-Andre Carle:

#orms is definitely the science of better: each software presented at #INFORMS2012 is better than its competitors :-)

(Thanks to Michael Trick for the tip!)

Okay, so what is “better” anyway? I get the sense that for many operations research insiders, “better” is another word for “faster”, but that is wrong, wrong, wrong. “Better” means different things to different people. For example:

  • More accurate.
  • Less prone to failure.
  • Easier to use by a broader set of people.
  • Faster to develop a solution.
  • Easier to integrate with other systems.
  • Better supported.
  • Cheaper.
  • Easier to customize and modify.
  • Easier to share.
  • Uses less memory.
  • Unencumbered by intellectual property concerns.

as well as…

  • Faster.

“Better” is a multiobjective problem: most of us actually want many of the things on the list. How we weight the various factors depends in part on what the software is being used for:

  • For academic research,
  • For rapid prototyping,
  • To create a model for a consulting engagement,
  • For a production system.
  • Some of these factors can be measured (and are, thanks to the tireless efforts of Hans Mittelmann and others) while others are more subjective. Even if we are focusing exclusively on “faster”, the picture remains complicated. In a production system what matters is how quickly users get the results they want. So we care about not only the time that the solver takes, but also:

  • How long it takes to retrieve the data and assemble the model to be solved,
  • The predictability of response time over different user requests,
  • How the solver performs in the face of many simultaneous requests.

Solver runtime differences of 5-10% don’t matter that much, generally speaking. I like to categorize how long an operation takes in real-world terms (I stole this idea from somebody else, but I don’t remember who):

  • instantaneous (subsecond)
  • the time it takes to check espn.com and/or twitter (5-10 seconds)
  • get coffee (a few minutes)
  • have lunch (30 minutes or so)
  • overnight
  • a weekend
  • ETERNITY 
  • It’s usually not worth the effort making an engineering decision based solely on performance if you can’t move to a different bucket. Otherwise you probably have better things to do.

About these ads
  1. Marc-Andre Carle
    18 October 2012 at 5:13 pm

    Hi,

    I pretty much like your categorization of run times. It seems to fit just well how people experience decision support tools. I had started to write a blog post about that but I’m going to link to yours instead and add my personal view on the matter.

    • 18 October 2012 at 5:26 pm

      Hi Marc-Andre,

      Glad to hear it! I am looking forward to reading your view on things as well. Let me know when it is posted and I will link to it here.

      Nate

  2. 22 October 2012 at 1:12 pm

    Hi Nate,

    Rather than repeat what you did in this post, I looked at the subject from a different point of view: http://www.thequestforoptimality.com/?p=93

  3. 23 October 2012 at 10:36 am

    Hi Nathan, I took yet another angle at it here
    I blogged about this and more her:
    https://www.ibm.com/developerworks/mydeveloperworks/blogs/jfp/entry/faster_to_what17?lang=en

  1. 22 October 2012 at 12:57 pm
  2. 26 April 2014 at 7:12 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 40 other followers

%d bloggers like this: