Software Engineering in Large Organizations

This afternoon I gave a talk at the University of Iowa ACM conference, where I spoke about software engineering in large organizations. It’s a topic I enjoy speaking and writing about, and I was particularly enthusiastic because the audience was mostly undergrads and grads in the CS department. I tried to resist the temptation to simply tell “war stories” for 75 minutes and I almost succeeded.

The premise behind the talk is that in a healthy organization, team success and professional development go hand-in-hand, but in practice the reality often differs from the ideal. A key to success and professional fulfillment is to build hard and soft skills that allow you to achieve team goals as well as individual growth in the face of these realities.

Team success and individual professional development are clearly beneficial to everyone involved, and not at all impossible to achieve simultaneously. An organization that makes long term investments in talented people working on clearly defined goals, extending the opportunity to accept and conquer big challenges is likely to succeed on both counts. (Not incidentally, it’s impossible to pull this off without creating a positive, encouraging, lively work environment.) Unhappily, it’s often the case that organizations collectively apply tactical thinking to ill-defined or changing goals, leading to poorly managed projects with periods of tedium followed by “death marches”, caffeine and cold pizza.

Employees of large organizations are not unique in facing these challenges, but the impact may be more acute because simple math says that they are likely to have less control over their professional environment. The serenity prayer comes to mind. Nevertheless, large organizations provide tremendous advantages. Big companies draw outstanding talent and are able to provide them all the tools they need to do their job. There’s a lot going on – the diversity of interesting, relevant projects at Microsoft continued to inspire and amaze me year after year. The same is true at Nielsen, and other companies.  Big companies tend to have formal processes in place for employee evaluation and development. They can be great places to learn new skills, be they technical or interpersonal.

In order to understand why software development at a big company is different from a startup, you need to think about what the job requires. Software development is a creative activity requiring engineering discipline. There are huge differences in the skill level of coders, even out in the professional world. Ubercoders do exist. But that is not the key factor determining success. I like to think of software engineers in terms of the following axes:

No matter where you work, you want to be a pro – in the upper right hand corner. Engineering discipline and creativity can absolutely coexist – and the traces of both are plainly evident in technology that truly inspires, be it the iPad, the Kinect, or whatever. Engineering discipline is simply more important (in a relative sense) than in smaller organizations, and for sound reasons. Large companies have different considerations. Big companies have big teams working together towards common goals. They all must march and work together. The cost of failure is often higher. Typically the team needs to support a large past body of work, such as a previous version. Mistakes can have consequences that last years. The list goes on.

A potentially uncomfortable but eminently fair example is with Windows Vista and Windows 8. I need to be careful here: I have never been a part of the Windows team and I don’t know anything that you don’t, and even if I did I would not reveal anything about a company that I no longer work for but still root for. Furthermore, Windows 8 has not shipped, and opinions may differ as to whether it will be enough of a success to stave off Apple, etc. But – I am sure that when it ships, Windows 8 will be a precise embodiment of a vision that was laid out years ago; that the choices that were made will be able to be justified with data; that it will ship on time and with high quality; that it will be something that the team who built it will be exceedingly proud of.  It is a matter of public record that this was not the case with Vista.  Why the difference? I will tell you what is not the case. It is not the case that Microsoft fired all the Vista people and hired new programmers (although there is new leadership). It is not the case that the existing team became significantly better programmers, testers, and designers. It is not the case that there were was a lack creative, interesting ideas during Vista development. (Au contraire, mon frere.) What happened was that the team realized a collective understanding of the importance of engineering discipline in all phases of the software cycle. Change is hard and not without cost, as is described in the recent Business Insider article on Microsoft (check it out). But I am sure that the costs will be repaid many times over by the work that has resulted from these changes. I am sure that many of those who stuck with the changes are better software engineers too.

This was the message behind the first portion of the talk. In the remainder of the talk I walked through the software lifecycle, giving my best account of how things work in a big organization and doing my best to explain why, with an aim towards identifying skills and techniques that improve one’s game. As time permits, in future posts I will share some of my thoughts from the remainder of the talk.



Author: natebrix

Follow me on twitter at @natebrix.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: