I don’t actually think predictability is a good thing in software engineering. This will probably come as a surprise to some people (especially managers), but I’ll explain what I mean.
In my view, a great software engineer is one who automates repetitive/manual labor. You would think that this is a pretty low bar to clear, right? Isn’t automation of repetitive tasks … like … programming 101? Wouldn’t most software engineers be great engineers according to my criterion?
No.
I would argue that most large software engineering organizations incentivize anti-automation and it’s primarily because of their penchant for predictability, especially predictable estimates and predictable work. The reason this happens is that predictable work is work that could have been automated but was not automated.
Example
I’ll give a concrete example of predictable work from my last job. Early on we had a dedicated developer for maintaining our web API. Every time some other team added a new gRPC API endpoint to an internal service this developer was tasked with exposing that same information via an HTTP API. This was a fairly routine job but it still required time and thought on their part.
Initially managers liked the fact that this developer could estimate reliably (because the work was well-understood) and this developer liked the fact that they didn’t have to leave their comfort zone. But it wasn’t great for the business! This person frequently became a bottleneck for releasing new features because they had inserted their own manual labor as a necessary step in the development pipeline. They made the case that management should hire more such developers like themselves to handle increased demand for their work.
Our team pushed back on this because we recognized that this developer’s work was so predictable that it could be completely automated. We made the case to management that rather than hiring another person to do the same work we should be automating more and it’s a good thing we did; that developer soon left the company and instead of hiring to replace them we automated away their job instead. We wrote some code to automatically generate an HTTP API from the corresponding gRPC API1 and that generated much more value for the business than hiring a new developer.
Technicians vs Engineers
I like to use the term “technician” to describe a developer who (A) does work that is well-understood and (B) doesn’t need to leave their comfort zone very often. Obviously there is not a bright line dividing engineers from technicians, but generally speaking the more predictable and routine a developer’s job the more they tend to slide into becoming a technician. In the above example, I viewed the developer maintaining the web API as more of a technician than an engineer.
In contrast, the more someone leans into being an engineer the more unpredictable their work gets (along with their estimates). If you’re consistently automating things then all of the predictable work slowly dries up and all that’s left is unpredictable work. The nature of a software engineer’s job is that they are tackling increasingly challenging and ambitious tasks as they progressively automate more.
I believe that most tech companies should not bias towards predictability and should avoid hiring/cultivating technicians. The reason that tech companies command outsized valuations is because of automation. Leaning into predictability and well-understood work inadvertently incentivizes manual labor instead of automation. This isn’t obvious to a lot of tech companies because they assume any work involving code is necessarily automation but that’s not always the case2. Tech companies that fail to recognize this end up over-hiring and wondering why less work is getting done with more people.
Or to put it another way: I actually view it as a red flag if an engineer or team gets into a predictable “flow” because it means that there is a promising opportunity for automation they’re ignoring.
Nowadays there are off-the-shelf tools to do this like
grpc-gatewaybut this wasn’t available to us at the time.↩︎… or even usually the case; I’m personally very cynical about the engineering effectiveness of most tech companies.↩︎
I like your criteria for technician. Engineers are tasked to think above all else, that burden is frequently shied away from and the technician role becomes seductive.
ReplyDeleteI've noticed this same thing. We code-generate about 95% of our code, so as a result we're very efficient. However, we aren't very predictable, since we're only left writing the 5% of code we don't know how to write, otherwise we would have automated it also. Some people find this frustrating, but would they rather we predictably took 10X longer?
ReplyDeleteThis is how we do it: https://www.youtube.com/watch?v=S4LbUv5FsGQ
The answer to this is often yes, counter intuitive though it may seem. It is human nature to prefer certainty over uncertainty. In some cases predictability is preferred to cost. All very context specific, of course.
DeleteLet me tell you a real story about a guy. He was hired as a shell programmer, performing tedious but important server tasks from 10pm to 6am and reporting his results via email (micromanagement). Sometimes the manager would ask him about the server stats in the middle of the night, just to confirm that he was working. You can imagine what tedious work he was doing. Therefore, like a good programmer, he successfully automated all the boring server tasks and email responses based on the questions in the body text. If something couldn't be done using his script, an SMS alert was sent to his cell phone. And he could sleep without worrying about his work. This worked for almost a year. When his manager found out what he did, he was fired. My opinion was that he should be promoted! :)
ReplyDeletealthough IMO your point is off-topic, i thought i'd still provide my input on your comment, to point out something: i get your sentiment / conclusion - and yes, i agree. however, on the flip side, the "shell programmer" was probably fired because he did something "behind the back" (without the knowing) of their manager - and it wasn't even just "a small thing" (low / no increased risk for the business) but related to something that was probably seen as "missing critical" as per your "important server tasks" input... and to make matters worse (from a manager perspective), this went on for months, in fact for almost a year, as you specified! don't forget, if anything would've gone wrong, how do you think this would've looked on the manager... and even if the manager would've ignored all that and just seen it from the programmer's perspective, if somehow (maybe the programmer went on vacation, got sick, decided to leave the company... aka became perm or even just temp unavailable) someone else would've gotten wind of that and that info made its way to upper management, how do you think they'd have seen that? not to mention that most (corporate) businesses have policies in place that dictate what and how such situations are to be implemented... and it's implemented in form of what's called "change management" and includes risk assessment, roll-back strategy and a worst-case-scenario (roll-back fails) strategy, all of which is executed multiple times over to iteratively implement something that definitely qualifies as a "major system automation" project, which needs to undergo a full dev, validation, test and then deployment lifecycle. sure, this programmer was "academic smart", but definitely not "business wise" - but perhaps that was that business' own fault (not having or not communicated company policies and procedures). nevertheless, the bottom line is: the manager, regardless of what you think, should always be in the loop of what you want to do outside of your specific job description (SOP), and if they don't take on your ideas, go a level up then... and worse case, keep doing what you're supposed to while looking for another job (sooner than later, that's why it's important not to sit around on ambitions and rather push for progress).
DeleteAdding another criteria to it, they should at first be documenteers as their code is only the practical infill. I hate 'devs' who leave and left blackboxes of code who no one understands.
ReplyDeletecoding isnt hard, designing isnt hard.. making a newspaper out of spagheti code thats hard.
I like what you're saying about a programmer's role and the element of unpredictability, or, as I like to put it, inspiration. I worry that people might be going a little too hard on the technicians, though. Not to say that I have any concrete ideas about how to keep automation from being a mere harm to them, or how to provide places for them.
ReplyDeleteWell said.
ReplyDeleteThe issue with "technicians" is that they market their job as of higher value to managers who ignore the facts.