Debunking AI Automation Myths: Why DevOps Matters More Than Ever

AI isn't coming for your job. Despite the hype around artificial intelligence automating developer roles, the reality is far different when we examine how software development actually works. Using a simple flowchart approach, I'll explain why understanding DevOps is crucial to seeing through the magical thinking that surrounds narrow AI's capabilities in the development process.

Understanding AI Reality

Narrow AI: Useful But Limited

Narrow AI is a form of intelligence that's really not intelligent. It's a way of doing pattern matching or doing things like full text search. Examples include voice assistants (which are pretty buggy), services that give you directions, or coding assistants that autocomplete functions. It looks like it has some form of intelligence, and these tools can be very useful.

For example, I think Cloud Code could be very useful for spitting out a README file, but what we see with these combined narrow AI solutions is that by default, they have lots of bugs. If you ask a narrow AI solution to do something, it's going to contain some bugs. And when you cobble them all together into a multimodal type solution, each component is a narrow AI and the bugs compound.

In many ways, you want to get in, ask for some kind of solution, and then get out as quickly as possible. It doesn't mean these tools aren't useful, but the bugs will compound if you rely on them too heavily.

The AGI Fantasy

There's no evidence at all that we're even close to Artificial General Intelligence (AGI) or that it's even possible. AGI would be human-level intelligence, and in order to have human-level intelligence or even animal-level intelligence, we would need consciousness.

Fortunately, in my early 20s, I was at Caltech and met an expert in the field of consciousness who was a former brain surgeon and professor in the computational neuroscience lab. He explained really simply what consciousness is: the ability to recognize what's happening. For example, if you're sitting in a chair and someone says, "Are you aware that there's pressure on your butt?" Well, now you're aware because someone told you and your consciousness was made aware of that fact.

Consciousness is the ability to recognize what's happening in your environment and switch context. There is no concept of that with narrow AI, and there's no concept of that even in the future with the approaches we're using for narrow AI. The idea that general AI is even close or even possible is complete fantasy and magical thinking. You might as well think that a great flood was responsible for changes in the earth.

The ASI Myth

Super AI or ASI would exceed all human intelligence. That's even more of a fantasy – maybe like a doomsday scenario or a scene from the Foundation series by Isaac Asimov. We have no evidence at all that AGI is possible, let alone ASI. There's actually evidence against it.

The DevOps Flowchart Test

I've developed a simple flowchart to evaluate claims about AI automating developer jobs:

1. Can You Explain What DevOps Is?

The easiest way to find out whether someone is lying, a magical thinker, or incompetent when they claim AI will automate coding jobs is to ask: "Can you explain what DevOps is?"

DevOps is a proven methodology similar to the Toyota way. Toyota has some of the best methodologies for building cars with low defects because they do things like "look and see" or continuous improvement. They have this methodology of always fixing things continuously (Kaizen), and that has translated itself into the software industry.

What DevOps really is is a way of continuously improving. You look at the build system, the testing, the linting, and the formatting. Every time you make a change, you're constantly improving your system. This has no AI component with it at all, other than maybe some statistical analysis with linting tools and complexity tools.

If someone cannot explain DevOps, how could they possibly understand the effect that narrow AI will have on software development? They don't understand what continuous improvement is. So in a flowchart perspective, if you cannot explain DevOps, you're incompetent. We just start there.

2. Does Your Company Use DevOps?

Now let's say someone can explain DevOps and understands why it's such a good methodology for improving software engineering. The next question to ask would be: "Does your company actually use DevOps?"

If they say their company doesn't use DevOps, but all jobs are going to be automated with narrow AI, they're unfortunately inexperienced and a magical thinker. How could you possibly say your company doesn't use DevOps – the methodology proven to be effective – and simultaneously think AI will automate jobs? You don't know what you're talking about and you're a magical thinker.

3. Why Would You Think Narrow AI Has Any Form of Intelligence?

If you can say you know what DevOps is and your company uses DevOps, now things get more interesting because you actually know what you're talking about.

So then the question is: why would you think that narrow AI has any form of intelligence? If you understand how DevOps works, you would be pretty scared about introducing something that's very buggy into the methodology of DevOps.

Likely, this will be very rare to find someone who thinks this. They're a magical thinker, and there are people who are magical thinkers. They think there are mystical things that happen in the world. They may be unaware of the scientific process, or maybe they really believe these magical things.

Why DevOps Matters

DevOps is in fact a proven methodology similar to the Toyota way. Toyota has some of the best methodologies for building cars that have low defects because they do things like "look and see" or continuous improvement. They have this methodology of always fixing things continuously, or Kaizen, and that has translated itself into the software industry.

What DevOps really is is a way of continuously improving. You look at the build system, testing, linting, and formatting. Every time you make a change, you're constantly improving your system. This has no AI component with it at all, other than maybe some statistical analysis with linting tools and complexity tools.

You're pushing these changes into production, measuring the result, and that feedback loop is effectively making your system better. The software engineering discipline has evolved around measuring and improving quality systematically.

The Reality of Job Automation

There are some nuances to consider when discussing job automation:

People Who Do Nothing

There are people that don't do any work in organizations. I don't think it's as common as people think, but if someone has been eliminated from a role because they did literally nothing – maybe they were on the phone or on Facebook all day – is that AI automating a job? Of course it's not. Someone didn't do anything, and now they were eliminated, and nothing happened.

Workers Who Create Negative Value

Likewise, there are situations where people are in roles at organizations where they do negative value. Maybe they create code at 2 a.m. while drinking, and they create bugs in your system. Those bugs take years to get squashed.

That's also pretty rare, but it does happen. I've seen it happen. Some people, every time they make changes to your software, actually make the system worse. Their elimination isn't AI automation – it's removing someone who was harming the codebase.

Measuring Software Quality

High Churn Files and Defects

One of the ways you could develop a methodology to identify poor software engineering is to look at churn. There is research that high churn files are actually correlated with defects.

In the case of the bad software engineer that's constantly making changes to the same file over and over again, the reason they're making changes to the same file over and over again is that they don't know what they're doing. If you're writing an algorithm, it maybe should be fixed within a few different changes, but if you keep changing it, it's because the code itself contains bugs. That's the intuition behind it.

You could look at, for example, which files constantly keep getting changed, especially if you have good DevOps patterns with testing and linting. Also look at complexity.

DevOps Metrics That Matter

• File churn rates: How often the same files get modified
• Complexity metrics: How complicated the code is
• Code coverage: How thoroughly the code is tested
• Deployment frequency: How often you ship to production

I think we're at very, very early stages of using narrow AI combined with DevOps. I'm excited to share some of the research I'm working on that explains this, but I think we can look at narrow AI solutions as definitely useful tools. I do use them, but it will be very interesting to look at some of the things we've already learned from DevOps.

Key Benefits of Understanding This Reality

1. Clear-eyed Assessment: You can evaluate AI tool claims without magical thinking or hype
2. Strategic Tool Use: You'll use narrow AI appropriately - getting in, getting value, and getting out before bugs compound
3. Continuous Improvement Focus: You'll maintain the proven DevOps practices that actually improve code quality
4. Quality Measurement: You'll use metrics like churn and complexity to identify problematic code areas

Conclusion: The Flowchart Assessment

This is an important way to dive into someone who claims "AI is automating coding jobs" – ask them to walk you through it. Walk me through what DevOps is. If they can't even explain what DevOps is, they're not a trusted source.

If they can explain it but their company doesn't use DevOps, you also have to question: how could you possibly tell me that you're not using the industry standard for software engineering, but some new thing is going to be better than what has already been proven effective?

And then the more rare scenario is someone who says they do understand DevOps, they do use DevOps, but they think all jobs will be automated. I'm really curious about why someone would think, if they truly understand DevOps, that narrow AI is going to automate developer jobs. I would suspect the person is a magical thinker or they're a grifter, but maybe they have some reason that they believe this. I'm skeptical though.

In a nutshell, those are my thoughts around how to debug and debunk "AI's automating coding jobs" by using this flowchart type analysis.

Want to hear the full discussion? Check out the complete podcast episode on DevOps and Narrow AI.