Chapter 2 started out with good intentions. Unfortunately, as was my style at the time, I never got to the point.

This time around, I will focus on the issues of classification in IT management and why IT confuses the business by frequently shoving square pegs in round holes. We will explore the practice of taxonomy which organizes observations for long-term study. To conclude, I will bring my observations forward twenty years by discussing the implications of artificial intelligence on technology management and measurement.

Confusing the Business

Maybe it is easier for the CFO, for example, because they have the backing of financial audits and many years of standardized historical performance. (Wigodsky, 2004)

The idea behind Chapter 2 was to present reasons why organizations needed to manage (and therefore measure) information technology differently. At the time, leaders measured and managed IT mostly by the big costs: hardware, software, and operations. Beyond that, it was anybody's guess what taxonomy we would find each time we met a new customer.

For decades, we constrained technology by design. Leaders created entire departments around vendors’ software products. Vendor "lock in" was a real problem, and replacing a vendor was a serious undertaking. The structure of each technology management organization had become more about the unique mix of vendors and less about the common services provided to the business.

IT was confusing the business: every line item on the budget needed explanation. There was not a straight line between technology components and revenue. As a result, comparisons between different strategies were less about business impact and more about features. There was extraordinarily little science applied to managing IT. Twenty years later, we still struggle from an inability to naturally compare these extraordinarily complex systems.

Pigeonhole Problems

Have you ever tried to make a challenging decision with a team of four? (Wigodsky, 2004)

In Chapter 2, I described the Pigeonhole Principle, but I failed when it came to describing its implications.

I pointed out the obvious: if you have enough holes (categories), you can fit lots of pigeons (technology components). Too few holes, and your pigeons get crowded. Too many, you waste space (like I did). Unbalanced and uncertain holes lead to confusion when placing your pigeons. When I authored the book, there were far too many pigeons shoved in far too few holes; we need to move beyond this metaphor to find the right categories to fit all the different technological components.

As a result, technology management was often a force fit. We tried to put all the hardware in a few holes and all the software in just a handful of others. “Operations” was often a gigantic catch-all for everything else. Terms like enterprise architecture and business architecture were still stuck in the intelligentsia and most organizations gave them little regard. Vendors offered huge financial incentives for bundles, and often the most innovative solutions were crowded out by premature commoditization. Once again, we alienated the business by organizing around categories with little meaning outside IT.

I am happy to report that in twenty years' time, ITIL and cloud computing caused a complete restructuring of IT classifications. Even with significantly more holes and pigeons, there is also more consistency between different organizations. We could not have scaled clouds like Amazon Web Services or Microsoft Azure without common classifications - it would be far too complex for consumers and investors if every customer had a distinct set of billable components. You can go to any cloud vendor to lease a virtual CPU or a virtual storage disk. It is even getting easier to move between vendors (at this layer, anyway).

It is still extremely rare that critical systems (groups of components) like those in the infrastructure are meaningful beyond IT. Software-as-a-service brings us the ability to hide these costs while still passing them on to the business function, but this ability often comes with challenges to technology governance. As cloud computing took off, this led to a widespread concern called "Shadow IT" where the business ignores the technology mumbo jumbo and just does their own thing. Shadow IT has shown up again in adopting generative AI; this time it's a lot more dangerous.

Subatomic IT: Lizard Hips and Spooky Action at your Fingertips

Like physicists and paleontologists, information technologists will be required to dig into our solutions and classify them, to make them more palatable for customers. (Wigodsky, 2004)

People love to classify. We want all our pigeons to fit nicely in their holes. When applied to scientific study, classification and taxonomy give us tools to organize our hypotheses and evidence, helping us to identify patterns, trends, and undiscovered knowledge. The significant disadvantage for IT management science seems to be that massive branches of technology classification often disappear altogether.

I sensed that the problem with putting IT into pigeonholes is that our highly customized pigeons are always in motion. Twenty years ago, you could not build most businesses without physical server hardware; that was often the largest budget item after operations. Today, more businesses have zeroed-out the hardware budget in favor of cloud CPUs and services. At the end of Chapter 2, I looked to other domains of science for guidance on how to create stable classifications/taxonomy for studying technology management.

In 1879, famed paleontologist Othniel Charles Mars named one of the largest dinosaurs, a sauropod (“lizard-hipped” dinosaur) named “Brontosaurus” (The Brontosaurus Is Back | Scientific American). Twenty-four years later, another scientist noted similarities between Brontosaurus and Apatosaurus, and over time the two dinosaurs converged on the evolutionary taxonomy. One hundred years later, the evidence seems to point to a new conclusion based on additional evidence: Brontosaurus has a longer, thinner neck than Apatosaurus. Once again, these are two different dinosaurs. Because paleontologists organize dinosaurs according to physical characteristics, such as the shape of their hips or the length of their necks, it is easier to add and remove layers necessary to define distinctiveness.

It took at least 100 years for physicists to develop and prove the Standard Model (What is the standard model? | Space) now used to describe and measure physics. If we assume that information technology management started in a British tea shop in the 1950s (How the computer changed the office forever - BBC News) that means that IT is getting very near to our century mark. If physics left breadcrumbs for IT, then it is time to start work on a Standard Model for IT. First, we need to address the elephant in the room.

Overfitting, Underfitting, and the Real World of AI

Today, there are real world, life-and-death implications of the pigeonhole principle. Ask any data scientist about underfitting or overfitting a model, and plan for a lengthy conversation (Overfit and underfit  |  TensorFlow Core). Ask the same question of a healthcare data scientist, and you will quickly turn to health equity and patient outcomes (e.g., death). Today, people in adjoining postal codes can have decades of difference in life expectancy. AI models may provide the path to determine why, and what to do about it, but only if we learn to treat AI like the machine it is.

There is inherent risk in AI, and particularly generative models (e.g., ChatGPT, Llama, Claude).

Consider Tay, one of the early generative models unleashed onto Twitter by Microsoft in 2016 (Tay (chatbot) - Wikipedia). It took under 24 hours for humanity to teach Tay to imitate all our most disgusting and vile beliefs. When a model shows bias or hallucinates, it is often because the pigeon-to-hole ratio is off. Fixing that issue is one of the biggest problems with attention-based AI, and it has gigantic implications on how we must treat these models when applying them to technology management and governance.

As I write this in July of 2024, generative AI is a toddler. Seriously! The similarities are stark. Stay tuned; I will talk more about generative AI toddlerhood in my next article, reviewing Chapter 3: Using RAPID to Identify Relationship[s] Between Architecture and Process.