The Limits of Standardization

Without standards we would all die. Without a standard kind of air to breathe, there would be mass suffocation. If there was no standard song to sing at birthdays, these events would be pure chaos. Nobody wants to live in a world without a standard keyboard layout so I can type on your keyboard. So stop setting your layout to Dvorak, Ed. That’s really annoying and even you say it doesn’t make you type faster. Standards improve our lives by allowing us to make assumptions about how to accomplish certain things and ultimately form the basis of our work culture. However it’s important to realize that standards are not free and should be evaluated in terms of their costs and benefits.

Incentives to Standardize

Standards are not always proposed with the best interest of users in mind. Standards are created by people with their own goals and interests that might not perfectly align with those of the user and a greater political context has to be taken into account when evaluating a standard. Some companies create standards to sell or license for profit. These standards are designed to be profitable first, and may consequently be useful, but the profit motive may lead to design decisions that might not be good for the user. When a company creates a set of standards that are mutually interoperable but not compatible with the wider ecosystem of related standards, we call that a walled garden. Creating walled gardens can be very profitable because once you buy in, it can be very hard to get out.

The most prominent walled garden for consumers today is the Apple ecosystem. Instead of using the standard USB charger, Apple has created their own charger standard that can only be used with iPhones. This is a great business move for the company for several reasons. For one, instead of paying for licensing of the existing standard, creating their own allows them to license the standard to other manufacturers at a large premium over the existing one. Another thing is that if the iPhone user has bought into Apple hardware, it will be more expensive for them to switch to a phone that uses competing standards because they already own a lot of accessories that don’t work with any other type of phone. They are essentially locked in to Apple now and the locks can be expensive to break. These sorts of lock-ins are bad for consumers and ultimately for markets because they reduce consumer choice and stifle innovation by entrenching the owners of the walled garden in their market position. This is the price we pay as a society to give the company incentive to create a standard in the first place.

At their worst, standards seem to be made intentionally difficult to implement, although I believe this effect is really a natural consequence of there being no incentive to make the standard easy to follow. For instance, the Microsoft Word document format is a standard that can theoretically be implemented legally by other word processing applications. However, alternative word processors like LibreOffice have a very difficult time actually creating an implementation because the standard is not well suited for public use. There’s no incentive for Microsoft to expend the effort to make their document standard accessible to competitors. It would be more efficient to have a common open standard we can all use, especially considering that Word documents are used in the course of some civic functions.

Using a standard controlled by another entity gives them a lot of power over your project. You should only choose standards where you trust that the incentives of the creators of the standard align with the interests of your business.

The Reduction of Minds

Aside from enabling interoperability, another important way standards work is to reduce the number of minds that are thinking about solving a problem. Standards provide readymade abstractions like TCP packets or HTTP headers so you don’t have to think about these things if you just want to run a website for your dog grooming service or whatever. The core internet standards are mostly considered a solved problem and nobody even thinks about any other way that could be done. That’s a great thing when the standard you’ve come up with is good enough and the problem is fairly static like data transfer. But when the problem is dynamic and the future is unclear, and especially when the process is social, premature standardization can lead to a whole host of problems.

Standardization is essentially a controlled automation of thought. When thought it automated, it reduces the amount of consciousness being put on a problem. Since changing standards are difficult, it raises the bar for the benefit required for some improvement to be proposed for the standard. With premature standardization, you lose the ability for many people to come up with competing incremental improvements so it puts a lot of pressure on the creator of the standards to get it right on the first shot. And most often this is an impossible task to complete. When incremental improvements are delayed, technical debt builds and can ultimately lead to a breaking point where a different standard is needed. An then you have yet another competing standard.

Sometimes it’s better to have a period at the beginning of the project where there are no standards at all and just let everyone figure out the problem for themselves. An explicit lack of standardization in some areas is a defining characteristic of the Federal system of the United States government. Certain powers of the government are delegated to the states to allow for experimentation with the rules to find out what is the best way to do things. When processes are decentralized, we can have disagreements in theory and try out a lot of different ways of doing things and let the outcome of the experiments be our guide to good policy. Many times having all those extra minds thinking about a problem is really not wasteful, but the best thing they can be doing to improve the situation.

The best standards arise organically from a diverse group of organizations each with their own experience solving the problem in their own way. Standards that are created by a committee at a single organization sometimes lack the flexibility to be used outside their original context.

Application to the Workplace

As a manager, you are probably a conscientous and orderly person who likes everything to fit into nice little boxes and it makes you uncomfortable when things get messy. So to make the world a bit more understandable, you begin to make standards. There’s a standard web framework everyone should use, a standard http server library, a standard number of approvals for code changes, and a standard code coverage target. At the end of the day you have a big pile of standards and you feel like you’ve accomplished a lot of work.

The problem is that most of the time, the decision process for coming up with these standards is simply the intuition of a few people and sometimes these people get things very wrong. It’s easy to implement a standard on a whim in a meeting when none exists, but to change the standard requires justification which can be very difficult to provide when the original standard was imposed without much justification at all. Sometimes it’s just a matter of one person’s intuition against anothers and the person with the original thought may not even work for the company anymore. Team standards are like any other knowledge asset in that they need continuous maintenance to justify their existence. Rules accumulate and become technical debt like code. Before you know it, you have a ton of crufty old rules that people are blindly following and nobody is getting any work done. This is the number one job complaint I hear from my developer friends.

While it might seem counter intuitive, sometimes chaos is not a problem to be solved, but should be embraced as a normal part of the creative process. Sometimes the best thing you can do is to just sit back and let everyone fight it out and see what comes out of it. The results may surprise you. In the process you’ll be gaining knowledge of all the things that don’t work which can be just as useful to know as the things that do. Never forget that you have the great advantage of real human beings solving your problems for you. And while the human way of solving problems in groups can be a very messy, this should not be treated as an engineering problem, but simply a natural limitation of the human experience.