Sources of complexity in software
There are two sources of complexity in software: the complexity inherent in the domain (essential complexity) and the complexity we add as programmers due to the platform or due to bad programming practices (accidental complexity).
►► Audio, Video, and Transcript available: https://lispcast.com/sources-of-compl...
►► Subscribe on iTunes: https://itunes.apple.com/us/podcast/t...
Transcript
What is the main obstacle to writing correct software on time and on budget? Hi, my name is Eric Normand, and these are my thoughts on functional programming.
There is a book called "The Mythical Man-Month" that talks about one of the main challenges of software is its complexity. That software is just complex. It's just going to take time and energy to make it correct, to make it work for us. I think that's fair to say. In the book, Fred Brooks, the author, talks about two kinds of complexity, essential complexity and accidental complexity.
Essential complexity is a complexity you cannot avoid and accidental complexity is complexity that you add, that you didn't need to. There might have been a way that you could've gotten along without it.
For what it's worth, that's a fair way to divide it up. There is a paper about 14 years ago called "Out of the Tar Pit." This is a very influential paper in the functional programming world, because it does a little bit better at dividing up the complexity, of defining the two types of complexity than Fred Brooks did.
For instance, Fred Brooks said, "Oh, there's just a lot of state in software and that's unavoidable." As functional programmers, talking about state as unavoidable, we know that's not true. You can avoid it.
This paper defines essential complexity, I think, in a more useful way where essential complexity is complexity that is due to the domain. If you are going to be making a piece of software about rocket science -- rocket science is just complex by itself -- your software has to have at least that much complexity, as much complexity as rocket science.
All of that is due to the domain. It's essential. You wouldn't be doing rocket science correctly if you didn't have that complexity in it. However, there is another kind of complexity, which is all the complexity that we add as software developers. This is everything that has nothing to do with rocket science.
This is threads, the file system, exceptions, syntax errors, types. All that stuff is complexity that is added on to the system if it adds complexity. Some things you could argue. These are actually removing complexity.
In general, if there is some complexity and it's due to something that when you name it, it is not naming a rocket-science thing it's naming a computing-thing, that is actually accidental complexity.
What this gives us is a minimum floor for our complexity. Everything else above that is all accidental. We can work with it to reduce the complexity or at least manage it better. Know that we're introducing complexity but we're managing it.
I say we're introducing complexity. For instance, the platform you're running on is going to have complexity. It's going to have accidental complexity by itself. That means you're making Web-based rocket science software.
Now you have to deal with browsers, the DOM, the AJAX requests, HTTP. All this other stuff that has nothing to do with rocket science, that just has its own semantics and some of those semantics have complexity to them.
You have to deal with timeouts of your network requests. You have to deal with all the different response codes when maybe you only needed two. These are the kinds of complexities that start adding up.
In general, we see in most software that the accidental complexity swamps out. It's way bigger than the essential complexity. That means that in your rocket science software, most of the complexity is due to the implementation and not to rocket science itself, which is a sad fact, but that's also optimistic.
It means there's a lot of room for improvement. That we're doing this to ourselves as programmers. We're adding in...Some things are maybe functional or operational requirements that are not part of rocket science.
You might have a functional requirement that says, "We have to implement all of rocket science." That's great. Then you might have a requirement that says, "It has to run in the Web browser." This has to be Web-based software. Now, it's a requirement.