Configure This!

My blog posts related to IT strategy, enterprise architecture, digital transformation, and cloud have moved to a new home: ArchitectElevator.com.

Having been to a number of conferences and their attached expo floors recently it was not surprising that I was not able to escape the line "this tool does not require programming, everything is done through configuration". Hearing this tag line for the third or fourth time got me wondering, "what really is the difference between coding and configuring?"

Good Intentions

Just to be sure, this is not intended to be vendor bashing (even though it can sure be tempting sometimes :-) ). Seriously, buying tools can save a lot of development work that we can buy off the shelves. These tools make our lives easier because they do a lot of the grunt work for us. Libraries and tools also serve an important community function by widening the reach of a specific technology. Yeah, a lot of us guys who like to have dinner with the Microsoft Indigo team or the Java OSS crowd for fun sure don't mind dabbling with WSDL elements and writing our own XML serializers. However, if your main task is to solve business problems and your boss won't let you go to TechEd because he (or she) considers it a waste of time, having a tool that lets you stay out of the nitty gritty can be a lifesaver.

Levels of Abstraction

The primary technique of making developers' lives easier is to raise the level of abstraction at which they have to work. Very few of us still write assembly code, read individual file blocks from a disk, or put individual packets onto the network. This level of detail has been nicely wrapped behind high-level languages, file and socket streams. These programming abstractions are very convenient (try doing without them!) and dramatically increase productivity. All I have to do is know a file name or a URI and bytes come streaming in and out. Of course, if abstractions are this useful, the question is whether we can add new abstraction layers to increase productivity further. If we are able to provide enough abstraction, we might be able to do away with coding and can create a solution simply by configuration. Or can we?

Variability Points

In our quest to provide more and more abstraction and to simplify the programming model further and further we come up to an interesting challenge: We want to simplify our world but we do not want to lose too much flexibility. If I need to store individual data records on my hard disk (for example to create a rapid direct-access indexing system) a file stream abstraction doesn't do me much good. To make our life actually easier, it's not enough to just paint a thin visual veneer over the existing programming interface. Instead we have to actively reduce complexity, and therefore likely some flexibility. This exercise is quite similar to the architecture guessing game I talked about before. If we take away the wrong things, our abstraction is limited in its usefulness. If we take away too few things, we didn't accomplish much in terms of simplifying our (programming) life.

Domain-Specific Languages

Two ways lead us out of the abstraction and variability conundrum. For one, you can try to get more people to think along your mental model. This'll make them less likely to desire features that you eliminated when you created the abstraction. This is the approach of the evangelists and, to some extend, the patterns authors. Alternatively, you can avoid limiting yourself to a single general-purpose abstraction, instead providing different abstractions for different purposes. This approach is great as long as you can make creating these abstractions easy, which is usually not the case. This is the very problem the Domain-Specific Language (DSL) folks are trying to solve by providing Language Workbenches. The workbenches are tools to create new languages that provide specific abstractions. DSLs are still in very early stages but promise to become quite useful.

Configuration or Programming?

But back to our original question. What kind of abstraction deserves to be called "configuration" versus "high-level programming"? We are often led to believe that everything that sits in an XML file can be considered configuration. This can be true for very simple scenarios, but XSLT programs are also XML files, and those things are definitely programming, and not for the faint of heart at that. A better criterion that comes to mind is whether what you create is executable. If the algorithm is predefined and I only supply a few key values this is more likely to be considered configuration. On the other hand, calling a method with parameters is not really any different and is certainly considered programming. Also, a "configuration" value can decide between different algorithms, so it has some impact on the program execution. Many development platforms actually allow us to accomplish the same tasks via configuration or code, depending on our preference (Microsoft Indigo is a good example where this is done fairly consistently).

So how about distinguishing by whether the things you enter are executed in a specific order. delineating pure data from a set of sequential instructions. This looks like a reasonable separation, even though some definite gray areas remain, especially when using declarative programming languages, for example to configure rules engines or our favorite language XSLT. Declarative languages focus on specifying relationships between constructs and do typically not presuppose a specific execution order as procedural languages would do. Also, aren't coding instructions just data for the execution engine? Apparently, it is still largely a matter of perspective.

When you make a system configurable, consider carefully whether you are really just configuring or whether you are actually programming at a high level of abstraction. Considering your mechanism as programming can remind of the value of proper tool support like editors or validation.

We once built an application from individual components that were composed via channels, with "configuration" files telling a component which channel(s) to receives messages from and which ones to send message to. Being a common setup in EAI tools, where adapters and such are configured to talk to each other via specific channel names, we stored the data in local XML files. After a while, though, a lot of errors in the system occurred due to misconfiguration: components would not talk to each other because the channel names were misconfigured or the flow of messages was wrong because we chained together the right components in the wrong order. This started to get us thinking that what we created was not a simple configuration file but really a high-level programming model for the composition layer of our system.

Ending the shadow life of the config files and making them first class citizens caused us to check these files into source control like any other source file, and to create a set of validation and management tools to perform efficient debugging and troubleshooting (see my related earlier post) . It also reminded us that providing this "configuration" is useful but it does not automatically mean that it is trivial or does not require skilled developers.

So, I'd summarize my advice on configuration as such:

If changing settings can break your application, you better consider that coding.

Configuration Progamming

Now that I am back on-line I can include a link to a dictionary entry on Configuration Programming. Here it states that:

Configuration Programming: An approach that advocates the use of a separate configuration language to specify the coarse-grain structure of programs. Configuration programming is particularly attractive for concurrent, parallel and distributed systems that have inherently complex program structures.

This definition highlights that academia have been thinking about the gray area between configuration and programming for some time. One example of a language designed specifically for the described purpose is Darwin.

Conclusion

Abstractions are a good thing, but let's not get carried away thinking that calling the abstraction "configuration" is suddenly going to eliminate the need for developers. Instead, treat configuration as a first-class citizen that requires design, testing, version control, and deployment management.