As a software engineer, it’s important to understand what leaky abstractions are, how to spot them, and how to mitigate against them. The term often remains unused by teams, most likely due to gaps in knowledge or training, or for fear of causing offence to colleagues. Understanding its core concepts will help you become a better developer, allowing you to spot potentially poisonous changes that may wreak havoc on your systems in the not-too-distant future. So what does the term mean? How can we spot it? What are some good examples? And when we do spot it in a codebase or architecture diagram, what mitigating actions can we take?
Put simply, a leaky abstraction is any API in your codebase or architecture that reveals too much information about its underlying implementation. One example most of us have seen is an API that returns its fields in all upper case letters, each of which matches the underlying database’s column name. Or an API whose functions and parameters match precisely those of its underlying implementation. Good API design – and by API I mean any interface we use to communicate with a software program, not just an API in the sense of a RESTful service – shields us from the information we don’t need to know. It asks only for that which it needs, and obfuscates unsightly legacy software that might ultimately be serving up its functionality.
When implemented correctly, an abstraction holds up against any and all use cases you can throw against it. If you’ve missed something, it is easier than not to plug in. Writing software to the right level of abstraction, avoiding leaks, is our toughest challenge as developers, but it also offers the greatest rewards. Developers love to be correct, and writing code or a design that holds up to any and all scrutiny feels great. But more than that, it proves that we have understood the problem and have created a solid solution that tackles our problem well enough for now.
An example of a leaky abstraction comes from my current contract, which was a leaky abstraction introduced by a third party into one of the solutions I was working on. We had held meetings regarding service contracts, and had proposed a minimal contract consisting of an entity type and id. . Consequently, the HTTP API would need to do some fetching of resources under the covers, but in doing so it would shield the outside world from needing to know more than the bare minimum. This allowed existing systems in the architecture to consume this service without any need to retrieve additional information in order to call the new API. Unfortunately, the third party didn’t grasp this concept, and decided to introduce several additional fields. Now, it’s at this point I should mention that this was an API over a well know big-name document management platform, the client’s system-of-choice for storing documents. The structure in which the documents were held involved some client metadata, some subsystem data from the client’s CRM system of choice, and one or two other friendly-names from said system. These references, metadata and IDs were now all required as part of the API contract. The rationale was that the API shouldn’t need to fetch additional data from the CRM system ‘because it would be too slow.’ This sounded alarm bells in me. A classic leaky abstraction: The document management system’s underlying file system was bleeding profusely out of their API. I set into motion trying to explain this to the third party. The issue was that any consumer of this API now needed to know a hell of a lot more than it ought to. Want to upload one attachment to a note on a back office incident? You now need to fetch customer details along with the parent hierarchy of IDs relating to the note you’re administering. These changes would permeate throughout the system. I tried to explain that the ‘slow’ call made in the API would now be spread into tens, if not hundreds, of calls from outlying systems that need to fetch the aforementioned data, however the third party – and unfortunately the client – simply did not get the concept. I had failed to appropriately explain the issue and it’s potential for cost. Several weeks later we wound up specifying a new piece of the solution which would cost the client a lot of money to implement, all because the leaky abstraction had not been plugged.
So how can you spot a leaky abstraction? Start with an empty API, and introduce a method or endpoint that has a requirement. Now what’s the smallest amount of information you can introduce that allows you to achieve the APIs goal? Make sure you understand whose job it is to know what information. In the above example, whose job is it to know the underlying folder structure of your document management platform? Is it a website that allows users to add notes to incidents, or the API whose job it is to shield users from such mundanity? Consider why you are building this API in the first place. You are attempting to build something of value; a streamlined way to access a capability. The most streamlined way possible is a clear, concise, and expressive API, with an ability to perform small, discrete tasks to manipulate its internal state while shielding the outside world from how you are manipulating that state.
Perhaps the best question we can ask, again using the example above, is how our API contract look were we to move away from the current document management platform? If we can say that our concepts and naming conventions stand up well, then we have a solid abstraction for document storage within our domain. If we realise we have parameters called parent folder or vnd_doc_sp_search_id, then we have a leaky abstraction.