There's always a bigger goat: don't let big problems stop you solving smaller problems
Photo Credit: Bailey Mahon via Unsplash
In the story of the three billy goats gruff, the goats want to cross a bridge guarded by a troll. They manage this by each telling the troll that there is a bigger goat just behind them until (spoiler alert!) the biggest goat comes along and butts the troll into the sky.
Sometimes, when we are trying to make the case for enterprise technology capabilities, it feels like we are the trolls, and that we are so scared of the biggest billy goat that we won’t tackle the smaller goats. When we look across our technology landscapes, we see mess, waste and mayhem, and wish that we had some of the foundational capabilities that would help clean things up. Yet we hesitate, because we know that every time we build something we will uncover another problem, and another problem, and another problem, until we get to problems that are so big that we cannot imagine how to solve them.
Let’s take an example. Despite decades of middleware, data lakes, service buses, event based architectures, portals, queues and gateways, the business of getting systems to talk to each other and share data and functions remains difficult. No presentation on digital transformation, data-driven business or the AI-enabled enterprise is complete without a reference to the valuable data that is locked up inside legacy systems - but rarely contains a solution to get at that data.
This is not because such solutions don’t exist: we have many products, patterns and approaches to enable reliable and secure interaction between systems. But implementing those solutions is hard, and especially subject to the billy goat gruff problem. The technologist may propose the implementation of an integration mechanism, only to be told that the real problem is that we don’t have consistent data schemas. The architect may propose a simple API standard, only to be told that the real problem is that systems aren’t configured for secure access. The security professional may design an identity and authorisation mechanism, only to be told that the real problem lies in data sharing and privacy regulations. The Chief Data Officer may put in place policies and standards for compliant and responsible use of data, only to meet the biggest billy goat gruff of all - a presumption that there is a culture against data sharing, and that people would rather live in siloes.
Sometimes people are so discouraged by this parade of billy goats that they lose the will to make any improvements at all. What’s the point of deploying an integration mechanism if you don’t have the data structures, security and policies you need - and if people are not culturally disposed to share data in the first place? Sometimes people valiantly attempt to jump the queue and tackle the big cultural billy goat lurking at the back. If they can get people to care about the availability and usage of their data, then perhaps they can create the momentum to sweep away all the other problems. Unfortunately, in the absence of practical solutions, cultural change usually turns out to be just too hard: people are most likely to be indifferent (‘Come back when you’ve got something that works.’) or to offer meaningless consent (‘Of course I’d love to share more - but I can’t.’)
I think that our best approach is to tackle one billy goat at a time. Few capabilities are built in one go: they are constructed out of components and built in stages. Each stage will have some value. In our data sharing example, an integration platform may do little more than contain the proliferation of point-to-point interfaces. That is something worth having, even if it does not solve the big problems of compliance and culture.
Furthermore, most technology components, especially those which provide general purpose capabilities such as integration, turn out to be more useful than we expected them to be: when we build them, we can never be sure of the full range of purposes for which they will be used. One step along the path turns out to be many steps.
And, as we steadily build things that are useful, we shrink the size of the goats waiting in the rest of the queue. A well-designed integration platform takes away toil, and gives people the capacity to design good APIs. Good APIs can be integrated with identity and authorisation mechanisms. When the flow of data is visible, well defined and well secured, it is easier to achieve and demonstrate compliance. And when all these things are in place, perhaps even the big, scary cultural billy goat will look smaller. Things that work and are useful are remarkable persuaders.
We started with a fairy tale, and we can end by framing our conclusions in the simple language of fairy tales:
Useful things are useful.
Useful things are more usefully useful than we think.
Start with the billy goat in front of you.
