I often talk about the moving “commodity line”, this line affects essentially all technology, including designs. Essentially, when any new technology comes out it will start highly proprietary, complex and expensive. Over time the technology moves towards openness, simplicity and becomes inexpensive. At some point any given technology becomes goes so far in that direction that it falls over the “commodity” line where it moves from being unique and a differentiator to becoming a commodity and accessible to essentially everyone.
Systems architecture is no different from other technologies in this manner, it is simply a larger, less easily defined topic. But if we look at systems architecture, especially over the last few decades, we can easily system servers, storage and complete systems moving from the highly proprietary towards the commodity. Systems were complex and are becoming simple, they were expensive and are becoming inexpensive, they were proprietary and they are becoming open.
Traditionally we dealt with systems that were physical operating systems on bare metal hardware. But virtualization came along and abstracted this. Virtualization gave us many of the building blocks for systems commonidization. Virtualization itself commoditized very quickly and today we have a market flush with free, open and highly enterprise hypervisors and toolsets making virtualization totally commoditized even several years ago.
Storage moved in a similar manner. First there was independent local storage. Then the SAN revolution of the 1990s brought us power through storage abstraction and consolidation. Then the replicated local storage movement moved that complex and expensive abstraction to a more reliable, more open and more simple state.
Now we are witnessing this same movement in the orchestration and management layers of virtualization and storage. Hyperconvergence is currently taking the majority of systems architectural components and merging them into a cohesive, intelligent singularity that allows for a reduction in human understanding and labour while improving system reliability, durability and performance. The entirety of the systems architecture space is moving, quite rapidly, toward commoditization. It is not fully commoditized yet, but the shift is very much in motion.
As in any space, it takes a long time for commoditization to permeate the market. Just because systems have become commoditized does not mean that non-commodity remnants will not remain in use for a long time to come or that niche proprietary (non-commodity) aspects will not linger on. Today, for example, systems architecture commoditization is highly limited to the SMB market space as there are effective upper bound limits to hyperconvergence growth that have yet to be tackled, but over time they will be tackled.
What we are witnessing today is a movement from complex to simple within the overall architecture space and we will continue to witness this for several years as the commodity technologies mature, expand, prove themselves, become well known, etc. The emergence of what we can tell will be commodity technologies has happened but the space has not yet commoditized. It is an interesting moment where we have what appears to be a very clear vision of the future, some scope in which we can realize its benefits today, a majority of systems and thinking that reside in the legacy proprietary realm and a mostly clear path forward as an industry both in technology focus as well as in education, that will allow us to commoditize more quickly.
Many feel that systems are becoming overly complex, but the opposite is true. Virtualization, modern storage systems, cloud and hyperconverged orchestration layers are all coming together to commoditize first individual architectural components and then architectural design as a whole. The move towards simplicity, openness and effectiveness is happening, is visible and is moving at a very healthy pace. The future of systems architecture is one that clearly is going to free IT professionals from spending so much time thinking about systems design and more time thinking about how to drive competitive advantage to their individual organizations.