Let’s zoom out today to discuss beyond AI in software engineering.
I have always believed that Generative AI based on the auto-regressive Transformer architecture is a new tier of technology, but never dived deep into the first principles of why I felt that way. Empirically, the “intelligence” of GenAI has led to a whole new suite of capabilities that were traditionally impossible. But it goes way beyond that. GenAI changes the way knowledge is being processed, which in turn affects the materialisation and de-materialisation of our creativity.
Materialisation and Technology Link to heading
In this discussion, I refer to the materialisation of creativity that results in new technology. The reductionist generalisation of innovation could be characterised as the following pattern: a problem with a long prehistory, sudden simultaneous breakthrough by rivals, and then incremental evolution over time. We shall call this inventive process “materialisation” today - simply because we productively materialise an idea, a concept for measurable outcomes. This pattern hasn’t been broken since we first discovered fire. Every innovation creates a network effect and multiplies the adoption rate of technologies that are more abstracted. This is the eternal propellant for civilisations.
Centuries of De-Materialisation Link to heading
Since the invention of language and the printing press, we have begun the journey in the opposite direction as well - De-Materialisation. Transforming physical inventions that are materialised into less-physical forms that can then be transplanted and their ideas transferred for replication by other humans. This creates the network effect of singular breakthroughs and creates a fertile ground for further evolution by faceless and uncooperated contributors.
Most of the information technology in the 20th century focused on accelerating the de-materialisation journey. From printed words to Web 1.0 with text-based pages, then multimedia and internet massively extending one’s reach of knowledge.
But all the information technology so far are one-way functions. They either facilitate the process of materialisation (virtual to physical) or de-materialisation (physical to virtual), never both. Software is the most direct form of materialisation technology where we can transform virtual ideas into physical experiences without the traditional need for materials. On the other hand, Wikipedia as a product dematerialised probably trillions of articles and pages.
One may argue that we did create dematerialising technology using materialising technology e.g. Youtube is created by Software. But it is always a pendulum swing between materialisation and dematerialisation, never a singular step process. Until GenAI came along.
A New Virtualisation of the World Link to heading
We have been storing representations of this world in low-dimensional data structures with our computing technology so far. But GenAI represents “knowledge” in high-dimensional embeddings. This is a significant change. The embeddings can be a universal representation and a singular model of the world without the need for multi-variant decoding unlike language. In a low-dimensional data representation of the simple idea “The sky is blue”, expressed in English words as you read it, is a siloed representation that is limited in its opportunity to influence. The receiver of the idea needs to be actively engaged (listening) and also be able to comprehend it (understands English).
But the vector embedding model creates a significantly more universal way to document and transcode concepts and ideas. One does not need to compress ideas and concepts in a lossy representation such as a low-dimension data structure like language, where nuances are lost for sure. For the first time, we can fully encapsulate an idea in every aspect. And you will be able to replicate this idea without loss (language-based replication is always inaccurate).
Then, Materialise! Link to heading
In the application of GenAI technology, we are currently trying to remove the friction from acquiring knowledge to applying knowledge with tangible outcomes. From the impressive chatbot in 2022 to all-capable agents in 2025. From virtual to physical, the materialisation process. So the process of materialisation is now quicker, cheaper, and more accessible by humans. This leads to more creations in the world which can be the foundation for further evolution. The rapid process of de-materialisation can consume these new creations and amplify the network effect across distance and time.
Till we run out of silicon on this planet, this may fuel a flywheel of innovation that can take our civilisation to the next level. After all, we might live up to the version of future envisioned in 90s sci-fi movies soon!