The era of compounding language

I was having a debate about whether the arrival of AGI is going to result in a singularity/foom/exponential-growth scenario. I'm not sure about it but I'm sympathetic to the idea that there's diminishing O(log(n)) like returns to efforts that at some point might slow things down.

This is because the space of knowledge and intelligence is exponentially big when you dive into the details and look at all the edge cases and interactions between pieces of recognizable order. So even if you have exponential improvement in science and technology, you might only get linear growth of coverage of the details of that space. But an exponential pace of improvement is very difficult to maintain. That's why you often get s-curve progress.

This made me wonder why then did human scientific progress subjectively seem close to linear in the last few hundred years?

This would imply a continuously exponential growth of knowledge which should be very difficult to maintain.

My impression is that this did really happen.

We got into a regime where one discovery in science or technology, new models, abstractions and new languages like math or other technical jargon enabled further discoveries which in turn enabled even further discoveries. Advances basically compounded on themselves, the current state of the art always relying on a growing foundation of past discoveries to boost the next advancement. The invention of the printing press may have marked the start of the era of compounding knowledge driven by evolving propagated words, the population of educated researchers exploding at the same time further compounding progress.

The compounding effect seems to have been key to get super-linear knowledge growth and thus closer to linear expansion of coverage of the knowledge space. This may or may not be able to continue indefinitely, though language models may keep it going for a while.

This led me to think about engineering processes. Could this be a fundamental pattern in technological advancement we can leverage? How do you prevent progress from slowing down to a crawl from getting bogged down by O(log(n)) curves in complex domains?

One way might be to find those areas where advancements can compound on each other.

If you believe in software and code as the ultimate way to encode knowledge, this might explain why a build and iterate agile process works so well. It's a direct application of this principle. An iterative process means each new iteration can compound on the previous ones. At each step, you get to reuse the knowledge that you built previously to thrust the current iteration. Code is a blueprint, a plan for software and that plan gets to use new abstractions and domain language from previous steps. This makes code a planning language that gets more expressive and powerful as you progress. This may be also true to a lesser extent of non code plans, especially if they are written in technical languages. These plans also benefit from compounding refinements in expressivity of evolving expert jargon.

The frequency of multiplicative compounding seems key to progress. Abstractions or DSLs that build on top of each other might be necessary to make good progress in large complex spaces.

Early steps in this process, when still far from a final design, might still contain valuable early abstractions that open a path towards optimal solutions, abstractions that can more easily be refactored or refined into the optimal solutions. This might explain why the Ralph Wiggum loop works in software development using AI agents.

It's likely possible to structure code to promote this via an architecture that enables good abstractions that can build on themselves and maximize how much knowledge is iteratively reused, rebalanced, refactored and compounded.

The principles that made the printing press and scientific languages powerful can apply at every scale from the process and architecture of an evolving codebase to the accumulated knowledge of civilization. Whether knowledge can keep compounding forever remains an open question.