I read it as, people will direct AI to search for solutions, then use these refined solutions to search for more solutions. A bit like how we use libraries and packages, and improved languages, to enhance traditional programming practices.
Taking an extreme helicopter view I think I can see it, but on the other hand I'm not convinced it's a very interesting observation. Throughout history we used machines to make more complicated machines.
Edit with this quote:
Because you only have to provide Software 1.0 implementation for a small number of the core computational primitives (e.g. matrix multiply), it is much easier to make various correctness/performance guarantees.
This is true, but it raises another question which is (to me) comically hand waved aside:
how do you make correctness guarantees of the output of the neural net? It's not addressed, probably because it's very hard to do so.
It's like the NAND gates inside CPUs and GPUs. Since they are so simple building blocks, they are very easy to verify.
It does not follow that the business logic I write to run on these things, is easy to verify.
The same goes for neural nets, but more so.
I'm not saying these new AI tools aren't useful, they are. But it's easy to misunderstand what they can do.
The "correctness" of neural networks is a vasy field of active research, with many prominent minds in deep learning as well as traditional CS theory working on it. We have many results for small scale networks, but I don't know of any results that can "prove" the correctness of an image classifier, for example. After some point, the correctness of such methods becomes very ill-defined, since unlike the normal programming world where everything is mostly in binary, here you will have to answer questions with some variation of "this is 98% likely" with no scope for 100% certainity.
They definitely don't. There's no way GPT-3/4 replaces software development, but it will save us some typing with fancy autocomplete and prose-ey documentation!
If I didn’t know better, I would think the author of this sentence didn’t know what any of those words mean.