>Code can be very easily verified by linters and type systems so the problem of verification is much simpler than in consumer use cases without linters and type systems.
you are confusing (syntactic) validation from verification. verifying code is an incredibly hard problem.
You can get a lot of value out of a models even if they are not capable of AI development because most people aren't doing things that are as complicated as AI development.
I don't think AI development is complicated. It's just a bunch of functions with parameters which are optimized with gradient descent. The AI development loop is extremely simple and most AI "research" is basically stacking standard tensor operations and seeing what works. It's surprising there is no company that is applying AI to AI development since it is essentially a game with symbols and very well defined measurable outcomes.
you are confusing (syntactic) validation from verification. verifying code is an incredibly hard problem.
You can get a lot of value out of a models even if they are not capable of AI development because most people aren't doing things that are as complicated as AI development.