I'm curious what your career trajectory was like. I'm surprised that your experiences are so different than mine (see my comment below). In the early years, we had tons of time to just play around (e.g. Paul Graham wrote Hackers and Painters in 2004).
My theory is agile turned software writing into a production line, well it attempted too. Hard to fit experimentation into the everything must be a ticket process/mentality and endless ceremony meetings. Also I think the quality of developers decreased, not sure if agile caused this or it's some sort of work around for it.
> My theory is agile turned software writing into a production line, well it attempted too.
Right conclusion, wrong origin. Let me explain.
Business management theory has been rooted in the lessons learned from Ford manufacturing for over a century. This has worked well for industries which manufacture goods using physical resources, of which most qualify.
However, software engineering is not bound by those forces. Adding more developers to an effort does not shorten delivery nor increase productivity (quite the contrary, actually, and well documented in "The Mythical Man Month"[0]). But adding "line workers" to a factory, assuming sufficient raw materials are available, will shorten its delivery cycle.
Because assembly line workers have a quantifiable job, easily measured based on physical factors, and fairly easily scalable (assuming sufficient factory capacity).
> Also I think the quality of developers decreased, not sure if agile caused this or it's some sort of work around for it.
IMHO, there is no substitute for understanding the problem needing to be solved. No SDLC paradigm can make a developer which eschews this successful.
Quality has definitely decreased, and I think it's the natural consequences of specialization. Most modern devs I've worked with (even/especially those from big tech companies >1B val) know their on particular niche well, but flounder when faced with a different kind of problem. They have a hammer, so everything is a nail. The power of modern infrastructure and orchestration systems has eliminated their need to understand the full stack in order to "deliver value".
From my POV, hacker culture is going away. Because it does not Scale in the way capitalists want it to scale. And the same capitalists are foaming at the mouth at the notion that they might be able to replace expensive engineers and developers with AI.
Our niche has been captured by global stakes, and those stakeholders are all too happy to believe that they can scale innovation without all of the previous "cultural baggage" that, IMO, is the only reason we have the systems that we have today.
I don't think hacker culture is going away, I think it's just drowned out by software eating the world in a capitalist economy. It used to be that software and computers in general didn't pay any better than any other white collar job, and were generally more arcane and less familiar to people, so only those of us with an inherent interest were drawn to it. I believe there are more of us than ever, there's just orders of magnitude more people drawn in for the money and power.
I certainly feel some nostalgia for the old days, but while I'm not thrilled by a lot of directions the internet has taken, I don't think there's ever been a better time to be a hacker in terms of tools available and what can be achieved by an individual or small group. Getting attention for your work is another matter, but distribution has always been hard, the internet making it easier to deliver bites just led to that much fiercer competition. The fact that there was a short-lived window where technical barriers favored hackers was just a coincidence of history, not a stable state that it makes sense to try to replicate.
I always understood that Agile was supposed to reduce the bureaucracy, not increase it. It seems to have been embraced, extended and extinguished by the sort of people who were pushing Waterfall in the previous era.
>Also I think the quality of developers decreased, not sure if agile caused this or it's some sort of work around for it.
I think it's mostly a function of developer quantity and the pervasive "anyone can do this" attitude. (My assessment: most people probably could, but fundamentally aren't comfortable using their brains the right way.)