Is that not the human experience? I have no “agency” over the next thought to pop into my head. I “feel” like I can choose where to focus attention, but that too is a predictable outcome arising from the integration of my embryology, memories, and recently reinforced behaviors. “I” am merely an observer of my own mental state.
But that is an uncomfortable idea for most people.
If this was true, you could lay back relaxed and watch where your brain takes you. But we experience life as a never ending stream of choices, usually between what's easy and what's right, and pursuing the right choice takes a constant effort. We are presented with problems and have to figure out solutions on our own, with no guarantees of success.
This "I'm just an observer" idea may be true at some higher level, if you're a monk on the threshold of nirvana, but for common folk this mindset leads nowhere.
The „making choices“ could just be an illusion where it’s already clear from the inputs what you’re going to do. You rejecting determinism could already be determined and you couldn’t even choose to believe it. That’s not falsifiable of course, but your point doesn’t really contradict the idea of determinism due to the possibility.
But that is an uncomfortable idea for most people.