To be fair, very few people can really give informed consent on this topic. There's a million and one ways that you could be taken advantage of in a scheme like this.
This is true, but the right to inspect software has meaning even if it isn't used. It can be used by other people besides the proverbial "average Joe", and non-techs can ask tech friends about software. Every technical person also started out as a non-technical child.
This is also a baseline, not the endgame. Software freedom is a necessary but insufficient condition for user freedom. I wrote about other conditions as well. HN discussion: https://news.ycombinator.com/item?id=25982860
Yeah that's why there are people who argue non-free software should be illegal entirely but it's much harder to convince people of this (although Apple and Google certainly have made it an easier case to make.)
That the subject matter is too complex to comprehend should be an additional reason to ban this practice entirely, not to accept it under the guise of "fairness".
