I often think that one of the biggest changes that could come would be the mass adoption of through statistical analysis classes as part of graduate programs.
There's far, far too many people I've know who just don't understand it.
I also see many forces working against it. For one, it'd show up the ignorance of a lot of the academic world.
I've known high level people, like a MD, who simply didn't fully grasp some basic concepts of statistical analysis (like P-value hacking, questioning sample size, comparisons between raw numbers, etc)
Statistics should be part of pretty much every step of education IMHO. Even in elementary school you can teach useful things about reading statistics, high-school math allows more, and universities should cover it too (again with different focus depending on subject and depth of math classes). It's among the most universally useful in life parts of mathematics.
Everyone in these industries takes statistics in college and maybe grad school tool. The problem is (1) that statistics is hard math and these people aren't math majors (2) if you do statistics correctly you can never say anything bold with confidence, which kills your career in any field except statistics itself
In the sense that there are courses like Physics for future presidents, it must be possible to at least attune peoples bullshit receptors to some degree - even if that is without going into the mathematics.
We now live in a time when media algorithmically shows you what you want to see, and politicians have effectively worked out that truth is almost totally unnecessary. Society will adapt, but it could end in disaster.
For example, one thing that most people don't get is that every measurement has an assumption. If you don't know the assumption you may make bad conclusions. Few people even ask this question when making measurements.
Stats courses for non-statisticians should focus on teaching people how to tell when they need to call in an expert. (Which is most of the time, of course. And that is not unique to statistics.)
And academic journals in medicine etc. need to require that research involving the use of statistics has been reviewed and okayed by specialist statisticians, and prioritise publishing null results. The last would save future researchers a lot of time.
I should have been more careful in wording my original comment.
I know there are statistics classes given in most professions. But those classes generally are graded on a bell curve and many people I've known have passed such classes without really 'getting much'.
Also, my comment should have specified statistics as part of critical thinking and analysis of complex phenomena.
A problem with training requirements seems analogous to Goodheart's Law. If you make something a requirement for graduating, then you have to make it easy enough for everybody to pass, in which case it turns into fluff.
But this isn't even statistics. We learned to read log plots in high school chemistry class.
To insert a bit of sanity: from a reply by the author, she indicates that she wasn't trying to introduce a novel concept with estimating an area using right trapezoids. I don't have access to the journal, so I can't read her entire comment.
My comment wasn't, 'doctors don't know math and I'm so smart', but rather the oposite. I was surprise was when I found a doctor who didn't have a firm grasp of basic principals of statistical analysis. And how through my life I've encountered a lot of high level professionals who far exceed me in success, confess to not understanding certain concepts I considered are objectively very basic.
I did statistics two semesters for a Software engineering major but i still cannot understand these graphs!
I know maths, theoretical and applied but statistics is such a difficult thing because you dont have to prove anything, you have to communicate a message to a human person. Even worse, to find these hidden messages within the data.
> I often think that one of the biggest changes that could come would be the mass adoption of through statistical analysis classes as part of graduate programs.
Can anyone recommend a good introductory course, ideally with a path to more advanced courses?
Whenever someone mentions raw numbers alarm bells go off in my head and I immediately assume there's some agenda. This should be normal, especially when people do such atrocious comparisons like x events in America vs y events in Iceland. Yet we see it done all the time.
Sometimes you cannot directly compare two numbers because they are on two different scales. It is basically like saying 10cm is bigger than 5in because there are 10, ignoring the fact that if you normalized the values, 5in would be a larger distance.
Of course, it is usually never as obvious to the casual observer as using inconsistent units of linear distance.
Worse than this, sometimes you can't even compare numbers on the same scale. Inherit assumptions make things difficult to compare. For example: some event may depend highly on the interconnectedness (maximal clique in a graph) than population and this a per capita comparison isn't relevant.
But this is much more than statistics, this is understanding measurements and events. This is really the fundamental problem here, not a lack of understanding statistics (though good statisticians are experts in asking these types of questions).
While I knew the concepts of log I first truly understood it when I started looking at everything as log2. In hindsight I feel like a moron, obviously it is about doubling and halving risks is what it's about (I'm an MD so everything is a risk ratio in my field).
I remember a colegue that was presenting results during their defense of a thesis where one risk ratio 0,45 and the other 2,1. I asked which effect was biggest and they automatically replied 2,1. I'm pretty sure that 80%+ of my colegues would make the same mistake. Everyone understands double/half - we should try to teach people this as the word log is just too intimidating for so many.
The word "logarithm" is an interesting accident of how they were calculated for so long. Exponentiation is easy: at its core it's just repeated multiplication. But doing the "opposite" is much harder, it is not just, say, repeated division. Some of the oldest algorithms were logarithms. Some of the most complex algorithms an average person might study for a large swath of history were algorithms.
One of the most complex functions a slide rule enabled was simply some logarithms.
When such computations were hard and slow, people filled entire books with logarithms tables. Multiple volumes sometimes filled entire stacks in early engineering libraries.
I feel like this illustrates one of the biggest problems when we teach math from a "pen and paper first" perspective: exponents are easy to teach with pen and paper as soon as people are used to multiplication, logarithms which are intricately linked to exponents (as the reverse relationship) get saved for one of the last things to be taught, if they are taught at all, because not only is teaching them with pen and paper hard, it's now even harder that slide rules and logarithms tables are out of fashion, so a lot of teachers skip them.
Computers make long slow calculations so much easier. We have the power to give people some very deep visualizations into things like how directly exponents and logarithms are related, we maybe shouldn't let the historic complexity hide the visual simplicity so much when we teach these early concepts. Logarithms are always going to feel complex and ugly to those that study them with pen and paper alone. We have the technology to improve that.
There's far, far too many people I've know who just don't understand it.
I also see many forces working against it. For one, it'd show up the ignorance of a lot of the academic world.
I've known high level people, like a MD, who simply didn't fully grasp some basic concepts of statistical analysis (like P-value hacking, questioning sample size, comparisons between raw numbers, etc)