Posts tagged "psychology"
by far the greatest weakness in the “publication regime” in social sciences today is the systematic disregard for basic principles of valid causal inference, a deficiency either in comprehension or craft that is at the root of scholars’ resort to (and journals’ tolerance for) invalid samples, the employment of designs that don’t generate observations more consistent with a hypothesis than with myriad rival ones, and the resort to deficient statistical modes of analysis that treat detection of “statististically significant difference” rather than “practical corroboration of practical meaningful effect” as the goal of such analysis (especially for experiments). This problem is 1,000x as big as “fraud” or “nonreplication” (and is related at least to the latter, which is predictable consequence of the substitution of NHT rituals for genuine comprehension of causal inference)
slam dunk at Evaluating evidence from published research « Statistical Modeling, Causal Inference, and Social Science Statistical Modeling, Causal Inference, and Social Science

In one of his stories, science fiction writer and curmudgeon Thomas Disch wrote, “Creativeness is the ability to see relationships where none exist.” We want our scientists to be creative, but we have to watch out for a system that allows any hunch to be ratcheted up to a level of statistical significance that is then taken as scientific proof.

Even if something is published in the flagship journal of the leading association of research psychologists, there’s no reason to believe it. The system of scientific publication is set up to encourage publication of spurious findings.

Statistics and psychology: Multiple comparisons give spurious results. - Slate Magazine (Andrew Gelman, df = 1)
Policy makers, like most people, normally feel that they already know all the psychology and all the sociology they are likely to need for their decisions. I don’t think they are right, but that’s the way it is. On the other hand, people who have not studied economics are fully aware of their ignorance. The use of mathematics adds a touch of magic to economics. Indeed it makes perfect sense for economists to be the interpreters of policy-relevant research, because they understand and are trained to use big data. This, and the fact that policies always involve tradeoffs and almost always involve money, explains the dominant role of economics in policy.
Daniel Kahneman’s Gripe with Behavioral Economics - The Daily Beast (don’t get me started about the bit where economists “understand and are trained to use big data” — and the problem is not the “big”)
Carter says our chance of success was 50 percent. 50?! I thought it was much higher. Another gut check. Would we have gone with Tony at 50 percent?
Argo hostage story: Mark Lijek’s true account of fleeing Iran. - Slate Magazine
Here’s my list of startup advice: Be alive. Be male. Be young. Don’t have health issues. Be born in America or move there. Enter the cycle after a recession. Speak English. Enter a growing/new field where the level of competition is low and so is the sophistication of your competition. Surf cost trends down from expensive to mass consumer markets. Work bottom up - on small things. Be of above average intelligence. Have family support. Have a college degree. Oh and most importantly of all: Get fucking lucky. The hindsight/survivorship biases in combination with faulty causality and the narrative fallacy will completely hose your thinking - so be careful.
Startups: never have so many understood so little about the statistics of varian… | Hacker News

Both Psych File Drawer and the Reproducibility Project were started in part because it’s hard to get a replication published even when a study cries out for one. For instance, Daryl J. Bem’s 2011 study that seemed to prove that extra-sensory perception is real — that subjects could, in a limited sense, predict the future — got no shortage of attention and seemed to turn everything we know about the world upside-down.

Yet when Stuart Ritchie, a doctoral student in psychology at the University of Edinburgh, and two colleagues failed to replicate his findings, they had a heck of a time getting the results into print (they finally did, just recently, after months of trying). It may not be a coincidence that the journal that published Bem’s findings, the Journal of Personality and Social Psychology, is one of the three selected for scrutiny.

Is Psychology About to Come Undone? - Percolator - The Chronicle of Higher Education