Design Experimental and Non-Experimental Seminar Series – 10/11
Design Experimental and Non-Experimental Seminar Series
Speaker: Maya Mathur
Assistant Professor
Quantitative Sciences Unit, Biomedical Informatics Research Division, Stanford University
Friday, October 11, 2024
11:00AM – 12:00PM
HCP Conference Room 224E
180 Longwood Ave, Boston, MA
Join via ZOOM
Meeting ID: 943 5080 8482
Password: 287186
Beyond the funnel plot: New methods to address p-hacking and other biases in meta-analysis
Abstract: Meta-analytic evidence contributes substantially to policy and healthcare guidelines, but the results of meta-analyses can be severely compromised by selective reporting. As traditionally conceived, publication bias arises from selection that operates on a collection of individually unbiased estimates. For example, affirmative studies (i.e., those with significant, positive estimates) might be more likely to be published than nonaffirmative studies (i.e., those with nonsignificant or negative estimates). This is a form of selection across studies (SAS). However, meta-analyses can also be compromised by selection within studies (SWS), in which investigators “p-hack’’ results within their study to obtain an affirmative estimate. When there is SWS as well as SAS, existing methods for publication bias can be severely biased in either direction. I will discuss two new analysis methods that accommodate joint SAS and SWS; these both involve analyzing only the published nonaffirmative estimates. I will also discuss within-study biases, such as confounding, and the surprising way they can interact superadditively with selective reporting.
