STRAW VOTES ON CERTIFICATION AND INTERACTIONS

by Dr. Rebecca Gelman

At the February 1994 Biostatistics Faculty meeting, two written surveys were conducted: one on certification of statisticians and one on the use of subgroups and interactions in clinical trials. Faculty and students attending the meeting were eligible to participate, and most did.

If the ASA had been conducting a vote on certification on the day of the February Faculty Meeting, 12(67%) respondents would vote against it, 5(28%) would vote for it, and 3(17%) would not vote. In answer to the question "if certification existed today would you yourself become certified?", 9 answered no, 6 answered yes, and 4 answered maybe. In answer to the question "if certification existed today would you volunteer to proctor exams and/or review portfolios?", 13 answered no, 3 answered yes, and 3 answered maybe.

The survey on subgroups and interactions used the answers Y for "yes, often", S for "yes, sometimes" and N for "no". A subgroup test was defined as a test for a treatment difference within subgroups determined by prognostic factors; an interaction test was defined as a test for treatment-prognostic factor interaction. The following tallies of answers were obtained to selected questions. Have you ever done such tests when the overall treatment difference is significant? (subgroups, 9Y,7S,1N, interactions 9Y,8S,1N), is not significant? (subgroups 4Y,6S,6N, interactions 5Y,8S,4M), when the test does not correspond to a prespecified hypothesis? (subgroups 3Y,9S,4N, interactions 2Y,11S,3N). When asked "What do you do about multiple comparisons?" the most answers were nothing (3Y,9S,4N), Bonferroni or other adjustment for multiple independent tests (5Y,9S,3N), adjustment for multiple non-independent tests (2Y,6S,7N), Bayesian or other shrinkage towards overall result or null (0Y,4S,12N), randomly split data set into model forming and model testing parts(0Y,8S,6N). Our conclusion is that the answer "sometimes" almost always wins.

Last modified $Date: 1994/12/01 20:23:33 $ by Evelyn Ophir, ophir@hsph.harvard.edu