| Type | Journal Article |
|---|---|
| Author | John E. Brush |
| Author | Mark Lee |
| Author | Jonathan Sherbino |
| Author | Judith C. Taylor-Fishwick |
| Author | Geoffrey Norman |
| URL | https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2757877 |
| Volume | 2 |
| Issue | 12 |
| Pages | e1918023-e1918023 |
| Publication | JAMA Network Open |
| Date | 2019/12/02 |
| Journal Abbr | JAMA Netw Open |
| DOI | 10.1001/jamanetworkopen.2019.18023 |
| Accessed | 12/21/2019, 7:23:26 AM |
| Library Catalog | jamanetwork.com |
| Language | en |
| Abstract | <h3>Importance</h3><p>Clinicians use probability estimates to make a diagnosis. Teaching students to make more accurate probability estimates could improve the diagnostic process and, ultimately, the quality of medical care.</p><h3>Objective</h3><p>To test whether novice clinicians can be taught to make more accurate bayesian revisions of diagnostic probabilities using teaching methods that apply either explicit conceptual instruction or repeated examples.</p><h3>Design, Setting, and Participants</h3><p>A randomized clinical trial of 2 methods for teaching bayesian updating and diagnostic reasoning was performed. A web-based platform was used for consent, randomization, intervention, and testing of the effect of the intervention. Participants included 61 medical students at McMaster University and Eastern Virginia Medical School recruited from May 1 to September 30, 2018.</p><h3>Interventions</h3><p>Students were randomized to (1) receive explicit conceptual instruction regarding diagnostic testing and bayesian revision (concept group), (2) exposure to repeated examples of cases with feedback regarding posttest probability (experience group), or (3) a control condition with no conceptual instruction or repeated examples.</p><h3>Main Outcomes and Measures</h3><p>Students in all 3 groups were tested on their ability to update the probability of a diagnosis based on either negative or positive test results. Their probability revisions were compared with posttest probability revisions that were calculated using the Bayes rule and known test sensitivity and specificity.</p><h3>Results</h3><p>Of the 61 participants, 22 were assigned to the concept group, 20 to the experience group, and 19 to the control group. Approximate age was 25 years. Two participants were first-year; 37, second-year; 12, third-year; and 10, fourth-year students. Mean (SE) probability estimates of students in the concept group were statistically significantly closer to calculated bayesian probability than the other 2 groups (concept, 0.4%; [0.7%]; experience, 3.5% [0.7%]; control, 4.3% [0.7%];<i>P</i> < .001). Although statistically significant, the differences between groups were relatively modest, and students in all groups performed better than expected, based on prior reports in the literature.</p><h3>Conclusions and Relevance</h3><p>The study showed a modest advantage for students who received theoretical instruction on bayesian concepts. All participants’ probability estimates were, on average, close to the bayesian calculation. These findings have implications for how to teach diagnostic reasoning to novice clinicians.</p><h3>Trial Registration</h3><p>ClinicalTrials.gov identifier:NCT04130607</p> |
| Short Title | Effect of Teaching Bayesian Methods Using Learning by Concept vs Learning by Example on Medical Students’ Ability to Estimate Probability of a Diagnosis |
| Date Added | 12/21/2019, 7:23:26 AM |
| Modified | 12/21/2019, 7:23:56 AM |
| Type | Journal Article |
|---|---|
| Author | Hermann Brenner |
| Author | Olaf Gefeller |
| Volume | 16 |
| Pages | 981-991 |
| Publication | Stat Med |
| Date | 1997 |
| Extra | Citation Key: bre97var tex.citeulike-article-id= 13263810 tex.posted-at= 2014-07-14 14:09:23 tex.priority= 0 |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
non-constancy of sensitivity and specificity caused by fact that most diseases represent continuous processes
| Type | Journal Article |
|---|---|
| Author | D. B. Pryor |
| Author | L. Shaw |
| Author | C. B. McCants |
| Author | K. L. Lee |
| Author | D. B. Mark |
| Author | F. E. Harrell |
| Author | L. H. Muhlbaier |
| Author | R. M. Califf |
| Volume | 118 |
| Pages | 81-90 |
| Publication | Ann Int Med |
| Date | 1993 |
| Extra | Citation Key: pry93val tex.citeulike-article-id= 13264712 tex.posted-at= 2014-07-14 14:09:40 tex.priority= 0 |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
| Type | Journal Article |
|---|---|
| Author | David L. Sackett |
| URL | http://dx.doi.org/10.1001/jama.1992.03480190080037 |
| Volume | 267 |
| Pages | 2638-2644 |
| Publication | JAMA |
| Date | 1992 |
| Extra | Citation Key: sac92pri tex.citeulike-article-id= 13264784 tex.citeulike-linkout-0= http://dx.doi.org/10.1001/jama.1992.03480190080037 tex.posted-at= 2014-07-14 14:09:42 tex.priority= 0 |
| DOI | 10.1001/jama.1992.03480190080037 |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
| Type | Journal Article |
|---|---|
| Author | M. A. Hlatky |
| Author | D. B. Pryor |
| Author | F. E. Harrell |
| Author | R. M. Califf |
| Author | D. B. Mark |
| Author | R. A. Rosati |
| URL | http://www.sciencedirect.com/science/article/pii/0002934384904376# |
| Volume | 77 |
| Pages | 64-71 |
| Publication | Am J Med |
| Date | 1984 |
| Extra | Citation Key: hla84fac tex.citeulike-article-id= 13264303 tex.citeulike-attachment-1= hlatky₈4<sub>f</sub>actors₉76652.pdf; /pdf/user/harrelfe/article/13264303/976652/hlatky₈4<sub>f</sub>actors₉76652.pdf; d9105c885bc73798ea3b8e6abc911d672f938291 tex.citeulike-linkout-0= http://www.sciencedirect.com/science/article/pii/0002934384904376# tex.posted-at= 2014-07-14 14:09:32 tex.priority= 0 |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
| Type | Journal Article |
|---|---|
| Author | L. Irwig |
| Author | A. Tosteson |
| Author | C. Gatsonis |
| Author | J. Lau |
| Author | G. Colditz |
| Author | T. C. Chalmers |
| Author | F. Mosteller |
| Volume | 120 |
| Pages | 667-676 |
| Publication | Ann Int Med |
| Date | 1994 |
| Extra | Citation Key: irw94gui tex.citeulike-article-id= 13264355 tex.posted-at= 2014-07-14 14:09:33 tex.priority= 0 |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
| Type | Journal Article |
|---|---|
| Author | J. A. Hanley |
| Author | B. J. McNeil |
| Volume | 143 |
| Pages | 29-36 |
| Publication | Radiology |
| Date | 1982 |
| Extra | Citation Key: han82 tex.citeulike-article-id= 13264208 tex.posted-at= 2014-07-14 14:09:31 tex.priority= 0 |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
| Type | Journal Article |
|---|---|
| Author | Irene Guggenmoos-Holzmann |
| Author | Hans C. van Houwelingen |
| URL | https://www.onlinelibrary.wiley.com/doi/abs/10.1002/1097-0258%2820000715%2919%3A13%3C1783%3A%3AAID-SIM497%3E3.0.CO%3B2-B |
| Volume | 19 |
| Pages | 1783-1792 |
| Publication | Stat Med |
| Date | 2000 |
| Extra | Citation Key: gug00inv |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
death of sensitivity and specificity
| Type | Journal Article |
|---|---|
| Author | D. Bamber |
| Volume | 12 |
| Pages | 387-415 |
| Publication | J Mathe Psych |
| Date | 1975 |
| Extra | Citation Key: bam75 tex.citeulike-article-id= 13263723 tex.posted-at= 2014-07-14 14:09:21 tex.priority= 0 |
| Date Added | 7/7/2018, 1:38:33 PM |
| Modified | 11/8/2019, 8:01:59 AM |
| Type | Journal Article |
|---|---|
| Author | Michael L. Barnett |
| Author | Dhruv Boddupalli |
| Author | Shantanu Nundy |
| Author | David W. Bates |
| URL | https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2726709 |
| Volume | 2 |
| Issue | 3 |
| Pages | e190096-e190096 |
| Publication | JAMA Network Open |
| Date | 2019/03/01 |
| Journal Abbr | JAMA Netw Open |
| DOI | 10.1001/jamanetworkopen.2019.0096 |
| Accessed | 3/3/2019, 8:14:46 AM |
| Library Catalog | jamanetwork.com |
| Language | en |
| Abstract | <h3>Importance</h3><p>The traditional approach of diagnosis by individual physicians has a high rate of misdiagnosis. Pooling multiple physicians’ diagnoses (collective intelligence) is a promising approach to reducing misdiagnoses, but its accuracy in clinical cases is unknown to date.</p><h3>Objective</h3><p>To assess how the diagnostic accuracy of groups of physicians and trainees compares with the diagnostic accuracy of individual physicians.</p><h3>Design, Setting, and Participants</h3><p>Cross-sectional study using data from the Human Diagnosis Project (Human Dx), a multicountry data set of ranked differential diagnoses by individual physicians, graduate trainees, and medical students (users) solving user-submitted, structured clinical cases. From May 7, 2014, to October 5, 2016, groups of 2 to 9 randomly selected physicians solved individual cases. Data analysis was performed from March 16, 2017, to July 30, 2018.</p><h3>Main Outcomes and Measures</h3><p>The primary outcome was diagnostic accuracy, assessed as a correct diagnosis in the top 3 ranked diagnoses for an individual; for groups, the top 3 diagnoses were a collective differential generated using a weighted combination of user diagnoses with a variety of approaches. A version of the McNemar test was used to account for clustering across repeated solvers to compare diagnostic accuracy.</p><h3>Results</h3><p>Of the 2069 users solving 1572 cases from the Human Dx data set, 1228 (59.4%) were residents or fellows, 431 (20.8%) were attending physicians, and 410 (19.8%) were medical students. Collective intelligence was associated with increasing diagnostic accuracy, from 62.5% (95% CI, 60.1%-64.9%) for individual physicians up to 85.6% (95% CI, 83.9%-87.4%) for groups of 9 (23.0% difference; 95% CI, 14.9%-31.2%;<i>P</i> < .001). The range of improvement varied by the specifications used for combining groups’ diagnoses, but groups consistently outperformed individuals regardless of approach. Absolute improvement in accuracy from individuals to groups of 9 varied by presenting symptom from an increase of 17.3% (95% CI, 6.4%-28.2%;<i>P</i> = .002) for abdominal pain to 29.8% (95% CI, 3.7%-55.8%;<i>P</i> = .02) for fever. Groups from 2 users (77.7% accuracy; 95% CI, 70.1%-84.6%) to 9 users (85.5% accuracy; 95% CI, 75.1%-95.9%) outperformed individual specialists in their subspecialty (66.3% accuracy; 95% CI, 59.1%-73.5%;<i>P</i> < .001 vs groups of 2 and 9).</p><h3>Conclusions and Relevance</h3><p>A collective intelligence approach was associated with higher diagnostic accuracy compared with individuals, including individual specialists whose expertise matched the case diagnosis, across a range of medical cases. Given the few proven strategies to address misdiagnosis, this technique merits further study in clinical settings.</p> |
| Date Added | 3/3/2019, 8:14:46 AM |
| Modified | 3/4/2019, 6:58:04 AM |