Virtual Pathology Elective, Real Education: The PathElective.com Experience as a Model for Novel Pathology Pedagogy and a Primer for Curricular Evolution. (2024)

As a result of the COVID-19 pandemic, medical education delivery transformed overnight. In-person experiences were converted to virtual experiences, with varying success. (1) Formal medical student exposure to pathology is often limited to didactic learning within the first 2 years and then optional elective experiences in the third and fourth years. The lack of a required clerkship results in matriculating medical students with highly variable pathology-related experiences. This is further strained by current nationwide efforts in curricular reform that aim to decrease preclinical education to fewer months overall. (2) The recent switch of United States Medical Licensing Examination step 1 examinations to pass/fail reporting also has ramifications for how medical students approach pathology learning before setting foot into clinical settings. (3) Taken together, these factors have direct and serious consequences for pathology pedagogy. Now more than ever, pathology educators need to be innovative and flexible in making sure appropriate pathology and laboratory medicine education remains well integrated throughout our students' education.

One important aspect of staying nimble in education delivery is our ability to execute novel pedagogy and then learn from that experience. Nationwide, students are taking control of their pathology education, (4,5) are more absent from class attendance, and are choosing to not watch school provided recorded videos. Alternate curricular sources such as Pathoma, the First Aid series, Sketchy Medical (Sketchy Group, LLC), AMBOSS, etc, are the backbone of our students' education, and these curricular modalities have not been formally studied for effectiveness.

At the beginning of the COVID-19 pandemic, in response to the urgent need for continued pathology elective experiences, Lilley et al (6) developed a novel, free, elective website for medical students and all health science students, the implementation and early effectiveness of which we published earlier. (6) Since its start in May 2020, the PathElective.com curriculum material has been incorporated into medical student and resident training programs at numerous institutions across the world, serving as a unique means of delivering high-quality pathology and laboratory medical education at multiple levels of training. In this manuscript, we provide website data on student usage, course choice, change in medical knowledge, and attrition rates. Our goal is to provide educators data from a large pool of students to share what works for students when they voluntarily choose to learn from free educational material. These data provide insight into trainee experience and choice, with implications for number of lessons, length of lessons, lesson content, level of depth, a student's willingness to complete virtual quizzes, and the appropriateness of the content for the virtual space, which may help guide curricular evolution within formal pathology education.

METHODS

PathElective.com was developed using Squarespace (Squarespace, Inc). All traffic data, including page views, visits, unique visitors, and geographic location, were obtained through the internal functionality provided by Squarespace. (7) Course content in areas of anatomic pathology and clinical pathology was designed, composed, and curated by a course coordinator, typically a pathologist or experienced medical laboratory professional. The course content was input into the website by the website developer and reviewed by the course director before publishing. Content was protected using GoPayWall (GoPaywall), a subscription service that allows subscribers to access the curated educational content. Subscription to PathElective is free, but the free sign-in protection on content was developed to ensure users had a valid email to which their certificates would be emailed and to make sure students agreed to our website's terms and conditions and consented to the use of their quiz data being used in research. Students and trainees from around the globe can sign up to be a member of PathElective and view the educational content on the website for free.

Students were assessed before and after interacting with course materials using a dual-form crossover quiz design on Google Forms (Google LLC). The dual-form crossover quiz design was chosen as the means of assessing the assimilation of new knowledge and has been favored as a means of assessing students in standardized test settings as well.8 Randomization between the forms was accomplished using the user's birth year (Figure 1, A). The different forms, though they contained different questions, assessed the same content areas and attempted to assess similar levels of questioning (Supplemental Table 1; see supplemental digital content containing 2 tables and 1 figure at https://meridian.allenpress.com/aplm in the May 2024 table of contents). Questions were assessed for difficulty by 2 reviewers before the quiz was launched. Additionally, quiz data were analyzed via paired t test after the study period to ensure the average score for each of the forms did not differ significantly from one another. Summary statistics were also collected between forms to ensure no statistical differences were present between forms. Prelesson and postlesson quiz data were paired and anonymized, and percentage of improvement was determined using a paired t test. A paired t test was selected to correct for variations in starting score due to the diversity in student backgrounds. These data were organized into a forest plot with the null hypothesis set to a 0% improvement. The 95th percentile was calculated and plotted on the forest plot to visualize statistical significance using Prism GraphPad by Dotmatics.

Upon completion of the postlesson quiz, a satisfaction survey was completed by students (Figure 1, B). The satisfaction survey included a Likert-scale assessment of (1) overall lesson quality, (2) lecturer quality, (3) social media engagement, (4) lecture quality, and (5) satisfaction with the online format. In addition to Likertscale data collection, voluntary written feedback was elicited at the end of the survey. Likert-scale satisfaction data were analyzed using the calculation of descriptive statistics for each category for each lesson. Though the satisfaction data are ordinal, means were used in the analysis since all courses had similar, if not identical, median analyses that were not able to distinguish the subtle differences in satisfaction across courses and categories. These means were visualized using Microsoft Excel (Microsoft Corporation). Means were compared visually based on a scale of total possible responses (1-5) to assess overall student satisfaction, against the mean across all categories (cross-category) to assess categories that need improvement, and against the mean for each category to assess courses that need improvement (Supplemental Table 2).

Based on the variety in course layouts provided by course directors, some courses contained a single lesson, while others contained multiple lessons. For courses composed of multiple lessons that were all released within a week of one another, a novel analysis was performed aimed at examining the attrition rate of students as more modules are added to an online course (Figure 1, C). This analysis was developed to assess the level of participation, measured by number of students who took the postlesson quiz, in each lesson within a course and to determine whether or not students tended to complete a course with multiple lessons, and if so, to what extent. The number of students who engaged with each lesson was recorded during the first 18 months. The percentage of change in student participation between lessons was determined and aggregated for each of the courses. The average and 95% CI were determined for each of the lesson additions across all courses. The data were plotted and statistically analyzed with polynomial regression using Prism GraphPad. The degree of polynomial regression and line of best fit was determined using the extra sum of squares f-test with an alpha set to .05, which was also calculated using Prism GraphPad. In addition to polynomial regression analysis, attrition was also visualized and plotted using violin-plot analysis to depict the entire spread of the data.

This protocol, as well as the consent procedures, were reviewed by the Loyola University Chicago Institutional Review Board (Maywood, Illinois) and was determined to be exempt (LU#215076).

RESULTS

During the study period (May 1, 2020 to December 31, 2021), PathElective.com received 622 681 page views (number of times a page within the website was loaded including internal links), 139 081 visits (number of times the page was loaded by a visitor coming from an external site), 67 670 unique visitors, and 10 278 registered users who earned 15 305 certificates. Sixty-eight and one half perfect (68.5%) of visitors (95 300/139 081) arrived at the site via a direct URL input, 17.2% (24 862/139 081) through a search engine, and 10.9% through social media (16 361/139 081). Of the social media channels, most visitors came from X (formerly known as Twitter [X Corp.]) (6.7%, 9286/139 081) followed by Facebook (Meta Platforms, Inc.) (3.5%, 4927/139 081), Instagram (Meta Platforms, Inc.) (0.8%, 1152/139 081), YouTube (Google LLC) (0.3%, 487/139 081), LinkedIn (Microsoft Corporation) (0.2%, 280/139 081), and Reddit (Advance Publications) (0.1%, 205/139 081). Most visits (44.4%, 61 740/139 081) were from the United States of America. The top 4 high-traffic states in the United States were Illinois (5701 visits), with Chicago (3050) hosting a majority of visits; Texas (5069), with Houston (1646) hosting a majority of visits; California (4321), with Los Angeles (670) hosting a majority of visits; and New York (4303), with New York City (955) hosting a majority of visits. After the United States, India was the second most popular country (10.8%, 14 901/139 081) from which PathElective was accessed, mostly in Tamil Nadu (1939), Karnataka (1829), Delhi (1694), and Maharashtra (1473), all exhibiting a similar level of website traffic.

Student performance was assessed as above and organized in Figure 2. In total, 7609 paired assessments were analyzed, with 6150 assessments originating from anatomic pathology (AP) courses and 1756 originating from clinical pathology (CP) courses. The course with the single highest number of student participants was lesson 1 of Gross Pathology, with 563 participants. The CP course with the highest level of participation was Hematology, with 323 participants. On average, student performance on course assessments improved by 13.8% (95% CI, 10.2%-17.5%). Separately, AP courses experienced a 14% (95% CI, 10.5%-17.5%) average improvement in score, while CP courses experienced a 13.2% (95% CI, 9.2%-17.5%) average improvement in score. All courses experienced a statistically significant increase in student performance except Dermatopathology: lesson 40 (n = 80), Cytopathology: lesson 3 (n = 75), and Cardiac Pathology: lessons 2 and 3 (n = 82 and 75, respectively). The course with the highest improvement in student performance was Gross Pathology: lesson 4, with an improvement of 30.2% (95% CI, 27.2%-33.1%) (n = 323). Courses with significantly higher than average performance were Autopsy: lesson 1; Gross Pathology: lessons 3, 4, and 6; and Hematopathology: lesson 1. Courses with significantly lower than average performance were Cardiac Pathology: lessons 1, 2, and 3; Cytopathology: lesson 3; Dermatopathology: lessons 1, 2, 4, and 5; and Gross Pathology: lessons 1 and 5.

Satisfaction data obtained after completion of each lesson quiz assessed overall quality of the lesson, quality of the lecturer, level of social media engagement, qualities of the constituent lectures, ability to get help when needed, and satisfaction with the online format. When taken together, the aggregate average of all quality metrics across all lessons was 4.46 (out of 5). The averages of each category for every lesson were organized into Supplemental Table 2. All courses received, on average, above a 3.88, with a range of 3.88 to 4.72 (Figure 3, A). When comparing all categories and courses to the aggregate average, it is evident that the category with the lowest average scores across all courses was social media engagement (Figure 3, B). When comparing each of the courses for each satisfaction category, courses with higher than average satisfaction metrics included Dermatopathology, Gross Pathology, Neuropathology, Illustrations of Histology, and Hematology (Figure 3, C). Courses with lower than average satisfaction metrics included Breast Pathology and Cardiac Pathology. Other courses that had lessons with some lessons falling higher and others falling lower than average according to satisfaction data included Cytopathology, Hematopathology, Clinical Microbiology, and Molecular Microbiology. It is important to mention that the differences between courses that were above average and below average were within 0.3 to 0.59 points. However, these small differences were noted to help improve course layouts, lectures, etc. to improve the learner experience. Additionally, written student feedback was relayed to the course director and was typically used to make certain topics clearer and provide additional resources.

For courses with more than 1 lesson (n = 12), student retention/attrition data were collected and analyzed. For the nonlinear regression of the mean taking into account standard deviation and sample size, the second-order polynomial function was significantly better than the first (P = .002), but the third-order polynomial was not significantly better than the second order (P = .21). The unique polynomial relationship with each added lesson highlights the precipitous drop during the transition between lessons 1, 2, and 3. After lesson 3, the drop in student participation was less precipitous. The latter half of the data set was limited by sample size, so the projection of the regression is less reliable. The equation yielded from the polynomial regression analysis was Y = --1.643 -- 26.19(x) + 2.838 (x)2, where x is equal to lesson number of lessons after the first lesson (ie, lesson no. - 1). The [R.sup.2] for the polynomial regression was found to be 0.656 (Figure 4, A and B).

DISCUSSION

Data harnessed from the PathElective.com module usage and satisfaction surveys provide a powerful and unique perspective that can help guide pedagogical approaches in medical education. Keeping a finger on the pulse of what students voluntarily choose to learn from, which modules are most effective, and which teaching frameworks lead to the highest increases in postmodule student scores can provide valuable insight that can be harnessed to guide postpandemic hybrid learning and medical school curricular reform. Data from the geographic origin of the students, satisfaction scores, and attrition rates in light of performance characteristics provide directed data for the PathElective. com site to adjust and refocus modules that are rated lower or wherein students are not performing as well as in others.

When analyzing the geographic distribution of students accessing PathElective.com, it is no surprise that the 2 major foci of interest were the United States and India, given the initial interest reported in Lilley et al 2021. (6) However, when analyzing the geographic distribution of students within the top 4 high-traffic states (Illinois, Texas, New York, and California), the hypothesis that interest would be centered around medical schools and residency programs remained only partially true. Interestingly, the geographic distribution of students accessing PathElective.com in Illinois, Texas, and New York exhibited a scattering of access points across the states, outside of the major cities that house large medical centers and medical schools. This finding could be due, in part, to students learning remotely during the height of the COVID-19 pandemic but could also support the claim that PathElective.com is being accessed by a broader audience than those in medical school or residency--the audiences that predominated the platform's user database in earlier studies. (6) Additionally, website traffic data support the hypothesis that a registration requirement decreases the number of people who access the website's educational content since, out of the 59 928 unique visitors, only 10 278 registered to access the educational content behind the registration wall on PathElective.

The ability to assess 7609 paired assessments allows us a powerful lens to evaluate modules and focus energy toward those modules that are not as effective as others. Overall, the paired assessment data show most improvement linked to those modules wherein information would be new for students. Autopsy, gross pathology, and basics of hematology are all subspecialized topics typically not covered in detail during medical school education. Autopsy education is always of great interest to students given its presence in popular media and the intrigue of being a detective of disease after death. This likely drives the interest, engagement, and improvement in scores for that module. Similarly, the importance of understanding of gross pathology for trainees in pathology, surgery, and pathologists' assistant education drives heightened interest in that module. These modules also utilized variable pedagogical methods (multimedia and interactive), which suggests that more engaging content could play a role in online medical education. While we did not systematically examine course content, pedagogy, and the difficulty level for specific questions, doing so in the future will help dissect these data even further.

Overall, usage of the website resulted in high satisfaction scores across all categories except social media engagement. The categories of overall PathElective.com experience, quality of the website, and functionality were generally scored as very satisfied or satisfied. This positive skew in satisfaction data was found to exist across all questions assessed. In fact, the average for each category was above 4 out of 5. The weakest category, however, was social media engagement. The reasons for this are probably multifold. On the site, we have encouraged students to interact with their course instructors and other PathElective users on social media. The idea behind this is to establish a feeling of a shared experience, even if virtual. This is aided by the fact that all our faculty members have an active social media presence. While we emphasize this point to the trainee, the onus is on them to make that interaction a reality. It is difficult for our site faculty to identify, keep track of, and initiate social media interactions with all PathElective.com users. Many students also feel inertia or hesitation in initiation of these interactions as well. (9) Additionally, it is important to consider the range of scores being analyzed. For the social media category, the range of averages across all courses was between 3.88 and 4.46 and the range of all course averages was between 3.88 and 4.75 (Supplemental Table 2). Though social media performed below average when compared to other categories, the average scores were still above the "neutral" choice of 3 out of 5. Taken together, our emphasis on how useful social media can be for their overall experience, but lack of a concrete follow-through on how to make that happen, is likely the underlying reason for the slightly lower satisfaction rating. These data allow us to consider modifications in strategies of engagement moving forward, which could include faculty having "virtual office hours" via X or Instagram Live, or even using voice-based platforms such as X Spaces or ClubHouse (Alpha Exploration Co), which have been shown to have added utility in such settings, for example, in their use during journal clubs. (10) There were no correlations found between any of the assessed satisfaction domains and average score improvement (Supplemental Figure 1).

Our novel attrition analysis offers a first-of-its-kind look into student participation and behavior in online courses with multiple lessons. As hypothesized, our data support the claim that students tending to terminate participation in online classes correlates with the increasing number of lessons; however, with our limited data set, the prediction of student retention past 4 lessons is less reliable. Additionally, most students tend to start at lesson 1 and work their way through the course even though the courses offer a modular, adaptable experience. The exception to the latter claim was seen in gastrointestinal pathology and cardiac pathology. Interestingly, there were no similarities between these 2 courses that would explain this difference, but in the gastrointestinal pathology course, each lesson takes an organ-based approach to teach its system's pathologies, and in the cardiac pathology course, the first lesson covered basic histology, which seems to have been skipped by a number of students. The trend of skipping the introductory module was not replicated in other courses with introductory lessons (ie, Dermatopathology, Head And Neck Pathology, Pulmonary Pathology, and Breast Pathology). Taken together, the regression plotting the mean and 95% CI, and the violin plots visualizing data distribution, median, and interquartile ranges give a holistic view of student activity in the voluntary, self-paced, online courses with multiple lessons in one course.

PathElective.com has proven to be an effective educational resource for medical students, residents, fellows, and physicians across their training journey that has been well received. These data support the use of PathElective in a wide range of in-person and virtual educational settings as an adjunct to their curriculum. Future research will focus on the utility of PathElective in select educational settings and in certain student populations as well as on the effect of different teaching styles on student performance outcomes. Further research also needs to be done to investigate the equitability and inclusivity of PathElective across the United States and globally to assess the ability to access the platform and any barriers there are that typically prohibit students from pursuing further education in pathology and medicine more broadly.

Members of the PathElective team who have contributed significantly to the content creation or curation, development, and/ or design of the website but not to the construction of this paper include Aadil Ahmed, MD; Vijayalakshmi "Viju" Ananthanarayanan, MD, MBBS; Natalie Banet, MD; Andrew M. Bellizzi, MD; Narendra Bhattarai, MD; Matthew Cecchini, MD; Joe Chaffin, MD; Bonnie Choy, MD; Nicole A. Cipriani, MD; Amy Deeken, MD; Christopher Demas, MD; Kyle Devins, MD; David J. Escobar, MD, PhD; Lucy Fu, MD; Sarah Garner, MS; Lorenzo Gitto, MD; Isabel Gomez, MD; Brittany Grantham, MS; Emily Hagen, BS; Daniela Hermelin, MD; My-Linh Ho, MD; Daniel Johnson, MD; Constantine E. Kanakis, MD, MSc; Walter Kemp, MD, PhD; Azmer Khan; Nigar Khurram, MD; Giselle Lopez, MD; Nolan Maloney, MD; Maximo J. Marin, MD; Patrick J. McIntire, MD; Sanjay Mukhopadhyay, MD; Cory Nash, MS; Megan Parilla, MD; Ami Patel, MD; Maryam Pezouh, MD; Raghavindra Pillappa, MD; Meredith Pittman, MD; Sameera Rashid, MD; Dana Razzano, MD; Kara Roncin, MD; Jaclyn Rudzinski, MD; Katherine Saunders, DO; Teresa Scordino, MD; Anurag Sharma, MD; Alexis Snyder, MHS; Michael Swete, DO; Cody Thomas, MD; Emily Towery, MD; Anna Trzcinska, DMD; Imran Uraizee, MD; Kartik Viswanathan, MD, PhD; Kristy A. Waite, DO; Michael Williams, MD, MS; Sara E. Wobker, MD, MPH; Eva M. Wojcik, MD; Joseph R. Wiencek, PhD; Kristy Wolniak, MD, PhD; and Brandon Zelman, DO.

doi: 10.5858/arpa.2022-0259-OA

References

(1.) Hassell LA, Absar SF, Chauhan C, et al. Pathology education powered by virtual and digital transformation. Arch Pathol Lab Med. 2023;147(4):474-491. doi:10.5858/arpa.2021-0473-ra

(2.) Emanuel EJ. Reforming American medical education. MilbankQ. 2017;95(4): 692-697. doi:10.1111/1468-0009.12291

(3.) Whaley RD, Booth AL, Mirza KM. Changes in USMLE step 1 result reporting: a pass or fail for pathology programs? Acad Pathol. 2021;8:2374289521998029. doi:10.1177/2374289521998029

(4.) Wu JH, Gruppuso PA, Adashi EY. The self-directed medical student curriculum. JAMA. 2021;326(20):2005-2006. doi:10.1001/jama.2021.16312

(5.) Ahmed A, Wojcik EM, Ananthanarayanan V, Mulder L, Mirza KM. Learning styles in pathology: a comparative analysis and implications for learner-centered education. Acad Pathol. 2019;6:2374289519852315. doi:10.1177/ 2374289519852315

(6.) Lilley CM, Arnold CA, Arnold M, et al. The implementation and effectiveness of PathElective.com. Acad Pathol. 2021;8:23742895211006828. doi:10.1177/23742895211006829

(7.) SquareSpace. Traffic analytics. Help center. https://support.squarespace. com/hc/en-us/articles/217999797-Traffic-analytics. Accessed December 10, 2022.

(8.) Sanders S. A Brief Guide to Selecting and Using Pre-Post Assessments. Washington, DC: American Institutes for Research, The National Technical Assistance Center for the Education of Neglected or Delinquent Children and Youth; 2019.

(9.) Tushir A, Duy Nguyen T, Biehl C, Mirza KM. Challenges and opportunities in using Twitter as a pedagogical tool in pathology and laboratory medicine education. Am J Clin Pathol. 2022;157(6):836-843. doi:10.1093/ajcp/aqab219

(10.) Ahmed A, Mirza KM, Loghavi S. Elevating Twitter-based journal club discussions by leveraging a voice-based platform: #HemepathJC meets clubhouse. Curr Hematol Malig Rep. 2021;16(5):418-421. doi:10.1007/s11899-021-00644-2

Cullen M. Lilley, MD, MS, MA; Christina A. Arnold, MD; Michael A. Arnold, MD, PhD; Adam L. Booth, MD; Jerad M. Gardner, MD; Xiaoyin "Sara" Jiang, MD; Sanam Loghavi, MD; Kamran M. Mirza, MD, PhD

Accepted for publication June 1, 2023.

Published online August 28, 2023.

Supplemental digital content is available for this article at https:// meridian.allenpress.com/aplm in the May 2024 table of contents.

From Loyola University Stritch School of Medicine, Maywood, Illinois (Lilley, Mirza); Department of Pathology, University of Colorado, Anschutz Medical Campus, Aurora (C. A. Arnold, M. A. Arnold); Pathology Department, Children's Hospital Colorado, Aurora (M. A. Arnold); Department of Pathology, Northwestern University Feinberg School of Medicine, Chicago, Illinois (Booth); Departments of Pathology and Dermatology, Geisinger Medical Center, Danville, Pennsylvania (Gardner); Department of Pathology, Duke University School of Medicine, Durham, North Carolina (Jiang); Department of Hematopathology, MD Anderson Cancer Center, Houston, Texas (Loghavi); and the Department of Pathology and Laboratory Medicine, Loyola University Health System, Maywood, Illinois (Mirza). C. A. Arnold currently works as a coach in Colorado.

This project was funded in part by the Association of Pathology Chairs Society of '67 Pathology Trainee Project Grants in Health Services Education.

The authors have no relevant financial interest in the products or companies described in this article. All authors and acknowledged contributors have participated in the design, development, and/or content creation/curation of PathElective.com.

Corresponding author: Kamran M. Mirza, MD, PhD, Pathology and Laboratory Medicine, Loyola University Chicago, 2160 S 1st Ave, Maywood, IL 60153 (email: [emailprotected]).

Caption: Figure 1. Depiction of PathElective methods. The figure depicts methods involved in the assessment (A), satisfaction (B), and attrition (C) analysis performed. Abbreviations: Avg, average; Cat, different categories of satisfaction being assessed; F, the extra sum of squares F test; Max, maximum; Min, minimum; n, the number of students participating in each lesson; [DELTA], the change in student audience between lessons. Created with BioRender.

Caption: Figure 4. Student attrition rates with each additional lesson. The number of students lost with each successive lesson addition (A) are graphically represented by mean (black dot) and 95% CI (gray). The formula for the polynomial curve, as well as the [R.sup.2] of best fit for the mean, is presented. Violin plots (B) are visualized to represent the median (dashed line), interquartile ranges (dotted line), and distribution of replicates (width of violin plot).

Figure 2. Assessment improvement by course. Forest plot visuallyrepresenting student score improvement after engaging with thePathElective course content. The lessons, percentage of change inscore, mean with 95% CI, and paired t test P values are presented.Courses that did not participate in the dual-form crossover designare demarcated by an asterisk (*). Abbreviations: AP, anatomicpathology; CP, clinical pathology. Course Change in Mean P value Score (%) [95% CI]AP Courses Autopsy: Lesson 27.72% < .001 1 (n=49) [22.18, 33.26] Breast Pathology: 9.57% < .001 Lesson 1 (n=210) [5.88, 13.26] Breast Pathology: 6.27% .02 Lesson 2 (n=110) [0.87, 11.68] Breast Pathology: 8.48% < .001 Lesson 3 (n=92) [3.66, 13.30] Breast Pathology: 13.08% < .001 Lesson 4 (n=78) [7.88, 18.28] Cardiac 4.31% .04 Pathology: Lesson [0.23, 1 (n=65) 8.38] Cardiac 1.71% .17 Pathology: Lesson [-0.72, 2 (n=82) 4.13] Cardiac 1.20% .5 Pathology: Lesson [-2.29, 3 (n=75) 4.69] Cytopathology: 9.87% < .001 Lesson 1 (n=152) [6.65, 13.09] Cytopathology: 10.81% < .001 Lesson 2 (n=75) [5.85, 15.77] Cytopathology: 2.83% .36 Lesson 3 (n=53) [-3.22, 8.88] Cytopathology: 15.00% < .001 Lesson 4 (n=42) [7.29, 22.71] Cytopathology: 11.90% < .001 Lesson 5 (n=63) [6.33, 17.48] Cytopathology: 10.49% .008 Lesson 6 (n=41) [3.15, 17.83] Cytopathology: 12.67% .04 Lesson 7 (n=30) [1.05, 24.29] Dermatopathology: 3.82% < .001 Lesson 1 [2.79, (n=254) 4.84] Dermatopathology: 6.45% < .001 Lesson 2 [4.39, (n=166) 8.50] Dermatopathology: 12.31% < .001 Lesson 3 [9.06, (n=104) 15.56] Dermatopathology: 0.25% .89 Lesson 4 (n=80) [-3.25, 3.75] Dermatopathology: 4.71% .004 Lesson 5 (n=85) [1.64, 7.78] Gastrointestinal 9.88% < .001 Pathology: Lesson [6.00, 1 (n=163) 13.76] Gastrointestinal 16.41% < .001 Pathology: Lesson [13.21, 2 (n=117) 19.61] Gross Pathology: 7.73% < .001 Lesson 1 (n=563) [5.74, 9.71] Gross Pathology: 16.09% < .001 Lesson 2 (n=391) [13.99, 18.18] Gross Pathology: 29.00% < .001 Lesson 3 (n=390) [26.44, 31.56] Gross Pathology: 30.15% < .001 Lesson 4 (n=323) [27.25, 33.06] Gross Pathology: 7.14% < .001 Lesson 5 (n=311) [5.17, 9.10] Gross Pathology: 20.76% < .001 Lesson 6 (n=277) [17.57, 23.95] Genitourinary 17.53% < .001 Pathology: Lesson [13.84, 1 (n=93) 21.22] Genitourinary 14.52% < .001 Pathology: Lesson [9.80, 2 (n=84) 19.25] Genitourinary 10.79% < .001 Pathology: Lesson [5.34, 3 (n=63) 16.25] Genitourinary 16.72% < .001 Pathology: Lesson [11.18, 4 (n=64) 22.26] Gynecologic 10.90% < .001 Pathology: Lesson [5.90, 1 (n=67) 15.89] Gynecologic 14.89% < .001 Pathology: Lesson [9.96, 5.1 (n=45) 19.82] Gynecologic 24.00% .005 Pathology: Lesson [14.67, 5.2 (n=25) 33.33] Head and Neck 11.60% < .001 Pathology (n=81) [6.98, 16.23] Hematopathology: 20.74% < .001 Lesson 1 (n=229) [17.92, 23.56] Hematopathology: 16.07% < .001 Lesson 2 (n=107) [11.04, 21.11] Hematopathology: 21.19% < .001 Lesson 3 (n=101) [16.11, 26.27] Hematopathology: 19.75% < .001 Lesson 4 (n=81) [13.57, 25.93] Neuropathology: 20.00% < .001 Lesson 1 (n=37) [12.33, 27.67] Pediatric 18.95% < .001 Pathology: Lesson [13.71, 1 (n=105) 24.20] Pediatric 8.06% .007 Pathology: Lesson [2.42, 2 (n=67) 13.70] Pediatric 18.12% < .001 Pathology: Lesson [12.46, 3 (n=69) 23.77] Pulmonary 12.84% < .001 Pathology: Lesson [8.69, 1 (n=109) 17.00] Pulmonary 21.33% < .001 Pathology: Lesson [16.18, 2 (n=60) 26.49] Pulmonary 15.81% < .001 Pathology: Lesson [8.78, 3 (n=43) 22.85] Illustrations of 14.39% < .001 Histology (n=179) [11.12, 17.66] Clinical 15.30% < .001 Microbiology: [9.75, Lesson 1 (n=122) 20.90] Clinical 7.40% .003 Microbiology: [2.59, Lesson 2 (n=122) 12.17] Hematology (n=323) 9.62% < .001 [7.95, 11.29] Flow Cytometry 11.11% < .001 (n=108) [5.87, 16.35] Coagulation 11.82% < .001 (n=181) [9.20, 14.45]CP Courses Clinical 16.03% < .001 Chemistry: Pre- [13.25, analytic * 18.81] (n=262) Clinical 18.23% < .001 Chemistry: [12.63, Analytic (n=158) 23.83] Clinical 20.79% < .001 Chemistry: Post- [13.22, analytic Part 1 28.36] (n=95) Clinical 8.02% .04 Chemistry: Post- [0.68, analytic Part 2 15.37] (n=81) Blood Bank: Cross 13.86% < .001 and Screen [11.33, (n=207) 16.40] Molecular 14.78% < .001 Microbiology: [6.94, Lesson 1 (n=46) 22.63] Molecular 13.53% .007 Microbiology: [4.28, Lesson 2 (n=35) 22.78] Molecular 12.00% .001 Microbiology: [5.24, Lesson 3 (n=35) 18.76]Grand Means AP Total (n=6150) 14.01% .02 [10.49, 17.53] CP Total (n=1756) 13.23% .002 [9.17, 17.29] Total (n=7906) 13.83% .02 [10.20, 17.47]

Figure 3. Student satisfaction data by course and assessmentcategory.A Course Average and scaled heat map Overall Lecturer Social Quality MediaAutopsy: Lesson 1 4.60 4.75 4.05Breast Pathology: 4.21 4.44 3.94Lesson 1Breast Pathology: 4.45 4.59 4.15Lesson 2Breast Pathology: 4.43 4.50 4.17Lesson 3Breast Pathology: 4.42 4.50 4.13Lesson 4Cardiac Pathology: 4.46 4.53 4.06Lesson 1Cardiac Pathology: 4.48 4.55 4.13Lesson 2Cardiac Pathology: 4.54 4.57 4.24Lesson 3Cytopathology: 4.47 4.58 3.99Lesson 1Cytopathology: 4.43 4.60 4.13Lesson 2Cytopathology: 4.31 4.46 4.02Lesson 3Cytopathology: 4.53 4.58 4.15Lesson 4Cytopathology: 4.58 4.62 4.17Lesson 5Cytopathology: 4.49 4.52 4.17Lesson 6Cytopathology: 4.47 4.49 4.20Lesson 7Dermatopathology: 4.62 4.70 4.14Lesson 1Dermatopathology: 4.72 4.75 4.36Lesson 2Dermatopathology: 4.65 4.68 4.38Lesson 3Dermatopathology: 4.58 4.64 4.32Lesson 4Dermatopathology: 4.65 4.65 4.40Lesson 5Gastrointestinal 4.65 4.70 4.21Pathology: Lesson 1Gastrointestinal 4.56 4.59 4.09Pathology: Lesson 2Gross Pathology: 4.51 4.57 3.88Lesson 1Gross Pathology: 4.58 4.62 4.07Lesson 2Gross Pathology: 4.67 4.69 4.11Lesson 3Gross Pathology: 4.64 4.65 4.15Lesson 4Gross Pathology: 4.59 4.60 4.14Lesson 5Gross Pathology: 4.64 4.68 4.19Lesson 6Genitourinary 4.53 4.60 4.17Pathology: Lesson 1Genitourinary 4.53 4.56 4.16Pathology: Lesson 2Genitourinary 4.56 4.56 4.15Pathology: Lesson 3Genitourinary 4.52 4.59 4.13Pathology: Lesson 4Gynecologic 4.55 4.62 4.28Pathology: Lesson 1Gynecologic 4.57 4.60 4.19Pathology: Lesson5.1Gynecologic 4.52 4.59 4.14Pathology: Lesson5.2Head and Neck 4.59 4.63 4.10PathologyHematopathology: 4.63 4.69 3.91Lesson 1Hematopathology: 4.33 4.53 4.02Lesson 2Hematopathology: 4.42 4.52 3.96Lesson 3Hematopathology: 4.42 4.51 3.99Lesson 4Neuropathology: 4.67 4.74 4.07Lesson 1Pediatric Pathology: 4.54 4.61 4.20Lesson 1Pediatric Pathology: 4.53 4.56 4.18Lesson 2Pediatric Pathology: 4.60 4.60 4.13Lesson 3Pulmonary Pathology: 4.60 4.72 4.26Lesson 1Pulmonary Pathology: 4.51 4.62 4.29Lesson 2Pulmonary Pathology: 4.57 4.61 4.30Lesson 3Illustrations of 4.71 4.73 4.46HistologyIntroduction to 4.50 4.59 4.36ClinicalMicrobiologyMolecular 4.49 4.51 4.32MicrobiologyHematology 4.60 4.65 4.26Flow Cytometry 4.50 4.54 4.07Coagulation 4.59 4.63 4.27Clinical Chemistry: 4.58 4.62 4.30Pre-analyticClinical chemistry: 4.44 4.50 4.25AnalyticClinical Chemistry: 4.51 4.50 4.26Post-analytic Part 1Clinical Chemistry: 4.52 4.49 4.29Post-analytic Part 2Blood Bank: Cross 4.55 4.64 4.16and ScreenMolecular 4.42 4.51 4.18Microbiology: Lesson1Molecular 4.56 4.56 4.25Microbiology: Lesson2Molecular 4.49 4.49 4.18Microbiology: Lesson3 Lecture Help Format QualityAutopsy: Lesson 1 4.55 4.43 4.58Breast Pathology: 4.11 4.16 4.33Lesson 1Breast Pathology: 4.42 4.28 4.46Lesson 2Breast Pathology: 4.41 4.32 4.44Lesson 3Breast Pathology: 4.42 4.29 4.44Lesson 4Cardiac Pathology: 4.46 4.27 4.47Lesson 1Cardiac Pathology: 4.52 4.30 4.53Lesson 2Cardiac Pathology: 4.53 4.38 4.54Lesson 3Cytopathology: 4.47 4.37 4.53Lesson 1Cytopathology: 4.44 4.35 4.48Lesson 2Cytopathology: 4.32 4.27 4.43Lesson 3Cytopathology: 4.52 4.37 4.52Lesson 4Cytopathology: 4.57 4.44 4.65Lesson 5Cytopathology: 4.46 4.39 4.54Lesson 6Cytopathology: 4.47 4.36 4.47Lesson 7Dermatopathology: 4.61 4.46 4.67Lesson 1Dermatopathology: 4.70 4.47 4.69Lesson 2Dermatopathology: 4.64 4.44 4.60Lesson 3Dermatopathology: 4.54 4.42 4.57Lesson 4Dermatopathology: 4.63 4.48 4.63Lesson 5Gastrointestinal 4.64 4.38 4.66Pathology: Lesson 1Gastrointestinal 4.53 4.34 4.58Pathology: Lesson 2Gross Pathology: 4.49 4.36 4.55Lesson 1Gross Pathology: 4.56 4.42 4.60Lesson 2Gross Pathology: 4.66 4.51 4.65Lesson 3Gross Pathology: 4.65 4.50 4.61Lesson 4Gross Pathology: 4.59 4.47 4.59Lesson 5Gross Pathology: 4.63 4.50 4.61Lesson 6Genitourinary 4.54 4.41 4.60Pathology: Lesson 1Genitourinary 4.52 4.34 4.55Pathology: Lesson 2Genitourinary 4.55 4.42 4.57Pathology: Lesson 3Genitourinary 4.53 4.39 4.54Pathology: Lesson 4Gynecologic 4.55 4.47 4.58Pathology: Lesson 1Gynecologic 4.53 4.45 4.61Pathology: Lesson5.1Gynecologic 4.54 4.39 4.52Pathology: Lesson5.2Head and Neck 4.60 4.40 4.60PathologyHematopathology: 4.65 4.30 4.64Lesson 1Hematopathology: 4.36 4.18 4.39Lesson 2Hematopathology: 4.38 4.27 4.43Lesson 3Hematopathology: 4.38 4.24 4.43Lesson 4Neuropathology: 4.67 4.43 4.74Lesson 1Pediatric Pathology: 4.52 4.33 4.52Lesson 1Pediatric Pathology: 4.50 4.34 4.53Lesson 2Pediatric Pathology: 4.60 4.39 4.57Lesson 3Pulmonary Pathology: 4.61 4.40 4.62Lesson 1Pulmonary Pathology: 4.50 4.43 4.60Lesson 2Pulmonary Pathology: 4.54 4.39 4.51Lesson 3Illustrations of 4.69 4.66 4.71HistologyIntroduction to 4.50 4.44 4.46ClinicalMicrobiologyMolecular 4.42 4.41 4.42MicrobiologyHematology 4.60 4.50 4.60Flow Cytometry 4.51 4.38 4.51Coagulation 4.60 4.52 4.58Clinical Chemistry: 4.54 4.50 4.57Pre-analyticClinical chemistry: 4.42 4.43 4.40AnalyticClinical Chemistry: 4.51 4.43 4.46Post-analytic Part 1Clinical Chemistry: 4.50 4.47 4.52Post-analytic Part 2Blood Bank: Cross 4.51 4.36 4.53and ScreenMolecular 4.43 4.41 4.47Microbiology: Lesson1Molecular 4.57 4.53 4.59Microbiology: Lesson2Molecular 4.48 4.42 4.51Microbiology: Lesson3B Cross-category meanC Category-specific meanThe 6 assessed satisfaction domains are represented with thecourse average and scaled heat map (A). The courses and domainswith any variation from the total, cross-category mean (B), andcategory-specific mean (C) are represented using the average asblack, as well as maximum and minimum as green and red,respectively.