Planning for a Great Residency Project
By Lisa Lohr, PharmD, BCOP, BCPS
Department of Pharmacy Services, University of Minnesota Medical Center, Fairview,
Minneapolis
Conducting a practice-based major project can be one of the most rewarding experiences for a pharmacy resident. It provides opportunities to learn real-world skills in project management, process improvement, and practice-based practice-based research. Working together to reach a common goal can provide many learning opportunities for the resident and preceptor. Using a structured plan for conducting a residency project will enhance the educational experiences as well as allow for project completion during the residency year.
Generating an idea for the project
Generating the idea for the project will require a lot of input from the preceptor. Because work on the project should start as soon as possible in the year, many residency programs generate project concepts before the resident even begins the program. Some of the best residency projects arise out of practice-based problems, after looking backward to find the root of the problem. Project ideas can also be developed out of departmental strategic goals and quality improvement initiatives, as well as national benchmarks, such as Joint Commission requirements and the National Patient Safety Goals.
The best projects are concise, interesting to both the resident and preceptor, and relevant to pharmacy practice. Projects with a focus that is too broad or that contains too many objectives can be very problematic. The scope of the project must remain contained to provide a quality learning opportunity and to make sure that the project can be completed during the residency year. Handing projects from one resident to the next does not allow the residents to participate during all phases of the project. "Project drift" can occur when the project's goals shift because of new obstacles or when newcomers to the project propose a new focus to the project. This should be avoided. In projects based on implementation of a process change, thought should be given to designing the intervention at this stage. The intervention should be accomplishable given the resources available during the project year and afterward, so that the project will yield useful outcomes, and should rely as little as possible on the availability of outside resources (eg, information technology [IT] support, physician time). The purpose of the study and primary and secondary outcomes of the study should be set, and the resident should be able to state them in one or two simple sentences.
Designing the study
The next step in a successful residency project is the creation of a comprehensive timeline. Essentially, this outlines the steps involved in project completion and the estimated time needed. Time spent in this phase will pay off later in increased efficiency. It allows for identification of when resources (eg, resident project time, IT support) will be needed. Responsibilities and roles of the project team members should also be assigned. Regular meetings should be scheduled to track progress as well as identify and solve roadblocks so that the project can continue successfully. Delays will be encountered in every residency project, and setting an aggressive timeline can allow for unexpected delays and ensure that the project is completed.
After setting a timeline, the next step is designing the actual study. Residency study projects can take many forms. Although it is possible to have residency projects that do not involve data collection, more complex projects that systematically investigate a situation or confirm accomplishment of a goal are preferred. Published studies generally are grouped into descriptive reports, observational studies, and experimental studies. Descriptive reports include case reports and case series that investigate a particular clinical problem. These offer a great introduction into publishing, providing the resident experience in formal case presentation and manuscript preparation. Observational studies in clude cohort studies, case-control, and cross-sectional studies. These studies do not involve an intervention, but examine the association between a risk or exposure and the development of an outcome. To gain useful information, these studies are usually large, requiring substantial time and re sources, and are usually not appropriate for 1-year residency projects. A randomized clinical trial will provide the most scientific proof of the research question. In these studies, the investigator actively intervenes and measures the effect of that intervention. Although it is the gold standard of research, a randomized clinical trial is very time-consuming and expensive, making it, most likely, a better fit for a research fellowship.
Quasi-experimental study designs are appropriate for many residency projects. Although scientifically limited in establishing causality, they are able to measure the effect of an intervention. The posttest-only analysis simply measures the outcome after an intervention and is the weakest of the group. This analysis type may be used if no data were collected before the intervention. The posttest with control comparison analysis uses an outcome measurement in the experimental group as well as the control group, but does not show that the outcome improved after the intervention. The pretest and posttest design is probably the most commonly used. It measures the outcome before and after an intervention, but is prone to temporal bias. Temporal bias occurs when something else that could influence the outcome (eg, new lab test methodology, new physician staff) happens concurrently with the intervention. The pretest and posttest with control design is stronger than the posttest with control design because it measures the outcome before and after the intervention, both in the experimental group and in a control group that did not have the intervention. This controls for some confounders but doubles the resources needed for data collection. Other considerations in designing the study are the definition of the outcome measures, the inclusion and exclusion criteria, sample size, setting, and data sources. Types of outcome measures include economic (direct costs), clinical (clinical effects, physiologic measures, length of stay), or humanistic variables (quality of life, patient satisfaction). The outcome measure should be as precise and as relevant as possible given the available resources. For example, mea suring venous thromboembolisms (VTEs) by venography after hospitalization is more clinically relevant, but it may only be possible to mea sure the surrogate measure of the provision of appropriate VTE prophylaxis during hospitalization. Inclusion and exclusion criteria should be chosen to best represent the population of interest. A more homogeneous sample, chosen with a lot of exclusion criteria, is less variable but it less likely the data can be extrapolated to the general population. A heterogeneous sample, chosen with few exclusion criteria, would be more variable and less likely to yield a specific result, but more representative of the whole. Comparator groups should be as similar to the study population as possible and may include patients at a different location, those offered "usual care," or historical controls.
Planning for data collection should include the source of the data, such as medical records or billing records. A common problem in residency projects is the collection of too much data, just in case it is needed. This is time-consuming and can discourage the resident. If you guard against "project drift" and collect only the data to match the outcome measures, this phase will go smoother and faster. The resident should think ahead at this point to how the data will be analyzed and presented. This will help him or her focus on the outcomes of interest. For example, if the proportion of patients with creatinine clearance <30 mL/min is needed, collect that data as a yes/no field, instead of recording all the laboratory values on admission. Sample size in a randomized controlled trial is defined by statistical calculations, but in quasi-experimental designs, the sample size is usually estimated based on the personal experience of the preceptor and available time and resources. The datacollection tool should be prepared at this point, including only the relevant information needed. The data-collection tool should be tested on a few patients before fine-tuning it. A plan for removing patient-specific data should be made, to protect the patients' privacy. Data-collection tools can range from simple onepage paper forms to complicated database- driven electronic tools, based on the needs of the project.
Getting approval
The next step in the process is writing the protocol and the institutional review board (IRB) submission. If all the aspects discussed have been considered, writing the proposal should be fairly simple. A usual proposal outline includes an introduction/justification, purpose/outcomes, methods, plan for analysis, data-handling plan, and references. Obtaining IRB approval ensures that the project is ethical and protects the human subjects and their privacy. It is required for projects that generate data that will be published or presented outside of the institution. Most IRBs classify projects according to the risk they pose to the patients. Many residency projects will qualify for the less intensive "exempt" or "expedited" reviews, because they pose minimal to no risk to the patient.
Implementing the project
After approval by the IRB and residency advisory council, the project can proceed. If the project is using a quasiexperimental design, data collection is usually the next step. It is important for the preceptor to monitor the resident closely during this phase. Many residency programs allow for special project time to be assigned, outside of rotational responsibilities. This is one of the phases where the timeline can go haywire, so planning for delays and close preceptor monitoring is necessary for a successful project. Gearing up for the collection of data in the post-phase may require some motivational skills on the part of the preceptor.
Implementation of the intervention can be very rewarding for the resident. It may include committee decision making, presentations to administrators, designing therapy protocols, conducting process analysis, assisting in developing IT solutions, teaching pharmacists, physicians and patients, or re-engineering a problematic process. As much of this step as possible should be delegated to the resident. These career skills can contribute to the resident's future plans.
Presentation of the project
After analysis of the data and formulation of conclusions, the next step in a successful residency project is the preparation of the material for verbal and written presentation. Condensing the study method, data, and conclusions into a 10- to 20-minute presentation can be challenging. Many aspects can be summarized, given the likely audience of the presentation. Most data are best presented in graphic form. Complicated tables and replication of the data-collection form or treatment protocol onto a single slide should be avoided. Adequate time should be devoted to a discussion of the conclusions, implications of the data, and the plan for future work. After initial slide review by the preceptor, the resident should be given opportunities to practice the presentation in front of an audience at least twice before the regional residency conference. In addition, the resident should take every opportunity to present the project internally to appropriate committees, departments, or task forces. Even if there is not time or opportunity for publication, the resident should write a manuscript based on the study, using a format appropriate for publication, to gain the experience of formal, scientific writing.
A successful residency project can provide many important real-world learning experiences for the resident and foster the mentoring relationship between the resident and the preceptor. Advance planning can turn the residency project into a meaningful experience for both the resident and preceptor.
