Homeopathy has been practiced for the last 221 years, and for most of that time curing the sick was all that mattered. But over the last few decades formal documentation and analysis of treatment has become a necessity due to the increasing focus on evidence-based medicine.
The moment we hear about evidence-based medicine, we start thinking about randomized controlled trials (RCT). But these are only one of the many research tools and methodologies available and they are not suitable for answering many research questions. One of the other popular research methodologies in clinical research is ‘Outcome Research’.
An outcome study is a generic term that refers to investigations of the results of therapeutic interventions regardless of the type of investigation used.[i] Outcomes research is a body of literature that identifies, measures, and evaluates the results of health care services in general and includes clinical effects, economic data, and quality of life information.
In this paper, I’ll first outline the difference between Randomized Controlled Trials and Outcome Studies, give more details about Outcome Research and then explain how homeopaths can build effective outcome studies even in a very small clinic setup, with an example. Since this paper is meant for clinicians, I’ll purposefully avoid some technical research and statistics jargon.
Difference between RCT and Outcome Studies
Outcome studies are believed (by some) to be less scientific because they do not follow the standard format of a randomized controlled trial (RCT). However, outcome studies can be just as rigorous as RCTs, but the scope of the studies may vary. The primary types of research questions that RCTs and outcome studies are designed to answer differ considerably. RCTs are suited to answer questions about safety and efficacy. RCTs take place in highly controlled settings designed to maximize patient compliance and limit extraneous confounding factors. They are typically short in duration and focus on laboratory or biomedical endpoints.
Outcome studies are designed to answer “real-world” questions pertaining to how drugs are used in the broad population.[ii] This information can be used to answer policy or clinical management questions. Outcome studies focus on cost impact, clinical outcome, and quality-of-life issues. These studies are most useful when real- world variables (e.g., cost, quality of life impact, long-term morbidity) are allowed to have an effect on the data, because outcomes do not depend solely on the activity of the drug but also on the patients and their environment. RCTs are often referred to as efficacy studies, whereas outcome studies may be referred to as effectiveness studies. The differences between RCT and outcomes studies are summarized in the table below[iii].
Outcome studies are gaining importance in all areas of medical practice due to managed care’s growing emphasis on evidence-based medicine. Most outcome studies are designed to evaluate a program’s success rate as well as identify areas of treatment that may need improvement. Moreover, outcome studies serve as a valuable motivational tool for therapists and clinicians because these studies substantiate the positive impact of the professionals’ work.
Benefits of Outcome Studies
- A first important advantage is that – unlike funded research studies – clinical outcome investigations can continue without preset funding cut-off dates, enabling outcome data registries to grow ever larger (and thus more valuable) over time.
- A second important advantage is that resulting outcome studies document changes following interventions in real world settings, thus addressing the issue of actual clinical effectiveness.
- Finally, outcome research may reveal promising – but previously unsuspected – trends that may fruitfully guide the formulation of questions to be investigated in subsequent scientifically controlled research.
PROPERTIES of OUTCOME STUDIES
- Properly conducted, outcome studies can accomplish much that is done also by scientifically controlled experimental studies. The main inherent limitation is that – in contradistinction to scientifically controlled experimental study designs – they cannot establish causality; in its stead, they can document correlations.
- In common with scientifically controlled experimental designs, outcomes studies can document the existence of clinical changes following specified interventions, the magnitudes of the changes, their directions, statistical significance, along with analyses of statistical power.
- Outcome studies can (and often should) use the very same assessment instruments and protocols as scientifically controlled experimental research, making them comparable in terms of psychometric properties such as study validity, reliability, and sensitivity.
- Outcome studies can also probe intervention effects longitudinally to study maintenance effects; and they can also probe generalization of treated behaviors with proper design.
Process of Outcome Research
- Identifying the patient outcome
- building the team
- selecting the instrument
- measuring the patient outcome
- analyzing the data
- summarizing the findings
- applying the findings to practice
- planning future patient outcomes projects
Designing an Outcome Study
In the section below, I’ll describe a simple yet effective approach to designing and implementing an outcome study that will deliver meaningful results in a small clinical setup. The main goal is to evaluate and improve the hospital’s/clinic’s treatment programs. Such small clinical setup trials (also called office-based clinical trials) are not something new. They are frequently used in conventional medicine in Phase IV of new drug trials, where physicians test the drugs in their practice. But in such cases the overall trial design is usually an RCT, and except for the administration of intervention, all aspects of the design and implementation are usually controlled by the pharmaceutical companies funding the research.
But here we are talking about planning and implementing an outcome study in an independent clinical setup.
We will divide the whole process into three parts:
- Planning Stage
- Implementation Stage
- Analysis & Presentation Stage
The four basic questions for the planning stage are:
- What are the objectives of the outcome study?
- What data needs to be collected and what instruments* will be used?
- When to collect the data?
- Who is involved?
(* The word ‘instrument’ here does not refer just to a mechanical tool but to all tools like questionnaires, laboratory reports, clinical assessment and tests done with mechanical instruments, which are used to collect data and monitor the effect of the treatment.)
Providing detailed answers to these questions not only helps design the outcome study but also ensures that the information gathered is reliable and provides appropriate feedback for improvements.
What are the objectives of the outcome study?
The most frequent goal of any clinical outcome study is to find and monitor the success rate of a clinical program, approach or intervention.
Measuring a treatment’s success rate is the very first challenge of setting up an outcome study because success in any program can be defined using different variables in different ways. Is success related to reduction of symptom severity, or reduction of some pathological markers, or adherence of the patient, or the financial benefits of the treatment?
It is important to define from the very beginning how much change is expected in a specific patient population over time. Do we want to see any difference that is statistically significant over time, such as a decrease in symptom severity from first consultation to the last? Or do we want to reach a specific target value that we know reflects a clinically meaningful change, such as a complete change in a pathological marker?
A clear understanding of the meaning of success is a key first step. These definitions become essential not only to identify the goals of the outcome study but also to select the variables to measure that success.
What data needs to be collected?
Once success has been defined, the quantifiable variables are selected to measure that success. Usually, quantifiable variables are extracted from well-established instruments/guides that measure symptom severity and disease evolution for each patient. The selection of these instruments should meet four criteria:
* Accurate and reliable.
* Widely available.
* Easy to use and interpret.
In essence, the chosen instruments should be recognized as accurate and reliable tools in their respective fields. For e.g. if a study seeks to ascertain the severity of asthma in children in the city, it can use as instrument, a symptom-based questionnaire used in epidemiological research like the ISAAC questionnaire or another instrument developed with an internationally accepted clinical classification of asthma severity (GINA) as its basis. For most disease conditions, internationally accepted instruments are available to plan your research around.
Based on the selection of instruments, it is easy to see how the information obtained from the outcome study can be used to provide valuable insight into which treatment components are effective and which may need to be revised to improve the success rate.
When to collect the data
Once the objectives have been set and the instruments have been chosen, the next step is to decide when to collect the data to obtain meaningful results.
There are two main types of outcome studies; retrospective and prospective. Retrospective studies are faster and easier because they consist of either analyzing data that has been collected in the past or contacting former program participants to ask about past treatment.
Prospective studies are the more common choice; they consist of collecting data at consistent time intervals, preferably starting with the registration or first consultation.
A primary advantage of prospective outcome studies is that they obtain factual information of the patient at baseline. The importance of establishing a baseline cannot be overstated, as this is the point that serves as a reference for most subsequent outcome measures. Learning as much as possible about the status of the patient at the start of treatment is vital because the admission point/first consultation represents the patient’s baseline. Since the goal of treatment is a reduction of symptoms over time, any changes from this initial baseline value become important measures for evaluating treatment success.
Similar to first consultation information, final consultation information also is valuable because it allows a measure of the effects of treatment immediately after the patient finishes the treatment.
Who is involved?
The third planning step is to determine which staff members will be involved in gathering, analyzing and interpreting the data. The two key persons in the research team are the team leader and the outcome study coordinator. The team leader is responsible for the overall direction of the project and should be a good manager as well as an educator and visionary.
The study coordinator serves as the liaison among clinicians, counselors and the project leader, and is paramount to the outcome study’s success. In addition to being skilled in collecting data and committed to the study, the coordinator must have strong people skills and must interact well with both therapists and patients. The study coordinator can also be responsible for interacting with medical records staff to obtain relevant demographics and insurance information, as well as contacting patients after treatment to gather follow-up data.
Other members of the outcome study team who also play important roles are the therapists, who provide meaningful insight regarding the definition of success and the clinical variables to measure the success rate. They also need to be willing to present the results to patients, family members, administrators and peers.
There is no doubt that the most effective data gathering process is one that can be integrated into the daily workflow. This means that doctors, medical directors and staff are all woven in the process of data collection at different levels. Done this way, it only uses a small fraction of their time to collect the data. For most outcome studies, the initial request for patient participation should become part of the patient registration or first consultation process.
Last but not least, every outcome study requires a statistician to assist in developing a comprehensive yet simple process for collecting, analyzing and interpreting the data. As statisticians usually are hired as outside consultants, it is important that the data analysis process is reliable yet easy to follow. Along with helping to ensure meaningful study results, a simple process allows staff to pass on the data gathering tasks to other team members over time.
Implementing the outcome study
Now that the goals of the outcome study have been defined, the instruments have been chosen, the data collection process is in place and the participating staff has been identified, the final step is to implement the study.
The team leader is responsible for designing and monitoring the study’s overall operation as well as communicating the plan and the progress to all members of the team. The research coordinator’s first task is to prepare the informed consent form, which asks for patient participation at admission/registration to the program. Medical directors must train the counselors on how to approach the patients for enrollment in the study and the proper techniques to administer the instruments.
The team then decides on a starting date and the research coordinator assumes the role of monitoring progress and entering the data into a format that the statistician has previously specified. Patients should be given a unique identification number to keep names confidential.
A good starting point for an outcome study is to collect information at registration or first consultation. Once that protocol is established and working well, the study can move on to collecting follow-up data on each subsequent patient visit.
Collecting data at various intervals after treatment is over is a challenging task. The most common methods are personal follow-ups in outpatient setup, sending the patient a questionnaire or telephone interviews. The telephone interviews give much better results than the questionnaire.
Analyzing and presenting the information
Once the information has been gathered, the final step is to analyze and interpret it. A simple tool to analyze the data is the paired t test, which determines if there is a meaningful mathematical change between two consecutive time periods in a given patient population. The paired t test can evaluate the overall effects of treatment in decreasing symptom severity from admission to discharge date or from discharge to follow-up. The advantages of the paired t test are:
* The same population is being compared at two different time periods.
* It is a simple way to tell whether there is a change in a single outcome variable over time.
* It tells whether the changes are statistically significant.
Another advantage of the paired t test is that it can be analyzed with common software packages such as Microsoft Excel as well as the more sophisticated software.
Planning an Outcome Study in a Small Homeopathic Setup
After understanding the basics of implementing an outcome study, I’ll elaborate the methodology further through a mock example. In this example, I’ll be referring to a homeopathic clinical setup, but the article is valid for any branch or system of medicine. I’ll be adding some additional guidelines for outcome studies of therapies that are individualistic and holistic in nature.
Big homeopathic hospitals are still a rarity, so we will build this outcome study for a smaller clinical setup, where there are one to four physicians involved in treatment.
Deciding the Objective
When you are trying to build a clinical outcome study, make sure you chose a condition for which you have/get significant number of patients, because the power (n) can make a study statistically significant or insignificant.
Let us say, I want to build an outcome study to measure the effect of individualized homeopathic treatment in pediatric asthma. Once I have decided the broad area of my research, I’ll have to answer many questions during the planning stage:
- When will the study start?
- What will be the duration of the study?
- What will be the size of the study?
- How many people would be involved?
- How will the diagnosis be confirmed?
- What would be the inclusion and exclusion criteria?
- What instruments (questionnaires and tests) would be used?
- What would be the frequency of follow-ups?
- Which software would be used for data analysis?
- What are the financial costs involved?
All this information needs to be properly documented before the start of the study. If you are not trained in research, it is better to take the help of someone who is working in clinical research. Reading existing trials and talking to the physicians in the concerned department of a big hospital/research center can give you a lot of information for the proper and effective planning of an office-based outcome study.
I can choose to include in the study all pediatric cases of asthma coming to the OPD/clinic in the next 12 months with a follow-up period of 12 months. So effectively, this study will continue for 2 years. The data analysis part will start only after that.
Although outcome studies are not very costly, you still need to make an assessment of the costs involved, which can include salaries, office expenditure, cost of acquiring software and instruments, cost of pathological tests required for monitoring the cases etc. For e.g., if we wish to get a spirometry done every three months, then it has to be decided beforehand whether it will be done from your funds or by the patient and what would be the costs involved.
An outcome study, even in an office based setup, needs at least three people for the implementation – one physician involved in treatment, one clinical assistant for data recording and managing patient schedules, and one statistician (part-time) for data analysis. Depending upon the size of the study and the size of the practice (individual office/polyclinic/hospital OPD), there could be many more people involved.
In selecting an instrument, users must consider the different types of instrument that are available and how they meet the requirements of the proposed application[iv]. Selecting the right instrument is of vital importance. If you chose an instrument that is not accepted by large international organizations, your research will not be held in good esteem. For e.g. there are many accepted instruments/questionnaires for evaluating cases of pediatric asthma. Some of them can be found here:
More general information about instrument types can be found here:
Now depending upon what parameters I want to track, I’ll chose the instrument accordingly. For e.g. if I have to monitor the effect of my treatment, I should choose an instrument that monitors the data like frequency and intensity of episodes, frequency of symptoms between episodes, and intensity of impairment between episodes.