Homeopathy has been practiced for the last 221 years, and for most of that time curing the sick was all that mattered. But over the last few decades formal documentation and analysis of treatment has become a necessity due to the increasing focus on evidence-based medicine.
The moment we hear about evidence-based medicine, we start thinking about randomized controlled trials (RCT). But these are only one of the many research tools and methodologies available and they are not suitable for answering many research questions. One of the other popular research methodologies in clinical research is ‘Outcome Research’.
An outcome study is a generic term that refers to investigations of the results of therapeutic interventions regardless of the type of investigation used.[i] Outcomes research is a body of literature that identifies, measures, and evaluates the results of health care services in general and includes clinical effects, economic data, and quality of life information.
In this paper, I’ll first outline the difference between Randomized Controlled Trials and Outcome Studies, give more details about Outcome Research and then explain how homeopaths can build effective outcome studies even in a very small clinic setup, with an example. Since this paper is meant for clinicians, I’ll purposefully avoid some technical research and statistics jargon.
Difference between RCT and Outcome Studies
Outcome studies are believed (by some) to be less scientific because they do not follow the standard format of a randomized controlled trial (RCT). However, outcome studies can be just as rigorous as RCTs, but the scope of the studies may vary. The primary types of research questions that RCTs and outcome studies are designed to answer differ considerably. RCTs are suited to answer questions about safety and efficacy. RCTs take place in highly controlled settings designed to maximize patient compliance and limit extraneous confounding factors. They are typically short in duration and focus on laboratory or biomedical endpoints.
Outcome studies are designed to answer “real-world” questions pertaining to how drugs are used in the broad population.[ii] This information can be used to answer policy or clinical management questions. Outcome studies focus on cost impact, clinical outcome, and quality-of-life issues. These studies are most useful when real- world variables (e.g., cost, quality of life impact, long-term morbidity) are allowed to have an effect on the data, because outcomes do not depend solely on the activity of the drug but also on the patients and their environment. RCTs are often referred to as efficacy studies, whereas outcome studies may be referred to as effectiveness studies. The differences between RCT and outcomes studies are summarized in the table below[iii].
Outcome studies are gaining importance in all areas of medical practice due to managed care’s growing emphasis on evidence-based medicine. Most outcome studies are designed to evaluate a program’s success rate as well as identify areas of treatment that may need improvement. Moreover, outcome studies serve as a valuable motivational tool for therapists and clinicians because these studies substantiate the positive impact of the professionals’ work.
Benefits of Outcome Studies
- A first important advantage is that – unlike funded research studies – clinical outcome investigations can continue without preset funding cut-off dates, enabling outcome data registries to grow ever larger (and thus more valuable) over time.
- A second important advantage is that resulting outcome studies document changes following interventions in real world settings, thus addressing the issue of actual clinical effectiveness.
- Finally, outcome research may reveal promising – but previously unsuspected – trends that may fruitfully guide the formulation of questions to be investigated in subsequent scientifically controlled research.
PROPERTIES of OUTCOME STUDIES
- Properly conducted, outcome studies can accomplish much that is done also by scientifically controlled experimental studies. The main inherent limitation is that – in contradistinction to scientifically controlled experimental study designs – they cannot establish causality; in its stead, they can document correlations.
- In common with scientifically controlled experimental designs, outcomes studies can document the existence of clinical changes following specified interventions, the magnitudes of the changes, their directions, statistical significance, along with analyses of statistical power.
- Outcome studies can (and often should) use the very same assessment instruments and protocols as scientifically controlled experimental research, making them comparable in terms of psychometric properties such as study validity, reliability, and sensitivity.
- Outcome studies can also probe intervention effects longitudinally to study maintenance effects; and they can also probe generalization of treated behaviors with proper design.
Process of Outcome Research
- Identifying the patient outcome
- building the team
- selecting the instrument
- measuring the patient outcome
- analyzing the data
- summarizing the findings
- applying the findings to practice
- planning future patient outcomes projects
Designing an Outcome Study
In the section below, I’ll describe a simple yet effective approach to designing and implementing an outcome study that will deliver meaningful results in a small clinical setup. The main goal is to evaluate and improve the hospital’s/clinic’s treatment programs. Such small clinical setup trials (also called office-based clinical trials) are not something new. They are frequently used in conventional medicine in Phase IV of new drug trials, where physicians test the drugs in their practice. But in such cases the overall trial design is usually an RCT, and except for the administration of intervention, all aspects of the design and implementation are usually controlled by the pharmaceutical companies funding the research.
But here we are talking about planning and implementing an outcome study in an independent clinical setup.
We will divide the whole process into three parts:
- Planning Stage
- Implementation Stage
- Analysis & Presentation Stage
The four basic questions for the planning stage are:
- What are the objectives of the outcome study?
- What data needs to be collected and what instruments* will be used?
- When to collect the data?
- Who is involved?
(* The word ‘instrument’ here does not refer just to a mechanical tool but to all tools like questionnaires, laboratory reports, clinical assessment and tests done with mechanical instruments, which are used to collect data and monitor the effect of the treatment.)
Providing detailed answers to these questions not only helps design the outcome study but also ensures that the information gathered is reliable and provides appropriate feedback for improvements.
What are the objectives of the outcome study?
The most frequent goal of any clinical outcome study is to find and monitor the success rate of a clinical program, approach or intervention.
Measuring a treatment’s success rate is the very first challenge of setting up an outcome study because success in any program can be defined using different variables in different ways. Is success related to reduction of symptom severity, or reduction of some pathological markers, or adherence of the patient, or the financial benefits of the treatment?
It is important to define from the very beginning how much change is expected in a specific patient population over time. Do we want to see any difference that is statistically significant over time, such as a decrease in symptom severity from first consultation to the last? Or do we want to reach a specific target value that we know reflects a clinically meaningful change, such as a complete change in a pathological marker?
A clear understanding of the meaning of success is a key first step. These definitions become essential not only to identify the goals of the outcome study but also to select the variables to measure that success.
What data needs to be collected?
Once success has been defined, the quantifiable variables are selected to measure that success. Usually, quantifiable variables are extracted from well-established instruments/guides that measure symptom severity and disease evolution for each patient. The selection of these instruments should meet four criteria:
* Accurate and reliable.
* Widely available.
* Easy to use and interpret.
In essence, the chosen instruments should be recognized as accurate and reliable tools in their respective fields. For e.g. if a study seeks to ascertain the severity of asthma in children in the city, it can use as instrument, a symptom-based questionnaire used in epidemiological research like the ISAAC questionnaire or another instrument developed with an internationally accepted clinical classification of asthma severity (GINA) as its basis. For most disease conditions, internationally accepted instruments are available to plan your research around.
Based on the selection of instruments, it is easy to see how the information obtained from the outcome study can be used to provide valuable insight into which treatment components are effective and which may need to be revised to improve the success rate.
When to collect the data
Once the objectives have been set and the instruments have been chosen, the next step is to decide when to collect the data to obtain meaningful results.
There are two main types of outcome studies; retrospective and prospective. Retrospective studies are faster and easier because they consist of either analyzing data that has been collected in the past or contacting former program participants to ask about past treatment.
Prospective studies are the more common choice; they consist of collecting data at consistent time intervals, preferably starting with the registration or first consultation.
A primary advantage of prospective outcome studies is that they obtain factual information of the patient at baseline. The importance of establishing a baseline cannot be overstated, as this is the point that serves as a reference for most subsequent outcome measures. Learning as much as possible about the status of the patient at the start of treatment is vital because the admission point/first consultation represents the patient’s baseline. Since the goal of treatment is a reduction of symptoms over time, any changes from this initial baseline value become important measures for evaluating treatment success.
Similar to first consultation information, final consultation information also is valuable because it allows a measure of the effects of treatment immediately after the patient finishes the treatment.
Who is involved?
The third planning step is to determine which staff members will be involved in gathering, analyzing and interpreting the data. The two key persons in the research team are the team leader and the outcome study coordinator. The team leader is responsible for the overall direction of the project and should be a good manager as well as an educator and visionary.
The study coordinator serves as the liaison among clinicians, counselors and the project leader, and is paramount to the outcome study’s success. In addition to being skilled in collecting data and committed to the study, the coordinator must have strong people skills and must interact well with both therapists and patients. The study coordinator can also be responsible for interacting with medical records staff to obtain relevant demographics and insurance information, as well as contacting patients after treatment to gather follow-up data.
Other members of the outcome study team who also play important roles are the therapists, who provide meaningful insight regarding the definition of success and the clinical variables to measure the success rate. They also need to be willing to present the results to patients, family members, administrators and peers.
There is no doubt that the most effective data gathering process is one that can be integrated into the daily workflow. This means that doctors, medical directors and staff are all woven in the process of data collection at different levels. Done this way, it only uses a small fraction of their time to collect the data. For most outcome studies, the initial request for patient participation should become part of the patient registration or first consultation process.
Last but not least, every outcome study requires a statistician to assist in developing a comprehensive yet simple process for collecting, analyzing and interpreting the data. As statisticians usually are hired as outside consultants, it is important that the data analysis process is reliable yet easy to follow. Along with helping to ensure meaningful study results, a simple process allows staff to pass on the data gathering tasks to other team members over time.
Implementing the outcome study
Now that the goals of the outcome study have been defined, the instruments have been chosen, the data collection process is in place and the participating staff has been identified, the final step is to implement the study.
The team leader is responsible for designing and monitoring the study’s overall operation as well as communicating the plan and the progress to all members of the team. The research coordinator’s first task is to prepare the informed consent form, which asks for patient participation at admission/registration to the program. Medical directors must train the counselors on how to approach the patients for enrollment in the study and the proper techniques to administer the instruments.
The team then decides on a starting date and the research coordinator assumes the role of monitoring progress and entering the data into a format that the statistician has previously specified. Patients should be given a unique identification number to keep names confidential.
A good starting point for an outcome study is to collect information at registration or first consultation. Once that protocol is established and working well, the study can move on to collecting follow-up data on each subsequent patient visit.
Collecting data at various intervals after treatment is over is a challenging task. The most common methods are personal follow-ups in outpatient setup, sending the patient a questionnaire or telephone interviews. The telephone interviews give much better results than the questionnaire.
Analyzing and presenting the information
Once the information has been gathered, the final step is to analyze and interpret it. A simple tool to analyze the data is the paired t test, which determines if there is a meaningful mathematical change between two consecutive time periods in a given patient population. The paired t test can evaluate the overall effects of treatment in decreasing symptom severity from admission to discharge date or from discharge to follow-up. The advantages of the paired t test are:
* The same population is being compared at two different time periods.
* It is a simple way to tell whether there is a change in a single outcome variable over time.
* It tells whether the changes are statistically significant.
Another advantage of the paired t test is that it can be analyzed with common software packages such as Microsoft Excel as well as the more sophisticated software.
Planning an Outcome Study in a Small Homeopathic Setup
After understanding the basics of implementing an outcome study, I’ll elaborate the methodology further through a mock example. In this example, I’ll be referring to a homeopathic clinical setup, but the article is valid for any branch or system of medicine. I’ll be adding some additional guidelines for outcome studies of therapies that are individualistic and holistic in nature.
Big homeopathic hospitals are still a rarity, so we will build this outcome study for a smaller clinical setup, where there are one to four physicians involved in treatment.
Deciding the Objective
When you are trying to build a clinical outcome study, make sure you chose a condition for which you have/get significant number of patients, because the power (n) can make a study statistically significant or insignificant.
Let us say, I want to build an outcome study to measure the effect of individualized homeopathic treatment in pediatric asthma. Once I have decided the broad area of my research, I’ll have to answer many questions during the planning stage:
- When will the study start?
- What will be the duration of the study?
- What will be the size of the study?
- How many people would be involved?
- How will the diagnosis be confirmed?
- What would be the inclusion and exclusion criteria?
- What instruments (questionnaires and tests) would be used?
- What would be the frequency of follow-ups?
- Which software would be used for data analysis?
- What are the financial costs involved?
All this information needs to be properly documented before the start of the study. If you are not trained in research, it is better to take the help of someone who is working in clinical research. Reading existing trials and talking to the physicians in the concerned department of a big hospital/research center can give you a lot of information for the proper and effective planning of an office-based outcome study.
I can choose to include in the study all pediatric cases of asthma coming to the OPD/clinic in the next 12 months with a follow-up period of 12 months. So effectively, this study will continue for 2 years. The data analysis part will start only after that.
Although outcome studies are not very costly, you still need to make an assessment of the costs involved, which can include salaries, office expenditure, cost of acquiring software and instruments, cost of pathological tests required for monitoring the cases etc. For e.g., if we wish to get a spirometry done every three months, then it has to be decided beforehand whether it will be done from your funds or by the patient and what would be the costs involved.
An outcome study, even in an office based setup, needs at least three people for the implementation – one physician involved in treatment, one clinical assistant for data recording and managing patient schedules, and one statistician (part-time) for data analysis. Depending upon the size of the study and the size of the practice (individual office/polyclinic/hospital OPD), there could be many more people involved.
In selecting an instrument, users must consider the different types of instrument that are available and how they meet the requirements of the proposed application[iv]. Selecting the right instrument is of vital importance. If you chose an instrument that is not accepted by large international organizations, your research will not be held in good esteem. For e.g. there are many accepted instruments/questionnaires for evaluating cases of pediatric asthma. Some of them can be found here:
More general information about instrument types can be found here:
Now depending upon what parameters I want to track, I’ll chose the instrument accordingly. For e.g. if I have to monitor the effect of my treatment, I should choose an instrument that monitors the data like frequency and intensity of episodes, frequency of symptoms between episodes, and intensity of impairment between episodes.
When the basic preparation is done and you start the study, the first thing you will need is an informed consent from the patient. It is easier to get consent in an outcome study because you are not ‘experimenting’ with the patient. You are just monitoring the patients more scientifically and rigorously for treatment outcomes.
Now we come to the part where data collection actually starts. And it also brings us to the part where we need to consider the special holistic and individualistic approach of homeopathy and many other alternative therapies.
Individualism, Holism and Research
Homeopaths are often taught to ‘treat the person as a whole’, ‘to look beyond the disease’, that ‘disease does not matter, it is the cause and underlying susceptibility that matters’. But such things do not gel well with research, because in clinical research you always need some common baseline against which you measure the efficacy of the treatment.
Now in an outcome study your treatment might be based on an individualistic and holistic approach but you will still need to follow some accepted research guidelines. There is no harm in using the disease as a parameter of the treatment’s success, because if your patient is improving as a whole, the disease too will subside.
So while I give individualized treatment to all my pediatric asthma cases, I’ll still use the same pathological criteria for diagnosing, evaluating and assessing the progress of such cases as is used in conventional medicine studies. Using those internationally accepted instruments gives scientific validity to your research results without interfering with the individualized approach.
When the patient comes, after initial screening (based on inclusion and exclusion criteria), your assistant can first get all the questionnaires filled and laboratory tests done to gather the baseline data. After that you can go for an individualized consultation and treatment.
A good thing about outcome research is that you can measure several outcomes simultaneously. While you use the disease related instruments to document changes in disease condition, you can also have another questionnaire that is more homeopathic and documents other aspects of the case like general wellbeing, sleep, dreams, diet, thirst, bowel habits etc. You should also record the use (and change in frequency and dose) of conventional drugs or other therapies, if they are used simultaneously.
In each follow-up, we will assess the progress of the case using the same instruments.
At the end of the study period, we will give all the data to the statistician to analyze it and report the trends observed.
Once the statistical summary is presented to you, you can write a research summary of your outcome study and submit it to peer-reviewed journals for publication.
There are some things that you need to remember while doing a study in your own practice:
- Be honest. Don’t screw your data just to prove that your treatment works. That is utterly against the spirit of research. And good analysts can easily spot the mistakes and misrepresentations in your research data. To improve the validity of your results, you can ask a colleague or expert in the area of study to act as an external observer. The observer can monitor the progress of cases with you as a neutral body. Video recording of the interviews and follow-ups can also help validate your data.
- Document and report not just the positive effects but also any negative occurrences and outcomes that you observe.
- Use the research outcomes to improve your own clinical practice. Outcome studies serve as valuable feedback about your effectiveness and can highlight areas where you need to do more work.
Lastly, research is meant for people who are committed, are diligent, respect scientific vigor, and have lot of perseverance. Don’t start a project, if any of these characteristics is missing. I hope this information will prove helpful in planning office-based outcome studies. Time has come when the effectiveness of homeopathic treatment is validated and quantified. Qualitative case reports that we usually take pride in, are considered the weakest form of scientific evidence. With lack of big hospitals and funding for large-scale trials, it is necessary that research data be gathered from individual practices. There are close to half a million homeopaths in this world. If we can produce 20 independent outcome studies on pediatric asthma (or any condition for that matter) from different parts of the world, their cumulative weight would be much more than any RCT and the results can then be compared with similar outcome studies done in conventional medicine.
Let us start documenting our cases more rigorously and more scientifically. That is the first step towards any meaningful clinical research!
[i] Riegelman, R. K. (2005). Studying a study and testing a test: How to read the medical evidence (5th ed.). Philadelphia: Lippincott Williams & Wilkins.
[ii] Ho PM, Peterson PN, Masoudi FA. Evaluating the evidence: is there a rigid hierarchy? Circulation. 2008;118(16):1675–1684. – See more at: http://www.o-wm.com/content/evidence-based-medicine-an-overview-key-concepts#sthash.uWxzfyLx.dpuf
[iii] Bomdardier C, Maetzel A. Pharmacoeconomic evaluation of new treatments: Efficacy versus effectiveness studies? Ann Rheum Dis. 1999;58 (suppl 1):182–5.
[iv] Fitzpatrick R, Davey C, Buxton MJ, and Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment 1998;2:14.