March 2014 Article of the Month
This month's article selection is by Chaplain John Ehman,
University of Pennsylvania Medical Center-Penn Presbyterian, Philadelphia PA.
Flannelly, K. J. and Jankowski, K. R. B. "Research methodology: research designs and making causal inferences from health care studies." Journal of Health Care Chaplaincy 20, no. 1 (2014): 25-38.
SUMMARY and COMMENT: This month’s article is an explanation of methodology that should be useful in helping CPE students to become more research literate. It highlights common approaches of spirituality & health studies and focuses on the strength of evidence for causal inference in experimental, quasi-experimental, and observational studies – an aspect of reading research that is, of course, critical but too-often underappreciated by chaplaincy students. For readers a bit more versed in research, the article still works well as a refresher about the hierarchy of quality of evidence.
The article first lines out observational designs: descriptive and correlational survey studies (with a comment about cross-sectional methodology), analytic studies using a case-control strategy and retrospective and prospective cohorts, and case studies. The latter are said to be "an ideal form of research for chaplains" [p. 30]. The authors hold up observational approaches as having the "potential to be done quickly, with uncomplicated designs, and minimal monetary investment," while yielding productive insights and being "exceptionally helpful in the development of new theory" [p.26]. Flannelly and Jankowski are both editors of the Journal of Health Care Chaplaincy and, throughout, they draw upon the publication history of the journal to illustrate the pertinence of these research approaches for chaplaincy.
Turning next to experimental research, the authors comment that it is "often considered to be the more complicated type of research even though this is not always true" [p. 30]. They note that "[e]xperimental research can provide the best support for cause and effect explanatory relationships" [p. 30] and its long use in medicine. Here the article shifts to particular criteria for drawing causal connections, from the work of John Stuart Mill:
First, the presumptive cause, or causal agent, must precede the effect in time. The cause must occur before the effect. Second, the effect must occur whenever the presumptive cause is present… The effect always happens if the cause has occurred. Third, the effect must not occur when the presumptive cause is absent…. Fourth, the presumptive cause must be isolated from other potential causes of the effect. Fifth, to ensure that the presumptive cause is isolated from all other potential causes, it must be produced artificially, which in this situation precludes observing the natural occurrence of the causal agent. [p. 30]
A study by Paul S. Bay, et al., "The effect of pastoral care services on anxiety, depression, hope, religious coping, and religious problem solving styles: a randomized controlled study" [featured as our March 2009 Article-of-the-Month], is used to illustrate an experimental design.
The idea of quasi-experimental design is given a short treatment [--see pp. 32-33] before the authors move to the core section on "making causal inferences from research" [--see pp. 33-36]. A basic hierarchy is presented, with experiments at the top, followed by quasi-experiments, and then prospective cohort studies, retrospective cohort studies, case-control studies, cross-sectional correlational and descriptive studies, and finally case studies. The strength or weakness of a method to determine temporal order for causal inference is considered briefly and straightforwardly in each instance (though the methods are presented in a somewhat mixed order for narrative purposes). The authors conclude:
Time is crucial to establish cause and effect, and research findings should always be evaluated with attention to the criteria for drawing causal inferences from experimentation and other types of research. Knowledge of research methods and reasoning will decrease the likelihood that researchers will fall into the error of making cause and effect statements that are not justified by the methodology employed in a study. [p. 36]
This article is not meant to be a comprehensive introduction to research methodologies but works well as a clarifying commentary on strengths of evidence for establishing causal relationships. It recognizes the value and usefulness of observational methods for chaplains while placing such methodology against the rigorous standards of experimentation and cautioning against hasty assumptions.
[Editor's Note: The table, Research Designs in Descending Order of Their Strength of Evidence [p. 26], unfortunately contains a formatting error, appearing to indicate that Observational Research is a subset of Quasi-experimental Research. Given its place at the very beginning of the article, the table may thus mislead readers as to the content which follows.]
Suggestions for the Use of the Article for Discussion in CPE:
The article should be useful with CPE students to paint a broad picture of research methodologies and raise the issue of causal claims in the literature. However, students should be warned about the misprinting of Table 1 [p. 26] in order to preclude confusion. For programs with monthly curricular components on research, the article might fit well about a third of the way through the curriculum but in advance of any plans for student projects. It could be paired with a report of research, such as the article by Bay, et al., [featured as our March 2009 Article-of-the-Month] used as an illustration in the text [p. 32]. General discussion could revolve around what it takes methodologically to justify causal inferences. Students may desire causal conclusions from articles but prefer to read observational studies. How feasible is that? Might students take this as a good reason to give more attention to reading experimental research? If they are planning a research project, might they see value in observational methodology but temper ambitions to draw from it causal claims? Students intrigued by an experimental approach could focus on Mill's criteria [pp. 30-32] and the task of applying such rigor to chaplaincy research. Discussion could also elicit students' thoughts about how one type of research could lay the groundwork for another, to build toward causal understandings -- for example: a case study leading to a survey study, leading to an experimental study -- so that the hierarchy that is set out in the article could be seen as a potential ladder for successive projects. This may help students conceptualize a very modest research venture as a first step to work that would extend beyond their CPE program.
Related Items of Interest:
I. For a slightly dated but still very useful methodological survey of the state of chaplaincy research, focusing on the qualitative approaches that dominate pastoral care research literature, see the following article (also featured as an Article-of-the-Month in January 2012). Note that two of the authors are Kevin J. Flannelly and Katherine R. B. Jankowski.
Galek, K., Flannelly, K. J., Jankowski, K. R. and Handzo, G. F. "A methodological analysis of chaplaincy research: 2000-2009." Journal of Health Care Chaplaincy 17, nos. 3-4 (2011): 126-145. [(Abstract:) The present article presents a comprehensive review and analysis of quantitative research conducted in the United States on chaplaincy and closely related topics published between 2000 and 2009. A combined search strategy identified 49 quantitative studies in 13 journals. The analysis focuses on the methodological sophistication of the studies, compared to earlier research on chaplaincy and pastoral care. Cross-sectional surveys of convenience samples still dominate the field, but sample sizes have increased somewhat over the past three decades. Reporting of the validity and reliability of measures continues to be low, although reporting of response rates has improved. Improvements in the use of inferential statistics and statistical controls were also observed, compared to previous research. The authors conclude that more experimental research is needed on chaplaincy, along with an increased use of hypothesis testing, regardless of the research designs that are used.]
II. For articles that generally address the importance of research in chaplaincy, see our August 2012 Article-of-the-Month page. One of the articles noted there focuses especially on case study methodology (and is cited also in our current article):
Fitchett, G. "Making our case(s)." Journal of Health Care Chaplaincy 17, nos. 1-2 (2011): 3-18. [(Abstract:) Health care chaplaincy needs to develop a body of published case studies. Chaplains need these case studies to provide a foundation for further research about the efficacy of chaplains’ spiritual care. Case studies can also play an important role in training new cha- plains and in continuing education for experienced chaplains, not to mention educating health care colleagues and the public about the work of health care chaplains. Guidelines for writing case studies are described, herein, as is a project in which three experienced oncology chaplains worked together to write case studies about their work. Steps that chaplains, and professional chaplain organizations, can take to further the writing and publishing of case studies are described.]
III. Flannelly, K. J. and Jankowski cite a number of books about research in their bibliography, and such long-form resources are worth seeking out. Three additional books that may be of interest: two that have for years been important in introducing CPE students to research (see Vandecreek) and one by a leading voice in medicine (see Koenig).
Koenig, H. G. Spirituality and Health Research: Methods, Measurement, Statistics, and Resources. West Conshohocken, PA: Templeton Press, 2011. [(From the publisher:) …[A] comprehensive overview of this complex subject. Dr. Koenig is one of the world’s leading authorities on the relationship between spirituality and health, and a leading researcher on the topic. As such, he is distinctively qualified to author such a book. This unique source of information on how to conduct research on religion, spirituality, and health includes practical information that goes well beyond what is typically taught in most undergraduate, graduate, or even post-doctoral level courses. This volume reviews what research has been done, discusses the strengths and limitations of that research, provides a research agenda for the future that describes the most important studies that need to be done to advance the field, and describes how to actually conduct that research (design, statistical analysis, and publication of results). It also covers practical matters such as how to write fundable grants to support the research, where to find sources of funding support for research in this area, and what can be done even if the researcher has little or no funding support. The information gathered together here, which has been reviewed for accuracy and comprehensiveness by research design and statistical experts, has been acquired during a span of over twenty-five years that Dr. Koenig spent conducting research, reviewing others’ research, reviewing research grants, and interacting with mainstream biomedical researchers both within and outside the field of spirituality and health. The material is presented in an easy to read and readily accessible form that will benefit researchers at almost any level of training and experience.]
VandeCreek, L., ed. Spiritual Needs and Pastoral Services: Readings in Research. Decatur, GA: Journal of Pastoral Care Publications, Inc., 1995. [(From the publisher:) What do spiritual needs, health care, pastoral services, religious resources, statistics, and quantitative research methods have in common? Traditional literature in the pastoral care, counseling, and education movement probably would say "Not a great deal." Indeed, the pastoral arts and sciences have tended to be high on art (especially praxis) but relatively low on science (especially Quantitative research). In this book, Dr. Larry VandeCreek has gathered a number of samples of previously published research reports which inform hospital chaplains, congregational pastors, hospital personnel, and theological students about ways in which knowledge gained from carefully designed research can enhance their project of care. The quantitative research reports in this volume explore topics such as the role of spirituality in terminally ill hospitalized patients, the demonstrated helpfulness of prayer in health crises, the impact of hope among women with breast cancer, the role and impact of chaplaincy services in general hospitals, and several other topics in which the relationships between spirituality and health care is explored.]
VandeCreek, L., Bender, H. and Jordan, M. R. Research in Pastoral Care and Counseling: Quantitative and Qualitative Approaches. Decatur, GA: Journal of Pastoral Care Publications, Inc., 1994. [(From the publisher:) The first part of this book, focusing on Quantitative Research, was originally published as A Research Primer for Pastoral Care and Counseling (by Larry VandedCreek). In the Foreword to this expanded text, Margot Hover writes as follows: "This is truly a book for all...seasons -- for those led apprehensive or even kicking and screaming into pastoral research as well as for those who enthusiastically indulge their curiosity in this way. It is for those who love to tinker with numbers as well as for those who look for patterns in the kind of documentation we excel at gathering -- verbatims, anecdotes, case histories. ...I grabbed Larry VandeCreek’s book the first time I heard the term 'pastoral research' as part of my job description, and my residents and I have relied on it ever since."]