Structured Abstract
Background:
Stakeholders in the health care enterprise are interested in the comparative effectiveness of treatment alternatives. Rehabilitation is a key component of the health care system that is expected to grow as the population ages and increasing numbers of patients survive with chronic illnesses and disabilities. At present, however, rehabilitation lacks sufficient rigorous research to guide clinicians and consumers toward effective treatments. The field has a plethora of measures of case mix and outcome, but the treatment process itself is a “black box” specified only crudely with metrics such as length of stay, sessions of physical therapy, and course of vocational rehabilitation. The rehabilitation treatments that must be compared in effectiveness research are not specified in sufficient detail to be measured and studied with respect to their differential impact on outcomes important to consumers. In 2008, we embarked on a NIDRR (National Institute on Disability and Rehabilitation Research)-funded 5-year effort to improve classification and measurement of rehabilitation interventions. This culminated in a conceptual framework for a rehabilitation treatment taxonomy (RTT), described in a January 2014 supplement to Archives of Physical Medicine and Rehabilitation. This work in progress has received enthusiastic support, commentary, and dissemination from many rehabilitation disciplines. We included representatives of these disciplines in periodic discussions and workshops to help shape the effort described in this report.
Objectives:
The objective of this project was to incorporate the conceptual framework of the RTT into standardized operational procedures so that clinicians, educators, and researchers across all rehabilitation disciplines could define and specify rehabilitation treatments according to their immediate effects, mechanisms of action, and hypothesized active ingredients. Thus, 1 tangible objective was the development, initial testing, and dissemination of a Manual for Rehabilitation Treatment Specification (see Methods). A further objective, using the results of that testing, was to continue developing the RTT toward the goal of achieving a common language and classification system for all rehabilitation interventions (now renamed the Rehabilitation Treatment Specification System—RTSS), thus allowing meaningful grouping of similar treatments and meaningful comparisons across distinct treatment approaches. The ultimate goals of the RTSS, once adopted, are to
enhance the ability of clinical educators and supervisors to ensure that treatments are delivered as intended;
improve the ability to replicate and disseminate effective treatment research;
improve the ability to synthesize treatment evidence across replications of the same or similar treatments;
support cross-cutting research on treatment principles that apply to larger categories of treatment;
enhance communication among members of different disciplines coordinating care; and
increase the value of administrative data sets in treatment research by increasing the information value of treatment documentation.
Methods:
Directed by the principal investigator and 2 senior co-investigators, a multidisciplinary core team and 5 work groups compiled a sample of 51 treatments that a multidisciplinary sample of rehabilitation clinicians was most interested in evaluating and comparing. We built vignettes, which contained brief clinical summaries, around these treatments and used them as the basis for treatment specification. The vignettes focused on a range of treatments delivered in both inpatient and outpatient settings to adults as well as children in the following disciplines: occupational therapy (13), physical therapy (13), speech-language pathology (12), nursing (7), and psychology (6). Core team members attempted specification of these treatment vignettes, identified ambiguities and gaps in the set of rules and procedures that were drafted to guide them, and recommended refinements to those procedures in an iterative manner. The core team discussed the challenges encountered in these attempted specifications in weekly meetings, and 1 team member was assigned to draft written resolutions or revisions to existing manual rules. After multiple cycles of attempted specification, rule revision, and respecification, the core team arrived at a full draft of the Manual for Rehabilitation Treatment Specification, which described the key concepts of treatment specification and provided a reproducible procedure for such specification, along with a set of 5 fully specified vignettes as teaching illustrations. We provided a group of 40 rehabilitation clinicians from mixed disciplines with web-based training in using the manual. We used feedback from trainees about the manual's clarity and utility to shape a final version. The project advisory board met by teleconference and in 2 face-to-face meetings to provide feedback on the evolving manual, suggestions for design of the RTSS clinician training cycle, and final input on next steps toward implementing the specification system.
Results:
Despite the previous period of NIDRR funding, a number of challenging conceptual obstacles emerged during the process. Developing and revising rules to guide the specification of volitional treatments—treatments that require effort and engagement on the part of the patient—were among the most challenging tasks. In addition, we developed procedures and rules for (1) determining the number of treatment components, (2) defining the target of treatment for each treatment component, (3) determining the treatment group to which each treatment component belongs, and (4) specifying the treatment component's active ingredients and their dosing parameters. Feedback obtained from the advisory board, from participants in the RTSS clinician training cycle, and from attendees at multiple professional presentations confirmed that the concepts contained in the manual were valuable and useful in supporting clinical education and supervision, clinical reasoning, and research reporting and synthesis. Advisory board members and training participants also agreed that repeated practice with feedback and discussion would be necessary to allow development of independent skills in treatment specification. Thus, the advisory board recommended continued efforts to implement these concepts into curricula for rehabilitation clinicians and researchers as well as a series of focused implementation projects that could document the positive impact of the RTSS and encourage its broader adoption.
Conclusions:
We developed and summarized the conceptual framework for the RTSS in the Manual for Rehabilitation Treatment Specification. Those who have engaged with the concepts have found them valuable for both clinical reasoning and research reporting, but the rules and procedures needed for independent treatment specification require practice and skill development. We will need to take additional steps to train relevant stakeholders to implement this system.
Note: In this report, technical terms are in bold when first used and are defined in the Glossary in the Appendix.
Background
Rehabilitation is in urgent need of a stronger evidence base to guide the provision of clinically effective treatments and services and to allow consumers of rehabilitation services to make informed choices.1-8 Building the evidence base in rehabilitation, as in other areas of health care, requires rigorous treatment research using randomized controlled trials, quasi-experimental studies, health services observational research, and other study designs. However, unlike health care fields that rely heavily on pharmacologic and surgical treatments, whose active ingredients are clearly defined, the treatments that must be studied in rehabilitation commonly are not themselves specified in sufficient detail to be operationalized, measured, and compared accurately.9-12
Treatment Specification for Comparative Effectiveness Research
Comparative effectiveness research (CER) and informed selection of rehabilitation treatments by consumers are limited until the treatments being compared or selected are operationally defined. CER of treatments that are defined at the institutional level (eg, inpatient rehabilitation vs subacute rehabilitation) or discipline level (eg, 16 hours of physical therapy [PT] vs 4 hours of PT) may produce null results because of the variable contents of those labels. Even when CER studies demonstrate differences in the impact of treatments defined in these ways, it is difficult to generalize those results to new instances of those institutions or disciplines, since the actual treatment ingredients delivered may vary. We argue that the most useful way of defining rehabilitation treatments, like pharmacologic and surgical treatments, is with reference to their active ingredients. Whereas one can productively ask about the relative effectiveness of aspirin and metoprolol (active ingredients) in preventing recurrent myocardial infarction, framing the question in terms of pills vs capsules (inactive ingredients) would not advance clinical research. This project addressed a critical gap in the specification and measurement of rehabilitation interventions, which must be resolved in order to fill the many specific gaps in the evidence base within this branch of health care.
Clear specification of rehabilitation treatments is not sufficient to guarantee useful patient-centered comparisons, but we would argue that it is a necessary step. Many treatment targets in rehabilitation are of immediate patient relevance (eg, the ability to be safely mobile, to dress and feed oneself, to be free of pain, to communicate with friends and loved ones). Comparing treatments intended to achieve those targets requires the potential treatments to be defined in an objective and replicable way. Doing so does not guarantee that well-conducted CER will be completed, or that evidence will be sufficient to make an informed choice. Failing to clearly define the treatments under study makes it impossible to advance CER. Some rehabilitation treatments address targets (eg, joint range of motion, gait biomechanics, working memory capacity) that are not of immediate patient relevance but that relate to treatment aims that are patient relevant (eg, ability to dress oneself, walking speed, ability to follow a conversation). In these instances, the clinician will need to assist the patient in weighing and comparing treatment alternatives by clarifying how they relate to meaningful patient outcomes. This is no different from many areas of medical practice in which the target of treatment (eg, lowering cholesterol) is not of direct patient relevance, but the treatment aim (avoiding myocardial infarction and stroke) is, and the clinician should clarify the link.
The Importance of Treatment Theory in Treatment Specification
A critical assumption of this work is that all rehabilitation treatments and services are best specified with reference to the treatment theory underlying each component of an intervention. Treatment theory has been defined as a “set of propositions that describe what goes on during the transformation of input into output, that is, the actual nature of the process that transforms received therapy into improved health.”13 As such, treatment theory hypothesizes that specific active ingredients are required for a given functional change. We have further defined 3 components of a treatment theory that together are necessary and sufficient to explain the treatment's effects.14 This tripartite structure consists of (1) a treatment target, which is a specific and measurable aspect of patient functioning targeted for change; (2) a set of active ingredients that, in some combination, are hypothesized to effect the desired changes in the target (note that inactive ingredients may also be present but do not change the target); and (3) the mechanism of action, which specifies the hypothesized processes by which ingredients bring about these changes, as illustrated in Figure 1. In this scheme, the ingredients and the target are both observable and measurable, while the mechanism of action is often invisible and must be inferred.
Figure 1
Tripartite Structure of a Treatment Theory for a Single Treatment Component.
Treatment theory, which explains the direct effects of treatment, may be distinguished from enablement theory, which refers to the more distal functional changes (treatment aims) that may occur because of a treatment-induced change in 1 or more treatment targets. For example, prescription eyeglasses might be provided to increase visual acuity (target). Improved vision may result in better driving, but that relationship is indirect (an aim), since driving performance may also be affected by attention, processing speed, aspects of motor control, etc. A central tenet of our work is that the specification and measurement of rehabilitation treatments should be organized around the concept of treatment theory. Enablement theory, in contrast, is useful for the selection of a treatment likely to have a desired distal impact on an individual patient, as shown in Figure 2. In 1 case, the treatment is directed toward a target of improved vision, and thus driving performance would be an aim. One might not prescribe eyeglasses to enhance the aim of better driving in a patient who also has extreme distractibility or upper extremity weakness. However, there may be adapted driver training approaches that would consider driving performance itself as a target, as illustrated by the arrow in the third row of Figure 2. Thus, driving is an aim for the first treatment but a target for the second.
Figure 2
Enablement Theory Related to Treatment Targets and Aims.
Figure 2 provides a schematic illustration of enablement theory. With respect to the International Classification of Functioning, Disability, and Health (ICF) framework, multiple factors at the impairment (ie, body structure/function) level may determine function at the activity level, and multiple factors at the activity level may affect factors at the participation level. Two potential treatments are illustrated with targets at 2 different levels. Even a highly effective treatment at 1 level cannot guarantee successful accomplishment of an aim at a distal level because of the additional causal factors relevant to that aim.
In rehabilitation, ingredients are clinician actions and behaviors, verbalizations, substances (eg, drugs), procedures, and other entities selected for their ability to affect a target via a prespecified mechanism of action. By specifying the active ingredients of treatment, a treatment theory also specifies which ingredient variations constitute meaningful variations in treatment dose. Ingredients vary qualitatively (eg, a clinician may provide explicit feedback or not, or terminal vs concurrent feedback) and sometimes quantitatively as dosing parameters (eg, feedback may be delivered on a specific schedule of frequency/intensity).
We also argue that treatment targets need to be better specified to measure rehabilitation with the precision needed to evaluate its effects empirically. In clinical parlance, treatment targets may be framed as, for example, “improved ambulation,” but for some patients it may be important to ambulate faster; for others, it may be to use a more symmetrical gait pattern, to incorporate an assistive device correctly, or to be able to ambulate for a longer distance. The clinician is likely to select different ingredients for these different specific targets. Our use of the word target connotes a more exact conceptualization of the immediate outcome of treatment, one that allows for precise measurement and precise selection of the types of ingredients (aids, instructions, cues, error feedback, reinforcement, environmental setup, etc) that will bring about the desired result.
Current Approaches to Rehabilitation Treatment Specification
Thought leaders in this field have long noted the “black box” problem of rehabilitation,15-19 meaning that its ingredients have not been sufficiently specified and quantified nor their effects compared in ways that will assist in treatment selection. Attempts to compare services with reference to the level of facilities (eg, inpatient rehabilitation unit vs skilled nursing unit), disciplines or staff training levels (eg, services from physical vs occupational therapy, or a physical therapist vs a physical therapy aide), or time (eg, length of stay, hours of treatment) fail to unpack the black box because they provide no information about the actual content and process of the treatment experienced by patients. The common clinical practice of defining treatment by the problem it is meant to address (eg, gait training, memory remediation, vocational rehabilitation) similarly provides no information about the methods or processes used to address the problem. Thus, the current state of affairs affecting rehabilitation research, program evaluation, and ultimately practice is analogous to asking whether, for example, cancer treatment is effective or ineffective without considering the differences among the many available formulas and intervention modalities that are targeted to specific forms of the disease.
Rehabilitation is not alone in this dilemma. The difficulties in specifying so-called complex interventions, many of which depend on active intrapersonal and interpersonal processes such as volition, learning, and the therapeutic relationship, have been recognized in other fields, including psychotherapy, behavioral medicine, public health, and nursing.20-29 Accordingly, the UK Medical Research Council's guidance for developing and evaluating complex interventions calls for improved methods of specifying and reporting intervention content to address the problems of lack of consistency and consensus.30 In influential early work, Donabedian called attention to the importance of structure and process factors in determining health care outcomes.31 In his scheme, structure includes physical facilities, health care technology, and financing. Process refers to such factors as the nature of assessment procedures, frequency of team meetings, and adherence to care pathways. Although measured structure and process factors have been shown to be associated with rehabilitation outcomes in a number of studies, they are rather crude predictors of outcomes overall.32 This is likely because such processes as team meetings can bring together wider expertise and increase the probability that a good treatment plan will emerge, but the team meeting itself doesn't enhance the patient's functioning, and not every team meeting results in the selection of an optimal treatment.32 In addition, even the best treatment plans need to be specified in enough detail for team members to carry them out accurately and consistently. Thus, there remains a need to better define and operationalize rehabilitation treatments to determine which are optimal for specific patients and specific functional problems amenable to rehabilitation services.
An example from the research team's experience may help highlight this need as well as the gaps waiting to be addressed. In 2011, the Department of Defense commissioned the Institute of Medicine (IOM) to analyze the evidence base supporting cognitive rehabilitation for traumatic brain injury (TBI).33 As the panel began reviewing the available evidence, the issue of treatment definition and specification became an immediate obstacle. For example, memory remediation consisted of repetitive drills, training in the use of internal mnemonic strategies, training in the use of external memory aids such as notebooks and cellphones, and other methods. In assessing the evidence base for memory remediation, should all of these disparate treatments be meta-analyzed together? Is the target of treatment really the same for all of them? Treatments such as drills appear to be intended to build memory capacity. But a memory notebook has no direct effect on memory capacity. Instead, it is designed to facilitate the successful completion of a set of everyday activities that are limited by memory deficits, such as keeping appointments and completing the tasks on to-do lists; a memory notebook would not be predicted to improve facial recognition or memory for routes traveled. Thus, the effectiveness of a treatment involving a memory notebook should be measured against completion of the intended activities rather than, for example, standardized memory testing or evaluation of facial recognition.
This example represents 1 instance in which lack of a meaningful method for defining and specifying treatment posed a major obstacle to useful synthesis of rehabilitation treatment research, thus precluding the development of reliable guidance for clinicians regarding which interventions to offer their patients. But a problem exists not only in the synthesis of research; if in the primary research the investigator fails to accurately specify the treatment target, he or she may miss important treatment effects. As in the example of the IOM panel, a memory notebook might be judged ineffective by assessing its impact with respect to patients' abilities to remember people's faces. If the wrong ingredients are chosen to represent the rehabilitation treatment, the same problem may occur (eg, if a smartphone were selected on the basis of size and shape rather than the usability and relevance of its internal features). Since efficacy or inefficacy of a rehabilitation treatment depends on the actual ingredients received by the patient, the field of rehabilitation needs an approach to treatment definition based on the nature and quantity of ingredients used for affecting specified functional targets.
Project Objectives
The overall objective of this project was to advance the conceptual framework for rehabilitation treatment specification, developed with previous funding, and to transform it into a procedural manual (the Manual for Rehabilitation Treatment Specification) that could guide clinicians and researchers in performing treatment specification in a standardized fashion—that is, to develop a Rehabilitation Treatment Specification System (RTSS). We addressed this overall objective with the following specific aims:
Aim 1
Develop a reliable and effective Manual for Rehabilitation Treatment Specification and associated training materials describing a standardized set of steps that will guide researchers, clinical educators, and clinicians in applying the tripartite structure of treatment theory to
determine whether to decompose a treatment into mechanistically distinct treatment components;
specify the treatment target for a particular treatment component, and distinguish it from additional (“downstream”) clinical aims; and
specify in observable/measurable form the ingredients required to initiate the known or hypothesized mechanism of action of the treatment component.
Aim 2
Provide training based on the Manual for Rehabilitation Treatment Specification to a multidisciplinary cohort of rehabilitation professionals, use the evaluation of training outcomes to assess the reliability and utility of the method for treatment specification, and subsequently make final revisions to the manual and associated training materials.
Aim 3
Use 50 well-defined treatment examples produced in the first phase of the project (aim 1) to make recommendations for further development of a rehabilitation treatment taxonomy (RTT).
We intended for these aims to result in the full development of a specification system, based on known or hypothesized active ingredients, which is supported by clear and rule-based procedures. We anticipate a number of positive impacts of this kind of treatment specification system, once broadly adopted by clinicians, clinical educators, and researchers. From the perspective of research, such a system ensures that the elements of treatment thought to drive its efficacy are clearly described and that attempts to replicate research ensure delivery of those same ingredients. This system also ensures that evidence reviews in their synthesis phase sample studies of the same treatment (in a mechanistic sense) for analysis so that their conclusions help inform the treatment theory that those treatments share. Finally, once strong evidence exists for the effectiveness of a given form of treatment, an ingredients-based specification system ensures that clinical practitioners can deliver well-defined active ingredients.
With respect to clinical education and practice, such a system allows educators to highlight the essential aspects of delivering the effective treatment ingredients, allows clinical supervisors to evaluate appropriate adherence to treatment delivery plans and standards, and facilitates clinical reasoning about what ingredients are able to modify what aspects of functioning. By employing discipline-neutral nomenclature, the system also fosters interdisciplinary communication about shared treatment theories and active ingredients. These potential benefits are illustrated by a complete specification of a simple treatment in Table 4. However, this detailed illustration will be easier to appreciate after review of the key RTSS concepts.
Table 4
Example of How a Treatment Session Could Be Specified Using the RTSS.
Ideally, an ingredients-based specification system also should support clinical documentation and billing. Doing so would enhance the value of administrative data sets as sources of evidence about treatment effectiveness, since treatments would be documented in categories pertinent to their effectiveness. However, a very simple and quick system would be needed for this purpose, and the potential tension between the accuracy needed to guide research and the simplicity needed for documentation has not been explored in detail.
Stakeholder Engagement
Unlike many PCORI-supported projects, the primary stakeholders for this project are rehabilitation clinicians, educators, researchers, and editors/publishers rather than patients or consumers of rehabilitation services. This reflects the technical nature of the “black box problem” in that better treatment specification requires reflection on treatment theories and hypothesized mechanisms of action with which consumers of rehabilitation services are unlikely to be familiar. Achieving the project aims will result in a set of rehabilitation treatments that are operationally defined with respect to their hypothesized active ingredients. This, in turn, is expected to improve the training and supervision of clinicians to reliably deliver those active ingredients and to enhance the ability of researchers to aggregate evidence about the efficacy and effectiveness of the treatments so defined. Only then will it be possible for patients and consumers to weigh the pros and cons of different well-defined treatment alternatives.
With this in mind, we designed 4 primary methods for engaging relevant stakeholders: creation of an advisory board; (2) conduct of multiple focus groups; (3) training of a multidisciplinary group of clinicians; and (4) presentation and discussion of the work in progress at professional meetings. These methods complemented the experience of the multidisciplinary team of rehabilitation clinicians and researchers that conducted the project. Below, we describe the impacts of the advisory board and of our presentations at professional meetings. We describe the impacts of the clinician focus groups and the clinician training cycle under the Methods and Results sections, as these were data collection methods described in the original grant methodology.
Advisory Board
We created an advisory board of 18 senior clinicians, rehabilitation educators, professional association leaders, researchers, and journal publishers, as well as 2 rehabilitation consumers who are also involved as rehabilitation professionals or advocates (see Table 1). We selected these individuals to represent multiple disciplines and practice settings, to provide insight regarding the likely users of the system we developed, to provide feedback on ideas and drafts, and to assist in dissemination and implementation of project findings. Early in the project, we held meetings of the advisory board by teleconference to discuss the results of our focus groups and to receive guidance on how to address the challenges of specifying volitional treatments (treatments that depend on the patient's effortful behavioral performance for their efficacy). In January 2017, we held a 2-day in-person meeting of the advisory board to review an initial draft of the Manual for Rehabilitation Treatment Specification, to attempt manual-guided treatment specifications, and to plan for the cycle of RTSS clinician training. Late in the third year, we conducted our final in-person advisory board meeting to summarize the feedback received from the cycle of RTSS clinician training and our efforts at professional education and to discuss next steps for implementing the developments of the project.
Table 1
Advisory Board Members and Their Affiliations.
Professional Society Presentations
Because the primary users of the RTSS are rehabilitation clinicians, educators, and researchers, presentations of this work at professional society meetings are an opportunity not only for dissemination but also for meaningful engagement of project stakeholders. Accordingly, members of our team have presented the work at 20 professional meetings in the United States, Canada, multiple European countries, and Australia to audiences of physical therapists, occupational therapists, speech-language pathologists, and psychologists (see Table A in the Appendix). The format of these sessions varied, depending on conference schedule and space constraints. However, in many of the presentations, we built in interactive experiences with attendees that allowed us to obtain feedback about the perceived value of the RTSS, concerns and criticisms, and opportunities for implementation. In some instances, this simply involved reserving substantial discussion time to invite reactions to the conceptual material provided didactically. In other cases, we conducted participatory exercises intended to elicit feedback on specific points. For example, at a meeting of the American Congress of Rehabilitation Medicine, we ran a session on organ function treatments at which we invited participants to nominate standardized targets that they treat in particular functional areas. This allowed us to explore the feasibility of developing standard target menus in the future and to promote reliability in specification and standardized communication.
Stakeholders had significant impact on the project at multiple points. Much of this impact will be evident from the discussion of methods and results that follows, but we will briefly summarize 2 specific changes that resulted from interaction with our advisory board: our handling of volitional treatments and our design of the RTSS clinician training cycle. The project team found specification of volitional treatments to be extremely challenging. We attempted several different sets of procedural rules to balance the precision of specifying volitional treatments with the clerical burden of doing so. With each change, we presented the new set of procedures to our advisory board (we presented one of these iterations to the RTSS clinician trainees for feedback) and asked them to attempt treatment specifications using those procedures. Their feedback allowed us to settle on a workable compromise for volitional treatment specification. Second, our advisory board heavily influenced the design of the clinician training cycle. The board felt that it was important to make archived presentations available since many clinicians would not be able to commit to 6 meetings at a fixed time point. The board also recommended offering the small-group discussions in the evening (so as not to interfere with people's daytime responsibilities) on different nights (we offered “sections” of the same material on 4 days of the week) and recommended treating these small-group discussions as sources of qualitative feedback rather than applying quantitative measures of learning. Finally, advisory board members helped publicize and recruit participants for the RTSS clinician training cycle.
Methods
Overview of Original and Revised Methods
This project did not involve collection of original empirical data about treatment ingredients or the way they have been specified in the literature, but rather involved an intertwined process of clarifying and refining underlying concepts, developing specification rules and procedures based on those concepts, applying the concepts to various specification tasks, and revising concepts and/or procedures in response to challenges in their application. In contrast to the implementation of a more traditional methodology, almost from the very beginning these methods resulted in unanticipated findings and challenges that necessitated changes in subsequent procedures. Consequently, the originally proposed methods for this project underwent significant revision during the performance period (each reviewed and approved by the principal investigator). In the end, data collection took the form of intermittent solicitation of qualitative data from advisory board members, participants in the RTSS clinician training cycle, and attendees at professional society presentations. Here we briefly discuss the original methodology of the proposed project and the methods actually used to accomplish its aims.
Proposed Project Methodology
We proposed that, during the first 6 months, we would convene work groups of rehabilitation professionals to (1) generate a heterogeneous list of commonly used rehabilitation treatments administered within a single treatment episode, and (2) generate assessment items with which to construct a scale to measure the quality of treatment specification. This phase resulted in a list of 50 commonly used rehabilitation treatments selected to represent multiple disciplines and settings. We proposed that the core project team would work to specify these treatments in batches of 10 at a time, with specification meaning identification of the number of treatment components, the target of the treatment, the ingredients, and the known or hypothesized mechanism of action by which the ingredients exerted their effects. These 50 treatment specifications would allow the creation of menus of targets and ingredients to be expanded as new treatments were specified. We successively tested the interrater agreement among team members to confirm the reliability of specification methods as they were developed.
At the conclusion of this phase, we proposed to complete version 1 of the Manual for Rehabilitation Treatment Specification, the written document that operationalizes the process of treatment specification. We also developed a training curriculum to support the use of the manual. Both the manual and the curriculum, as well as the quality rating scale developed in the first phase, would receive stakeholder input and revision in an in-person workshop involving our advisory board. We proposed that, after suitable revision, we would embark on a program of training for clinicians, clinical educators, and clinical researchers from all disciplines involved in rehabilitation, delivered in a series of biweekly web-based sessions. We would deliver this training series 3 times over a 10-month period, with about 40 participants in each cycle (ie, approximately 120 individuals trained altogether), affording the opportunity to edit and clarify the material in response to trainee feedback between cycles. We would assess the effectiveness of the training by masked ratings of the quality of attendees' treatment specifications before and after participation in the series, and participants would be asked for suggestions on improving the clarity and quality of the training and the specification materials, to inform version 2 of the manual.
In the final phase of the project, we proposed to reconvene our advisory board for a final meeting to address the future of the specification system (eg, the nature of the further work needed, who will undertake it [using what means], and how this specification system should interface with related efforts undertaken by other professional organizations).
Actual Project Methodology
The actual methods we used to achieve the aims of the project followed the proposed methods in terms of their intended outcomes, but in certain phases we elected to forgo the proposed quantitative methods (eg, statistical testing of pretraining and posttraining ratings of specification quality) in lieu of qualitative data that we judged to be more appropriate to the stage of development of the treatment specification system.
Summary of Methods by Project Phase
Phase 1 (quarters 1-2)
Phase 1 was identifying treatments and measures of the quality of treatment specification. We shifted the planned in-person work groups with rehabilitation clinicians to teleconference format due to logistic constraints. We conducted an in-person, multidisciplinary pilot focus group at MossRehab, which helped shape the questions for the subsequent teleconferences. Teleconference focus groups were organized by a core team or advisory board member of that discipline. We conducted separate focus groups with physical therapists, occupational therapists, speech-language pathologists, rehabilitation nurses, and rehabilitation psychologists to solicit nominations of treatments that the core team could begin to specify.
We worked to ensure variability within each focus group in terms of populations treated (adult/child, neurologic, musculoskeletal, etc) and practice setting (hospital, outpatient). The discussion format encouraged participants to nominate treatments that multiple patient groups commonly use. The core team collectively reviewed notes from these focus groups as well as potential treatment nominations. In addition, we received structured feedback about the draft Likert-type ordinal rating scale for measuring specification quality. The proposed ratings focused on whether the reader of a specified treatment would be able to implement that treatment as described.
In these focus groups, we discovered that it was rare for clinicians to nominate specific standardized or protocol-based treatments and much more common for them to discuss approaches or treatment activities in general. Setting aside how well named or operationalized these examples were, it was difficult to extract 50 identifiable treatments. Rather, clinicians tended to talk in terms of the problems or goals they were likely to address and some of the ingredients or approaches they used to address them. Interestingly, in speech-language pathology, in which a number of named and manualized treatments do exist (eg, semantic feature analysis34; VNeST35 [Verb Network Strengthening Treatment]), focus group members did not nominate any of these. The facilitator probed whether clinicians were familiar with and used these treatments. Responses took the form “Yes, I use some of that” or “Yes, I sometimes apply those principles,” suggesting that many rehabilitation clinicians think of treatment planning less as the selection of standardized treatment elements and more as an improvisational cooking task in which the chef has significant latitude to select and add ingredients “to taste.”
Because of this, the core team abandoned the plan to arrive at 50 discrete and well-specified treatments in common use. Instead, we elected to insert a step of developing brief clinical vignettes written around a specific patient problem, rather than a well-characterized treatment, with the idea of specifying the likely treatment options that might be used for such a patient. Although this concretized the process of attempted treatment specification, it introduced a variety of irrelevant issues related to the unknown characteristics of the hypothetical patients in the vignettes. Thus, in our later attempts to use these vignettes for specification, it was sometimes difficult to determine whether the challenge was in the specification process itself or in not knowing, from the vignette, exactly which patient characteristics affected ingredient selection. In real-world treatment specification, the researcher or treating clinician would have access to the full set of facts and observations about a patient and his or her own opinions about the treatment being pursued.
During these focus groups, we received general endorsement of the proposed approach to rating specification quality as well as several helpful edits and clarifications for specific items, which allowed us to revise the scale for use later in the project.
Phase 2 (quarters 3-5) and phase 3 (quarters 6-7)
Phase 2 was the following: defining and specifying a set of treatments and operationalizing the process of treatment definition and specification. Phase 3 was the following: incorporating the rules and procedures for defining and specifying rehabilitation treatment components into a manual and associated training materials that will equip researchers, clinicians, and clinical educators to apply treatment definitions and specifications. For several reasons, we abandoned the plan for quantitative evaluation of interrater agreement in these phases. First, we discovered that it was difficult to apply inadequate specification rules to a full treatment specification. Rather, as we attempted specifications based on the early rules in place at that moment, specifiers were unable to complete the specifications with the rules available. Thus, it was uncommon for 2 team members to complete specifications that disagreed and more common that they both recognized obstacles to completing the specifications in the first place. Accordingly, the most productive way to identify issues that required new specification rules or rule clarification was through discussion of attempted specifications and the disagreements and challenges that arose.
We moved to a plan of assigning pairs of core team members to specify a treatment vignette and then to discuss with the whole group where they disagreed and what challenges they faced. This often led to modifying the specification rules to accommodate the most recent challenge and frequently to revisiting previously specified vignettes that required revision. In this way, the specifications of the vignettes and the development of the specification procedures and rules proceeded in parallel, but complete specification of treatments was difficult to achieve until near the end of the manual's development. In lieu of formal testing of interrater agreement, core team members typically achieved a consensus on the optimal rule and specification through repeated discussion. When the core team faced particular challenges in agreeing on an optimal solution, they selected these challenges for targeted discussion and resolution with the advisory board.
Phase 4 (quarters 8-11)
Phase 4 was evaluation and revision of the Manual for Rehabilitation Treatment Specification and associated training materials. We revised the proposed plan for 3 sequential rounds of clinician training to a single cycle with 40 participants, largely due to time constraints, and we shifted the emphasis of the training cycle to prioritize qualitative feedback on the manual's utility and clarity rather than empirical evidence of its impact on skill development. We made the latter change after realizing that the original plan to obtain empirical evidence of skill development was unrealistic. We learned during our first in-person advisory board meeting that skillfully applying the rules and procedures we had developed was challenging and required a degree of practice and feedback that might not be feasible during a short web-based training cycle. Thus, whether a measurable improvement in specification skill was a reasonable goal for such a brief training interaction was unclear. In addition, during discussions with many clinicians, it became clear that our conceptual framework changed perceptions of what implement the treatment as directed means. In other words, if clinicians, prior to our training, believed that performing the same activity today that was performed yesterday constituted treatment as directed, then they would rate a specification that described the nature of the activity as “good.” After training, they might come to see that they didn't really know what treatment target was being pursued with that activity, what level of challenge for the patient it should have had, what form of cuing or correction they should have used, etc. As a result, training might affect not only the quality of treatment specification but also the interpretation of the rating scale. Finally, it was difficult to design a quality rating process that was blind to whether the specification was completed pretraining or posttraining, since specific technical words and concepts would be likely to appear only in the posttraining specifications.
Because the goal of the training became qualitative rather than quantitative feedback, we reduced the training to a single cycle with 40 experienced rehabilitation clinicians and 4 weekly small-group sections. With help from our advisory board, we solicited clinicians from around the country who came from all rehabilitation disciplines and had experience in varied patient populations and treatment settings. Initial dissemination of information about the training cycle resulted in contacts from 105 individuals. We eliminated 6 due to their limited experience (4 had <2.5 years) or the fact that they were therapy assistants (2). We sent surveys of schedule availability to the remaining 99 clinicians and eliminated 28 for failing to return the survey or because of their limited availability as indicated by their survey responses, leaving 71 individuals. We invited 48 of these individuals to participate based on a match between their availability and when we could schedule the sessions, but 9 of these ultimately withdrew, leaving 39 who participated in the full training cycle. They received three 1-hour web-based lectures on the conceptual framework. The lectures were given in real time but archived for later viewing for those who could not attend the scheduled sessions. The individuals also received 3 homework assignments involving target and ingredient specification from vignettes provided. The vignettes and documentation sheets were distributed after each lecture and returned by email prior to the sessions when they would be discussed. Members of the core team reviewed the homework for specification errors and composed a discussion outline for the small-group sessions. We divided the 3 small-group discussions of their attempted specifications and of the manual into 4 evening sections held from Monday to Thursday to allow for schedule flexibility. Members of the executive committee facilitated the homework discussions via teleconference, supported by web-based slides summarizing the key points. Core team members served as scribes for the discussions and consolidated the notes from the 4 sections into a single summary of each assignment. (The 3 homework assignments are provided in the Appendix, along with an example of the summary slides created to discuss each of the assignments.)
The characteristics of the RTSS clinician training participants are shown in Table 2.
Table 2
Characteristics of Participants in the RTSS Training Cycle.
Phase 5 (quarter 12)
Phase 5 was informing further development of an RTT. We obtained extensive feedback on the Manual for Rehabilitation Treatment Specification and the concepts it contains from the RTSS clinician trainees through multiple methods. As noted, 2 scribes attended small-group sessions to record qualitative feedback, and their feedback was reconciled into a single summary that the full core team discussed to determine which changes in the specification rules and/or manual were warranted. The core team identified and reviewed participants' specification homework errors (defined by instances when their homework specifications deviated from consensus specifications prepared by the core team in advance), as these errors informed topics requiring further explication in the manual. We asked training participants to complete a posttraining survey that solicited structured feedback about the clarity and utility of various sections of the manual. Finally, trainees were invited to send specific proposed edits to the manual to the team at the conclusion of the training. We obtained feedback on iterations of the manual and many other issues via telephone and in-person meetings of the project advisory board. In addition, project team members presented on the evolving RTSS at multiple professional meetings, as mentioned previously, often including participatory activities that elicited feedback from potential stakeholders at these meetings. We synthesized all these sources of feedback and brought them for discussion to the final advisory board meeting in November 2017, along with a final revision of the manual that resulted from the feedback received up to that point.
This last phase of the project evolved largely as anticipated. The advisory board members participated in a final round of specification based on the updated procedures defined in the last version of the manual in order to provide feedback on the status of the concepts and the clarity of the procedures at the project's close. Importantly, the agenda of the final advisory board meeting was primarily devoted to feedback on 2 further steps for advancing this line of work: production of a set of manuscripts summarizing the key developments of the project during this funding cycle (4 manuscripts and an editorial are now in press) and discussion of next steps for advancing and implementing the products and concepts that we developed.
Results
Conceptual Developments
Enduring Concepts from the Prior Project Period
Many important concepts that developed during a previous round of funding by the National Institute on Disability and Rehabilitation Research (now the National Institute on Disability, Independent Living, and Rehabilitation Research) remained intact. However, this project emphasized development of a Rehabilitation Treatment Specification System (RTSS), rather than a rehabilitation treatment taxonomy (RTT). A taxonomy is any rule-based classification system. The more important issue for our purposes is what kind of taxonomy is likely to be most useful in advancing treatment research and practice, and we argue that it is a taxonomy built around treatment theory-defined active ingredients of treatment. At the same time, we recognized that developing a method for specifying treatments with respect to their active ingredients supports the development of such a taxonomy but does not achieve it. We still required a subsequent step to decide how to organize well-specified treatments into categories or dimensions. Thus, for the near term, we defined our work as focused on the specification process itself.
We remained committed to a specification system based on known or hypothesized active ingredients of treatment and relying on treatment theory. We based our commitment on the premise that only this kind of system can lend order to the attempt to organize and synthesize evidence regarding treatment efficacy and effectiveness. We also knew from our work on the previous project that no other specification system incorporated this perspective. The International Classification of Health Interventions (ICHI), the closest in purpose to the RTSS, defines treatments in gross terms, primarily with respect to the goal of treatment rather than the ingredients used to achieve that goal. Within the framework of treatment specification, we retained the core concept that the tripartite structure of treatment theory should underlie the RTSS, with enablement theory informing the selection of treatments that will have the desired distal effects in particular patients. We also retained and further developed the important concept of a treatment component, which is the largest unit of treatment with a single target. Thus, for example, a treatment for depression might consist of 4 distinct components focused on different targets and employing different ingredients: (1) increased patient understanding of the nature of depression; (2) more accurate patient knowledge of the name and properties of a prescribed antidepressant; (3) increased patient skill to reliably take the antidepressant medication; and (4) improved patient mood (from the antidepressant chemical prescribed).
Other concepts from the previously funded research were retained if they continued to prove workable as tools for unambiguous treatment specification, but we modified them, if necessary, in response to specification obstacles.
Treatment Groups
We retained the notion of distinct treatment groups, composed of mutually exclusive sets of targets and characteristic ingredients. However, we reduced our original treatment groups from 4 to 3 and renamed them in response to feedback from our stakeholders. We previously had a treatment group focused on changing organ structures and one focused on changing organ functions (parallel to the ICF's categories of “functional levels”).36 However, with the exception of a few cosmetic treatments (such as providing a nonfunctional prosthetic hand or a facial prosthesis), rehabilitation treatments rarely pursue changes in organ structure except to achieve changes in organ function. Accordingly, we eliminated the structure group. This resulted in the 3 treatment groups shown in Table 3, along with typical examples.
Table 3
Key Characteristics of the 3 Treatment Groups.
Another important conceptual development was the addition of habits to the skills and habits group (formerly called skilled performances). We recognized that the development or adoption of new behavioral routines enacted on an automatic or quasi-automatic basis—ie, new habits—is an important but sometimes neglected aspect of rehabilitation and one in which the desired behaviors respond to many of the same ingredients as skill development. A focus on the concept of habit development within this treatment group will encourage clinicians and researchers to attend to not only the ingredients responsible for improving skills in the short term but also the internal and external cues (“triggers”) that will maximize the long-term deployment of these skills habitually in the patient's natural environment.
Specifying Volitional Treatments
The core group spent a great deal of its energy on the challenging task of developing rules for specifying volitional treatments—that is, treatments that depend on the patient or consumer enacting certain effortful behaviors such as exercising, practicing tasks, or processing information. In contrast to passively applied treatments, which allow the clinician to specify in advance precisely which ingredients will be delivered to the patient, volitional treatments require the patient to self-administer some of the ingredients if the treatment is to be effective. For example, a clinician may design a set of exercises for a patient to perform, but the patient must repetitively lift the assigned weights in order to build strength. This introduces several challenges:
The clinician does not have full control over the ultimate delivery of effective ingredients in volitional treatments, yet formally acknowledging the patient's control over the receipt of these ingredients threatens to undermine the specification process altogether (only the patient could “specify” some of the ingredients).
In some volitional treatments, the only ingredients actually delivered by the clinician are instructional or motivational (eg, directions for a home exercise program), which have no mechanism of action on functional entities such as muscle strength. To adhere tightly to the tripartite structure of treatment theory, such treatments could have only targets of performing the assigned activity, which would completely leave out the important clinical question of whether the activity selected by the clinician was, itself, likely to be effective if performed.
One can identify pairs of treatment targets for volitional treatments: the volition target, which refers to whether the patient performed the treatment activity as instructed (eg, performed the exercises), and the direct target, which refers to whether the performance of that activity resulted in the functional change that was predicted (eg, increased strength). However, in many instances, it is difficult to determine whether the clinician's ingredients influence the volition target or the direct target. For example, when a clinician educates a patient about the purposes of his or her medications, which of the clinician's behaviors are ensuring that the patient is attending to and processing the information (the volition target), and which are ensuring that accurate medication information is conveyed (the direct target)?
We ultimately concluded that there was no perfect resolution to the specification of volitional treatments—that it was ultimately a balance between theoretical purity and practical utility. The core team drafted several versions of the procedures for specifying volitional treatments and discussed the pros and cons of the resulting specifications. We also solicited feedback on 2 of these iterations from the advisory board and on 1 iteration from the participants in the RTSS clinician training cycle. Finally, we arrived at a compromise in which supervised treatments specifically included volition ingredients but not a separate volition target, whereas unsupervised treatments required separately articulated volition and direct targets.
Specification of Treatment Targets
The core team also discussed rule-based specification of treatment targets. In particular, we explored whether finite menus of treatment targets could be developed to facilitate treatment specification with less clerical burden and with greater agreement. We began by exploring whether the body function codes of the ICF could serve as predefined treatment targets for our organ function treatment group and for the function-like entities in our skills and habits group (eg, improved balance, improved attention). Discussion of the existing ICF codes revealed several challenges in applying them:
To the knowledge of core team members, many ICF-defined body functions do not have corresponding treatments, so at best a subset of these codes could be used as treatment targets.
These codes do not address the measurable aspect of change, as required of RTSS targets. Thus, an ICF code of muscle power would have to be supplemented with a measurable dimension, such as increased.
In some cases, ICF entities do not map onto current scientific understanding of functional areas. For example, the way the ICF divides the cognitive domain of attention does not correspond to current concepts of neural network control of attention, such that it is unlikely that biologically informed treatments could target aspects of attention as defined by the ICF.
In many treatment areas, the precise entity being treated is difficult to determine without detailed discussions with treatment experts. For example, are certain voice exercises focused on the strength of the relevant muscles or the coordination of those muscles? Thus, we felt that treatment experts should develop predetermined menus of organ function targets in various areas, and that they should build on ICF categories if possible but not be confined by them.
In the case of activity-like skills and habits targets and representations targets, we recognized that the number of potential targets is infinite and, therefore, a comprehensive menu is unlikely. However, the targets in these groups can change in only a finite number of ways (eg, accuracy, speed). Thus, we will need to individually specify the targets in these areas, though we can draw the measurable aspect of change from a finite menu. We distilled the guidance we developed for specifying treatment targets into the manual both as a set of procedures and rules and as a set of example specifications from the original vignettes, which mapped descriptions onto well-articulated targets.
Specification of Treatment Ingredients and Dosing Parameters
We made similar advances in the guidance for the specification of treatment ingredients and their dosing parameters. We determined that active ingredients in the organ functions category were organ-specific physical stimuli. We proposed asking the same treatment experts who are engaged in articulating consensus-based organ functions targets to generate a list of the ingredients that actively alter the functioning of the organ(s) relevant to their treatment expertise. We determined that the key ingredient for the skills and habits group was the clinician's facilitation of performance, often coupled with various forms of guidance and feedback. The key ingredient in the representations group was the clinician's facilitation of information processing, either in the form of delivering new information or encouraging reflection and reprocessing of existing information. This information can be more neutral (as in education) or contain more emotional valence (as in motivational or behavior change discussions). Importantly, we determined that the ingredients that are active in achieving a volition target (ie, the successful performance of a treatment activity) are the same as those that are also associated with representations targets; in fact, volition targets are a special case of representations targets, in which the clinician intends for the information to lead to action rather than simply to understanding. In addition, we proposed that the COM-B framework,37 developed in health psychology, is a useful way to organize and conceptualize volition ingredients. The COM-B framework stresses the importance of Capability, Opportunity, and Motivation in leading to health-related Behavior change, and we have incorporated these dimensions into our guidance about the specification of ingredients for representations group targets as well as specifically for volition targets.
In addition to clarifying the types of ingredients associated with the 3 treatment groups, we also developed rules for specifying dosing parameters for different kinds of ingredients. Dosing parameters range from quantitative measures of ingredients on a ratio or interval scale (eg, number of practice repetitions, load during a muscle contraction) to those on an ordinal scale (eg, moderate tension applied during serial casting), and from those reflecting settings on an ingredient delivery vehicle (eg, the wattage setting and head size for an ultrasound transducer) to those reflecting treatment progression (eg, systematic changes in dosing during treatment that are planned in advance). We stress that the relevant notion of dose differs for different kinds of ingredients. Currently, the notion of dose is often framed with respect to time, as in specifying the number of hours of physical therapy or the number of days in inpatient rehabilitation. We have specifically discouraged specifying dose with respect to time, except in contexts in which the ingredient's mechanism of action does depend on duration of exposure (eg, duration of heating prior to tendon stretch). In other contexts, dose specification should rely on measurable quantities of the active ingredient(s), such as number of performance repetitions or the specific information content delivered.
We distilled the conceptual developments related to ingredient specification, like those related to targets, into the manual along with illustrative examples. We provide 1 such example in Table 4 to illustrate a number of the concepts and potential impacts discussed previously.
In this specification, what a speech-language pathologist traditionally might consider a single treatment (instruction in use of TEP [tracheoesophageal puncture] prosthesis) is divided into 4 distinct treatment components. Although the TEP and prosthesis affect organ functions (voicing), there is no organ functions treatment component. This is because orthotic and prosthetic devices have their own organ functions treatment components only when they exert their effects passively. In this case, the TEP prosthesis is unlikely to achieve any target unless it is incorporated into the performance of a skill and becomes one of the necessary ingredients for acquiring this skill. The first 2 components listed address performance skills and therefore are classified as skills and habits treatment components. There are 2 distinct skills and habits targets because the skills, materials required to execute them, and criteria for judging independence are different. As expected, however, the clinician provides opportunities for repeated performance of each skill, along with verbal, visual, and/or tactile instruction and feedback. Each target is named for the skill to be acquired, and target achievement is measured by correct performance without the need for assistance. Note that for many skills, increased independence (rather than complete independence) may be the target chosen. But in this instance, the clinician determines that training must continue until complete independence is reached. The practice schedule and total number of trials (dosing parameters) are not specified quantitatively because the trials will be closely clustered within a single session; rather than specifying the number of trials, training will be to criterion since the number of trials necessary is not known in advance. Note also that the precise feedback and physical assistance to be given are not stated, since these are dependent on the nature of the patient's difficulties and errors. Both skills and habits targets are volitional treatment components, but because treatment will be closely supervised, volition ingredients (eg, physical assistance, feedback to increase “capability”) are specified but no separate volition target is provided.
The third treatment component involves a representations target. Representations targets exist on a continuum from knowledge to action. The patient has already acquired the skill of cleaning the device. The clinician has chosen a target at the action end of the continuum, since she wants the patient not simply to know what he should do at home but to actually do it. Thus, target achievement could be measured (in principle) by the frequency and consistency of TEP prosthesis cleaning at home, which might be approximated by self-report, inspection of the condition of the TEP prosthesis at his next visit, etc. As expected for a representations target, we do not see ingredients involving repeated performance; rather, we see informational ingredients intended to alter capability, opportunity, and motivation (eg, instructions, the log sheet, information on the risks of not cleaning the device regularly). The last treatment component also involves a representations target, albeit further toward the knowledge end of the knowledge-action continuum. The clinician has made the judgment that repeatedly dislodging the device to allow practice of the skill of dealing with this problem is not feasible and, therefore, has settled for the probability that the knowledge alone will increase the patient's propensity to act as instructed if and when the time comes. Accordingly, she would measure the target through knowledge probes rather than performance assessment. The ingredients for these 2 targets do not have dosing parameters because they are specified as either present or absent (eg, log sheet, written handout) or as the key “bits” of information.
Returning to some of the potential impacts of RTSS specification, we believe that a published treatment protocol of the type illustrated in this example would provide the key details needed for other clinicians to implement the treatment, delivering the same active ingredients. Adherence to the treatment protocol (either in a research or a clinical context) could be assessed by observing whether the specified materials are provided, whether initial demonstration and subsequent corrective feedback are provided, whether practice is continued to criterion, etc. The protocol would not require the use of exactly the same words for conveying instructional information, on the assumption that the key points rather than the specific words are theoretically relevant. However, the clinician's treatment theory assumes that reading the written handout aloud while the patient reads along will enhance retention. The specification also sharpens clinical reasoning. It requires the clinician to consider in which contexts information alone will achieve the necessary outcomes and which ones will require practice.
This type of specification offers additional potential benefits for research. A researcher developing such a treatment would not select outcomes such as increased speech intelligibility or increased verbal interaction as targets, because these would be more distal aims that cannot be guaranteed with the ingredients shown or achievement of the targets listed. However, other researchers might provide treatments that facilitate patients' use of their TEP-based voicing in social interaction (eg, supervised practice in social settings) and legitimately choose measures of verbal performance or interaction as outcome measures. Availability of RTSS specifications would prevent attempts at evidence synthesis across these very different types of treatment, even if they were all retrieved while searching for “TEP/voice rehabilitation.”
This specification also suggests next steps for the individual clinician if treatment fails and more potential questions for the researcher. If the patient acquires the first 2 skills but does not maintain the device at home, this would contradict the clinician's hypothesis that the ingredients addressing capability, opportunity, and motivation would be sufficient to achieve regular performance at home and would suggest several potential courses of action. The clinician could consider whether more supervised cleaning practice prior to discharge might make the skill less effortful and the patient more likely to perform it at home (ie, that these COM-B ingredients would be sufficient to drive a less effortful skill); whether “stronger” COM-B ingredients (eg, more detailed discussions of risks, goal-setting maneuvers, discussing the most feasible times and contexts for cleaning) might drive the current level of skill at home; or whether enlisting a friend or caregiver to prompt scheduled practice and habit formation at home might be necessary. A clinician treating a similar patient who also had significantly impaired declarative memory could still consider pursuing the 2 skills and habits targets, since such skills can be acquired implicitly with practice by amnestic patients, but he or she would need to recruit caregivers or develop some reliable cuing system to ensure regular cleaning and response to dislodgement. A researcher could ask whether, given the relative simplicity of these skills, a standardized instructional videotape might provide sufficient information for the patient to perform the procedures without repeated practice. Thus, protocols involving instruction alone vs instruction plus practice could be compared. Moreover, teaching other manual skills of a similar nature, pursued by members of other disciplines, could be structured in a parallel fashion, such that a collection of very similarly structured treatments could be developed and shared and additional questions asked, such as “How is the amount of assistance required on the first trial related to the number of trials required to reach independence?” or “Would supervised ‘overlearning’ toward the development of a minimally effortful habit increase the chance of consistent performance at home?” Thus, the RTSS's focus on the ingredients that drive skill, habit, and attitude changes may help researchers see the ingredients that could or should be compared in CER, with the specific skill, habit, or attitude that currently receives the most attention taking a secondary place in research.
We are not prepared to discuss in detail the possible ways that an RTSS-based system could be feasibly incorporated into documentation and billing systems to enrich their values as sources of observational evidence. Clearly, asking each clinician to document treatment sessions at this level of detail would be onerous. One possibility would be to standardize and name such a treatment protocol (most likely, in this example, in 4 parts, since clinicians may choose to address different numbers of treatment components for different patients). Once completed, the name(s) (unlike disciplines or goals of treatment) would meaningfully represent the treatment components' active ingredients. Periodic adherence assessment might be required to prevent drift from the original specification. Alternatively, meaningful ingredient “chunks” might be documented with dosing parameters as appropriate. For example, one could have checkboxes in an electronic medical record that list “risks of not cleaning discussed” and “visual demonstration with mirror provided” for informational items related to representations targets as well as checkboxes for “voicing practice trials (specify number:__)” and “cleaning practice trials (specify number:__)” for skills and habits targets.
Potential impacts of RTSS specification cannot be illustrated completely with a single treatment. However, it is worth noting that advisory board members—who include clinicians, clinical educators, researchers, and evidence synthesizers and who have had the opportunity to work through multiple treatment specifications in multiple functional domains—perceive that the benefits in these areas could be substantial.
Development of the Manual for Rehabilitation Treatment Specification and Submission of Project Manuscripts
The manual was developed during the course of the project, used by core team members in their attempted specifications, and revised iteratively in response to those attempts. We assembled it into 3 full drafts as the project progressed in order to solicit stakeholder feedback. We distributed the first complete version to the advisory board in advance of the January 2017 meeting and asked board members to review it and then to use it for a set of practice specifications at the meeting. We revised the manual in response to the board's feedback and used this next version for the RTSS clinician training cycle. We used feedback from the clinician training cycle to create the version used in the final advisory board meeting in November 2017. The feedback from this meeting resulted in only minor changes, which we implemented in a final draft in the last days of the project. As noted in the Conclusions, we are developing plans for hosting the manual on a publicly available website and disseminating invitations for rehabilitation professionals to engage in beta-testing it.
Publications and Dissemination Activities
In the proposal, we had planned to submit 2 to 3 manuscripts describing the outcomes of the project. By the end of the project period, we had revised the plan to include 4 interrelated manuscripts and an overarching introduction and had arranged with the Archives of Physical Medicine and Rehabilitation, the publisher of the 2014 set of RTT manuscripts, to simultaneously publish these in the journal, subject to peer review. We circulated outlines for the 4 planned articles prior to the November 2017 advisory board meeting, and the group discussed each outline at the meeting, with the primary authors noting feedback. One article provides an overall update on the concepts and procedural implementation of the RTT/RTSS since the 2014 set of publications. One focuses specifically on the importance of volitional behavior in rehabilitation and in defining rehabilitation treatment. The 2 other articles focus on the particular impacts that may be anticipated from the RTSS on clinical practice and on research reporting and evidence synthesis. We also were invited by the Archives to write a commentary on a systematic review article concerning the poor state of treatment description in stroke rehabilitation.
Upon return from the advisory board meeting, the primary authors for each of the articles began working on initial drafts, informed by the board's feedback. These articles, which involved most or all of the core team members, were circulated for collaborator comments and edits and submitted to the Archives by March 9, 2018. Revisions based on initial peer review were due to the journal on August 31, 2018, and all have now been accepted. The invited commentary, which illustrates the contribution that the RTSS could make to some of the information gaps cited in the article on which it comments, has been published (see Publications for list of articles).
Next Steps in Dissemination and Implementation
Prior to the final advisory board meeting, we synthesized the multiple forms of feedback received from previous meetings with the advisory board, from the clinician training cycle, and from our various presentations at professional society meetings, to present for discussion with advisory board members. Taken as a whole, the feedback from rehabilitation researchers, clinicians, and educators was consistent: There was strong support for the value of the concepts developed during this project and a clear sense of their potential impact in both clinical and research realms. There was also consistent feedback that the concepts addressed by the RTSS are complex and challenging and require considerable discussion and practice to master. Our work with the advisory board and RTSS clinician training cycle demonstrated that accurate specification is a skill that requires practice and feedback, not the presentation of a simple set of factual knowledge. Indeed, even within our core team, which was immersed in this topic, we often arrived at stable treatment specifications only through discussion.
Though the procedures and rules of the RTSS are somewhat complex, there is consensus that the bulk of the complexity lies in the nature of the treatment domain that we are attempting to specify and in the fact that RTSS specification requires the clinician or researcher to make explicit certain behaviors or approaches that he or she may not have otherwise considered explicitly. In this context, it became clear that implementing the RTSS would require considerable additional work. To be successfully implemented, the RTSS concepts would need to be introduced into curricula and through training courses over time rather than in brief 1-shot workshops. Providing such exposure and training, in turn, requires curricular materials and a cadre of faculty sufficiently familiar with the system and sufficiently motivated to teach it. In the research realm, journal editors and funding agencies would need to exert pressure on researchers to use the RTSS in defining study treatments, and researchers and peer reviewers, too, would need exposure and training in the RTSS to comply. There also is a need to convene expert consensus groups to develop menus of treatment targets and ingredients used in various treatment areas, as discussed previously, in order to lessen the burden on individual specifiers and reduce variability in the words used in specification.
The final advisory board meeting was substantially devoted to analyzing how we could foster continued development and implementation of the RTSS and who might lead the various relevant efforts. We agreed that a number of different implementation projects should be considered in parallel as next steps, in order to build evidence for the positive impact of local implementation efforts and to identify and address additional obstacles to implementation as it is scaled up. There was considerable enthusiasm among members of the current project team, as well as members of the advisory board, to lead such projects. We hope that the combination of published articles characterizing the work, publicizing the availability of the manual for use in beta-testing, and the design of multiple specific implementation projects will continue to build evidence showing that this method of specification is both useful and feasible.
Discussion
Our project addresses a critical deficit in the specification and measurement of rehabilitation interventions that must be resolved in order to fill the many specific gaps in the evidence base within this branch of health care. Rehabilitation clinicians, educators, and researchers perceive concepts contained in the RTSS as valuable. However, stakeholders also have emphasized that use of the RTSS is a skill that will require ongoing exposure, practice, and active discussion. As a result, potential users of the system must be convinced of its value in order to motivate the various implementation efforts required to generate widespread adoption. The manual is a freestanding product amenable to beta-testing by these various stakeholders. We expect that the 4 manuscripts recently accepted for publication will help researchers and clinicians understand the potential value of the RTSS, leading them to use the manual and to provide feedback for its further development and implementation.
The current strengths of the RTSS include its comprehensiveness (ie, all treatments delivered by all rehabilitation disciplines may be specified by the same system) and its basis in treatment theory. This latter feature of the system forces attention to the proximal outcomes (targets) of precisely defined interventions, rather than macro, downstream effects that may require additional clinical efforts to achieve (aims). Importantly, the system also demands specification of active ingredients (eg, clinician actions, materials, modalities) that are expected to cause the changes in functional targets. The sorting of rehabilitation targets into 3 mutually exclusive (though large) groups is intuitive and helps align the abstract concepts of the RTSS with clinical thinking. The stipulation that both targets and ingredients must be measurable, at least in principle, serves to demystify the actions performed during the “art” of rehabilitation and to encourage measurement of treatment effects not only in the laboratory but also in the clinic. Moreover, the explicit separation of targets and ingredients addressing the direct effects of a treatment, and the patient's volitional behavior that must occur for those direct effects to be achieved, highlights an important distinction for both clinical practice and research advancement. Clinicians can use this distinction to adjust the ingredients needed to boost one or the other source of the treatment's overall effect; researchers can systematically vary volition-directed ingredients to determine their separate effects under different conditions and with different patient populations.
The RTSS in its current state of development also has limitations, some of which we described in earlier sections. We have received feedback (and have observed in our own research team process) that the concepts are complex and require much repetition and many clinical examples to be fully understood. As noted earlier, the selection of certain items for target and ingredient specification will need to be driven by selection menus and language conventions, which have yet to be developed. Thus, as it stands, the RTSS is limited in its capacity to produce identical treatment specifications from any 2 users, although those specifications might share significant overlap in the most theoretically important treatment elements. Partly for this reason, 1 anticipated impact of the RTSS—streamlined descriptions of treatment suitable for electronic medical record documentation and third-party reimbursement—awaits a future iteration, in which menus and other language constraints will have been fully implemented.
Despite these limitations, the RTSS represents a forward step in the ongoing effort to solve the problem of the black box of rehabilitation. It simultaneously offers more granularity and, by virtue of being theory driven, more opportunity for standardization across disciplines, settings, and populations compared with alternative systems. For example, the practice-based evidence studies conducted as multicenter observational investigations addressing stroke, joint replacement, spinal cord injury, and TBI rehabilitation38-41 have used discipline- and study-specific, clinician-generated classifications of treatments to describe each clinical encounter. The resulting codes, naming functional domains, devices, and modalities, tend to emphasize the problems to which interventions are directed (eg, balance, strength, cognition) rather than the process or ingredients of treatment. Moreover, potentially superficial differences among disciplines and specialties have been reinforced by creating separate systems for each study of a different rehabilitation population and for each discipline. Another, more global system, the World Health Organization's emerging ICHI,42 thus far provides treatment classifications too broad (eg, practical support with walking and moving, emotional support with walking and moving) to illuminate the actual treatment activities and their mechanisms of change.
The same holds true for an even more recent proposal for classifying aspects of rehabilitation, the International Classification of Service Organization in Health-related Rehabilitation (ICSO-R). The authors, a working group of the International Society of Physical and Rehabilitation Medicine, distinguish classification of (medical) rehabilitation at 3 levels: macro (health system and health policy), meso (service provision and organization), and micro (health condition and function), and they explicitly state that the ICSO-R is aimed at the meso level, ceding the micro level to ICHI.43 ICSO-R has 2 levels (a third one, consisting of value sets, is planned) and is not a classification or taxonomy per se but a listing of dimensions (9 under rehabilitation service provider, 3 under funding of the service, and 8 under service delivery) that are to be used to characterize organizations/programs delivering rehabilitation services. An operationalization of those 20 dimensions has not yet been provided, making application of the ICSO-R problematic in many aspects, as was made clear in 3 efforts to use the scheme.44-46
Yet another source of treatment specification might be found in such efforts as the Consolidated Standards of Reporting Trials for Non-Pharmacologic Treatment47 and the Template for Intervention Description and Replication,48 both of which offer journal article reporting guidelines for researchers engaged in nonpharmacologic treatment studies. However, these systems leave it up to the researcher to decide which procedures and other aspects of treatment are critical for reporting. By identifying the tripartite structure of treatment theory as the basis for treatment specification, in contrast to prior systems, the RTSS could potentially standardize both the concepts and the language used to define rehabilitation treatments so that they may be accurately researched, taught, administered, and replicated. Ultimately, a completed RTSS might be useful not only for medical rehabilitation but for other branches of rehabilitation (eg, psychiatric, vocational, drug and alcohol) and health care (eg, behavioral medicine) that rely on both behavioral and pharmacologic treatments.
Further research would be valuable to assess and further refine the reliability of the RTSS. However, a meaningful assessment of reliability will need to wait for the development of standardized menus of targets in certain areas and categories of ingredients so that minor wording differences do not confound assessments of reliability. Additional work also is needed to develop and test RTSS training curricula for clinician, clinician educator, and researcher stakeholders and to examine the effects of RTSS training and use on clinical reasoning and, ultimately, on treatment outcomes. At our summative advisory board meeting in November 2017, we devoted considerable time to brainstorming and then detailing, to the extent possible, additional beta-test research and clinical implementation projects that might advance the RTSS. For example, Dr Lyn Turkstra, who is deeply involved in graduate-level training of speech-language pathologists at McMaster University, suggested combining training in the RTSS with existing training programs and formally measuring students' perceptions of the system's contributions to treatment planning. Other suggestions included using the RTSS to develop and test volition training programs for difficult problems in rehabilitation (eg, long-term prevention of pressure injuries through self-management in spinal cord injury) and using the RTSS to improve carryover of treatment targets and ingredients in the transition from inpatient to outpatient care.
Of course, executing these plans will require additional time and funding that has not yet been secured. However, 1 recent NIH R21 (Exploratory/Developmental Research Grant Award)-funded project led by an RTSS core team member (Dr Jarrad Van Stan, another speech-language pathologist) will incorporate RTSS concepts into the development of a specification system for voice therapies. Another funded pilot project (developed by core team member Andrew Packel, a physical therapist) will examine the effects of in-depth RTSS training on improving the consistency of treatment during the many inevitable episodes of coverage by other clinicians. A third funding application is under review by the Department of Defense to apply the RTSS to specifying a cognitive rehabilitation program for individuals with mild TBI.
Further development of the RTSS will depend not only on projects of this nature but on wider dissemination and uptake by the rehabilitation community. Toward this end, a central location is needed where the manual can be maintained and where decisions about its ongoing use and further elaboration can be managed. The core team decided not to provide the manual as supplemental content to the published journal articles since this would place its copyright control in the hands of the journal. Rather, the core team has copyrighted the manual and given revocable authority to grant permission for its use to John Whyte, the project principal investigator. Discussions are ongoing with the American Congress of Rehabilitation Medicine to post the manual in a limited-access fashion and to serve as the organizing point for an interest group that would manage requests to use the manual; synthesize feedback resulting from its use; oversee the consensus modifications required by such feedback; and serve as an organizing forum for continued efforts to implement and disseminate the RTSS, the Manual for Rehabilitation Treatment Specification, and related training materials and supports.
Conclusions
Comparative effectiveness research cannot be conducted on rehabilitation interventions until those interventions are described and defined using a common language and systematic framework for specifying their functional targets and known or hypothesized active ingredients. A framework such as the RTSS, when further developed and tested, will enable patient and clinician consumers to examine and compare the strengths, limitations, indications, and outcomes of specific rehabilitation interventions and to customize those interventions for the ultimate goal of maximizing function for people with disabilities.
References
- 1.
JohnstonS, LiddyC, MillK, IrvingH. Building the evidence base for chronic disease self-management support interventions across Canada. Can J Public Health. 2012;103(6):e462-e467. https://link
.springer .com/article/10.1007/BF03405639 [PMC free article: PMC6975212] [PubMed: 23618029] - 2.
Fleming-CastaldyRP, GillenG. Ensuring that education, certification, and practice are evidence based. Am J Occup Ther. 2013;67(3):364-369. [PubMed: 23597695]
- 3.
Van LithT, SchofieldMJ, FennerP. Identifying the evidence-base for art-based practices and their potential benefit. Disabil Rehabil. 2013;35(16):1309-1323. [PubMed: 23116320]
- 4.
BrownCA, KuoM, PhillipsL, BerryR, TanM. Non-pharmacological sleep interventions for youth with chronic health conditions. Disabil Rehabil. 2013;35(15):1221-1255. doi:10.3109/09638288.2012.723788 [PubMed: 23050860] [CrossRef]
- 5.
ZauszniewskiJA, BekhetA, HaberleinS. A decade of published evidence for psychiatric and mental health nursing. Online J Issues Nurs. 2012;17(3):8.
- 6.
LuftAR. How to gain evidence in neurorehabilitation: a personal view. Biomed Tech (Berl). 2012;57(6):427-433. [PubMed: 23023795]
- 7.
GroahSL, LibinA, LauderdaleM, KrollT, DeJongG, HsiehJ. Beyond the evidence-based practice paradigm to achieve best practice in rehabilitation medicine: a clinical review. PM R. 2009;1(10):941-950. doi:10.1016/j.pmrj.2009.06.001 [PubMed: 19797005] [CrossRef]
- 8.
DijkersMP, MurphySl, KrellmanJ. Evidence-based practice for rehabilitation professionals: concepts and controversies. Arch Phys Med Rehabil. 2012;93(8 suppl):S164-S176. doi:10.1016/j.apmr.2011.12.014 [PubMed: 22683207] [CrossRef]
- 9.
DijkersM, KroppGC, EsperRM, YavuzerG, CullenN, BakdaliehY. Quality of intervention research reporting in medical rehabilitation journals. Am J Phys Med Rehab. 2002;81(1):21-33.
- 10.
Elorriaga ClaracoA, HannaSE, Fargas-BabjakA. Reporting of clinical details in randomized controlled trials of acupuncture for the treatment of migraine/headaches and nausea/vomiting. J Altern Complement Med. 2003;9(1):151-159. [PubMed: 12676043]
- 11.
CameronID. Models of rehabilitation - commonalities of interventions that work and of those that do not. Disabil Rehabil. 2012;32(12):1051-1058. doi:10.3109/09638281003672377 [CrossRef]
- 12.
van HeugtenC, GregórioGW, WadeD. Evidence-based cognitive rehabilitation after acquired brain injury: a systematic review of content of treatment. Neuropsychol Rehabil. 2012;22(5):653-673. [PubMed: 22537117]
- 13.
KeithRA, LipseyM. The role of theory in rehabilitation assessment, treatment, and outcomes. In: GlueckaufRL, SechrestL, BondGR, McDonelEC, eds. Improving Assessment in Rehabilitation and Health. Sage Publications; 1993:33-58.
- 14.
WhyteJ. The contributions of treatment theory and enablement theory to rehabilitation research and practice. Arch Phys Med Rehabil. 2014;95(suppl 1):S17-S23. [PubMed: 24370321]
- 15.
BodeRK, HeinemannAW, SemikP, MallinsonT. Patterns of therapy activities across length of stay and impairment levels: peering inside the “black box” of inpatient stroke rehabilitation. Arch Phys Med Rehabil. 2004;85(12):1901-1908. [PubMed: 15605324]
- 16.
DeJongG, HornSD, ConroyB, NicholsD, HealtonEB. Opening the black box of post-stroke rehabilitation: stroke rehabilitation patients, processes, and outcomes. Arch Phys Med Rehabil. 2005;86(12 suppl 2):S1-S7.
- 17.
WhyteJ, HartT. It's more than a black box, it's a Russian doll: defining rehabilitation treatments. Am J Phys Med Rehabil. 2003;82(8):639-652. [PubMed: 12872021]
- 18.
WadeDT. Research into the black box of rehabilitation: the risks of a type III error. Clin Rehabil. 2001;15(1):1-4. [PubMed: 11237155]
- 19.
BallingerC, AshburnA, LowJ, RoderickP. Unpacking the black box of therapy—a pilot study to describe occupational therapy and physiotherapy interventions for people with stroke. Clin Rehabil. 1999;13(4):301-309. [PubMed: 10460118]
- 20.
FixsenD, GrimshawJM, EcclesMP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009;4:40. [PMC free article: PMC2717906] [PubMed: 19607700]
- 21.
American Medical Association. CPT: Current Procedural Terminology. American Medical Association; 2004.
- 22.
International Council of Nurses. International Classification for Nursing Practice. Accessed September 4, 2019. https://www
.icn.ch/what-we-do /projects/ehealth /icnp-download/icnp-download - 23.
MartinKS. The Omaha System: A Key to Practice, Documentation, and Information Management. Saunders; 2005.
- 24.
BowlesKH, MartinKS, NaylorMD. Using the Omaha System to describe what's inside the black box. Int J Nurs Terminol Classif. 2006;17:21.
- 25.
BulechekGM, ButcherHK, DochtermanJM, WagnerC, eds. Nursing Interventions Classification (NIC). 6th ed. Mosby Elsevier; 2013.
- 26.
SabaVK. Home health care classification. Caring. 1992;11(5):58-60.
- 27.
SabaV. Clinical Care Classification (CCC) System Manual, A Guide to Nursing Documentation. Springer Publishing Co; 2006.
- 28.
BulechekGM, ButcherHK, DochtermanJM. Nursing Interventions Classification (NIC). 5th ed. Mosby Elsevier; 2008.
- 29.
CraigP, DieppeP, MacintyreS, MichieS, NazarethI, PetticrewM. Developing and evaluating complex interventions: the new Medical Research Council. Int J Nurs Stud. 2013;50:587-492. doi:10.1016/j.ijnurstu.2012.09.010 [PubMed: 23159157] [CrossRef]
- 30.
MichieS, RichardsonM, JohnstonM, et al.The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81-95. [PubMed: 23512568]
- 31.
DonabedianA. Evaluating the quality of medical care. Milbank Q. 1966;83(4):691-729.
- 32.
WhyteJ. Invited commentary on quality of care indicators for the rehabilitation of children with traumatic brain injury, and quality of care indicators for the structure and organization of inpatient rehabilitation care of children with traumatic brain injury. Arch Phys Med Rehabil. 2012;93(3):394-395. [PubMed: 22284403]
- 33.
Institute of Medicine. Defining cognitive rehabilitation therapy. In: Cognitive Rehabilitation Therapy for Traumatic Brain Injury: Evaluating the Evidence. National Academies Press; 2011:75-87.
- 34.
BoyleM. Semantic feature analysis treatment for anomia in two fluent aphasia syndromes. Am J Speech Lang Pathol. 2004:13(3):236-249. [PubMed: 15339233]
- 35.
EdmondsLA, NadeauSE, KiranS. Effect of Verb Network Strengthening Treatment (VNeST) on lexical retrieval of content words in sentences in persons with aphasia. Aphasiology. 2009;23(3):402-424. [PMC free article: PMC2744980] [PubMed: 19763227]
- 36.
World Health Organization. International Classification of Functioning, Disability and Health. Accessed March 26, 2018. http://www.who.int/classification/icf
- 37.
MichieS, van StralenMM, WestR. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;23:42.
- 38.
GassawayJ, WhiteneckG, DijkersM. Clinical taxonomy development and application in spinal cord injury research: the SCIRehab Project. J Spinal Cord Med. 2009;32(3):260-269. [PMC free article: PMC2718825] [PubMed: 19810628]
- 39.
HornSD, CorriganJD, BognerJ, et al.Traumatic brain injury-practice based evidence study: design and patients, centers, treatments, and outcomes. Arch Phys Med Rehabil. 2015;96(8 suppl):S178-S196. [PMC free article: PMC4516907] [PubMed: 26212396]
- 40.
DejongG, HornSD, GassawayJA, SlavinMD, DijkersMP. Toward a taxonomy of rehabilitation interventions: using an inductive approach to examine the “black box” of rehabilitation. Arch Phys Med Rehabil. 2004;85(4):678-686. [PubMed: 15083447]
- 41.
DeJongG, HsiehCH, GassawayJ, et al.Characterizing rehabilitation services for patients with knee and hip replacement in skilled nursing facilities and inpatient rehabilitation facilities. Arch Phys Med Rehabil. 2009;90(8):1269-1283. [PubMed: 19651261]
- 42.
International Classification of Health Interventions (ICHI). ICHI Beta-2 2019 International Classification of Health Interventions. Accessed September 4, 2019. https://mitel
.dimi.uniud.it/ichi/docs/ - 43.
GutenbrunnerC, BickenbachJ, KiekensC, et al.ISPRM discussion paper: proposing dimensions for an International Classification System for Service Organization in Health-related Rehabilitation. J Rehabil Med. 2015;47(9):809-815. [PubMed: 26353828]
- 44.
RøeC, KirkevoldM, AndelicN, et al.The challenges of describing rehabilitation services: a discussion paper. J Rehabil Med. 2018;50(2):151-158. [PubMed: 29260837]
- 45.
KiekensC, MeyerT, GimiglianoF, BaffoneC, GutenbrunnerCM; UEMS PRM ICF Workshop moderators and rapporteurs. European initiative for the application of the International Classification of Service Organization in Health-related Rehabilitation (ICSO-R). Eur J Phys Rehabil Med. 2017;53(2):308-318. [PubMed: 28488426]
- 46.
ProdingerB, Scheel-SailerA, EscorpizoR, StuckiG; UEMS PRM ICF Workshop moderators and rapporteurs. European initiative for the application of the International Classification of Functioning, Disability and Health: development of Clinical Assessment Schedules for specified rehabilitation services. Eur J Phys Rehabil Med. 2017;53(2):319-332. [PubMed: 27882909]
- 47.
BoutronI, MoherD, AltmanDG, SchulzKF, RavaudP. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148(4):295-309. [PubMed: 18283207]
- 48.
HoffmannTC, GlasziouPP, BoutronI, et al.Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687. doi:10.1136/bmj.g1687 [PubMed: 24609605] [CrossRef]
Publications
- •.
TurkstraLS, NormanR, WhyteJ, DijkersMP, HartT. Knowing what we're doing: why specification of treatment methods is critical for evidence-based practice in speech-language pathology. Am J Speech Lang Pathol. 2016;25(2):164-171. doi:10.1044/2015_AJSLP-15-0060 [PMC free article: PMC6195039] [PubMed: 27145191] [CrossRef]
- •.
FasoliSE, FerraroMK, LinSH. Occupational therapy can benefit from an interprofessional rehabilitation treatment specification system. Am J Occup Ther. 2019;73(2):7302347010p1-7302347010p6. doi:10.5014/ajot.2019.030189 [CrossRef]
- •.
WhyteJ, DijkersMP, Van StanJH, HartT. Specifying what we study and implement in rehabilitation: comments on the reporting of clinical research. Arch Phys Med Rehabil. 2018;99(7):1433-1435. doi:10.1016/j.apmr.2018.03.008 [PubMed: 29649449] [CrossRef]
- •.
HartT, DijkersMP, WhyteJ, et al.A theory-driven system for the specification of rehabilitation treatments. Arch Phys Med Rehabil. 2019;100(1):172-180. doi:10.1016/j.apmr.2018.09.109 [PubMed: 30267669] [CrossRef]
- •.
WhyteJ, DijkersMP, HartT, et al.Arch Phys Med Rehabil. 2019;100(1):156-163. doi:10.1016/j.apmr.2018.09.111 [PubMed: 30267665] [CrossRef]
- •.
ZancaJM, TurkstraLS, ChenC, et al.Advancing rehabilitation practice through improved specification of interventions. Arch Phys Med Rehabil. 2019;100(1):164-171. doi:10.1016/j.apmr.2018.09.110 [PubMed: 30267670] [CrossRef]
- •.
Van StanJH, DijkersMP, WhyteJ, et al.The rehabilitation treatment specification system: implications for improvements in research design, reporting, replication, and synthesis. Arch Phys Med Rehabil. 2019;100(1):146-155. doi:10.1016/j.apmr.2018.09.112 [PMC free article: PMC6452635] [PubMed: 30267666] [CrossRef]
- •.
DijkersMP. An end to the black box of rehabilitation?Arch Phys Med Rehabil. 2019;100(1):144-145. doi:10.1016/j.apmr.2018.09.108 [PubMed: 30267668] [CrossRef]
Acknowledgments
We gratefully acknowledge the members of our advisory board, who carefully reviewed and critiqued the project's conceptual developments and drafts of the Manual for Rehabilitation Treatment Specification, advised us on the conduct of our RTSS clinician training cycle, and offered many constructive recommendations for fostering further development of the RTSS in the coming years.
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#ME-1403-14083) Further information available at: https://www.pcori.org/research-results/2014/creating-manual-better-define-rehabilitation-treatments
Appendix
Appendix (PDF, 797K)
Table A. Presentations by Core Team Members (PDF, 132K)
Glossary
- Active ingredients
Attributes of a treatment, selected/delivered by the clinician, that are hypothesized to exert the treatment's effects on the target.
- COM-B framework
Framework borrowed from Susan Michie and colleagues, working in the field of health psychology. These researchers model voluntary behavior as a function of Capability, Opportunity, and Motivation, with further subdivisions within these 3 elements. This scheme has been particularly useful to the RTSS to create categories that help to prompt specification of ingredients needed for voluntary behavior, especially behavior enacted at a remove from the clinician (e.g., home programs or assignments between sessions).
- Devices
Devices are a subcategory of ingredients that include prostheses, orthoses, and assistive devices, either devised in therapy or off-the-shelf, such as cell phones. Some devices (e.g., positioning aids) can achieve their purpose passively, in which instances the device is considered to have a treatment target of its own, in the organ functions group. More commonly, however, a patient must develop the skill of using a device during relevant tasks; in these instances, the device's therapeutic attributes are defined to be ingredients directed toward a skills and habits or representations target. In either case, the ingredients should be defined in terms of the theoretically relevant attributes of the device rather than the device itself.
- Direct target
Change in the specific aspect of functioning which is predicted to result from performance of the treatment activity by the clinician (non-volitional treatments) or the treatment recipient (volitional treatments). The direct target for volitional treatments may be accompanied by a separately specified volition target (see entry) in cases where clinicians are unable to verify directly the performance of the behavior needed to convey the active ingredients for the direct target.
- Dosing parameters
Quantitative variations in ingredients, such as numbers of repetitions, intensity of practice, setting on device that delivers energy to tissues, criterion for success for progression and slope of progression in treatment, schedules of practice or reinforcement.
- Enablement theory
A formal theory or conceptual system that specifies how change in one aspect of a patient's functioning (eg, at the level of an International Classification of Functioning, Disability and Health [ICF] component: body structure, body functioning, activity/activity limitation, participation/participation restriction, personal factor, or environment) will translate into changes in another aspect, specifically a characteristic classified elsewhere in the framework being used. The RTSS makes a distinction between a treatment theory (which specifies the ingredients that achieve a selected target by means of a mechanism of action) and an enablement theory (which sets out how a change in one or more targets may result in various “downstream” aims).
- Habit
A behavioral routine that is repeated regularly and, after acquisition, tends to occur automatically in the presence of certain stimuli. Habits are important considerations for rehabilitation because the long-term goals of rehabilitation often entail patients' acting habitually in new, adaptive ways to sustain or continue improvements in functioning. Habits are included as targets in the skills and habits group because habit formation responds to many of the same active ingredients as skill learning.
- Inactive ingredients
Attributes of a treatment that do not define or moderate the impact of the treatment on the target. Ingredients may be presumed to be inactive when they are not addressed by the treatment theory (e.g., the color of the walls in which the treatment is conducted) or have been empirically determined to be inactive.
- Ingredients
Observable (and, therefore, in principle, measurable) actions, words, hands-on manipulation, common objects, chemicals, devices, or forms of energy that are selected/delivered by the clinician to a treatment recipient. See also active ingredients and inactive ingredients.
- Mechanism of action
Process by which a treatment's active ingredients induce change in the target of treatment. A treatment theory should specify/hypothesize how the active ingredients engage mechanisms of action to bring about desired treatment effects. That is, specification of the mechanism of action explains how the active ingredients alter the treatment target within the framework of the treatment theory. Unlike ingredients and targets, mechanisms of action are frequently not observable and must be inferred by the effects of ingredients on targets.
- Progression
The clinician's deliberate, systematic alteration of treatment ingredient(s) to maintain, over time, the degree of challenge to the body system/behavior(s) selected for change. The next highest level in a progression is often triggered by improvements in the target of treatment; therefore, the pace of progression (within a single treatment contact or over a course of treatment) typically depends on the pace of change in the treatment target. The form that treatment progression takes (and hence the nature of the challenge that is being maintained) is often specified by the treatment theory.
- Rehabilitation Treatment Specification System (RTSS)
Conceptual framework that can be used to specify any rehabilitation treatment, by connecting the actions of the clinician (ingredients) with the changes produced in the patient or other recipient of treatment.
- Target (of treatment)
Specific, measurable (in principle) aspect of the recipient's functioning or personal factor that is predicted in the treatment theory to be directly changed by the treatment's mechanism of action. A single target plus the treatment ingredients hypothesized to achieve it constitute a treatment component. Change in a target causally precedes any aims that may be achieved as a result.
- Taxonomy
System of classification or categorization based on common/distinct characteristics of the elements within the group to be classified. The resulting classification may have pragmatic, theoretical, and/or heuristic utility.
- Treatment component
Portion of a clinical treatment that includes one target, selected active ingredients, and associated known or hypothesized mechanism(s) of action, as defined by an underlying treatment theory. Treatment components are often, though not necessarily, administered in combinations in an effort to produce the desired changes in functioning faster or more completely.
- Treatment group
Broad class of treatments that are mutually exclusive with respect to treatment targets and mechanisms of action. See organ functions treatment group, skills and habits treatment group, representations treatment group.
- Treatment theory
Conceptual system that predicts the effects of the ingredients used in specific forms of treatment on their targets, specifying the law(s) of the relationship between active ingredients and changes in treatment targets. Because all treatment theories contain ingredients, mechanisms of action, and targets, they are described as having a tripartite structure.
- Tripartite structure (of treatment theory)
See Treatment theory
- Volition
Volition may be roughly equated with effort expended by the treatment recipient. Some organ functions treatments—physical exercise being a prime example—and all skills and habits and representations treatments require volition.
- Volition target
A target in the representations group that is expressed as a specific volitional behavior assigned by a clinician. Volition targets may stand alone or be paired with direct targets, in cases where the clinician cannot directly verify the occurrence/accuracy of the recipient's action: eg, in the case of a home program, homework assignment, certain forms of telerehabilitation. In such cases, a volition target must be enacted by the treatment recipient in order for the ingredients for the direct target to be implemented.
- Volitional treatments
Treatments where a (hypothesized) mechanism of action requires some effort, either mental or physical, on the part of the treatment recipient. They can be contrasted with non-volitional (passive) treatments where such effort is not required for the mechanism of action to unfold.
Institution Receiving Award: Albert Einstein Healthcare Network
Original Project Title: Better Rehabilitation Through Better Characterization of Treatments: Development of the Manual for Rehabilitation Treatment Specification
PCORI ID: ME-1403-14083
Suggested citation:
Whyte J, Hart T, Dijkers MP, et al. (2019). Creating a Manual to Better Define Rehabilitation Treatments. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/02.2020.ME.140314083
Disclaimer
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
