Last Modified: 01/06/2012
VII. Evaluating Tele-Intervention Outcomes
The telehealth literature supports claims that TI may be more effective, cost-effective and timely, with fewer cancelled appointments resulting in greater frequency and consistency of services (need citations of the literature here). Based on their collective experiences to date, NCHAM learning community members suggested a number of potential differences (shown below in Table 3) between early intervention services provided face-to-face or through tele-intervention. These claims, and similar claims in the published literature, are supported mostly by anecdotal evidence. Research to identify actual differences between services provided by tele-intervention and face-to-face home visits is sorely needed, particularly for children who are DHH. Research could provide credible evidence that tele-intervention results in similar or better outcomes for the child and the family. When this evidence is collected, and one considers the cost-effectiveness of TI, then children who are DHH and who are currently underserved due to location and the shortage of trained professionals may be able to achieve outcomes commensurate with their hearing peers. An evidence-based model of tele-intervention could make this dream a reality.
It is important to note, however, that the literature on tele-intervention evaluations is primarily based on pilot studies or pre-post designs. Recommendations for evaluating TI are provided here based on the collective planning of learning community members. Therefore, it provides a springboard for conducting future studies on TI for children with hearing loss.
On This Page
- Proposed Outcomes of TI
- Creating a Logic Model to Guide Evaluation
- Selecting Measurement Tools and Administration
- Recent TI Evaluations
Proposed Outcomes of TI
As with any type of intervention, evaluating TI is critical to ensure that it is in fact resulting in the desired outcomes:
- Increased parent knowledge, skills/competence in fostering development, and satisfaction with services and providers, increased competence with technology
- Increased Child developmental/communication outcomes
- Improved parent-child interaction
Additionally, it is important to measure the impact of TI on the following additional outcomes that are associated with delivery of TI:
- Increased frequency and intensity of intervention
- Provider skills/competence in coaching, incorp. of natural routines, and use of technology
- (Arlene)-- Remove this name therapist/interventionist satisfaction, use of time and decreased illness
- Decreased costs, considering travel time, opportunity cost to providers as well as families
One of the learning communities interviewed families about what they saw as the positive outcomes of TI. The families interviewed reported the following beneficial outcomes based on their personal experience:
- Greater knowledge of the language development process
- Increased skills and confidence in promoting their child’s language and listening
- Increased child responsiveness to parent
- Improved child language development, listening skills
- Increased skills of family members as coaches themselves
There are also factors that can influence outcomes that are important to capture:
- How much acoustic and visual clarity needed in the technology used?
- Can the provider offer needed emotional support to families when needed via TI?
- Does TI work for various interventions?
- Are some parents/children a better fit?
- Are some “interveners” a better fit?An important place to start in evaluating the cost-effectiveness of TI is to develop a logic model.
Creating a Logic Model to Guide Evaluation
A logic model serves to guide the identification of measurement tools to answer questions such as those posed above. A logic model is a way to graphically depict the resources, actions, and expected results of an intervention. Here are some resources to guide develop of a logic model:
- The University of Wisconsin-Extension offers useful examples and guidance for developing logic models
- W.K. Kellogg Foundation
- Learning for Sustainability
Download an example of a logic model developed by NCHAM [PDF]
Selecting Measurement Tools and Administration
A logic model serves to guide the identification of tools to measure outcomes. The table below corresponds with the NCHAM logic model, delineating the constructs to be measured, the tools to be used, and the frequency of data collection. Creating an outline such as the one provided below is important in creating your evaluation plan. Creating such a table helps ensure that your evaluation plan is comprehensive but also practical in regard to your capacity to employ all the measurement activities.
Recent TI Evaluations
- Baharav, E., Reiser, C. (2010) Using Telepractice in parent training in early autism. Telemedicine and e-Health v.16 (6) 727-731.
- Cason, J. (2011) Telerehabilitation: An adjunct service delivery model for early intervention services. International Journal of Telerehabilitation.v. v3 (1) 19-28.
- Kelso, G.L, Fiechtl, B.J., Olsen, S.T., Rule, S. (2009). The feasibility of virtual home visits to provide early intervention. Infants and Young Children, v.22 (4) 332-340.
- Mashima, P.A., Doarn, C.R. (2008). Overview of Telehealth Activities in Speech–Language Pathology [PDF]. Telemedicine and e-Health v.14 (10) 1101-1117.
- McCarthy, M., Munoz, K., White, K.R. Teleintervention for infants and young children who are deaf or hard-of-hearing. Pediatrics. 2010 Aug;126 Suppl 1:S52-8.
