>> [Alyson] ok.
It's the top of the hour.
Let's get started.
My name is Alyson ward, the Director of quality improvement at national center for hearing assessment and management, or NCHAM for short.
I am happy to host and be a presenter.
We will be discussing an EDHI learning community that we just concluded recently.
All around the impact of family support and engagement.
So today's agenda, we will do some quick introductions.
We have several presenters.
So we will do some quick introductions and then we will talk a little bit about the background for this work.
Tools for doing the work and then really dig into what happened with the learning community and the learning community process and then kind of what came out of the learning community.
So I will go ahead, I already introduce made self.
I will go ahead and turn the time over to Lisa Kovacs.
>> Hi, everyone.
I'm Lisa Kovacs.
I am the Director of programs and the FLC center direct.
I will turn it over to Mangy.
>> Hi, I'm Mandy Jay at national center for hearing and management as well, and have been a part of this learning community from the beginning.
Ok.
Want to go to Brigit.
>> Hello I am Bridget Walde and I work at the Department of Health with the early hearing detection and intervention program.
>> All right.
Anne?
>> Hello I am Anne Barlow, director for Minnesota hands and voices.
>> Thank you, Anne.
Tara?
>> Hi, I'm Tara Carrol.
I am the program manager for the first sound which is South Carolina's EDHI program at the South Carolina Department of Health and environmental control.
>> Great.
Thank you, Tara.
And Cara.
>>> Hi I'm Cara, Director of Beginnings, South Carolina.
>> Thank you so much.
You will be hearing from all of these presenters as we move through the presentation today.
I am going to kick us off with just a little bit of background about the learning community.
As most of you know, the federal maternal and federal health bureau or MCHB has long promoted the role of families as partners in EDHI programs.
The focus of family support and partnership has evolved over the years.
Some recent examples are the requirement that EDHI programs as part of their MCHB funding must spend about 25% of their grant funds in funding family support activities.
Another demonstration of MCHB's commitment to family engagement and support is the inception of the family leadership and language and learning center.
Or FL3 center for short, that is housed at Hands and Voices.
The EDHI MTMC and the FL3 collaborated on putting together learning communities as part of our funding applications in this last round of funding.
So far, we have held two -- well, three learning communities total.
We held two learning communities around the family EDHI project and then this is our third learning community and this is around the impact of family support and engagement.
And that's where we will be spending most of our time today.
The concept of this learning community has really been brewing for years.
There's a lot of anecdote Alan processed information around family support.
Primarily satisfaction related data.
And, we really wanted to dig into what really are the outcomes of family support and hopefully, over time, see more of that impact which the impact often takes several years to really demonstrate.
And so that was the major goal of this learning community.
So I want to just talk just a little bit about some of the tools that we used to kind of prepare for this learning community and Manda and Lisa and I did a lot of this background work prayer to convening the learning community.
We used a variety of tools.
Looked at different types of evaluation and impact and outcome evaluations are important.
We also looked at incorporating fault improvement methodology to this learning community.
We polled some existing definitions and I will talk about those in just a minute.
Then we looked at the conceptual framework domains of family support, which Lisa will discuss in just a little bit.
Then we will also walk you through the process of the learning community from like, you know, looking at existing surveys and developing more impact and outcome related questions.
So first, we really looked at types of evaluation.
And first off, what determines measures as process outcome and impact, really depends on the goal or aim of your project.
So, we will talk a little bit more about the goal and aim of this project in a few minutes, but we wanted to get a good sense of really what these different buckets of evaluation are comprised of.
So, we looked at process, which is typically where a lot of the family support data had been in the past.
About kind of who attended events, how often they attended, what types of events they were, you know, different things like that.
Satisfaction often ends up in the process category, as well.
And it helps monitor progress -- project progress over a short period of time.
Outcome data should take longer to develop.
Or longer to be able to measure.
And it's really what you want to come out of the project or the effort that you are engaging in.
It should also be reflected in your goal or your aim.
And then, it should change -- it should demonstrate changes in groups or individuals over time.
Then impact measures are longer term changes.
Effects often on larger communities like, or larger groups like schools or communities.
And outcomes can lead to impact, but it's not always directly linked.
But, and it also depends a bit on the target audience.
So we started looking at those components.
And then we really wanted to look at using a quality improvement methodology.
So, primarily using the model for improvement.
And I know a lot of you have heard about the model for improvement over the years.
But it's led by three overarching questions.
What are we trying to accomplish?
Which is the aim.
How will we know that a change is an improvement.
That's developing the measures.
Those process and impact that I was discussing.
And what change can we make that will result in improvement?
That's the ideas and changes.
Then testing the changes that we can see in the bottom, the plan, do, study, act cycle.
The goal of the PDSA cycle is to do it in short increments over a short period of time to test if what you are doing is making a difference before you invest a lot of time and money and energy into that change.
Then we also pulled from previously defined definitions in prior projects.
One was looking at what support is referred to.
And we pulled that definition from the FL3 needs assessment that NCHAM helped the FL3 conduct about three or four years ago at this point.
Maybe even longer.
And then we also looked at definitions of family-to-family support.
That came from the FEDI work groups that we did earlier and then we also looked at the definition of family based organization.
Which also came from I think it actually came from both FEDI and the FL3 needs assessment.
I will turn the time over to Lisa to talk about the conceptual model.
>> Hi, everyone.
It's Lisa.
So as Alyson mentioned earlier, one of the tools, the learning community used, was an evidence-based model titled the conceptual framework of parent-to-parent support for parents much children who are deaf or hard of hearing.
This model was developed by researchers from Western University, Dr. Sheila moody Henderson and Johnson between the years of 2014 and 2016.
The first step in their development of the model was a scoping review of current parent-to-parent support literature.
This included 39 peer-reviewed articles that were published about parent-to-parent support between the years of 2000 and 2014.
And then the next step included stakeholder opinion and feedback from 31 experts from seven different countries with experience in the provision of parent-to-parent support.
Participants completed an online questionnaire that informed an initial framework and then eventually the revisions to this final framework that you see on your screen.
As you can see on the image, the framework includes three main constructs.
If you look to both the left and the right-hand side of the hourglass.
That's three constructs include well-being, child and family well-being.
Knowledge, which would include advocacy, systems navigation and transitions and education.
And then the third construct is empowerment and that is the family's competence and confidence.
And then on the right-hand side of the screen, you can see that each one of these constructs included several different components within those constructs.
The learning community used these three constructs, well-being, knowledge and empowerment, to design our survey measuring the impact of family-to-family support.
Next slide.
The learning community starting place was we reviewed existing family-to-family support survey questions, so we requested from family-based organizations if they had a current family-to-family support surveyor evaluation, if they would be willing to provide it to us and we received
nine existing surveys that included 47 unduplicated questions.
So, prior to the first learning community meeting, we took those 47 unduplicated questions and we organized them within those three constructs of well-being, knowledge, and empowerment.
We found that many of the questions were measuring satisfaction from the family's experience being enrolled in family-to-family support.
But had not yet really reached what impact did receiving family-to-family support have on the family.
So again that was the aim of our learning community.
So the next step of the learning community was to review these existing questions, determine what questions might need to be edited or added and work through a consensus process in determining what those final questions were going to be.
Our goal was to get to, to come to, to whittle down to five questions for each construct with the goal of hopefully having the survey short enough that we would get a good response rate.
Next slide.
So, when we put the learning community application out to all the states and territories, we had 13 EDHI programs with our parent leader jointly apply to be a part of the learning community.
In order to make the learning community manageable, we whittled this down to accepting six of the programs, the application required that both an EDHI staff and a parent leader participate in the learning community.
And a couple of the questions we included in the application process included describe how this quality improvement project may help your family-based organization, EDHI program, and larger EDHI system in your state or territory and describe how the learning that you gained from
the learning community, how you might be able to sustain that learning after the learning community.
So, here you see we accepted six states and one territory on the slide.
So that's who participated in our learning community.
And so, now I am going to turn it over to both Minnesota and South Carolina to share a little bit about what motivated them to apply to be a part of the learning community.
>> Hello, this is bridget.
I am from Minnesota.
Within Minnesota, the EDHI program within the Department of Health provides funding to Minnesota Hands an Voices through a grant each year.
As part of the grant, Minnesota Hands and Voices develops a work plan.
Certain objectives included relate to quality improvement and also surveying families.
And, at the Department of Health, we work very closely with Minnesota Hands and Voices and because of this relationship, we were already kind of starting to think about whether the satisfaction surveys that we had been useing in the past were giving us the information that we were
really interested in and looking for.
So, an opportunity came along in alignment with what we were already interested in and also related to the activities, some of the activities in Hands and Voices work plan.
And then, because of our partnership together and this interest, we were able to apply and participate.
>> Thank you, Bridget.
South Carolina group, if you go ahead and share your experience.
>> So, this is Tara from South Carolina's EDHI program.
And we looked at this, we had some learning communities within our own state as part of the Herza grant and had good experiences with that.
We were looking forward to being able to collaborate with other states and see what we could gain in terms of coming up with, helping to formulate something that all states including us, could use to better track the service delivery and do changes for improvement.
Beginnings is our family-based organization that we are contracted with as part of the grant, the Hrsa grant.
And we have been working with them even before our contract but with the contract, the deliverables are aligned to be in alignment with the deliverables for the Hrsa grant.
So tracking this information is definitely part of that.
Cara, if you want to say something?
>> Yeah.
The Beginnings South Carolina is comprised of professionals, deaf adults and parents.
And here at Beginnings SC we understand the importance of family-to-family support and we are committed to improving our Parent Education services.
Our team applied because we value the information data collection provides.
We also recognize the importance like others have said of learning from others in this EDHI system.
>> Ok.
So we at NCHAM and the FL3 did some prework before we pulled the groups together to look at what our purpose and aim would be.
We wanted the learning community to focus on the questions for the survey so the purpose was to determine the most important domains and Lisa's already gone over that with well-being, knowledge, and empowerment, of family support and how to measure those domains through a survey.
And so, we a tight timeline with the goal of the learning community to develop and test a survey that measures the impact of family-to-family support.
And we were very specific in wanting it to be family-to-family support.
We know that families get support from other places but for this survey, we are looking at family-to-family support for children and families who have children who are Deaf and Hard of Hearing.
So next slide, please.
We tried to make this as, give as much information and make this as easy as possible for participants.
So we had four learning sessions.
And we scheduled those out for 90 minutes each.
And each session, everyone had prework that we reviewed the learning community aim and purpose.
We had discussion and team decision making.
And then talked about next steps, what we would do before the next learning community meeting.
And so, everybody was asked to review questions and provide input.
Everybody's input was greatly appreciated and very much needed.
s next slide, please.
So, what we were looking at with the survey is number of questions because we wanted people to not be overwhelmed with the amount of questions being asked.
It needed to be a realistic amount that wouldn't take forever.
So that's when we looked at the time to complete the surveys.
We needed plain language.
We wanted this survey to be read -- have the -- readability for all participants.
We did ask some demographic and content questions and then for purposes of our learning community, we did ask questions from the testers so that we could receive that feedback and know how to improve at the next stage.
So think as we have already discussed, we did two different testing phases.
And took input after the first phase to make some changes for the survey for the second one.
Ok.
I believe Tara, it's you.
>> Yeah, so we essentially -- can we go back to the other slide?
There you go.
So, we essentially did two PDSA cycles as Alyson described before.
That's the method of quality improvement that we use here.
So, the, with the initial cycle, we targeted to select just a couple of families, two to three families, targeted families, that the family organizations worked closely with already that they would have a good feeling that they
would participate and provide the feedback we were looking for and we tracked how many were completed and took that input.
Rolled over into another PDSA cycle where we expanded a little bit, included eight to 10 families.
Kind of chose those randomly.
And then tracked how the delivery method to send the survey out and how many were completed.
And the feedback that we got back.
>> Go ahead, Cara.
>> Trying to find my video.
Can't find my video.
Ok.
There I am.
We wanted to keep the survey short so that people would complete it as others have said before, so we limited our demographic questions to the ones shown on the screen.
This included where families live, their race, the language used in the home, the relationship to the child and the highest level of education obtained.
'.
>> And so, the final product from the learning community is a question Bank of impact, questions, that correspond to the three constructs of the framework that was introduced by Lisa.
And the idea is that EDHI programs, family-based organizations and family leaders can use these questions to create their own state or territory level surveys, or consider adding these questions to improve an already existing survey.
So as Lisa already discussed, the first construct in the framework is well-being.
And parent-to-parent support contributes to child and parent and family well-being.
The learning parent needs support related to child self determination, participation in goals and the learning parent also needs relational and emotional support, which is provided by the supporting parent.
These are the five final well-being questions.
And all questions use a Licart scale unless otherwise indicated.
There was mixed support throughout the process for either a three point or four-point scale.
When the five point was used, that was adding a neutral or N/A option to the scale of strongly agree, agree, disagree, or strongly disagree.
What we found was that in the five-point scale, some individuals choose to respond neutral across many questions, which didn't provide a very clear response, which is why we ended up going with the four-point scale.
>> So the second construct is the construct of knowledge.
And when you look at the whole thing in, as a whole, you will notice that within the three constructs, knowledge, scaffolds both well-being and the empowerment parts of it.
So, it goes along with the adage, when you know better, you can do better.
Next slide.
There it is.
When -- excuse me, the five questions on this template were designed to discover the parents have received and would be southerned knowledge that helps them to make informed decisions and identify and navigate the various systems and resources that will help them support their child's
growth and development.
And so you can see the questions.
Here.
And based on the results that we got back, you want to go to the next slide, you will see some of the feedback that came from the fifth question, which was an open ended question.
And you can see on this slide some of the common denominators of what their feedback offered.
And I would also like to read a couple of the examples.
They listed that they thought it was most helpful that they got great advice, list of resources, they especially liked that the resources that they received were local to their location.
Others highly valued learning how to become an advocate for their child and that they learned that they are not alone on this journey.
And they appreciated getting new ideas.
So, the feedback that we got from this question were all over the map, which really shows the breadth and depth of the support that families receive from us.
>> The third construct is the empowerment construct and that lacks like competence and confidence.
Next slide.
These are the questions on the survey that relate to empowerment.
Again, these are focusing on the care giver's confidence and competence as it relates to their child.
>> This is Lisa.
Sorry, I can't get my camera to come on.
But some feedback that we received from the users and changes that we made based on that.
So again as a reminder, we did two different PDA cycles and we asked these four questions to the families who participated in the survey.
And so question one was -- was the survey explanation clear?
And if not, what information was missing?
And all the respondents in both tests said that the explanation was clear.
So, we really didn't make many changes to the explanation.
The second question was were the questions easy to understand and family friendly language and if not, to please list the questions that were difficult to understand.
And through the two tests cycles, we did only receive one question that they felt that was not easy to understand.
And that question was originally "I know how to use the information provided to me to support my child."
And so we changed that question for the second test to "I am confident in how to use the information provided to me by the family-to-family support provider" and we felt that was more clear that we were trying to measure the confidence level of the family after receiving family-to-family
support.
The third question was about being able to complete the questionnaire in a timely manner.
So as I mentioned earlier, we really stuck to five questions per each of the constructs, but you also saw that we had six demographic questions and then we had these four additional questions, as well, to ask them about their experience taking the survey.
So overall, that was 15, 21, 25, um, total questions that we ended up in both of our surveys.
And so, the timely manner we had definitely met the goal.
Everyone responded that they were able to complete it in a timely manner.
So we just really feel especially those of us from family-based organization that our parents ourselves, we just feel that the very people we are trying to survey are probably very busy people.
They have a lot going on and we wanted to make sure that the survey is quick and easy for them to complete.
The final question was based on your experience receiving family-to-family support, is there any additional questions you would recommend we include or suggested edits to the current questions?
So although we didn't receive any suggested edits, to the questions that we had, we did make a few on our own.
A couple of examples after we had reviewed the question as few times through the learning community, we had a question that said "I am more prepared to explain my child's hearing level and their needs to family and friends."
And, so we changed that more to "I am more prepared to share my child's needs with family and friends."
And then we had another question where we said "I am comfortable contacting my family-to-family support provider if I have new questions or need additional support" and we changed that question to "I know how to contact my family-to-family support provider" because we
really decided, again, it wasn't comfort that we were trying to measure.
We were trying to measure their competence.
And then finally, another question "my understanding of the different language and communication options for Deaf and Hard of Hearing children has increased" and we changed that to "I have an increased understanding of the different language and communication options for
children who are deaf or hard of hearing" and then the additional questions based on this question number 4 on your slide that we got good feedback from the participants were to consider asking about what types of supports the family received, adding a comment box to where they
could provide input on how to improve the program, asking questions about like how long have you been in the program?
Or, did you receive support in a timely manner.
So I believe in the chat box, you will be -- there will be a pdf of the question bank, as well as we will be posting the question bank from this learning community work on both the NTRC and the FL3 centers' website.
So you will see on that question bank we have our five questions from the constructs that we believe measure impact.
We also have the demographic questions and then we have included additional questions that you might want to consider.
The goal of the learning community was so that we would -- could go through these two test cycles to provide you with some template questions that have already been tested.
But of course we want to also be able to allow you at the state and territory level to develop a survey that meets your program needs as well.
So, and then as was mentioned before, we did add a question about how families received the survey.
Because we are really interested in digging in a little bit more about that.
What's most effective?
E-mailing?
Texting?
Phone call reminders?
And we had a variety of responses of how families receive the survey.
So I'll turn it back over to Anne and Cara to determine how they handled the surveys at the state level.
>> This is Anne.
And like Lisa mentioned with the seven different participants we each had a variety of experiences.
I will be relating experiences here at Minnesota.
So for the first round part of our process was to reach out to a somewhat smaller group of parents.
We have 10 parent guides and we tasked each of the parent guides with finding one participant and if they couldn't identify one participant, we invited them to take the survey themselves, because they are all parents.
And most of them did find a family to take the survey, but we did have a parent guide that took the survey themselves.
And it was just interesting to -- that they offered a different kind of feedback with maybe a little bit deeper and knowing the goals of this group they answered the questions a little bit differently.
That was a happy accident as far as we were concerned.
Round two, we were asked to identify, or to send a survey out to a larger group of families and we also did not send out to the same families so we were able to collect from not only a larger but a fresh batch of families all of them were families that we interact with on a
regular basis.
And we did that so that we were hoping to get a good rate of return, which we did.
We also reached out to families that we have a fairly close relationship with so that we could count on them to answer the questions with the high level of honesty and accountability.
We chose to send our survey out electronically hoping for a higher rate of response than we do with snail mail surveys and that, indeed, was the result.
We got a very high rate of return by sending it out electronically.
And just a little bit about the, I guess I'm going to hold off on my next comment and I'll invite Cara to add in their comments from South Carolina, what their experience was.
>> Here in South Carolina, we used text, e-mails and encouraging phone calls to get parents to participate in the survey.
We chose families who were recent recipients of family-to-family support for the first test run and then we aapplied the same principle that included more families than we did round one for the second survey and the families we chose we hoped would get a quicker turn around.
Here at Beginnings, we are going to keep our current family needs survey in place.
But we will utilize the IFSE survey to collect data on the impact of family-to-family component.
An initial IFSE survey will be sent to the families three months post initial home visit.
And then again at the one-year service mark.
The data collected on well-being, knowledge, and empowerment at the three-month mark will help us check our impact early on.
And the data collected at the one-year mark will provide data for first sounds data collection purposes.
And I think I'm turning this back over to Anne to start talking about our barriers.
>> Hi, this is Anne again.
We are going to be, let's see, we are going to be making some changes to how we send out the survey and also to our survey which up to this point have been pretty much strictly a satisfaction survey which is not untypical but this learning community really helped us to reframe and
analyze the questions that we can add in in order to be more robust in our reporting to this state.
So some of the barriers that we received in the past as far as rate of return we have only been sending them out up until recent history as snail mail and have not experienced a good rate of return even with the families that we have had great connections with.
In recent months we have added in sending out the surveys as a link.
And that has led to a higher rate of return.
And what we are going to be look at very soon is sending the surveys out as a QR code and sending it with our materials and this is something that our Deaf and Hard of Hearing guide program has already started doing with their surveys for their families.
And they have had an excellent rate of return by using the QR code system and sending them via text as well as e-mail.
We are still wanting to include the satisfaction questions that we have had for our surveys.
It helps to inform us a as far as our internal processes, such as the timing of our outreach, whether or not families feel like the information that is shared with them is shared without bias, which is one of our missions.
Making sure that the families feel that they are being heard.
But the outcomes questions from -- that have resulted from this, is going to help us to be more robust in our reporting to the state and to the EDHI system.
So the questions that came out of this learning community are going to help us to identify not just what information is given to families, but whether or not the support they get, that the families get from us, help them to understand how to use the information to work toward
successful incomes for their child.
We also are taking a look at changing not only the format, but the timing of our surveys.
We have typically sent out a survey within a few weeks of our initial contact with families.
And then again at an annual point.
And what we are looking at doing is sending out surveys at milestones.
Ages of transition for example, age 3, 11 or 18, which are school-aged transitions.
Or at a time marker after their enrollment in our program such as after the first year, after three years of enrollment, and that would be a way for us to engage our impact on the families, as well.
>> I think Tara will talk next.
>> I was going to address the barriers from the perspective of South Carolina's team.
The no-brainer for participation in the learning community would be time, you no he, we are all certainly very busy with what we do.
And, um, and Alyson and Lisa and Mandy ran a very tight ship in terms of pulling all together and getting it done.
As you saw on the schedule in the previous slide, in a very short period of time, actually.
So, but, for South Carolina, certainly I can speak for First Sound and Beginnings have been strong partners all along and we all support moving things forward with family support and measuring the impacts that we can have these outcomes to share with stakeholders.
So, we certainly were committed and made the time to do those things.
Time is also a barrier probably for families to participate in the survey.
So, we think going forward is, making sure that families fully understand the value of completing the survey.
So that they don't choose not to do so.
We all get bombarded with surveys for just about everything that we do.
Every transaction online.
You pretty much get a survey wanting, especially if you order anything from amazon, you know you get surveys all the time.
So, making sure that they understand the impact and kind of integrating that into the family support process in that interaction with the families so that they understand and will feel more inclined to fill it out.
Ok?
>> Ok, Mandy, I believe you are up.
>> Ok.
There we go.
I lost the interpreter for a minute.
Ok.
So we did ask not just the families who tested the survey, but we did ask the learning community what they would do different moving forward and there were a couple of strong responses from them.
One was that hearing that we disagree on if a family disagreed about something or strongly disagreed, we want to know more.
How can we improve it if we don't know exactly what the problem was.
They suggested using a comment box so that we can add that in and people could provide a little more information if they felt comfortable with that.
And then, we wanted -- we thought we should reiterate the purpose and the aim that they were looking for family to family support impact throughout the survey.
Not just at the beginning, but before every time the constructs were mentioned it would say based on your experiences with family-to-family support, talk to us about or answer these questions based on that experience.
So I think that Cara from South Carolina will talk a little bit more about other lessons learned.
>> That's me.
So, at Beginnings, we are committed to improving our programming that we provide and so, this is another tool that we will be using to continually improve our services with families.
We still need to plan a way to use the survey and establish protocols for delivering the survey to families.
We will be sharing the survey at three months, as I said before.
And we will continue to prep parents for participation in EDHI.
It's important to empower parents and care giverrers to know that they are part of the EDHI system and their voice is crucial.
Complete this survey is one way that allows their voice to be heard and to impact change.
We are also looking to further collaborate with South Carolina Hands and Voices to use seniority say to collect data from their organization and their events about their family-to-family support.
We are hoping that this will provide some data which we can use to provide further financial support to South Carolina H. hands and Voices for the important connections they make with families, as well.
Of course we all receive many surveys so it will be important to find a motivator for familys to take the time to complete this one.
So far we thought of innocent sieves such as books for children, or giving an opportunity to win a prize after they completed the survey.
It was really a great process to be able it look at what we do currently and fine tune our work through this targeted survey to measure these outcomes and it was great to be able it work and learn from other states and participate in developing the survey content.
>> And from the perspective of Minnesota, Anne already shared a couple of our lessons learned and how we plan to use this survey in the future like using a QR code to share with the families and in the future, we are still going to ask some satisfaction questions but also pair
them with these impact questions so that we can make sure we are gathering all of that information.
And like Anne mentioned, as well, exploring how and when we send out the surveys in the future and from the perspective of the Minnesota Department of Health and the EDHI program, using this, Hands and Voices using this impact survey will be able it show us the value and the impact
of the work that Minnesota Hands and Voices is doing and we will be able to in turn share that with our fenders such as Hrsa and it will be really impactful in that way, as well.
>> [Alyson] ok.
So the final results, think we have been kind of leading up to this.
Gunner has posted it in the chat.
And Gunner if you will post it once more in the chat so it raises up on everybody's radar.
You have available to you now our final product which are the questions from the constructs that we just discussed and during this webinar, as well as some alternative questions that you may want to consider.
I believe earlier we talked about some things that we didn't end up putting in our survey like degree of hearing loss and questions like that.
That we thought actually might really give the family support organization or the state EDHI program a little bit more information about the families and provide some context around why families answered the way that they did.
But in terms of what we wanted to do in this learning community, was to really focus on getting those five questions per construct nailed down and tested.
These, as I mentioned, so Gunner just reposted in the chat.
They will also be available on the FL3 website and the questions will be -- the questions, the PowerPoint slides and the recording will all be posted in the next three days.
I will stop sharing my screen and we have some time for some Q and A.
Please go ahead and use the question-and-answer box which is at the end, or at the bottom of your screen, to post your questions and then I am going to field the questions to our panelists.
So -- here is one to kick us off.
You have shared great ideas on the various delivery methods.
I am curious if there are other suggestions on how to generate participation in surveys.
We send surveys out to over 200 families .and get less than 20 responses
Who would like to take a crack that the one?
>> This is Lisa.
I will just reiterate what I thought some of the really good suggestions were.
The QR code seems to be working really well.
And especially when you can send it out through a text message.
Think we are seeing really good responses that way.
I think it takes more than one time for someone to see it.
I don't think that was a suggestion we talked about yet.
But really, almost coming up with like a strategic plan of how you are going to deliver your survey.
The ways you are going to deliver it and then kind after time frame like when are you going to first send it?
And then put on a calendar that you will send it a second and possibly a third time.
Because it although seems like once people see something a second or third time, they are more tempted to do it than when they see it just the first time.
But I think families receiving it in a variety of ways, planning out multiple times of sending it out, and then also thinking about either a drawing, they get entered into a drawing for a prize or some kind of motivation, motivator like the books I think somebody mentioned.
It would be also some great steps.
Anybody else have anything else you would suggest?
>> [Alyson] I have could a couple of additions, Lisa.
This is Alyson speaking.
That's always a really, really hard question to answer.
Everybody has different experiences and response rates are unfortunately often quite low.
It's very normal to get a 12 to 15% response rate but it's hard to make generalizations based on that 12 to 15 percent to the rest of your population of families.
So certainly the more families that you have respond to your survey, the more confidence that you feel in generalizing the results to your general family population that you are serving.
A couple of ideas that we shared with the learning community during our sessions are, was to send three reminders.
So three reminders and give people two weeks.
So -- two weeks to complete the survey.
So sometimes you will see surveys that are open for three, four, five weeks.
But, kind of the science behind it is demonstrated that people really, after about two or three weeks, don't respond.
It becomes kind of white noise, those reminders become white noise and people start to ignore them.
So, the three reminders in two weeks, test out what your families are more likely to be responsive to.
Those different delivery methods.
The snail mail, the QR code, the e-mail, the text.
And see what families are most responsive to.
You may even want to mix it up where your first invite is via e-mail.
Your second reminder is via text.
So that's really kind of the beauty of quality improvement is you can see, you can test things before you just say this is what our family seems to respond to.
Anyone else have any other suggestions?
Ok.
So the next question is can you have a family complete a survey while you visiting and have them seal and sign the back?
I think what I am understanding is, you are with a family, let's say you are doing a family support session and you are meeting in a physical space.
Either one on one or maybe in a group setting.
That you are asking the family to complete the survey while they are there?
And then they can close the envelope and sign the back?
I think that's what I am understanding?
I mean I would say sure, you could do that.
I don't know, sometimes people are less likely to respond honestly if they have -- if it's not anonymous.
So you would have to determine what's more important.
Kind of accurate answers or more accurate answers I should say versus knowing who the answers are connected to?
I would think about that.
But you certainly can have people fill out surveys in a physical setting.
In fact, I have done that multiple times and you will see that like when you go to the fair or something like that and they will use those big mass gathering opportunitys to capture people's voices at a specific site.
So you certainly can do that.
I just -- I am kind of getting thrown off about why you would want to know who the individual is.
So -- that's kind of a different -- that's kind of a different topic.
Anybody else want to add anything to that?
>> This is Lisa.
Will just add that I do know that some of our family-based organizations are limited to the number of visits that they can do with families based on the funding that they receive.
So sometimes they do like a closeout visit.
Where they want to make sure the family, you know, has gotten all their questions answered.
They want to make sure that they understand, you know, what other kind of activities other than one on one support does the family-based organization do?
Like events?
So they want to keep them connected so it's almost like an exit visit.
And so that might be a really good time to say, you know, you will be receiving a survey from us and it is so incredibly important for us to get your input.
And, explain to them what you would be doing with that input.
That it helps us, you know, to improve our program, to sustain our funding.
That kind of thing.
So I can see doing it that way but I agree, Alyson, I don't know how honest a family would be if you sat down next to them and was having them complete the survey in your presence.
>> Yeah.
>> Ok.
Let's go to our next question.
So somebody asked how does receiving the QR code on your mobile phone work?
It seems like the link would be easier on a phone rather than a QR code to direct you to a link.
Anybody want to take a crack that the one?
Ok, go ahead, Anne.
>> This is Anne from Minnesota.
We have been using QR codes a fair amount in recent months and the QR codes, once you get used to how they work and the magic of them, it's actually really easy.
So the QR code, you take a picture of it, it takes you automatically to wherever the QR code leads you.
In our case, it takes them to the survey and then the survey actually, when they complete it, gets housed in our Teams file.
In that regard it does show up anonymously, which is a nice thing for us to be able to share with families that when you follow this link, it takes you to the survey site, you complete it.
And it's anonymous.
We -- Alyson, looks like you were going to and I was going to say, so do you send them a text message with the QR code in it?
I think that's part of the question.
Right?
Is if you are already on your phone, you can't really take a picture of your phone with the QR code.
That's what I am gathering from the question.
>> Oh, I get it.
Yep.
We have also -- well we have done it two different ways so far.
And we are still playing around with it.
So, we can send the QR code in an e-mail and the DHH guides have taken the QR code and laminated it and then at the end of their visit, they just show the -- they give the family the QR code as a laminated thing that they can snap an image of and then once they leave, the family
can come plate the survey.
So they aren't -- they are providing the information at the visit, but are not waiting for them to complete the survey before they leave the visit.
>> Thank you, Anne.
>> Is that helpful?
>> Yep, yep.
So I ran into that issue, too.
So I definitely understand why that question arose.
Text seems to make more sense to send a link, but if you are giving them something physical or there's a poster or something like that, where they can scan the QR code through their phone's camera, that's a good delivery method.
Ok.
So the next question is -- have you thought about motivating families to complete the survey by compensating them with a small gift card?
And I think we did talk a little bit about that.
Right?
Things to think about in the future.
So think that is on all of our, you know, on the radar.
Go ahead, Tara.
>> We talked about it, but, the reality of giving every single family a gift card financially might not be feasible.
But what we discussed alternatively was that completing it would be an entry for a drawing for a gift card.
>> Yes.
>> An option.
>> And this is Lisa.
The only other thing I would add is you have to be really careful about scammers.
When you are offering everyone who completes it a gift card.
So, if you are putting it on your social media, not just individual invitations, bots can get a hold of that and you can all of a sudden all of a sudden receive a thousand responses and you are like whoa.
Where did those come from?
Just be cautious about that.
>> It would also have to be conscientious of state laws about gambling.
That applies here.
In South Carolina.
You got to be careful how you word it.
>> That's a good point, Tara.
Ok.
So, someone asked a question, maybe this was mentioned, about how many families responded to this last survey?
I'm not sure if that means in the second PDSA cycle?
Lisa, you probably have that number, tallied and in front of you.
So I will let you answer that.
>> Sure.
So what we found is as you saw, the first cycle we sent it to two to three families per program.
But those, they all reached out to families that they were pretty confident were going to complete it.
And we had 17 respondents.
The second round, we sent it to eight to 10.
And we had 18 respondents.
And that was because these families were just random.
These were not families that we were so confident about.
So that definitely tells us that there will still always be that challenge and we need to think about how we motivate families to complete the survey to get the information that we need.
>> Great.
Thank you.
Ok.
This is a really excellent question, this next one.
And one that came up in our learning community discussions several times.
So -- this is great work and the then, you know, of course praising us for our work, which we will take, we love that.
And then the question is, I like the flexibility of being able to choose questions that are most relevant to my state or territory.
Many but because of that state or territories may not choose the same question.
Is there an effort underway at the system level to choose a standard set of questions from the question bank and have states and territories all answer the same question?
So there's comparison across states or across programs.
So, who would like to take a crack at that one?
I'll maybe Volun-tell Lisa.
>> Ok.
So this is Lisa from the FL3 center perspective.
Right now, there is no requirement that family-based organizations report data specifically to us.
And, so, think it would be really hard to get, you know, full participation for us to be able to see this across states and territories and I really believe this is in the infancy.
As we mentioned, most of the surveys that we received was measuring satisfaction.
So I think it's a great goal to get to.
Being able to some day come to impact questions that we would want all family-based organizations that are part of their EDHI system to ask annually of the families that they serve.
I just don't think that we are there yet.
Think we need to, you know, let family based organizations get some experience under their belt, work, I think this is the first step having some questions that we have already tested as far as impact questions.
And then maybe in a year or two, readdress looking at this at the systems level.
>> Yeah.
And I am going to just chime in there, because I agree with that approach, Lisa, because it's kind of moving it almost into the next larger scale PDSA scale.
I think if you look at the CDC, FSHS data that states submit, those questions have gone through I don't even know how many iterations over the years.
So it really takes a while to hone in how clear the questions are so that everyone responds the same or similarly.
I know we are at time.
There's one more question that I think is really important that we didn't really hit on.
Earlier.
And that is we hear frequently that families have so many people trying to help them that they don't know which person, agency, person/agency is providing that support.
Has any consideration been given to perhaps sending the survey to families with the actual name of the parent-to-parent support person instead of the more general family-to-family support provider?
So, we definitely within a into that.
Who wants to take a crack at this last question?
Ok, go ahead, Anne.
>> This is Anne.
I can just briefly say that's our current system.
When we send out the satisfaction survey, it has a place for us to write in the parent guide's name.
And it also is on our letterhead with our logo.
So we are hoping that we are connecting some dots for the family, recognizing that they are inundated with just so many people that are reaching out to them.
So we try a few different ways to make sure that they know who it is that they are getting it from and we also, the parent guides, when they are talking with the families, let them know there will be something comeing in the mail.
So hopefully all of those pieces put together help them to identify us.
>> Thank you, Anne.
Anybody else want to add to that?
And we didn't add that in this -- in testing this survey, because this was actually testing the survey itself, aened not necessarily directly discussing the specific family-to-family support agency.
But it does make sense to cite that, right?
When you are sending your survey.
But because we were using it across the six different states and one territory, it didn't make sense for us to use it that way.
Ok.
All right.
Let's go ahead and wrap up.
I want to thank all the panelists for joining today and presenting.
I also want to give a big shout out to our captioner and our interpreter and our IT support at NCHAM.
Again, the slides and the survey questions and the recording will be posted on infithearing.org.
If you look in the chat, Gunner has put up a short, think it's like five-question evaluation survey of this webinar as well as, it will help you generate a certificate of attendance if you need that.
Thank you so much for joining today and have a marvelous Thursday.