#095: How We Approach Evidence (Internal and External)

Marisha: Hello there and welcome to the SLP Now podcast, where we share practical therapy tips and ideas for busy speech-language pathologists. Grab your favorite beverage and sit back as we dive into this week's episode.

Marisha: Hello there, and welcome to the SLP Now podcast, your host Marisha, and this month we have Monica Lynn joining us to talk about all things evidence-based practice. So in episode 95, we did a review of the EBP triangle, just some of our initial thoughts. And then for the rest of the month, we are going to be diving into the different parts of the triangle. So without further ado, let's dive in into this week's episode.

Marisha: Let's dive into all things, evidence. So I don't know if I missed this in grad school or just forgot about it, but it wasn't until recently, I will have to confess, that I started thinking about the evidence as internal and external. Which was just like, obviously, but it was a really cool perspective shift for me. And so let's just start talking a little bit about what that could look like.

Marisha: Monica, do you want to share a little bit about how you tackle the side of the triangle and what you think about?

Monica: Yeah, so I think it's probably different for me, because I really like reading research articles and if one of my SLP friends is like, do you know about this? I'm like, hold that thought. I know a little bit, but I will find the perfect research article for you, and it's a side hobby for sure. So that might not be the same for everyone. It's kind of like thinking about where do you get it? Where do you go first when you need external evidence? Do you go to Google and type that in? Google scholar is a great thing for that, if you're just looking for research articles, but I think one of the biggest hurdles is justifying something that's easily digestible. Especially you had a full day of work.

Monica: You're really tired and trying to read this research article and get out of it what you need to might be really difficult. So some of the things that I do to get easier external evidence are I have an informed SLP subscription. That's really easily digestible. They have an audio one that I do the most frequently, so monthly and you can just listen to it, and then they have little summaries of them that are the most relevant. So a lot of times I'll listen to it, their monthly one, and then the ones that are really relevant to me, I'll go back and then just read those articles instead of having to read a bunch. I also subscribed to the Table of Contents notification for journals.

Monica: So the one for ASHA, I believe you can go and you can get an email notification every time a new one comes out. And then that way you can scan through all of the new ones that come out and then just read those relevant ones. So for things that come up the most often, so like the areas that I work with the most with my population, I have a folder on my computer that have those so that I can go back and reference those the most. We talked about the ASHA evidence maps. Those are sorted by topics and then the different areas of the triangle. So you can also... They've got a search bar because if you just click on it, there's lots and lots and lots. There's lots to scroll, but you can always search through those.

Monica: SLP Now also has the academy and research summaries, which is where I really started to look because it is a perfect one-page summary for those research summaries. Especially when I need something really quick. A lot of times I feel like we know what we're doing is working and we've learned about it in grad school, we've heard it about it in the CEU, but I just want to double check, just to make sure. And so something that's one page like that has been perfect. And then the academy has also been really awesome to watch with my interns because sometimes I feel like when I am doing things, like we're doing sessions together, like we're talking about little bits and pieces here and there, but it's nice to have everything together where it's comprehensive and then it's also been really great as a talking point. Especially because I've got new, fresh knowledge in grad school.

Monica: And then I learned a lot from them because they've got something that they've been learning in class to connect altogether. So between all of those, I feel like it helps me to really stay up to date on the external evidence in a way that doesn't feel like I have to go search it out myself all the time. So it's been a little bit easier just because our responsibility as SLPs is to stay up to date on the research and to make sure we're doing that, it just feels so overwhelming to do on top of our jobs, that I think it's been an easier way to do it.

Marisha: Yeah. The Informed SLP has been tremendous because they do 99% of the work for us. They scour all of the journals. They pull out the clinically relevant ones and then they help us with the analysis and not all of the journal articles are accessible for free, but it's really nice because they can do that initial legwork for us.

Marisha: And then if we need more context or if we want to use our own clinical judgment and analysis, we can still go to the source. All we have to do is type in the title and we can find it more easily than trying to dig through and find the right search terms and all of that. So that is a huge help. I also subscribe to the Table of Contents for different journals. In the show notes, we'll link to where you can do that and tell you how to do that. But I think that's another cool way to keep up with new articles that are coming out. And it's nice because in the email they just list the name of the article and the authors, and then you can just click there to open them. And I just have a folder where I automatically save those articles and if I don't have time to read it right away, then I just flag the stuff that applies to my caseload.

Marisha: So then I can go revisit it if I need to. So I think that's super helpful to keep up with the current research. I feel like those two combined will give you everything that you need to know on all of the current research. But a lot of times as SLPs, our caseload changes, we move to different settings. So sometimes it is like starting from scratch. And so definitely as a CF too, like I had just gone through grad school. What we learned in grad school was very different than what we need to apply EVP for our caseload, I feel like. So one thing that has been super helpful for me, because it is overwhelming, whether you're a CF or you've been doing this for 20 years, it's a lot of information. I've definitely gotten down on myself, like Marsha, you should keep up with more of the stuff or do more of the things or you don't know what you're doing with this kid or whatever.

Marisha: And obviously I've got my strong foundation and I do my best to keep up with everything, but I think just to take a step back, I really like filling in a caseload at a glance, that just helps me manage my brain a little better. So basically what it is, is just, I just list it by grade. So I have a section for preschool, kindergarten, first, second. And you just put in whichever grades you're working on. And then I go through my students' goals. So let's say I go through the preschool IEPs and you can make it as general or specific as you want. Like if you put phonological awareness, apraxia of speech, or if you want to put specific goals like categories, whatever they're working on, I just start a list. And then if there's multiple students that have that goal, I just add a little tally. If I'm feeling overwhelmed and it's like, okay, this week, maybe I don't have time to read anything this week, but next week I have a little bit of time where I can read one or two articles.

Marisha: I can go back to that caseload at a glance sheet and be like, oh, well I have 10 kids working on this area that I don't feel as good about. So let me focus my energy there. Then you are a little bit more strategic in how you're spending your time versus just trying to fight fires. And I think that's super helpful. And then when I'm trying to build my skills in a specific area, like if I haven't had a kiddo with apraxia of speech in a couple of years, and I have someone on my caseload, that's one case where I would want to go back to the evidence maps and start looking into that. And so that caseload at a glance just helps prioritize it. So maybe there's only one or two kids who have apraxia of speech. If I don't have a lot of knowledge, I'm going to highlight that in red or something or flag that as higher importance.

Marisha: And so that just helps me decide where I spend my time when things are overwhelming. And then I think the evidence maps are super helpful because we want to think about the quality of the evidence. So the evidence maps include a lot of systematic reviews and meta analysis and randomized control trials. So when we're collecting all of this different type of evidence, it's important to keep that in mind as well, because it can get a little bit disorganized if we're pulling all the different sources. And you said you like to use a folder to organize your articles?

Monica: I do. I keep them... I have a folder on my desktop, but I am a paper journal article reader. I can't do it on the computer. I've tried it on an iPad. I just can't do it. So I have a big magazine holder type thing on my desk that I just have them readily available too that are all highlighted and marked up.

Marisha: Yeah. That's super helpful. I have a couple of different ways that I go through articles. Like The Informed SLP, I actually download the PDFs and drop them to my iPad, and then I annotate and highlight there. And then it syncs with my Google drive. If I'm searching for something I can do that. I don't know if most SLPs would want to do this or would benefit from it, but when I was in undergrad, I did a bunch of research and did my thesis and everything. And so I started using a tool called Mendeley. M-E-N-D-E-L-E-Y. I don't think I've ever paid for it, but it's really cool. And I use it now too, because I do presentations and stuff, so I want to keep track of all of my citations and have a nice database to be able to search.

Marisha: I still use it clinically if I'm like, okay, so I'm working with this type of student. Let me pull up the articles that I've read before and get a little bit of a refresher. But it's a really cool tool if you're into the digital organization, it helps us keep track of those pieces. But I think that's important in talking about how we keep it organized and all of that.

Monica: Yeah, for sure. I will definitely have to look into that.

Marisha: One other thing that I was thinking about. So when I attend seminars or watch an online course, at least the quality courses they'll include a list of references and so that can be a good way to like, if there was a presentation that was really helpful, we can go back to the research and look into that as well.

Monica: I do try to keep all of the PowerPoints from presentations if they have it just kind of accessible because sometimes those CEs are really long and it's hard to remember and digest that information.

Marisha: Especially at a conference because they're doing like 20, well, maybe not 20, but a lot of hours of learning and it's hard to process all that. Is there anything that we need to consider when it comes to the external evidence?

Monica: I definitely think so. I think we've talked about how to maybe be critical of the information that you digest, because it might not be super specific to your populations. You can't just take it from that and then apply it straight to your students, but just using all of your grad school knowledge, making sure that it's high quality research, making sure that you're not taking a single case study and applying it to your whole caseload. Also making sure that it's relevant.

Monica: I think that it's a balance between how relevant it is to your population, how high quality it is, and then sometimes the research may not be available for exactly what you want, especially if it is a rare disorder or really complicated case with a kid that has a lot of different support needs. You might not be able to find exactly what you need, so you might have to rely more on your clinical judgment and more on that internal data as well. So I think that's something that we could talk about right now is the internal data, how you do that and then how you use it.

Marisha: That was the perfect segue. So in terms of the internal data, and I don't know if I'm the only person who does this, but especially as a CF when I was like, I'm not sure if what I'm doing is working.

Marisha: If I wasn't feeling confident, I would pull a progress monitoring tool or assessment or something. Like always an informal thing if it was impromptu like that and just give me some data to make sure that this is actually working. So it is a huge for your assurance to me. I spent a lot of time watching courses and reading books and articles at the very, very beginning to help myself feel a little bit more confident. So in terms of how I approached my data, I attended an intensive with Dr. Strand and it was for apraxia of speech, but she talked about how she approaches data collection. And it's really similar to what I'll talk about here, but it just made so much sense when she was teaching that way. So the idea is we do a quick probe, and a probe is just progress monitoring, like a mini snippet of data where we don't give the student support just to see where they're at.

Marisha: And I tend to not write my goals with support because it's just easier to measure. And obviously it's not black and white and there's some in between there, but that's how I typically like to do my probes, if it makes sense. Given our clinical expertise and the other parts of the triangle too. So what I'll do is the students come in, we've got a routine around it, we collect that quick probe data. I know exactly where they stand with that skill. Like if they've retained anything or just like how they're coming in. And then I use that information to determine what the treatment is going to look like. And it's really nice because I just pull up my probes in the SLP Now app. And then I just take the data, super quick, enter that. And then I get to forget about that, put that away and really focus on being the best therapist that I can be.

Marisha: And so how I use that data, for example, if we're working on WH questions and a student achieves 0% accuracy when responding to who questions, I'm going to back up and do some teaching and do some more structured practice before we dive into some more contextualized practice, like answering who questions about a story. So we're going to back way up. But if they're at 50% accuracy, I might jump into the contextualized practice, but just make sure to give them a visual support or do a quick review before we do that. Or if they're at 80, 90, a hundred percent, that'll change the session significantly. I won't give them any support in context, unless they end up needing it. But that's how I use that, whether it's articulation, language, vocabulary, it only takes a couple of minutes if we're super organized, the students know that that's part of the routine.

Marisha: We get that out of the way and then put on our therapy hats. So that's how I approach it. What about you, Monica?

Monica: I think that I'm really similar and I'm glad you brought up Edythe Strand because I got to go to the same one and was just blown away by it. So if you ever had the chance to see her in person, she's just so dynamic, she is such an amazing speaker. So definitely go, but I definitely have the same thing. So I do something very similar. I do SLP Toolkit and I have to take data for every session because of my district and billing. So for those of us out there that do have to, I will take data qualitative and quantitative, and I only take the data that I'll use. So I'm not... Same thing. I'm not taking data the entire session.

Monica: I'll take a little bit of data, pretty much the same as you, take it in the beginning to kind of see where we're at and then focus on the session. And then that way, I can adjust the supports and visuals and everything like that. And then I can just make a note, I'll make a note after about what type of support they needed, just so I know for the next time where they're at. And then definitely the same. I'm a really big fan of progress monitoring. So just using your therapy materials, or if you use something that's similar, but maybe change the story, but you do it in a different way that you're using for different progress monitoring. Like you could use the cube as a progress monitoring tool, or you could use a lot of the SLP Now materials are set up in the same way.

Monica: So it's like, if you do a story with a comprehension task or whatever, then the next time you use a different story for the same type of task and then you're comparing apples to apples, I think is a good way to do it too. But I think that is the best way to check for progress. So it's like, you've made your goal and then now you need to see if you can adjust it because we're making goals for a year. A lot can happen in a year. So if you're looking at your students and halfway through the year, they've almost met it. If you have been doing that progress monitoring, then you know, you can set up the dreaded amendment and redo a little bit of that IEP. But the opposite I think is also true that if you're doing all this progress monitoring, and then you see that that student isn't making as much progress as you expected, then that way a couple of months in, or a couple of weeks in, you can make adjustments rather than waiting for a little bit longer.

Monica: And it's all going to be based on that internal evidence, that data that you're taking. So it's like if you're not getting a great outcome, you're monitoring that and then you can go back up and then change it based on maybe some external evidence that you kind of have in mind for that student and then make sure a little bit more progress is happening.

Marisha: Yeah. And I loved how you mentioned the qualitative data, because I think that's just as important or maybe even more important than the quantitative. Just because I put away my data binder or my data collection system, that doesn't mean that we're not taking data. We're being super analytical and noticing which types of support does the student benefit from? And I think being able to document, oh, the student really benefits from this verbal cue, and this specific verbal cue, this specific visual, and being able to document that and use that in future sessions and reference that is huge.

Marisha: And then if anyone were to transfer speech therapists or whatnot, then they would be set up for success with that. So I think that is huge. I love what you were saying about keeping an eye on the data to see if things are working. And that's one of the really cool things about digital data. In SLP Now, you set your baseline and then you set the goal accuracy. So if their starting at zero and the goal is to be at 80% in a year, it makes this trendline and then you can see if they're following along on that line. And then it makes it really easy. I mean, you see the data every time you go into a session, but it might be cool every month to look at that and be like, okay, this isn't working for this student.

Marisha: What can we do differently? I guess we do that every session really. We're always thinking about that. That's incredibly powerful. And just making sure that we have a way... It doesn't have to be digital, but having a way to see that progress or lack thereof and being able to respond quickly.

Monica: For sure. And I think that really goes along with talking about how you are the therapy tool, because you are constantly analyzing that as you're going through a session. Which is sometimes why it's hard to explain our job because it looks like a simple therapy activity, but really as we're doing it, we're making so many little micro adjustments as the session goes along that takes an incredible amount of skill.

Marisha: Yeah. I could not agree more.

Marisha: Thanks for listening to the SLP Now podcast. This podcast is part of a course offered for continuing education through speech therapy PD. So yes, you can earn ASHA CEUs for listening to this podcast. If you enjoyed this episode, please share with your SLP friends and don't forget to subscribe to the podcast to get the latest episodes sent directly to you. See you next time.