#114: Evaluations for Culturally & Linguistically Diverse Students: The Why

Speaker 1: Hello there and welcome to the SLP Now Podcast, where we share practical therapy tips and ideas for busy speech language pathologists. Grab your favorite beverage and sit back as we dive into this week's episode.

Speaker 1: Hello there and welcome to the SLP Now Podcast. This month, we are diving into evaluations for culturally and linguistically diverse students. So CLD for short, and this topic was inspired by Kallie Knight. I am so excited to dive this. She's got some really awesome tips for us to navigate these evaluations. And then just a little bit about Kallie. I told you she's got an amazing Instagram account and it's such helpful and inspiring content. I love how she approaches all things SLP. And just a little bit more about her. She is a school based speech language pathologist in Texas. She is in the trenches with us. Maybe she can tell us a little bit more about her caseload. I'm really excited to hear from her. So hello Kallie.

Kallie Knight : Hi. I'm happy to be here

Speaker 1: Before we dive into all of the nitty gritty strategies and content, I thought it'd be helpful to just kind of back up a little bit and chat about why this topic even matters. And I think that will kind of align with what was in your reel on Instagram as well.

Kallie Knight : For sure. And I think understanding the why is going to guide and make everything we would do in addition or differently for these students or clients. It's all just going to make sense when you understand the why. And so, especially for these students, when we look at like a stereotypical evaluation that we would probably do, we're likely going to use a standardized norm-reference tool. Yes, you'll probably use a couple of different informal measures too, if you're able to. But one of the big things we use, whether you're in a school or clinic, lots of people use standardized testing and it's kind of just a necessary evil. Sometimes we have to do it, but what we have to realize, not only for culturally and linguistically diverse students, but even for some of our kind of mainstream, main culture students, there are a lot of issues with norm-reference test, more issues just come with our culturally and linguistically diverse populations.

Kallie Knight : So for example, if we want to go over psychometrics, like we want to go over classification accuracy, sensitivity and specificity, which basically is just how accurate is the test identifying people with disorder or not disorder correctly. And if you were to go into the manual or really look into most of these norm-reference tests, that classification accuracy is actually pretty poor. What we're really looking for to be good is about 80% across the board. And most of our norm-reference tests, even for those that the test is normed for is under 80%. So that's a little bit of a red flag.

Kallie Knight : But on top of that for a lot of our culturally and linguistically diverse students, and that could be someone who's from a different culture, from a different language all together, socioeconomic status, like culture goes far more than just you live in a different place than me. But there's a lot of bias there. There's content bias. There's obviously linguistic bias and I could go into that for forever. But these are just some of the issues that when we have these culturally and linguistically diverse students take a test that was essentially normed on and made for oranges and they're apples, we're not really getting a good comparison.

Speaker 1: Yeah. And just to bring this home a little bit, like how can that hurt our CLD students?

Kallie Knight : I mean, I think it depends who you talk to. Some people might say, well, it's not going to hurt them to be in speech therapy and maybe not. But especially from a school based perspective, when I'm looking only at eligibility or what's going to be best for a student, it's going to be the least restrictive environment. And for a student who does not have a disability, which happens to be a lot of our culturally and linguistically diverse students, that's not special education, which is where we give speech therapy. That nuance looks a little bit different, if you're not in a school setting, but I am. So a lot of what I'll talk about might kind of apply to the school setting. So at the end, they're just not getting the help they need. And just think about those other things, like when you get put into special education, I know most kids love to come to speech, but there can be other sociological factors or like feelings associated with going to special education when they don't even have a disability.

Speaker 1: Yeah, absolutely. Like, I love what you said. Like it won't hurt for them to be in speech therapy. It'll only help. But especially in a school setting, if they're seeing us, they're missing time out in the general education curriculum, which is where they're supposed to be doing their learning. And if they miss out on like a critical component of the math ... I mean, school districts now have a lot of regulations on when students can be pulled, but if they're missing out on, even if it's some other part of the curriculum, like that could lead to some other issues down the road as well.

Kallie Knight : Absolutely.

Speaker 1: Marie Ireland did a presentation at the SLP Summit. If you ever have the opportunity to hear her speak, definitely do that. And we won't go into too much detail here, but she shared like even more consequences in terms of like correlations. Like if students have IEP or the diagnosis and what that looks like long term. So there's impacts that we might not even see, that could come up down the road.

Kallie Knight : Absolutely.

Speaker 1: Okay. Awesome. And then just back channeling a little bit. So in terms of the psychometric properties, I know that some SLPs hear that and they're like, where do I look? Like, what do I look for? So I know it's different for every test, but do you have any suggestions or resources for SLPs who are trying to start looking into that?

Kallie Knight : Yes. So, I mean, obviously if you want to find where it should be, and I say should be because unfortunately not every test reports these kind of psychometric features, and in some cases that should be a red flag in and of itself, but you can find it in the test manual. They should have a section on validity and reliability, and there should be usually a section on sensitivity and specificity and it should outline it. The [ASHER 00:06:41] Leaders Project has a couple of really, really nice in depth reviews on a couple common norm-reference tests, like the PLS and the self. And they'll not only give the sensitivity and specificity, but a lot of other information with like pros and cons.

Kallie Knight : And then I can't remember who made this, but someone or some program or community, someone made a big list where they basically went through all the test manuals and made a list of this is the test. This is the classification accuracy. This is who it's normed on. This is who it's represented in the normative sample. And I can't remember who did it, but I do have access to it. So if there's a way for me to give it to you and get it out to people after this, we can do that.

Speaker 1: I think it's Virginia Department of Education. I think that's part of the research that Marie Ireland was also a part of. But we'll definitely put that in the show notes so that even if it doesn't exist, we'll find a way to have some kind of resource to get us started here.

Kallie Knight : And there's probably more than one document like that, but that makes it easier for SLPs to just grab and go. But I think sometimes it's good to be informed and know if you don't have that quick reference, that you can still find that information and the test [inaudible 00:07:56].

Speaker 1: And let's say a test has like, I don't know how many of them are out there, but let's say a test has 90% sensitivity. Because you said that 80% was the goal for that.

Kallie Knight : Right. That is what is typically considered adequate.

Speaker 1: And then even with 90%, there's 10% who meet the criterion, but may not be. So let's say a test has that 90% sensitivity, but it's normed on, what would be an example of a norm sample?

Kallie Knight : Like a lot of our normative samples have a lot, a lot of people who are monolingual English speakers. So that's one, if we're looking at linguistic diversity. There are some that have some bilingual students, but it's usually Spanish or one or two kids from other language backgrounds. And that's certainly not enough for me to give a fair comparison. Again, like the apples and oranges thing. And on top of that, you need to look at things like socioeconomic status. Like even those kind of things where you're not coming from a completely ...

Kallie Knight : Sorry if you hear my dog squeaking. If you come from a different country or obviously like, oh yeah, you have a different culture, but there's even different cultures from different parts of the United States from different socioeconomic statuses. And most of the time this is, or it should be outlined in the normative sample. And so you're able to go in and look in the test manual and be like, okay, is there a sufficient amount of kids here that represents the student I'm evaluating? And for a lot of our culturally and linguistically diverse kids, it's often no. The answer is no.

Speaker 1: Do you ever use that assessment then?

Kallie Knight : If I wasn't working in a school where it's kind of a necessary evil and they just want to see the score, I would probably skip it all together. However, because it's something I have to use, I can still give it. And when we talk maybe later about dynamic assessment, I can show you some things you can do to still get a little bit of bang from your buck if you're going to give a norm-reference test. But when you report the scores, I just say interpret with caution. And you can explain why, like there was cultural bias, there was linguistic bias and content. And if you want, you can explain some of those or you can just state that. And I often say this person also wasn't represented well in the normative sample. So just interpret with caution. And then you go into the rest of your evaluation that hopefully has a lot of informal information that contextualizes their performance. Because there's a good chance they could have done poorly on the norm-reference test and have a disorder or they could not.

Speaker 1: Perfect. I love how you laid that out. That makes some much sense. Perfect. What are some common mistakes that we might be making? I mean, we talked about some of them, so not considering the normative sample or looking at the psychometric properties of the norm-reference tests, but are there any other common mistakes that you see when evaluating CLD students?

Kallie Knight : I would say the biggest one is probably the tools we use just because people know I need to use my standardized test because that's what we use. And I don't even want to say a mistake because a lot of people just, I think don't know that is the case and that there are better things to use. Or sometimes they know that they should use something better and don't know what it is. And that's when I'm like, you know what? And then you just learn about it, and it's okay, and you don't need to worry about it or feel guilty. But we'll talk about this probably a little bit later if we're talking about informal measures. But sometimes doing things like language sampling and just kind of doing like a static language sample, but not doing any comparative analyses. Just looking at the kid's language sample and analyzing it the same way you would any other kid, a good chunk of the time you might make some errors, but we could go on for a while.

Speaker 1: Okay. Perfect. Anything else that just like general big picture that SLPs should look out for or consider?

Kallie Knight : I think something else that is important to consider is just everyone does come from different cultural backgrounds and we have to realize that sometimes we don't understand everyone's cultural nuances and what is isn't important in their lives and what experiences they haven't had. So that's, I think another one of those ignorance mistakes sometimes. Sometimes learning about other ... In learning about other cultures, we can be more aware, but I think some of the times everyone comes to the table with different experiences, different beliefs, different values. And if we haven't experienced that culture before, there's always mistakes that we can make there. But some of that is just, you have to learn as you go because you don't know what you don't know.

Speaker 1: Yeah. And what do you feel like has helped you learn more about the different cultures? Is it just experience working with different families or ...

Kallie Knight : I would say that's definitely the best way because even if you try to go online and learn about culture, it usually gives you an idea of a general macro culture. And that doesn't mean that complete aligns with that individual family's values. And so I think there is obviously some good, like maybe doing some book learning or learning on the internet about different cultures, but I think where you really learn and get good contextual practice is just learning from the families themselves. And they're usually more than happy to help you, especially when you're like, I'm just going to let you know. I don't know what backgrounds you come from. I don't know what's important to you. So you tell me what's important to me, what your life experiences are like, and I've made a lot of mistakes and I think that's where I've learned. And I think when you're okay with making mistakes and learning from them, that's probably the best thing to do.

Speaker 1: Yeah. Oh, I love that. And I think in general, people really like talking about themselves.

Kallie Knight : Yeah.

Speaker 1: Like if I were to go see a doctor or a therapist or like whatever support I'm seeking, like if they ask questions and try to get to know me, like I think I feel very seen. And I think that really helps establish that relationship. So we build the trust by like being curious and asking those questions, but we also kind of get to continue building on that.

Kallie Knight : Totally goes along with like just doing a good in depth, thorough case history. So we just touched on that, but I guess I didn't say that explicitly. That's another mistake that sometimes people make is not doing any case history at all, or parent interview or something like that. So I'm glad you touched on that.

Speaker 1: Okay, perfect. So let's wrap up just kind of the intro part of this. So we talked about some of the common learning opportunities when we're working on CLD evaluation. So we may not using the correct tools or considering the psychometric properties and norming sample of norm-reference test. We may only take a static language sample and we'll talk more about that in a future episode. And then another learning opportunity is to take time, to understand the culture and then also to do a good client history background, parent interview/all of those things. So we will talk about you alluded to this too, but if we do use a norm-reference, even if the norm population isn't representative of the student, we can use dynamic assessment too. And that's a really great tool. So we'll talk about that in the fourth episode in this series. So stay tuned for all of the good stuff, but yeah, I'm so excited.

Kallie Knight : Me too.

Speaker 1: Thanks for listening to the SLP Now Podcast. If you enjoyed this episode, please share with your SLP friends. And don't forget to subscribe to the podcast to get the latest episode sent directly to you. See you next time.