[Music] good afternoon everyone it's March 19th 2024 and this is the curriculum subcommittee of the Brookline Public Schools uh and uh we'll start if it's okay with approval of the 22724 minutes uh do I have motion thank stepen do I have a second I can I can second it I'll second it um Stephen you vote Yes Natalia yes and I vote Yes as well thank you um I'm going to switch this around just a moment on our agenda we have discussion for data requests for the ninth grade unleveled English we actually are not going to do that today uh and Jody if you're there you want to just give us a quick explan about that it's not that it's not important but she's on the phone and she's gonna join us in a few minutes okay so um I can just share with you that Jody shared with me that um the ninth grade team english team and the uh High School administration would like uh to step back for a moment and I think they're going to look to partner with perhaps um a higher level level group I mean not higher higher education U maybe a college group that might help them think about this and so they aren't quite ready for that input from us so um they've asked that we could postpone it so they wouldn't attend today for that and um we will put it back on the docket sometime later after they've had a chance to to gather some thoughts as well is that okay with everybody okay okay thank you um but hold on to thoughts about that Stephen because I know you've got good thoughts so that's that's good okay I I had put a bunch of materials together for this meeting on that can I just do any I'll just hold them on I'll hold them I'll hold on to them okay thank you just I appreciate that you did that we I just got the word on this I think yesterday last night so that's why you didn't get any more notice um okay Michelle and Matt are you going to run us through the data dashboard we are yeah so um I'm gonna let Matt start yeah and um oh so yeah so thanks for uh having Michelle and I hang out with you this afternoon uh we are really excited about a new partnership that we have with a group called open Architects and what Michelle and I wanted to do as we're in the early stages of rolling this out is we wanted to build in some time for all of you to actually experience uh the tool so you sort of get a feel and sort of a taste of what educators are going to have available for them uh before before um going into the tool Michelle and I also just want to provide a little bit of a context for um how it is that we're using data right now in the district and how it is that open Architects is really going to help us solve for a couple of problems that we're experiencing uh when we're using uh data um as um was great and Michelle and I have absolutely felt this in particularly in the last couple of years that there's this increasing desire and want and engagement around uh data broadly within schools um and we're really seeing um administrators classroom teachers uh Specialists um uh central office staff of course uh really using data to understand the strengths and the needs and the goals not only at that 30,000 foot view at the district level but really sort of um going down at the school level the grade level the classroom level um and the individual student uh level um as well really common as we are building and refining programs for students that were using things like attendance and grade data to help um develop those programs Michelle and I have done a lot of work uh this year uh with csts around um supporting a problem definition and how it is that data really help support that um oftentimes when we're thinking about a problem solving process that that problem definition piece is often times thought of as maybe being the most important part of that process given that all of the interventions and supports that we're going to provide um student is going to be based on that definition um and then finally um when we're thinking about how it is that kids are responding to tier one instruction how does that kids are responding to tier two and tier three supports making sure that we have a database for uh measuring that um and what we know uh from a uh um a decision-making process that when we are using data uh to make decisions they're going to be way more accurate they're going to be way more effective than a process that is more purely clinical in nature um and it can happen sometimes where we are making judgments that's just purely sort of based on hunch or based on clinical judgment and that is prone to bias and error and so we're not necessarily saying hey you can only use data to make decisions but we know when we marry uh data and that clinical judgment that's where we tend to get the very best decisions for kids for programs and it really sort of moves us towards the goals that we have um across thect um often times um and we see this in the Strategic plan uh we'll have reference to mtss or multi-tiered systems of supports um and fr us um using that as a structure within schools is really important for efficiently and effectively using our resources to making sure that we're meeting the needs of every single student uh you really can't have an mtss architecture without really good Data Systems in place because a fundamental part of mtss is that at any given moment we understand how it is that kids are responding to core instruction we understand that how kids are responding to intervention and then we're shifting resources interventions instructions based on how kids are responding to them and so we have some uh good work that we're U moving toward um in that in that space um and of course we have some more work uh to go yeah so we want to recognize um the experience of data in Brookline when I first came all we talked about with the mcast and there were a lot of really concern big concerns so what this is built from is this is a six steps to accepting data that Educators people go through as they're learning the data and one of the things that I think we noticed as we were looking at this is that we've really moved towards that Intrigue and digging in whereas a lot of the conversations early on were about like well the questions are bad um we don't teach it in the format and that's really typical of a of a school district or any organization to go through as they're learning but it was really good to look to see that actually we are hearing more of those conversations around Intrigue and digging in and so when we started to look at these processes that we could bring in with a really good dashboard we got excited and so one thing I'll add too and and I think it's important that we change the relationship that people have with dat I think times I know Michelle and I when we're positioned in talking with teams about information it can feel critical I think sometimes people think if we're coming in to talk about data we're going to use it as an entry point to talk about all the things we're doing wrong or all the things that we need to change and a little bit what we want to demo later is that we can absolutely use this process to celebrate the heck out of the really good work that is doing because we always want to lift up and do more of what is working and then of course it also um introduces entry points for for changing practice and the last thing with that is also if you look at the language of changing practice it's I know what my students aren't understanding I can write lesson plans it's not about me as a person or my pratice my me as an individual whether I'm a good or bad it's knowing through the student lens and the student outcome how to change my practice and that's where ultimately we would want to head and that's where the mtss process supports the work so we're gonna also so I picked this picture out for a reason I was thinking about this I was like if i s through some of our data presentations this is what I would look like because they're boring and so what we want to make sure we're doing in the past we presuppose what you want to hear in our about our data and which data point you want to hear and part of what we're doing today is allowing your access to the data to really see what it can do what we have so that we are having conversations with you um instead of more of a presentation style because again I don't want to look back at school committee tapes and see this on the screen so as we move into our piece today we want to make sure that you know that our data sources have grown immensely um when both I think we talk a lot about when we both first arrived it was like a little bit uncomfortable to talk about putting in a data system the data sources that are um bolded on here are bolded for a reason they are data sources that give us very clear pictures of students so that they're either normed against a national Norm they're number based or if even if it's student survey it's specific data one of the reasons that some of those other pieces aren't dark right now it's not that those won't show up in the open Architects but they're they're not measured against something that's similar we're not looking at St against the same piece and we have this conversation all the time that once we start to move our schools towards some similar practices so that if you go to school a or you go to school B and I know this conversation has come up recently with as we talk about eighth grade recommendations we'll have a better opportunity for students to have had to have these be more comparative but right now we are sticking when we talk about the comparison data to those darkened pieces all of these well not the teacher evaluation but anything for students we are eventually going to be able to load into the system so we are currently working with the company to add all pieces of this data as I sent to you in the email we're only going to look at two of them today and then also it's what we do with it that's really important so the first thing we always do is we do look for celebrations so any data meeting I sit in I think what went well I always always want to know that because data isn't meant to to fog people it's meant to um give us a picture and then identify the dist the existence of problems and Define the nature of the problems so when Matt talked about the problem definition a lot of the conversation we've had this year is about how do we identify the needs of our students especially for instance if a student comes to CST are we looking at the whole student are we looking at individual problems because they got flagged for a math test or an Ela assessment or are we really looking at how do they feel within their Panorama data what is the Dibbles what does it all tell us what does their attendance data tell us and then eventually we want to as we're going through the process we make a plan how are we going to work on this so one of the things we've talked about a bunch of times is looking at the pyramid and saying like we say that X number percentage of students need support sometimes that's the base of the pyramid and we should be looking at tier one so whether it's an intervention or something we have to change in the classroom and then being able to monitor that so to see uh the impact of those interventions so we really want to be able to do it a lot with our data and I think this system is going to provide us that opportunity but we also want you to see what it is we do with it so pass on to Ma so um a challenge that we experience and a challenge that a lot of school districts uh experience is that um it's great that we're collecting a lot of data we have a lot of information but they are living in different places and so that in that previous screen you just saw a bunch of different sources of data those are oftentimes housed in very different platforms right so we have Aspen we have Panorama the devil's data is housed in a an amplified platform um we have canvas for grade data and what can make it really hard are for team to sort of take that whole child perspective and understand what are grades what are attendance how our kids feeling um and all of those sources of data are really really important to come together to understand um what a student speeds U are and so uh with open Architects is sort of meant to address that very real problem that school districts um experience and what they essentially do is they're they're I've used this term sort of they're plugging into our data sources where every evening they're pulling out reports from all of those data sources and then collapsing them into a single dashboard um what you're going to see in a moment um is actually not an example of a collaps dashboard you're going to see you're going to see Diles data you're going to see attendance data separate separated but what they are building out right now and you sort of see an example of this in the corner where all those columns are actually different sources of data so and you'd be able to see at one snapshot how it is that a student um is functioning across multiple domains and a thing a responsibility that Michelle and I feel very real is that if we're going to be asking folks to use data it has to be digested make sure that it's really really meaningful and we have an understanding of what it is that that students need so this is why we're we're excited for it and so um here are actually there's even more dashboards to be available uh for folks but what you're going to experience is how is is the depth uh and the breadth of analyses you can conduct in each of those areas while also giving that overall stamp shots for students so we're going to play around the dibles and and the attendance yeah and just so you know we aren't expecting every Ro group grp to be able to use each of these so the idea is to match the the dashboard with the proper role group so like we will keep a separate attendance dashboard but it may be school secretaries and principls to have that versus the some of the other student large dashboards that Matt showed that had multiple things and that may be just teacher levels so we have that freedom to work within these yeah and and I think what we're excited about flipping one of the outcomes from or work together today is as you're going through the dashboard sort of thinking through okay this is interesting here are the questions that we can pretty easily answer as a result um of open Architects and having a lot of that guide the conversations uh that we're having um and also hearing from teachers and principles and vice principles and basically everyone as they're interfacing with this just having a really good understanding of what we're able to know about Kids U both in the moment and across time yeah so I think we want to move into trying it out yeah yeah so uh we have some the three questions here that we talked oh sorry Suzanne go ahead well so here's what's kind of of interest to me and I I've been talking about this for years is that it works best I think when grade level team y can have conversations about what's what go these three questions what are we celebrating existence or problem and what will we take it's hard to do that as a teacher I think alone it is I mean it's just it's a little overwhelming like you've said so just could you just kind of briefly go over again the opportunities teachers have to work together I I guess I'm most concerned about the K through five group because I think Middle grades and high school certainly that's a great question and what's good is that there are structures in place the most common time is common planning time or CPT might hear that that acronym yeah is a time where teachers come together and as where Michelle and I first got excited um with this is around CST so that teaming structure of it because as as you um play around with it so Envision was what we sort of Envision is a meeting where educators are looking at a common screen and looking at common students and very clearly are being able to identify at the grade level at the class level at the student level where are strengths where our needs where we setting goals and so very much thinking this is going to help facilitate that teaming process to have conversations about kids yeah so we to your point though we're not quite there yet so like Matt and I have been working mainly with the CST and we're starting to work with u we've shared with some principles we've had some principles sit in on our conversations to to determine how we do exactly what you're talking about and with this year's uh Dibbles data so we didn't have this dashboard so teachers were having to work through the Dibbles we did set aside like three specific uh data meetings where at one of the faculty meetings during each at beginning middle and end that's this is what the process we would like to get teachers to where they are doing exactly what you're talking about but you're absolutely right these should be our regular questions as we look at student work as teams I think like when I showed you that six six uh areas of like growth to get to it's new for us so we're starting to get get those practices in place but I wouldn't say like we've we've gotten across the district yet not not yeah right so CS teams are child study teams correct yes yes and those are who do do teachers request those that they have a student is there a particular student or do they go through their whole class at CST that's a really good question I think it's different depending on we've heard all different scenar but typically yes so it is an educator sometimes it is a classroom teacher sometimes a principal will refer an individual student to be presented to the child study team sometimes processes allow two students per meeting to be discussed but typically it's a focus meeting of a student that's being talked about right that's different to me than having a common planning time correct okay no but the goal is what we've learned I think what we've learned is we have more people interested in the data so that they are in more interested in doing this on a more regular basis so that and like our our um like when we did the when we had the ELA the first two data meetings this year so the beginning of year middle of the year uh Matt provided the teachers in the older grades with a similar structure to go through Panorama data so everybody was looking at data at the same time and so teams were looking by grade level at the amplify data the the dibles and other teams were looking at the Panorama so we're we are trying to build that culture like you said to look at this work together with a framework that gets us to supporting students in the classroom okay Stephen and Natalie are you with me on this do you know what I'm talking about a little I mean I think Suzanne I was goingon to say you know in I've had experience with my children in IEP meetings which is like a group and those I I wish like sitting through that being like Oh wouldn't this be wonderful for all kids to have this opportunity but um I guess the the bottom line is like time and all of that so and I mean I I was going to ask what I am struggling with and I'm trying to say is why are we're reviewing this as a curriculum subcommittee but this seems to be looking at data AC you know is it because we're supposed to be focusing on the curriculum educational I'm I'm trying to know what to focus on why do we do this yeah why is it in this subcommittee and not the whole SCH that's where I'm at too actually I would love let's back up a little bit yeah thanks this data in general data helps us determine what we are doing in the classroom so if we see within but we can't look at our data in one dimension so it's we don't make major curriculum decisions just using the mcast we don't make instructional decisions only is in the mcast because it's after the fact it's one number and it's it for you know we need to have more sources so the reason we're bringing this to you is now we have some more sources so as we're making decisions instructionally uh for materials you've asked about for instance uh you've asked about seal materials in curriculum we want to show you the data that leads us towards how we use data to lead us towards those decisions and for what's happening in the classroom and so what we're doing is we wanted to we wanted to model what we do to look at data when we think about these decisions so today's data that you're going to see it's District level it's not individual student it's there's some school level data but it's not individual teacher it's about how we look at the entire picture and like noticing things so Stephens a lot of times you will ask us specifically about some of the skills from the of reading that that we may not be tackling best here we can look at how are our students doing in those specific skills to really say is that true test it and we can bring that to you to say here's why we've made a decision or why we haven't because we have a broad set of data and what what Michelle and I were thinking about the attendance and um the dibles platform so obviously the dibles DAT is going to be a really important metric that we're using to evaluate uh literacy instruction one of the things that we used if a student if the literacy outcomes aren't great are or is attendance potentially an issue of why they haven't responded right so so we without those two data sources could be an example of that itative process that we would go through to evaluate curriculum broadly then also sort of other instructional practices so I think some of your question was why here and not in the whole school committee Oh I thought you meant why here in general I'm sorry there's a little bit of that probably why why even yeah but also why here or not in the full school committee and it it is possible this could go to the full school committee if we think that that's really important to do but one of the things and one of the things we're trying to do in subcommittee is to take deep Dives and it's harder to do that in in the whole committee so um this is kind of give us because we're on this committee this subcommittee an opportunity to take a deeper dive than we might take in the in the whole committee that are we good to sort of move into the practice oh Natalia did you have a question no that was a thumbs up move forward oh good okay so um I'm gonna give me so we're each gonna show you we're gonna Orient you to the dashboards and in doing so I think I'm logged in I think I'm logged in whoops we're gonna Orient you to the dashboard because it's not easy to just I don't have my glasses can I before you get to that um so on your presentation you had the the three things that you're looking for um the celebration the problems and what was the third thing again oh the so the third one we're not going to get to today but just to show you the full look is the first thing we're going to do is look at celebrations and then we're going to look at like what things cause you to pause like where do you apply yeah I think the the place where I got stuck just now in my brain was um like I guess um so the way the way I use dashboards for my own work is I I guess I have um like a success narrative and so I I need to use my dashboards to make sure I'm making progress towards the key indicators that would define My Success so so in something like attendance I would want to say like what is successful attendance look like so that that doesn't really fall into these three um ways of cutting at the dashboard so I think Stephen if we think about it at a district level and we think about the Strategic plan and we think about the data that we need to collect for the Strategic plan around attendance like when we report out we will be using this data so the Strategic plan has measurable outcomes related to attendance um it has measurable outcomes related to Performance and now what we'll be able to do is more easily have more than just the MCS um available to us to discuss student performance in a way that's more meaningful for the committee does that help yes yeah I guess I just I couldn't slot that way of looking at the dashboard into the ways that you'd present to Michelle and I was taking like the big picture look my brain was getting stuck absolutely but when you described what you were talking about I was like actually that fits right in so okay if you take a look at the way the D data and what we're going to do is we're going to each walk you through to orient to the specific data that's there and then we're going to give you some time to to do your own roaming and looking for those like celebrations or something that and something that causes a pause and be able to talk to us while you're doing it so that you have that opportunity to really dig into the type of work that we're doing with this um I a celebration I'm thinking Steph could be that we hit our attendance goals or uh maybe one that the state has set for us um so that that a celebration would be making goals identifying the problems might be not making goals and so then we would go into interventions that's how I think about it yeah so what I what I have here is uh obviously the attendance uh dashboard and so when I presented at a full school committee um recently this is actually what I used to do a lot of that update and so some of these images may be pretty familiar to you so the way that most of the dashboards uh work is that you'll be brought immediately to this summary page and so here we're seeing at the district level some of the key indicators of attendance and so over here this is showing um average attendance rate broken down by grade this is the average percentage of school days in which kids are enrolled here you're seeing that chronic absence rate broken down by grade level as this this kid I think right now the number if students have missed uh 12 days of school they're considered chronically absent it's sort of an interesting period of time right now it's a little volatile so when I presented the full school committee this number was 11.1% right now it's at 10.6% it so this sort of moving all over the place so if you come back in a couple days you might see this number move um over here you see sort of throughout the school year and immediately starts with attendance rate but it's interesting if you were to come over here and you can see when cold and flu season hits that obviously we see increases but now hopefully we're going to be in a period where there're less uh colds less flu and so um historically we see this flattening out over the the warmer months and so in terms of a celebration right when I look at this I see um a pretty noticeable um celebration here around ninth grade and how strong this attendance uh number is um and when I think about so so why why is that the case I know that the ninth grade team the work that they're doing in that building is all geared around belonging making sure that kids are in the space they're in class are supported in the classroom in every single space um if you're ever in the building during a transition you will see Deans and teachers in the hallways making sure that kids are going where they're supposed to be going and all their interactions with kids are built around to be supportive right so part of a program evaluation piece and seeing how strong this um percentage is that's a real celebration and a really strong comment on the work that they're doing what you'll see um this is sort of interesting um is this is bro broken down by by day so again we're looking at the full um District level um we don't see it here but there are opportunities to break it down by school the the um boxes that are shaded um more Darkly indicate unusually high um absenteeism in that day typically happens on the day before a school vacation or immediately afterward and you see that reflected over here as well in these red um lines um or o sodies right before or after a break and then here demographics is where sort of pulling out um uh different student groups right so really easily being able to see um attendance by group in here you're seeing broken down by race uh this red part are the percentage of students um you can't see it just because it's a little small you also see the number of students um who are chronically absent Within These categories and so as I noted during um uh school committee right we're these discrep IES here are mirroring many of the discrepancies we're seeing in our academic outcome data our belonging data and a lot of sources of data right just a continued um area here that we need uh to explore into support and the final and you see these are all filter options that you can move through up here and also down here and then finally here as we're thinking about mtss we're thinking about tiers right our students well supported by tier one instruction our kids needing tier two and tier three supports this is a way broken down by grade level and if if we were could see student level data OBS you're not going to to protect confidentiality if you clicked on any of these buckets You' see the kids in that grade level who are falling in these areas so for example if we were in um at a specific school and during a common planning time meeting we wanted to know which of our kids are chronically absent we would click on this you'd see that full listed names and then down below you would see that platform where you could also see Panorama data Dibbles data and have that whole istic view um for how a student is doing and then s of move into that problem um solving process any questions activated with this dashboard REM me I'm sorry could you remind me again the definition for chronic absence yeah and so really important distinction is so there's no difference between excused or unexcused um and a students who have missed 10% of school days in which they're enrolled right so typically right now if students have been enrolled since the first day of school that numberers 12 um days of school and so for most students who are enrolled in a full school day uh year 10% would be 18 absences over the course of a school year um the big thing is just sometimes people ask there's no difference between excused and unexcused so that's why that graph went up and down exactly right so yeah so there are for example if if a kiddo missed five days because of covid and then then had a family emergency then had the flu right you could in a very non-c concerning way get to 12 absences at this point in time um right now it's typically historically when we see chronic absenteeism uh Peak um at this uh point of time last year we were at 17 and a half% of kids were chronically absent we ended the year at 14 a half percent so right now we're sitting at around 10 and a half percent and so my assumption is that we would land closer to 10% by the end of the year thank you that's helpful yeah so I have put up the dibl uh dashboard and one of the things that Matt and I are spending some time doing right now is working with the group to talk to them about some of the things that aren't so helpful on the dashboard and they're being really helpful and we've asked teachers and we are getting feedback um and that's what what's really awesome as Matt and I can see as people log in and we're seeing tons of like counselors using this and the CST teams and it's not even finished but I do want to give you some uh insight into this dashboard so this is our current dibles and one of the things that I just want to make sure that you recognize is there's some data here that doesn't belong to the district it's the norming data so right here this information here is how we're being normed and so that is the National Norm that is not the District's scores but what's really nice is you have that as a reference point as you look through the rest of the data the other piece of that is um that is normed you know that's the University of Oregon's research with the Iowa so it's a great uh place to use as a benchmark but it is not give a little behind the scenes so Michelle and I we actually had a meeting with them earlier in the week we have another meeting with them on Monday we were able to give them very explicit feedback so for example one example Michelle and I were just talking about for the meeting is that we think it would be helpful in this graph for example is to have our data layered next to it so that we can really quickly see how we're comparing to these averages and so that's a design feature that we're going to be able to give uh that feedback too and they're going to be able to make those refinements so what's really what's really nice about this is that down here what you'll see is you have both our Sy 23 and our sy24 data Benchmark by period so this is all grades oh Stephen before I get too far out just really dumb basic question so we're using open architect for our dashboard interface the the customization of the interface is done via meetings with an open architect liaison consultant architect yes yeah um is that like a an ongoing consultancy that we have is there or is this part of the package part of the package part of the package part of the package um when when do we select open architect as the dashboard I'm just I just want some of like the super Basics because I actually haven't that the real basic is that the state gave us a a dashboard for free over the past two years with mcass and then um the state has a um a a partnership with them to allow us to purchase a other modules so we had already been using it as a free product so it didn't make sense to to move to a different product since we knew this one but we're spending not this is not criticism this is me just trying to understand but we're spending on a consultancy so we can customize the dashboard so it meets our specifications we were just paying for um the ability to so we had access to the mcast and access data and so now what we're paying for is to plug into all of our Data Systems so they can all come together uh into a comprehensive dashboard but another behind the scenes so they are recommended by desie Michelle and I met them at the same time we were looking at are there other companies that do something similar there aren't a ton of places that are necessarily within this educational space of pulling in data um and so finan they a single Source yeah and so financially it sort of made made sense for us um to explore so our contract I think goes for the rest of the school year and then we'll go back together but the contract is not about the Consulting and the reason we have to meet with them is because we have to set up those uh conduits between our data and their data so they do the backend work so backend yeah so they're connecting various databases with with our dashboard so who's doing the front-end work who's customizing the The View so they do as well so we tell them hey we actually don't like how this graph looks you shap change it to somewhere else and like okay they go back so they're customizing with the front and the back end yeah yep okay thanks I just wanted some of the basics on that yeah no and we don't do any work I mean the the most awesome thing is their agreement with Dy they pull data they pull some of our data right from desie they pull from Aspen they pull from amplify we do nothing but give them the the conduit numbers they need or the account to get into it and so it's very little work on our part to do this and this is your one of a two-year um subsidized package that desie is granting us no this is year so that we part of our package was subsidized we as part of our disproportionality we purchased this year's this year that was disproportionality we purchased for this year the rest of it so that we can look at the disproportionality data because this is a way to to flag students or to actually get the right data and then we'll need a new contract with them for next year yeah is the plan to extend the current contract or to yeah yeah based on our early experience okay thanks I didn't mean to hijack I just I didn't know that no um so when we get back into the Dibbles so one of the nice things here is this is all grades last year we gave beginning middle and end of the year assessment this year we gave beginning and middle and this is the the blues are the percentage of students that are at or above Benchmark that is all three grades but the really nice part of this is I can come up here and I can change the grade level so if nothing is selected so you know you're getting all the grades but if I just put grade two in what I have immediately is just grade two data in all of these areas so I have the subtests here and those descri there are descriptions there of what is each subtest what I want to just give you the warning on is on the ran which is the rapid naming test every only students who are of deep concern will get the ran and that's something Diles does it will prompt you to give a particular student the Ran So when you look at this and you see the number of students it is not the whole class it is just certain students everything else those numbers total up to be much larger so here 272 students are at Benchmark so then the other piece also is just like as a celebration here what you'll see here is that in second grade between the beginning and the middle of the year we have significantly increased our um our at and above Benchmark and that is just a celebration of of the work that students are experiencing and going through to get there uh Natalia sorry and I and this is I I'll maybe ask you a different time but the ren it seems like it's only well below or below why is because the students who get identified to take it are students that are at risk don't I mean I just from my experience uh I've seen ran that then says above in the results it's goe so I think the teams made the decision um so the the rant task should be relatively insensitive to training and experience with it so I believe the decision was made that if students had um two occasions in which they did perform at or above Benchmark on ran the decision was that they weren't going to give that task again and so this is a subpopulation of kids who did not perform at Benchmark on the ran and they decided to readminister it to those students I'm confused but it it's not relevant to this conversation maybe no I would love to understand it more but yeah and it's it's it's a practice that we're trying to determine that we doing it the best way yeah the one thing so like sort of the the theory there is if the students performing well on RAM it's highly unlikely that in the future they perform poorly on that task so as a way to save a minute of administration time for students and the and the educator that they'll then just readminister that task for students who previously have not performed at at Benchmark on a task so Natalia those are the 38 students that they suspected are struggling and they gave him that test but so yeah so all kids all kids got it and this is a sub um group of kids for whom there were additional concerns um so they gave that that they readministered that task to them it's on the readministration yes and so the other thing that you can see is also that you can move through um looking at different periods in the year and you can look at different years again like Matt said is that if there if we had the student data the individual student data open to we can click on any bar and look at who the students are um you also can up here play with the filters and you can determine the school you can look specific specifically at a domain so if you wanted to see how they were doing a composite or maze which is the um comprehension you can do multiple uh you can look at different specific uh subtests to look at a student and you might want to do it for multiple reasons you might be think about the instruction you've been working on in the classroom and want to look in those specific areas especially because this is our earliest readers and then the other thing just to tell you and this is like how we were talking about um feedback this table down here is one that Matt and I decided today we were going to give some feedback on because again these are the National numbers they aren't our numbers and one thing we think that would be really helpful is to have the national number next to the Brookline number so that we can do some uh closer comparing without having to do a ton of U competion oh so that's really important to know those are the National num yeah so this one this took us this was one so this is a new dashboard to us but this one and this up here are the natural numbers and so what we want to do is actually have Brook line show in both of those areas so we can look a little bit more so as an example right so down here you see this is uh in grade two right so 65% of students nationally perform at or above this the composite score as compared 74% of students in Brook line so I plan is to give you an opportunity to do some of your own playing in there and be able to ask us some questions while we're here and you can you can indicate when either you've hit your limit or you have questions either way right so we need to log in Y is the goal to get to know the dashboard or to get to know the subject matter or a little both or I think the dash awesome question I think one thing that Michelle and are thinking about is this to get a feel of okay here are the questions for example when we're about to have a presentation right like here are the questions I I'm pretty confident that Michelle or jod or mat can get to pretty quickly and so that it's going to help guide some of those conversation so more of a feel for the dashboard um itself I think is what we're thinking as an outcome for the exercise and if you see anything interesting in there and you want to flag it for us that's great too yep okay so why don't we work backwards a little bit Jody how long will you need how how much time are you giving us to do this right now like I need about five minutes for the S well that's all you need that's all I that's not very much you figure we have questions H oh maybe 10 for questions okay okay okay we got plenty of time then you have plenty of time and if you again if you hit a threshold like we are happy to you we're going to leave this open to you for a few days so that you can spend a little time in it like we weren't planning on shutting it this dashboard was made specifically for you so we can still get the other data right but we we after you shut us down we won't be able to access the dashboard is that what you're saying what I right so what I'm saying going this dashboard will'll leave open for you we have other dashboards we don't have to change permissions on this so we'll leave it open um you know and yep I heard you groan Stephen are you okay why would you suggest that they would shut it down for us'll be we were doing this as a Dem we wanted to see how this went I see had thought about yeah we hadn't thought about like do you want it all the time I mean this is really simp data that you see from us the same data we released but you'd have in the minute data yeah yeah I mean I right I think it would be like I think it would be awesome to have us have like a a basic level of dashboard literacy so that we are bothering you for this kind of information coupled with like a solid expectations management primer where we don't then like try to try to get information feel entitled to ask you for lots of information based on that information but I mean I I think this you know the more informed we are based on the data the better we can be I would think yeah yeah I mean so again when you do benchmarks by domain I'm kind of I'm over here and we say that letter sounds 177% are well be below of our second yeah I noticed that one that so so these yeah so these numbers are concerning right letter sounds and that was mid year uhuh so the 177% of our students do not have have not made Benchmark on letter sounds in second grade so it's actually 20 uh six% yeah if you add that below nine but what's what's interesting about that is then to if you look to the right and you see sort of how did we fall within the national and so Su what Suzanne what would be your next question then about that 26% so you see there's 26% that are are yeah because I mean it's not a blanket 20 6% right it's not so I mean I would start looking of course at at individual students and see what what they have and what they don't have but I mean letter sounds is pretty basic this so this task is so um so dibles actually doesn't have a pure letter sound fluency task so for example a task where you just get a series of letters and you would articulate the phony graphium correspondence what they do is they present it so that end WF stands for nonsense word fluency so you have a series of nonsense words and the student is essentially decoding that word the expectation at this point is for example if a student saw like was like Wu Z they would go W and that would be three correct letter sounds expectation and you're going to see it is eventually they're going to recode that entire word and just say was afterwards and so for us so this is really a decoding um uh task right and I say in general when we're thinking about um from instructional standpoint if anything is sort of Beyond 80% is where we start thinking about tier one instruction and then of course in the moment yeah pulling out those individual students to see what additional practice it me and that could be in the I mean like a lot of times that's things like that are in the classroom it's additional practice at the basics that we do but what I did was I just flipped the the um filter a little bit because a couple of things this is every single student in the second grade so this can include students on this or no this does include all of our El students all of our students on IEPs so depending on their profile might mean we have to dig in a little deeper to see is that 177% representative of students who have a language-based disability or students who have are a level one in in language and that starts to change those numbers and we do have the ability to do that when we have all the students data listed here and that offer to a really powerful thing about uh Dibbles is because we're doing it at three measurement occasions in a year right there could be uh students in that 9% gray area who are below Benchmark so if we clicked on that we would actually see a list of all students who are in that um area then we'd be able to see what the beginning of year score was right so for a student who maybe moved from well below to below that's really meaningful growth um and actually could be a celebration and sort of evidence that we should keep on doing what we're doing versus kids for example who maybe moov down into the blow um uh blow composit score so that that growth indicator is something we're going to have rapid access to okay Sarah has a question then Stephen Sarah go ahead so what you just said was a little bit of part of that mat that I was interested in um was you're you're kind of talking the specifics of how you would use this um as feedback and what you would do with it so then right before you had said something else I didn't quite catch when there's a certain percentage of students that do something then tier one kicks in could you talk about that again yeah so this is sort of when we think about mtss we think about multi-tier systems of supports often times we when we operationalize so what does it mean for tier one to be effective sometimes the the level that we use with 80% of students in a given area are meeting the best benchmarks that' be evidence that that tier one approach is supporting the vast majority of students a lot of that is conceptual as you think about allocation of resources within a system and that when you need to allocate tier two and tier three interventions to more than 20% of kids that's work can be sort of a strain on the system so some of that is is um is conceptual but that could be sort of a gold standard if you will Benchmark that if you're pretty consistently in your areas having 80% of your kids across demographic groups meeting different um thresholds for performance that's a good indicator that tier one is working so it's a good indicator that tier one is working but then is there like I was just curious overall with all of this is if there's something that automatically kicks in that's a change in practice because of some sort of percentage is met or isn't met or is is this just about setting goals and keeping track of whether those goals get met without having anything change in practice so I think that I think it's a really good question and I think like Sarah one of the things that I noticed so I brought up kindergarten just to look more specifically at an example and one of the things that we see in kindergarten is right around here when we're looking at pheic awareness like I see 55% of students below or well below or 50 60 55% of students below or well below my first question is going to be actually that's too many students students like that's not an intervention that's a classroom issue and so I'm going to look at my instruction to your point it's not that what we want to get into the practice of is not something automatically kicks in an intervention automatically kicks in what I want to look at this is say where what have I done in this so far in my class is this number a surprise to me or do I have evidence that students were either getting there or not and then if if I have taught a lot of it and my evidence doesn't match this there's a couple of things I'm going to look back at who these students are and that's going to be part of my regular classroom instruction in addition I'm going to look at benchmarking to make sure that I am assessing it as I go along but I think the other piece is um if it's a surprise to me then I might need to be teaching it differently and that's when I saw I can go back to like materials I have the supports I have to determine that so it's it's what we're not looking for automatically saying all students need other people to be doing things or other interventions and when you say I you you meant that you yourself classroom teacher so so it could also be that the classroom teacher looks at this and is like oh I was really prioritizing that math project for that week exactly so I know that I'm gonna get better at this by March and so I'm just gonna be okay y exactly remember that skills develop by the end of the year we're looking to make that end of the year and to see the growth for students exactly uh stepen um yeah is there do you come up with data for 23 24 beginning of the year to pull any up so uh 22 23 24 oh not for kindergarten because the kindergarten window so the state requires us to do um two assessments a year that we that we are informed on requires to suggest three for kindergarten we determined that doing it right in the first month of school if if many of them hadn't been in any other formal education that was not the time to assess but we did it in 2223 right and we learned from it interesting okay we learned that it was like we didn't get enough from it to make it worthwhile because we're also not looking to put kids in immediately into intervention who hadn't been an early intervention really it didn't give us like a baseline to see how we were doing by the midline I think uh I mean you know here's the thing one of the reasons that we identified this assessment um well okay so there's that's that's sort of interesting you look at the middle year middle year beginning you know it's something to revisit yeah that's something to revisit yeah so Stephen I think that's and that was I think people were um going so I think the concern was from a developmental perspective were there some quote unquote false positive were there some kids who are performing poorly just because they're five and doing this kind of task can can be challenging versus that competing demand and it's so nice to have that growth indicator from the fall to the winter and so I know there were some um assessments that the kindergarten teams did do around letter sounds to have some source of data s kids are coming in but I think that's a thing that we'll we'll revisit I think we need to revisit and here what you're going to see is like ran everybody all right so yeah we're learning from your diving in some some interesting information for ourselves it was honestly curiosity if if you if you think that it's not giving you good information then I defer to you it's year two so actually what you just asked is a great thing for us to reflect on in one of the Bas we want other people looking at this with us okay Rel question looking at it I would question it I'm also assuming that the the beginning midline and end of year those are all um normed to that point of the Year too right it's not the same thing three times okay thanks thanks St Natalia so I mean I'm I'm struggling because I'm curious about details on the work but I want to sort of step back at the dashboard so Trends really matter and I see the trend for you know on the national like I'm curious who who do you expect to use these dashboards is it the classroom teacher is it the principal you know and and what are they really what are you encouraging them to focus on my challenge with dashboards is that there's so much information and it's important to highlight like we really are focused on on TR you said you know that we expect under 20% and if a classroom is doing worse you know there has to be a reason like maybe it is that there's a high number of children foring Learners or you know but who is responsible you know I feel like on with something like this some teachers may be really Keen to play around and some may not but what is the expectation yeah yeah I think we're we're we're we're building that as we go one thing that we're really excited about it's a question that Michelle and I've been kicking around a lot is that we imagine there's going to be different dashboards for different people in different views for different folks right so the dashboard that's going to be really helpful for a principal you're probably spot on it's going to be a very different dashboard that's helpful for a third grade teacher and so part of what we're we're going be doing throughout the spring is getting this um in people's hands and then having them experience and say oh this actually like and we we've done some of this where people have been very honest and saying I can't imagine myself using this part of it in my practice but I can't imagine it using this part um within my practice so that's the one thing we're excited about the relationship that there is is that customization because we know needs are going to be different I think your point is well taken about overload overload is not going to help us move our practice and so we've seen some dashboards we we saw one yesterday that we really like for teachers it's just it that one we weren't able to preview with you today because it wasn't quite ready for us they're they're feeding data into it so I I guess I would say in my experience it's been really helpful when there are one or two folks in the school that spend time on this and then share that with the classroom teachers so I you know it could be maybe a vice principal for the lower grades or it could be a literacy Specialist or you know a resource seter person who gets into this but my experience has been that it's really been helpful for classroom teachers when there's somebody who leads the conversation and and can point the trends and say let's talk about this and this because it does tend to be overwhelming and you could you know we're going down in the weeds in some areas and maybe those aren't the the weeds we should be in we should be in other weeds or something you know um so to have all this information I I think again is a you really need to think we need to think about um you know how is it shared and who shares it and and What Becomes of it um well I think that's why also Jody's point at the and Michelle's pointed at the beginning about using the Strategic plan as the master narrative at the beginning to which this reports out is a really good one that it cuts through a lot of the the the noisiness although I think this is a really elegant dashboard this feels like a pretty easy to read dashboard to me but it helps you just present out on on the it helps you visualize the data that needs to be visual visualized so you tell the story that you're trying to tell in the district like this is what a successful District looks like and these are the key indicators towards that story and if it takes away like our time that we spend because you don't have access to the individual student data but when those when that's part of this if you can click in a particular domain and say there's an intervention group because they have some similar uh challenges and it actually makes sense that's really quick instead of having to like look through each student and their their profile you have it like right at your fingertips so there there's some stuff we'll be able to do to help with that Natalia yeah just um a word of caution and I I worry that I fall into your category of uh you know I liked your six domains of acceptance but really depends on you know we have to be confident that this benchmarking this this test is both um done in a consistent manner that the teachers know how to do it that this really is telling us what we want to know because the other risk is basically that you overemphasize some random you know metric because you have the metric and it's not and it misses like what you're actually so like that confidence that Dibbles is really an appropriate assessment and that our teachers are because I I understand it's the classroom teachers that do it with the students there's a lot of you know I I you know a lot of variation and not you know on how you do it or the kid you know just a little bit like I I worry with dashboard sometimes because we don't ask those questions enough to make sure that you know they're this is but that's your respons you know I trust you no no you're so so I'm just I'm laughing because talk about SL so this is only gonna be as good as the data that's going into it right right and so we we've had conversations around how are we going to work consistently for example enter data in Aspen around attendance right so that we can feel really confident around what the data are showing us right we've even it was actually been helpful with the dashboards is we're seeing some Curiosities that's helped us ask some questions around data entry um which has been helpful uh for us more specifically with the dibles I think last year uh Michelle and the crew were incredibly thoughtful about the roll out and doing a lot of like our goal this year just administer it and administer with Fidelity and now this year moving into some additional utilization of it exactly because of what you're were talking about now and and one of the things that we've done to talk about the the implementation of the assessment is there was both a reading specialist and a teacher working in every classroom this year to ensure like alignment so that um because I know there's been a lot of conversation in the past about the vas being subjective and it was there's lots of it is subjective and I that was one thing that we intentionally looked for when we went after our the universal screener this is required by the state and when we looked at what we wanted one of the things was is the reliability because exactly what you said subjectivity like Matt said is only as good as what you put in okay other thoughts or questions on this what what I might suggest is is it Poss that you could leave our access open for a while so that I don't know that we're GNA get to it I mean because when I get into it I like get down into it so you know I would need more than five minutes to do that um we'll leave it will we have access for a while with this yeah unless we are told when can't so I I would suggest that we actually play around with it a little bit kind of get a feel for it maybe write down our questions or comments or send them off sure to you if that's okay and um then we kind of we can wrap up on this topic for today does that sound right you want am I want a last word about that I mean we have time for a last word if you want something else okay I mean there's a lot here there's a lot here to go over so that's great okay thank you so so much um I really appreciate that uh Jody I'm going to turn it over to you then excellent thank you so every three years the district is required to write um a plan that because of the student Opportunity Act which was the governmental act that gave more money and the foundational funding to school districts um our three years is up this year um so we have to actually submit a new plan April 1st we'll be bringing this to the full committee on the 28th but that's going to be a very long meeting I'm assuming um so I thought I would give this group kind of a brief overview of what's in it and the reason I can be so brief is we have our strategic plan which is oh also for three years and so the things that we're focusing on we have to focus on um certain subgroups whose performance we want to improve so we're focusing on the subgroups that we've identified in our strategic plan our black and Latina students and the um opportunity Gap that exists between their white and Asian peers um with this plan all of the steps it's really just a rewording and almost a summary of our strategic plan it'll be in your packet for the meeting on the 28th but I wanted to give you a heads up that it was coming look for it look for its consistency with the Strategic plan and um if you have any questions let me know and I'll be prepared to answer them but I'm not anticipating that this will be controversial or something that provokes a lot of discussion because it does so closely mirror the Strategic plan questions thoughts I mean I guess I would just say this is at the state level right Jody this is for the state and so is it a separate form that you've completed it is a separate form you have to um fill it out in gems which is the Grant Management System it doesn't have any money attached to it which is interesting it's just a requirement of that foundational piece so um the budgetary items that I did put in it are the cost of our new reading program and a partnership that we are going to have with new teacher Center to design our mtss system so that it's truly robust for our students um those are the two um big buckets of work that we're highlighting um in the plan how closely does it resemble the Strategic plan I mean how how neatly does the plan map onto this uh this template um I had the Strategic plan open while I wrote it um and literally went side by side and did um even a little bit of cutting and pasting Andy it was it was so convenient I I remember Michelle struggling with the end of the year report um last year for this report and I I almost felt a little guilty because it was so easy to write because we had just done the Strategic plan so is it just is uh it just coincidence that it's the three-year plan and the Strategic plan is the three and there at the same time it is we locked out we locked out okay or I loed out as for writing it right and so the the budget that well you mention but you didn't have to put a budget in with this plan or you did you did so I did um a million5 for the cost of the full roll out of the ELA plan which seems like a huge number saying it out loud um given everything we're going through right now um and um I think it's a around 150,000 for the partnership with new teacher Center over the course of the next three years is that 1.5 million that's over three years or that's next year over three no that's over three years next year's only 500,000 and is that no fully funded in the current budget it is okay okay anything you want us to know that we need to help support when it comes to the full committee no I think I again I think it should look very familiar and feel very comfortable to people um just as a rewarding or sometimes the same wording as our strategic plan any Gabe are you joining us to say something no no I just had to take a phone call from Anthony and I'm back okay I are there any other questions that you guys all have I guess not I think we're good like in [Laughter] general that's up to Suzanne well we have new business this is part of new business if there's something else in new business no stepen sh no no um okay uh any other new business anybody have any other things they want to bring up okay then uh we actually can adjourn a few minutes early if that's okay with people um okay I move to adjourned we have a second second second Stephen yeah yeah Natalia yes and I vote Yes as well thank you very much to our presenters today I really appreciate all that work Matt and uh Michelle that you did thank you so much and we are GNA have fun you can spend your 15 minutes looking at your dashboard thank you bye everyone thank you everyone have a good evening