>> Kaleena Black: Hello, everybody. Welcome to this Library of Congress Literacy Awards webinar. I'm Kaleena Black. I am an Educational Resources Specialist at the Library of Congress. We're really glad that you're joining us today for the first of four panels as part of our Literacy Awards Successful Practices Series. Today's topic is on Serving Adult Literacy Needs. And thank you for being here. Kathy, if you can, go to the next slide. Just a little bit of housekeeping before we get started. We are recording this program, and we'll start the recording as soon as we are able to. And since this event is going to be recorded, any questions or other participant contributions may be made available publicly as part of the Library's archives. So want to keep that in mind. Next slide, please. So, like I said, we're so excited that you're here. You'll have the opportunity to talk to each other and the panelists via chat. And so, to get started with that, we'd love if you could just share your name, where you're joining us from in the chat. And please be sure to make sure that you're sending it to everyone so that we can all see your contributions. Great. It's great to see some folks that are local, folks from all over the country, all over the world. Yeah. Great that you're all here. Thank you. So please keep letting us know where you're joining us from. Again, just a way to let you know that we want you to participate, and we're happy that you're here. Next slide, please, Kathy. Just a little bit of background, the Literacy Awards was originated by David M. Rubenstein in 2013. And it honors organizations that are working to promote and expand literacy and reading in the United States and worldwide. The awards recognize groups that are doing exemplary, innovative and replicable work, and they emphasize and spotlight the need for the global community to come together and striving for universal literacy. I'm going to put the link in case you want to have a look and learn more about the program. I'll put a link in the chat. And up on your screen is a video that I'm also going to link to in the chat about this year's honorees. So you can watch that video to learn about the winners and the honorees this year. And then, on the next slide, this is just a screencap of the page on our site where you can learn more about the organization. So let me just pop those links into the chat really quickly. So, like I said, today's panel is on serving adult literacy needs. And it's going to be facilitated by Dr. Loriene Roy. She is a Library of Congress Literacy Awards advisory member and also a professor in the School of Information at the University of Texas at Austin. And our panelists today represent several of this year's honorees. And they include the Institute for the Study of Adult Literacy, the Learning Source, Seeds of Literacy and VALUEUSA. So, Kathy, you can go to the next slide. Before we jump into our panel and I turn this over to Dr. Roy, we really want to want to hear from you first and kind of just reflect a little bit on why we're all here and just get us thinking a little bit before the program really starts. So, if you could, complete the following sentence. Just fill in the blank in the chat. We would love if you could participate. So literacy makes blank possible. Okay. I see stability. I see reaching one's goals, success, innovations, joy, citizenship, everything, thriving. These are great. It's like you -- so you've been thinking about this. Critical thinking, success, being your best self, connections, the joy of life, empowerment, liberation. These are -- these are so great and inspiring and positive. Health, transformation. Yes. So just, you know, keep adding into the chat to keep reflecting on this. And I think we can also keep this in mind as we're hearing from Dr. Roy and the panelists. I think we'll circle back to this a little bit later. So thank you so much. And next slide, please. With that I am really thrilled to turn this program now over to Dr. Roy and the rest of our panel. >> Loriene Roy: Thanks, Kaleena. Bonjour. Hello. I'm Loriene Roy. I'm Anishinaabe, enrolled on the White Earth Reservation, a member of the Minnesota Chippewa tribe. I'm also a professor in the School of Information at the University of Texas at Austin. As we start our conversation, I want to acknowledge the Indigenous peoples on whose lands each of us is residing. I'm honored to be a member of the Library of Congress Literacy Awards advisory board. And I'm here with four individuals -- well, five; we have two representatives from one group -- whose organizations were selected as a 2021 Library of Congress Literacy Awards Program Successful Practice Honoree. We're going to have a conversation built on our panel theme, Serving Adult Literacy Needs. I'll now pass it over to each of you to introduce yourselves as well. And please share where you're calling from and provide us with some background on your organization. Let's go in order of how people are listed on the program. So we'll start with the Institute for the Study of Adult Literacy. >> Carol Clymer: Hi, everyone, and thank you to the Library of Congress for this -- hosting this and for the award. We're really grateful. So I'm Carol Clymer from -- the codirector of the Institute for the Study of Adult Literacy and the Goodling Institute for Research in Family Literacy. And we're at Penn State University in the College of Education. And I want to briefly tell you about our work and how we are structured and organized. So both of our institutes are highly integrated. What we learn in from and about adult education, we feed into family literacy knowledge and vice versa. We have in both of our institutes the mission to conduct research and to develop lessons and knowledge about evidence-based practice. We use that research to inform professional development. And so we do -- we have several grants from Pennsylvania where we run the professional development in the state for the adult education system. We also use that information to inform direct service. We have both a Career & Family Pathways Program, which means we provide adult education and family literacy services in three counties in the state of Pennsylvania, in central Pennsylvania. And then, finally, we use that -- all of this information to inform policy and advocacy and so to inform those who fund adult education and family literacy about really important practice and things that are important to the delivery of adult education and family literacy services. We work internationally. We're known for our work internationally on these four areas. Next slide, please. So the Institute for the Study of Adult Literacy was established first in 1985 by, of course, a researcher professor in the College of Education. And it's grown, you know, considerably since that time. And it's -- we are totally grant funded. So we raise all of our own funding to do our work. And we've historically worked on career pathways with workforce development, families, health literacy, coalition building, instructional technology, direct service; and we do that for low income and underserved populations. And we've been doing this work a lot -- before some of the things became buzzwords like career pathways and workforce development, many, many years. We provide a very -- a variety of services. As I said, we do a lot of large-scale evaluations. We develop instructional materials and curriculum. We provide technical assistance, professional development. We do customized assessments and evaluations. And we run the help desk for the management system in the state for adult education, national reporting system data that is required. I've included our website link there so you can, you know, learn more about what we do if you open that link. The next slide, please. The Goodling Institute for Research in Family Literacy was established in 2001 by a -- by Congressman William F. Goodling. He's from York, Pennsylvania and his colleagues. He's a US Representative. His colleagues gave him an endowment of funding for his retirement, which he contributed to Penn State. And through this endowment, we've been able to conduct research and evaluation projects, again, the same thing, professional development. We also offer a post-baccalaureate certificate in adult basic education and family literacy through the World Campus at Penn State. And then we do a lot of writing. We do a combination of scholarly articles, guides, research briefs. And then we also are developing print and digital instructional tools for use around the globe. Again, our website is here if you want to learn more about that. Next slide, please. So I wanted to just kind of share just very briefly some of the highlights of our work. You know, we have done adult education research on adult learners, on immigrants, on career pathways, on literacy and health. And one of the things that we really tried to do in our research is make it practitioner and user friendly, make it so that it's readable and you can use it in your everyday practice. We've done a lot of work -- this is our direct service programs, which I mentioned that we have in three different counties. But the -- I want to point out we have a really stellar program, I believe, and I think the prison does, too, at Muncie Women's State Correctional Institution in Lycoming County. It's a peer tutoring program there where, you know, we're working with inmates who have unmet literacy needs. They're trying to get into the GED program or into the vocational education program. And prison officials have said that the peer tutoring is really in some ways changing the environment around the prison because the tutors feel like they are really being useful and helpful. And the people who are getting tutored are learning from other inmates. And then we run a tutoring program that actually gets credit at Penn State, six hours of credit; and we use tutors from Penn State to work in our adult education and Family Literacy Programs. We've historically done a lot of work on workforce development, working with employers in Pennsylvania. And, also, a lot of those resources are on the Pennsylvania Adult Resources website because we provide professional development in the state to all adult education agencies to connect their adult education programs more closely to workforce development programs. And then, finally, we've been doing a lot of work on integrated education and training. We have a library. We have a guide for developing integrated education and training programs. For those of you who aren't familiar with that term, it's a way to accelerate adult education learners who are low basic skills so that they can simultaneously participate in vocational training or occupational training and build their basic or their language skills at the same time. Next slide, please. And then, finally, some brief highlights about our Goodling Institute work. We're currently running a William Penn Foundation Family Literacy Program in the city of Philadelphia. We're doing the evaluation and technical assistance. We have done -- I don't know if any of you have heard about the National Center for Families Learning, Toyota Family Service Learning Initiative. We did a six-year program evaluation, national evaluation of that initiative, helping them to find some positive outcomes of their -- of family service learning. We recently -- because we run a Family Literacy Program, we needed to switch from, you know, face to face to remote education. And we recently published an article in UNESCO on that experience. And I just mention it because I feel like, again, one of the lessons that we have learned in our work is how important Family Literacy Programs have been during the pandemic. As adult education instructors have seen their adult learners who are parents struggling with their children's learning at home, I think they've done more to help parents, especially immigrant parents navigate, you know, school systems and learn about was what was required to help their children with remote learning themselves. We also do evaluation in the prisons related to family literacy and have some publications on fathers that are incarcerated. And then, again, with our professional development and our practical work that we do, we have a number of guides, like how to do interactive literacy. That's parents and children learning together. We also have developed a list that we adapted from some work in Pennsylvania of indicators of program quality for family literacy so that programs can look at their own success in terms of these indicators. And then, finally, we've been trying to really encourage continued funding of family literacy. And so we've been doing a lot of work on what the status of family literacy is in the United States now. And we've written some -- a policy brief on that, which is changing the course of family literacy. So thank you, and I'll turn it over to Loriene. >> Loriene Roy: Thank you, Carol. You really show how your organization interprets this role in serving adult literacy needs. But you also pointed out so many other things that your organization is involved with. You could have been on a panel on workforce development or literacy for those who are incarcerated or health literacy. You also mentioned the National Center for Families Learning. And last year they received the David M. Rubenstein Special Response Award from the Library of Congress. And, in 2017, they received the American Prize. So you're illustrating how the community and family of literacy efforts are connected to each other as well. So thank you so much for starting us off. We're next going to hear from our representatives from The Learning Source, Aimee Callahan and Sarah Secrist. >> Sarah Secrist: Good afternoon, everyone. My name is Sarah Secrist. I am the Chief Operating Officer with The Learning Source, and I'm joined by my colleague, Aimee Callahan, who is our Chief Learning Officer. And we are so thankful to be here and to hear all about your great work and are so grateful to be receiving this award. The Learning Source was established in 1964. We are based out of Denver, Colorado; and we are actually the oldest and largest adult education agency in the state. We started out as a very small volunteer-run program for adults that were studying to get their GED, and now we have over 30 locations across the state. Next slide, please. A little bit about who we are and what we do. The mission of The Learning Source is to promote and provide educational opportunities through collaborative partnerships for adults and families to support their future achievements. So we really rely on our partnerships with other agencies to further our mission. We work with school districts, libraries, community colleges across the state to provide free community classes to adult learners in Colorado. This is a list of our current program offerings, so we provide adult basic education in high school equivalency classes. We have English language acquisition programming. One of our biggest and most successful programs is our Family Literacy Program, and that's at about 15 different schools in the Denver Metro area. We also have a robust student success coaching and career navigation department. They really serve as academic support for our instructional team and also helping our adult learners think about their next step in their educational careers. And then one of our new and most exciting areas of programming which actually Aimee is overseeing is our Integrated Education & Training or IET Career Pathways. So we are in the process of working with the State of Colorado to develop these industry recognized certificate programs to bridge that gap to higher education and get our adult learners into more sustainable career pathways. Very excited about the development of that, and we're hoping to launch some of that programming this coming January. Next slide, please. So a little bit more about who we serve. So we, again, have about 30 locations across the state of Colorado. We started out as a very Denver-based program, and now we sort of span different names. We have partnerships up in Fort Collins with the community college there, down to Pueblo. And we are expanding into more rural communities up in mountain region. We serve about 1500 adult learners and families each year. This is a little bit of a breakdown about who our learners represent. They come from a diverse range of backgrounds. About 32% or more are immigrants representing countries all over the world. Over 50% of our learners are at or below high school level education when they're enrolling in our programs, 56% or more are unemployed or not currently part of the workforce, 70% or more represent low-income individuals. And then many of them have barriers to education, whether that be English language proficiency or low literacy skills. So we're really focused on helping them move up in their skills to achieve, again, better education or employment opportunities for them and their families. Next slide. So -- so some of the ways that we have been able to address these issues with our learners, we have really focused on creating small leveled classes for our adults. We have a team of about 60 paid instructors and over 100 volunteers that are working to support those classes. We also have ongoing professional development opportunities for instructional team. We really want to help them think about their instructional practices and improving the quality of the programs and services we're offering the community. So just this past year, we started offering or creating professional learning communities for our instructors to focus on some of those best practices. And then each of our team members creates a very professionalized -- personalized professional learning plan to focus in on how they can improve their practice and share strategies with their colleagues. We also have -- we use standardized curriculum. So all of our core texts for both our ESL and GED classes are aligned with college and career readiness standards with the state as well as English language proficiency standards. And those are also aligned with the state assessments that we are required to use to demonstrate educational growth for our learners, which ties to our funding. Most of our programs are funded through the Federal Adult Education and Family Literacy Act or AEFLA grant. So that alignment with those assessments is really important. I think one of the things that has come out of the past couple of years is really looking at the format of our program offerings. Prior to 2020, most of our programs were in person. We did have a small distance learning program. Now that we've been able to sort of experiment with different program formats and getting feedback from our learners, we are offering more online hybrid and flexible course offerings to meet the needs of our busy adult learners, many of them who are working or have busy family lives. We're also looking at offering weekend classes moving forward to meet the needs of those that can't meet those weekday classes. We've been very intentional about integrating technology into our classes before we use that more as a supplement. But now all of our learners have access to some sort of online platform where they can practice their English language practice or studying for their GED on their own outside of class to get that extra support. Again, we also have a growing success coach and career navigation department so they can provide the academic support to our instructional team, really focused on that retention and persistence for our adult learners so that they can achieve their educational goals. We are developing our career pathways and, again, bridging that gap to post-secondary education. And we are excited to become a Northstar Digital Literacy site in the new year. This is supported by the Colorado Department of Education. But we know that, in addition to building literacy and language skills, many of our learners need that digital literacy support to really bump them up to that -- their employment education goals. So we will begin offering that this next year. And then this last slide here is a little bit about our outcomes and impact. So we are proud to say that over -- about 50% of our learners are demonstrating a full measurable skill gain on the state assessments, and that is something that is required by our grant. So we've been able to exceed some of those targets each year. Additionally, many more of our learners are demonstrating grade level improvement in reading, math or language and completing their educational goals. We have close to 100 students that complete their GED each year, and we host an annual graduation. Many of those learners are part of our corrections program. We have one of our oldest partnerships is with the city and county jail, and we have classes and the testing center there to help those learners complete that credential and get them on a better pathway moving forward. And then other community program graduates, many go on to post-secondary education and training. And then, in addition to that, we offer both live and online employability skills workshops that are available to all of our 1500 learners to help them, to complement their academic work and have them really think about what their career goals are as they move forward. So thank you again for having us be part of this webinar. We look forward to any questions you have about The Learning Source and applaud you all on your great work as well. >> Loriene Roy: Thank you so much, Sarah. I was really pleased to see that a number of your locations are in public libraries since I teach students whose career goals typically are to work in public libraries. And I'm also a past president of the American Library Association. So we all see the library connection, and I was pleased to see a number of the sites there. Thanks so much. We're going to now hear more from another of our panelists' contributions, and that'll be about Seeds of Literacy. So we're going to hear from Bonnie Entler. >> Bonnie Entler: Good afternoon, everyone. I'm Bonnie Entler. I'm the President and CEO of Seeds of Literacy. Seeds is an award-winning adult literacy program in Cleveland, Ohio, and nationally accredited by ProLiteracy. Next slide, please. I am honored to be here on behalf of our 1000-plus adult students and 250 volunteer tutors. Seeds reaches those in greatest need for the greatest potential. Seeds of Literacy's mission is to provide free personalized education to empower adults to thrive in our community. We accomplish this by offering free one-to-one instruction in basic education and high school equivalency prep to adults. We provide weekly registration, three class sessions per day Monday through Thursday and are open year round. Frequency, flexibility and individual instructions are the key to reaching students most in need of education. Next slide, please. Last year, 88% of our students lived at or below poverty level, 70% were female, 30% are male, 68% of our students are between the age of 24 and 59, 65% were African-American, 45% were White, 17% are Hispanic, 4% are Asian and 3% are others. More than 41% of our students are employed, and almost 50% of our students have dependent children. Last year, Seeds' students came from 29 other states and four other countries, attending classes in our virtual classroom and our two physical locations. You may be surprised by our incoming educational levels that we see everyday here at Seeds: 94% of our students enter with a reading level below ninth grade level; and almost everybody, 99.7% of our students, have math skills below a ninth grade level. And 88% of our students lack computer skills when they start classes here at Seeds. Seeds is different. Seeds reaches those adults left behind, sometimes twice by traditional classrooms as children who did not finish junior high or high school and as adults who do not complete traditional GED or basic education classes. Seeds' model is based on educational research on one-to-one instruction. Next slide, please. One-on-one instruction is the most effective teaching method for nontraditional students. We follow national Common Core State Standards and GED benchmarks in our curriculum and teach computer skills most requested by employers with a national standard certificate. Teaching [inaudible] lies in our one-to-one tutoring method which allows students to work at their own pace and at their own level. Personally assisted by volunteers, tutors, students take the test of Adult Basic Education, TABE, to benchmark both math and reading level. Staff creates individual instructional plans for each student based on the TABE results that match a student's needs with tutor's ability. Staff provides and monitors curriculum, track student progress, adjust methods and curriculums to ensure skill mastery. Students are reassessed to ensure that we're tracking progress effectively. Our students have been students [inaudible] our one-to-one instructional work for our students. Our students have thrived in the last year. Last year, we had 829 new students enroll in Seeds. We had 397 passed GED sections. We had 276 students pass at least one of the GED sections, and 94 students passed all four sections of the GED. Next slide, please. Our numbers continue to grow every year. We probably average about 100 GED each year. Very fortunate to see those numbers kind of grow even as the GED computerized exam in 2014. Unfortunately, here in Northeast Ohio, we have extremely high illiteracy rates here. We are in Cuyahoga County, which is the largest northeast county in Ohio, which has a 42% illiteracy rate. Here in the city of Cleveland, the largest metropolitan area in northeast Ohio, has a 66% illiteracy rate. In some neighborhoods, the literacy rates are as high as 93%. The stats are almost unimaginable and difficult to imagine places in Ohio where parents can't read to help their children with multiplication tables, where jobseekers can't pass employee tests written at 8th grade levels, where grandmother's can't understand the directions on a bottle of hopsner [phonetic]. As we all believe, literacy converts poverty into possibility. Adult literacy is the first step to economic self-sufficiency, better health and academic success for children. Our students are our motivators. Our students' courage to learn helps us to identify new ways to improve learning, increase retention, engage all our stakeholders. We feel that education has offered many challenges and rewards and have been honored to lead Seeds of Literacy in the last 18 years. Thank you for this wonderful recognition, and congratulations to all the other awardees. >> Loriene Roy: Thank you so much, Bonnie, for helping us learn a little bit more about what Seeds of Literacy is all about. I love seeing the Seeds successes on your website, all these great little comments from your graduates. And I am very grateful to all of our groups that are involved with helping people prepare for the GED. Neither of my parents graduated from high school. And my father worked as a janitor and a school bus driver until he had a stroke when he was about 50 and could no longer work. And so my mother then went and got her GED. And then she spent the rest of her working life as a teacher's aide for an Indian Education program. So getting that GED meant a world of difference to our family. Next, we'll hear from Marty Finsterbusch, who's the Executive Director of VALUEUSA. >> Marty Finsterbusch: Loriene, thank you for having VALUE here. So right now you're going to hear about the most unique organization in all of adult ed. VALUE is made up of adult learners, the people who went through the programs as students. We're the only organization nationally run by the students of adult ed. Our board of trustees are people who went through adult ed programs. I myself went through adult literacy program as a student. And so what makes VALUE unique, for over the last 20 years, we look at the -- we look at adult ed through the eyes of the students. While most programs look at adult ed looking down at it or looking at it, we the students look at adult ed by looking up, how far do we have to go, how do we navigate this system? When you look at adult ed that way, it looks totally different than from looking at it and looking down at it. So VALUE is very unique in that capacity. We -- our goal here is to promote effective policies, and we encourage adult learner leadership skills; and that we believe that anyone who ever wants to prove their skills in reading and writing or in English can and should be able to. So that is what VALUE is about. Next slide, please. So what do we do? What does a group of adult learners do? Well, we are a research center. We have been looking at adult ed and student involvement. How do you do that in your program? How do you recruit students? How do you keep retention rate in your program? And so VALUE has become the one organization that collects what adult learners are doing in the United States. And we help local programs, we help state groups and we help national groups that improve their services. So we become that resource, collecting that information and giving back to the field. We also have what we call a Leadership Institute where we bring student leaders and practitioners together every four -- or every other year, and every four year we go to DC and we do leadership skill training and building and learning how to work together. We also help educate legislators and elected officials about who is an adult learner and what does it mean to us from our viewpoint, not from programs' viewpoint but from us, the adult learners. And the other thing that we have done is develop a training that takes administrators and adult learners, bring them together, teaches them how to analyze their own programs, what needs to be improved in that program through the eyes of the students and the program and then teaches critical thinking, organizing skills, diversity training to the adult learner. But, at the same time, it really is a program improvement training. And so we have done this training all around the country, and we're in the process of working with ProLiteracy to have it published so then programs can start doing this training across the United States on their own without VALUE have to going in and during that training. So these are some of the things that we do on a regular basis. Next slide. Okay. So with the next slide, look at adult ed. People don't see adult literacy. You don't say, hmm, that person walking down the street can't read or that person can't spell. You don't see. Our nation doesn't see adult literacy. Writer's role [phonetic] is to bring that awareness to our field. And as some of them, by getting more students involved in the field of adult literacy and making it more comfortable for adult learners to be able to speak out, I'm telling you, when we started 20 years ago, our field, students felt ashamed speaking out about adult literacy. We've been working on changing that image. And so VALUE is also unseen in our field. We've been here for over 20 years, and I guarantee most people in our field hasn't heard about VALUE as said. But our impact is there. And so we have directly impact legislations. I have actually testified before Congress. We've met with legislators. They have -- we have actually changed pieces of legislation, but you don't see that. But the students are there doing that for our field. We've aided other national organization, national campaigns throughout the years. They come to VALUE, and we help them miss the pitfalls reaching out to adult learners, or how to talk to adult learners or how to get them in. And we've been doing that. We also meet the needs of local students. I'd be surprised how many students have called me up to say, my program is not listening to me. I'm getting frustrated. I'm going to leave and then talking with them, how that program works and how the best to navigate that. These are the things that VALUE do. We also do this for local programs when local directors call and say, Hey. I can't get anyone into my program. How do I get students into my program? Guiding them how to reach adult learners in their own communities. This is the things that VALUE does all across, but no one sees that because we're not out there, like, Hey. So we're behind the scene organization, helping improve adult education. And we help state groups. We've helped other national organizations doing campaigns. I can't get into all the things that we've done over the years. By just showing you that we're one of the unique organizations that you don't see. And we believe that the most valuable resource being underutilized in adult ed is the adult learners. And that's because our field hasn't yet seen that the adult learners are our equals in this field. We always see going down to help the students. We're saying, guess what? We're your equals, and we can help you do your job better. We can reach legislators better than this deal can. We can help educate the nation about adult literacy. And so that's what makes VALUE so unique. We're not a direct service provider. We're a provider for the field to make the field better to meet the needs of adult learners. And I'll leave it at that. I threw a lot at you real quick. >> Loriene Roy: All right. Thank you so much, Marty. You mentioned ProLiteracy. And ProLiteracy Worldwide was one of the David M. Rubenstein Prize recipients. They received the prize from the Library of Congress in 2019. I invite the audience and all of us to really take a good look at the websites of these organizations. I looked at your webinar, Marty, on leadership training for people who've been in your program. And that provides advice for anyone who would want to be an advocate for an issue. Really great advice on how to elevate a member of a program but also what happens personally when someone is singled out and how they can work with their family and their community and -- on this change that's coming over them, not just being someone who has acquired more skills in literacy but being that spokesperson. So that leadership training video was really helpful. So thank you, all of you, for providing the background and setting the stage for the conversation that we now can have. Do you have questions that you would like to ask the other panelists, anything that you would like to ask we'll -- from the panelists, we'll go to see if there any questions coming in from the audience. >> Carol Clymer: I'm going to ask Marty this question because I know it's a -- and I'll do this. Sorry. There. Hi. It's Carol. I'm going to ask Marty this question because I know it's something that we're always struggling with, and maybe you have some good tips for everybody. And it's been particularly difficult during the pandemic is that's recruiting students. Do -- you kind of mentioned that's something that you help programs with. But can you talk a little bit about, you know, what we should be doing now where people may be not wanting to, you know, do face to face or they may be reluctant to do hybrid instruction, anything, any tips you might be able to share with us? >> Marty Finsterbusch: What VALUE has learned over the last two years talking with other adult learners, that there is a group of adult learners waiting for our programs to reopen. They don't want to do online. They don't want to. I mean, I hear a lot of programs say, oh, online is going to get so many more people, and it's -- and it's going to, but there's still a core that is sitting back and waiting till this is over. So just know that our program numbers are going to go up. Physical or -- can never disappear. Yes, we can handle more people, and it's going to save programs a lot of money by having it online. But we're hearing from students saying they're waiting till the COVID is over. Now, if you can do a hybrid kind of thing, I think that's the best way to go because adult learners need to learn more technology and get used to and is afraid of it. One of the things that frustrates adult learners who can't read and struggle, people say that, well, just go online and register. Well, how do you go online and register if you can't spell? Well, you can buy it on Amazon. Well, how do you buy something on Amazon when you can't spell Amazon? So what I'm saying to you is you're going to get them in your door and then introduce them to your program online. They'll be more likely to do that once they walk through your door. But we're hearing students are holding back. They're holding back. >> Carol Clymer: Thanks. >> Marty Finsterbusch: I don't know if that's going to help you, but that's what we're hearing from -- 33 No, it does. We have that -- we have -- we have some that are going to more likely to come now because -- especially for our evening remote classes that that's working better. For hybrid, we have some that are coming in person and then on remote. And if they can't come in person, they go remote and vice versa, they kind of switch up. And -- >> Marty Finsterbusch: Are they your more you're advanced students too? >> Carol Clymer: No. They're more of the student -- I think this is true -- more of the students who can't navigate the computer and don't want to be doing all of the, you know, everything online. They want somebody physically to be helping them. >> Marty Finsterbusch: And that's the hybrid. >> Carol Clymer: Yep. >> Loriene Roy: Thank you, Carol. That was a great question. And that kind of leads to me asking everyone on the panel if they can share anything they would like to about how their work has fared during COVID. I know it has impacted even the work that I'm involved with at our university. We've seen a bumper crop of admissions, 850 applications. We generally admit about 100 new students a year. So we've had a huge rise in admissions. Part of it, we no longer require the Graduate Record Exam for our master students. But how has COVID impacted the work that you do? >> Carol Clymer: I don't -- I don't mind going. Our Family Literacy Program, it's been really, really -- I'll just talk about that one because we had to start going totally remote. And it's really -- it's not that easy to do what's called interactive literacy activities remote, you know, where you're trying to, you know, model for the parent to, you know, ask questions when they're reading to their children or, you know, that sort of thing. But we have found really a successful way to, you know, handle the remote situation in our Family Literacy Program is, for example, like, for the interactive literacy, we do a bedtime story. And, you know, the kids come at seven o'clock with their pajamas on and their parents and, you know, the whole, you know, remote learning activity is around that bedtime story. And so we've had to be -- I guess to answer your question specifically, we've had to be really, really creative. It's really pushed the envelope for the instructors to figure out how they could do things that they would do in person online. And I think in a lot of ways it's improved their instruction. In some ways, it's harder for the students, you know. We've -- I think we've experienced what everyone has experienced. We've had some individuals who have, you know, low digital literacy skills, but they don't anymore. It's really, you know, helped them to learn their digital literacy skills and to, you know, do more with technology. I'll stop there, let others talk. >> Loriene Roy: That's a great contribution. I know that I've had to look at ways to provide that social connection. Our students want to be with each other. They love the breakout rooms. And so I've had to be flexible in how much we can cover in every class. And this fall I had a class where we covered more content than I've ever covered in a course. And I had to run to keep up and prepare new material for them because they were demonstrating that they were eager to learn more. So we've heard from the Institute for the Study of Adult Literacy. How about The Learning Source? How has COVID impacted in any way the work that you're doing through The Learning Source? >> Sarah Secrist: I think that we share some of the same observations as the other programs in terms of being able to expand our program offerings and really getting feedback from students about what sort of format works best for them. A lot, as we've sort of moved back to in person and reengage students that had -- were unable to continue with us through the pandemic because of technology issues or that format just didn't work for them, we've heard that -- with our evening classes, which usually have the most interest but sometimes lower retention just because of the difficulty of kind of persistence with an evening class when people are busy working or families that we've kept a lot of those classes remote to accommodate those adults. So we're trying to experiment with different formats, different times of day, again, more intentional integration of technology so that our class work or the work of the students expands beyond the classroom walls so they're getting that direct instruction and interaction in class but then they have the opportunity to do practice outside of the classroom as well. So, again, yeah. A lot of the -- a lot of the same observations and things that other programs shared as well. >> Loriene Roy: Thanks, Sarah. How -- Bonnie, how about Seeds of Literacy? Anything changed as a result of COVID, or what's your experience been like? >> Bonnie Entler: COVID was challenging for everybody. But right before COVID hit, the board of directors and I were working really on kind of a growth plan. And never in our vision would we have ever thought of a virtual classroom. I can't thank my program staff enough. We were actually only shut down about nine days. We -- kind of a trial and error for about six months. We did a lot of recorded lessons. We did a lot of tutoring via phone. We really [inaudible] able to rally the community. We were able to get computers out to students who needed them. And kind of about a six month of trial and error, we were actually kind of able to offer what we're calling now our virtual classroom, which really does mirror our physical classroom. So we have those site coordinators who really are able to match the students and the tutors as they come on. They go into their private breakout room. We digitized their entire curriculum, trained all our volunteer tutors, not only technology wise but working with students that way. It was something that I would have never envisioned that our students would really able -- to be able to step up on to use any kind of technology, device, whether it's their phone or tablet or computer to be able to connect to us. We have been opened, our physical classrooms, since right after Labor Day, back to our 12-hour days we operate on. So we've got morning, afternoon, evening classrooms. And our virtual classroom still is our most popular, even for our beginning readers. So we will continue to keep that as an option as we continue to look for expansion opportunities in our physical sites. >> Loriene Roy: Great. Thanks. >> Marty Finsterbusch: VALUE, for VALUE, the most biggest impact for us is our staff. We get work study students from the community college and local college. And this past year and the other time, we didn't get this. We didn't get the students. When they come in here and we get staff support, which helps us get our website, get newsletters out, corresponds with memberships. So we're teaching adult learners -- I mean, community college students about how to run a nonprofit. They get to see the back end of it. They get to see levels that they would never get to see because I say, Well, here's our budget, boom, boom, boom. We're planning a national event in the Hill. This is how you do. We're planning event; we're planning this training how create these spreadsheets. And so we give exposure to work study students that they would never get anywhere else. And they're learning about adult ed. Well, Covid has dried up that pool. And so we've been missing our support staff in the office, and that's the biggest hindrance for us because we don't have enough manpower here to keep on reaching out and responding to everyone and getting newsletters out and sending out reports and so forth. So that's our biggest hurt for us for Covid is not being able to get college kids back into VALUE so they can learn about adult literacy and use their skills to help us learn. So that's the biggest thing that's been hurting VALUE. >> Loriene Roy: Thanks very much. Thinking of the challenges that each of you has faced and your many, many successes, can you think of a way that you might collaborate with each other? I can see so many levels of skills, so many levels of strengths. And even just spending time again on your websites, you have something to offer everyone. Anyone have anything they'd like to share? I often volunteer my students for activities. In the spring, I'll teach library instruction and information literacy. And we look for clients. We develop pathfinders or guides to literature. We develop lib guides in a kind of a sandbox area and then instruction videos, teaching people how to do things like search in a database. This past year, our client was our new undergraduate program. And our most popular item was just a list that answered questions about what it takes to prepare for a career in librarianship. And this was for our new undergraduates, 18- to 22-year-olds just entering our program on informatics. >> Bonnie Entler: I can highly recommend Marty's program. We had the fortunate opportunity to actually have Marty actually come out, work with some of the Seed staff as well as some of our students. And that has really helped us as an organization, as our program committee is always keeping our students' voice at the forefront. Every decision we make is revolved around what our students think about that. And if any program has that opportunity to bring Marty out, I would highly recommend it. >> Loriene Roy: His material on the website on leadership, that would impact any program and any educational program too. I can see content that would benefit my own students as well. >> Marty Finsterbusch: That training is coming. That's why we're working with ProLiteracy. VALUE cannot keep them running around the country going from program to program around the country. There's not -- we can't keep that up. So we have pilot tested. We cleaned it up, and now it's in the printing process. And we were hoping to have it up at the first of the year, but without the support staff here it's slowing that process down. So within another month or two, it should be on the market. And then any program can take this training and help evaluate your own programs and get your students more involved in your own program to help you meet your needs. And you talked about collaborating. I have to share the Barbara Bush strategic plan are also one of your funders. VALUE's one of the few organizations that are actually named in it. So if you haven't read this yet, or seen this, you might want to look at this plan also that's out there. So you talk about collaborating, I just want to throw that out there. >> Loriene Roy: No, thanks. Great for sharing. And we've had a member of our board very active with the Barbara Bush Foundation and as well as connections, continued connections to that effort. But we all have something to share with each other. And I love the examples that you've given, how your organizations are supporting each other. Anything else anyone would like to mention. >> Carol Clymer: I mean, we have a lot of resources on -- online and, you know, happy to share those plus professional development that we do, both for the state, nationally for local programs. And we do evaluations. I know Marty does that too. But we can do all different kinds of evaluations. I'd also love to know from my colleagues like how they are measuring their outcomes beyond just what we have to put if you get AEFLA funding, federal funding beyond just what you put into the national reporting system. That I think, you know, we -- Marty made a really, really important point. Our programs are invisible in many, many ways to many people and not just to the learners but, you know, to -- even to other programs. I mean, I'm delighted to be on this call today because this is, you know, the first time I've heard of The Learning Source. It's the first time I've heard of the Seeds of Literacy. And I'm really glad to hear about your work, but we don't always have -- we don't have, like, a well-organized field because we get money from different funders to do our work. We measure different outcomes, depending on the funders that, you know, fund our programs. And we go to COABE and we go to ProLiteracy, and we have those networks. But, you know, it's not the same as K-12. And so I -- you know, I really think it's -- it would -- it's important to learn how people are measuring their outcomes and in ways that aren't being mandated by the federal government because that's how we're going to continue to succeed and serve our learners and become more visible I think. >> Loriene Roy: Thanks, Carol. That is a terrific question. And just looking at applying for a Library of Congress Literacy Award, we ask people to respond and provide information about certain criteria: Innovation, sustainability, replicability, measurable results and evidence-based practice. And those two last criteria, measurable results and evidence-based practice, that's where we want to see the measurement, how to measure outcome. Let's see how other folks on the panel are addressing that -- that question, that area about measuring outcome. Learning Source, come back to Sarah. Anything you'd like to contribute about measuring outcome? >> Sarah Secrist: I think this is a question we continue to think about because, as other panelists have mentioned, we have this sort of standard outcomes that we're reporting for our grant. But I really -- I think something that we're looking to grow is really our -- how our students can become advocates for our programs in the field. So I think really interested to learn more about some of these leadership and training opportunities, thinking about how we can get -- grow our arm of advocacy within our program and really have our students be able to share, you know, their long-term successes as a result of the support that they got through our programs. So I think it's an evolving conversation. I don't know that I have the exact answer for those things. But I think those long-term outcomes, that's really what we're trying to work towards with some of this career pathway development and, again, engaging our alumni to be able to speak for the field and how important it is -- the work is that we do, beyond just that educational functioning level. >> Loriene Roy: I love the gathering of stories, what has this meant in someone's life. As we kind of think about learning from each other, it also reminds me of programs that provide funding that actually support or recognize collaborations. The Institute of Museum and Library Services, IMLS, is one. They really like to see how groups collaborate with each other. And so I can envision, you know, grant funding that includes a number of your organizations working towards some aim, some difference. I'm going to ask Bonnie from Seeds of Literacy to add any comments that she would like about the topic on measuring outcome. As the other programs shared, measurements are always a challenge. I think only in our world does anybody know what an educational functional level is. No one in the outside world knows what that is. So we have measured a variety of different things. And we measure all the typical NRS ones like functional levels, skill gains, all of that stuff. But we actually kind of break it down into some things that I think a lot more of our other stakeholders and funders understand a little bit more. We do use our TABE for grade level increases. We do share a lot of our attendance data from the number of people going through orientation and how they progress through the number of classes and hours that they contribute to really help stakeholders understand that our adults are busy and have lots of other barriers, so education is not always a top priority. But they may just step out for a little while, but they may come back six months later. So kind of share a lot of that data as long as -- as well as kind of all our reassessment data, our practice test data, things like that. We also teach students who have very limited reading abilities so someone reading under the third grade level. So we do some fluency assessments, and we kind of share that information with our stakeholders, too, so that they can understand that, you know, someone learning their letter sounds and all of that and progressing to be able to read some sentences, huge progress. But we're always working on sharing those kind of stories with our different stakeholders so they understand that a student may not achieve their [inaudible] in twelve months, but they may learn something new that will be helpful to them throughout their life and make them better community members. >> Loriene Roy: Thanks, Bonnie, so much. Marty, anything you'd like to add about -- >> Marty Finsterbusch: Yes. I'm going to comment on this issue that our field needs to do testing. And it needs to be standardized at some level so comparison. But sometimes a field has gone overboard in that capacity and that doesn't allow certain things. Adult learners, it's not only just about getting reading levels but can you survive? Can you do something that you couldn't do before for yourself, for your family and your kids? A lot of times that's not counted anymore, but that's what the adult learner come in for. So if someone comes in your program and they're 65 years old now and they just want to be able to learn how to read this, we smack them with big tests. We make them do -- jump through all these hoops, and we make all these requirements. So sometimes their testing drowns out what the program is there for, is helping someone get something because, when someone comes into a program, normally there's a need that they need to get met right away, not that we're going to sit you in a program for five years and we're going to test you and test you. So it's our -- there's a fine line here, and we don't address that. But sometimes our -- to get that data, We're creaming out the crops. The lower level students sometimes get left behind in this testing because the testing we have doesn't really meet the needs of that adult learner and it overwhelms them. So as soon as you walk into a program, here's a test. Here's a test that you're going to drown. Well, do I want to come back after that? So we need to look at that whole process constantly because, from the student viewpoint, it's awful getting smacked with a hard test as soon as you walk through a door before you know anyone and says -- and if you're at the lower level, that test is -- overwhelms you that we have. And so I'm just throwing that out there from the student viewpoint. That area is still sense -- it needs to be improved for our field because it's not fun for the adult learner in any way, shape or form. Just throwing that out there. >> Loriene Roy: Yeah. No, that's great. It reminds me that the Institute of Museum and Library Services follows and asks its grant recipients to follow an outcome-based evaluation, that is, they say, tell us how you're going to measure changes in attitudes, changes in behaviors, changes in skills. So it's a different way of counting. I want to now see if we have any questions from the audience. >> Kaleena Black: Great. I'll come back on. Hope you've had a chance to see some of the comments and responses to the panel. I did see one question from Leanne, I believe, was asking about volunteers. So she said, Everyone's mentioned program volunteers. Can each of you talk a bit about how you recruit and train volunteers? >> Loriene Roy: That's a great question. Volunteers. Let's start with -- >> Bonnie Entler: [Inaudible] if you don't mind. >> Loriene Roy: Yes. Go ahead. Oh, Bonnie. Thanks. >> Bonnie Entler: Seeds is about 24 years old. We're getting ready to celebrate our 25th anniversary next fiscal year, but we've always remained a very volunteer-based organization. So we have a pretty small staff. There's only about 20 of us serving 1000 students. But we use volunteers as tutors to every aspect of our program [inaudible] things like that. But tutors have always been the active part of our organization, allowing us to provide that individualized instruction so students are able to work at their own pace and at their own level. So our tutors go through a six-hour training process that we've created. And then they will observe with a current tutor on site before we put them into the classroom. We have tweaked that for our virtual tutors because we know no one can sit for six hours. So we do a kind of two one-hour sessions and add some homework onto that. We train tutors probably about every six months. Our training schedule is on our website. If anybody would like to join us virtually, we'd be happy to share our program with you. >> Loriene Roy: Thanks, Bonnie. Who else would like to say something about recruiting volunteers? >> Sarah Secrist: We, too, are a very volunteer-based organization. We use a lot of the sort of national databases, VolunteerMatch to recruit volunteers, trying to establish partnerships with local colleges and universities. We -- just over the last couple of years have a great partnership with Front Range Community [inaudible] >> Marty Finsterbusch: Lost a connection. >> Loriene Roy: Yeah. She's -- she was saying something great, and we were halfway there. But hopefully she'll come back. Anything else to add about volunteers, how to recruit volunteers? >> Carol Clymer: I just put something in the chat. >> Loriene Roy: Okay, great. And what did you put in the chat, Carol? >> Carol Clymer: I mean, our -- we do have a six-credit course that's in the English Department. And we recruit from all different areas of the University. And it's interesting, you know, we get science and engineering majors and a lot of college of ed majors, and it's a really fantastic program. Of course, it has a very specific training, you know, because we have a syllabus that we use. And it's built off of our training manual that -- that was done by another program in the state for professional development. And then they have a practicum, right, so they come in, tutor with us. And it's a really great way to get tutors. Not everyone has that, you know, luxury but -- of working, you know, with a university or college. I think that's what -- yeah. I forgot the name of the person who just dropped off, but I -- was it Leann? Leann, anyway, from The Learning Source. I think she was just about to say we have a good relationship with a community college. So I would suggest, you know, build relationships with colleges and universities as one tool for recruiting. It is harder to get tutors from the community. Part of that is because in the state of Pennsylvania our tutors have to have a bachelor's degree and so -- if they're going to be doing one-on-one tutoring. And -- yeah/, we can get some people with a bachelor's degree but -- and we go out to all kinds of different community groups to do presentations and, you know, the same things that, you know, do the online tutor recruitment. But it is a little bit different because of the commitment that people have to make. And -- oh, it was -- Kathy was going to say it. I don't know why I said Leann. Sorry. Anyway, I'll let her finish now. So that's a little bit about how we do recruiting. Kathy, you were about to say you have a good relationship with a community college, I think, before you cut off. >> Loriene Roy: That was Sarah, Sarah Secrist. But, yeah. Would you like to finish what you started, Sarah. >> Sarah Secrist: I apologize for that. My internet just all of a sudden went out. Yes. So we have a great partnership with Front Range Community College. They have a TEFL certificate program, so we've been able to serve as sort of a mentorship practicum resource for those students, and that has been a really great partnership to recruit new volunteers, even new staff. So we're looking at trying to kind of establish more of those partnerships to bring in new volunteers and also just share our opportunities with the community. And then we host twice-a-year trainings. We've been looking at trying to create more online trainings to allow volunteers to get involved with our program at various points throughout the year and then connect them to our different sites across the state so. >> Loriene Roy: Great. Thanks. Sarah, I wanted to thank you for mentioning recruiting students at colleges. Our students have to complete what they call the capstone. So each of them has to find a field supervisor willing to supervise some work they're doing during their last semester. So they donate 125 hours toward creating some tangible product. And they're always searching for good hosts, and they do terrific work. I'm going to ask Kaleena if we had any other questions. >> Kaleena Black: So we're getting close to the end, and I want to make sure we have some time for the wrap-up. But I think this last question would be -- I'm forgetting the name of who this came from. I think it was James asked, How would you suggest a small library start an English literacy program or a technology literacy program for struggling adults? It's a good question. >> Loriene Roy: Any advice for this? Sounds like a librarian. >> Bonnie Entler: I know one of your award recipients is the Northstar Digital Literacy thing [phonetic]. We've been using them since they started, and they've been -- incorporate the assessment into our program, and it's been a wonderful resource I would highly recommend.