Explore the latest CodeHS autograder updates in this webinar. Learn how new features streamline grading and provide better feedback for students in CS courses.
Full Transcript
Read the complete transcript of this webinar
Hello, hello, hello everybody.
Happy to have you all here in our autograder updates webinar.
And as you're coming in, I will add to the chat. I'm going to add the slides if you want to kind of follow along. that link should drop you should take you to the slides. but also feel free to put in the chat, kind of introduce yourself, let us know who you are, where you're attending from today.
And, we'll go ahead and get started here in about a minute or so, with the content of it. But yeah, feel free to introduce yourself in the chat. And we're excited you logged in today to learn about the autograder updates with us.
And again, yeah, while y'all are introducing yourself in the chat, I will introduce myself. I met a few of you. My name is Steve. I'm a PD specialist with CodeHS. I am based in Houston, Texas. I taught computer science for 11 years. seven of which I use the code chess platform and super excited about anytime I get to be involved in an autograder PD. I think these updates are amazing. I think they are a lot of super cool capabilities and with me presenting them today is one of our software engineers, Sarah. I'll let Sarah introduce herself briefly before we because she'll be you'll be hearing from her a lot later, but Sarah, if you want to introduce herself before we keep moving on. Hi, I'm Sarah Heden. I'm a software engineer and I've been working at CodeHS for about three years.
awesome Sarah. Yeah, Kim, I will fix the slides link here just in a moment. sorry it did not show up as a link for you, but yeah, we'll get the link fixed, but in the meantime, keep introduce yourself in the chat. we will keep going over kind of our agenda. So, basically what we're going to talk about today and then I will get the slide link to you all in a bit. But we are going to go over what the code's autograders are. We're going to learn how to access the autograders. We'll be talking about the different visibility settings. Lindsay just dropped it in as a link as well. So hopefully that that link should work. U, we'll go over the visibility settings and feedback levels. Those are two of the new big changes we've made to the autograder interface. some cool new features that you can do just to control how your students interact with the autograders. and then the last thing we'll go over is the preset configurations. Those were there before, but we've added a lot of features depending on your language. And we'll also talk a little bit about giving you resources for how you can make your own custom autograders as well. So action-packed 45 minutes here today. that's what we'll be covering. Sarah is going to walk you through all of those. And there again, they're all some super cool updates to features or new features that we've rolled out that just will allow you to have more control over how your students interact with the different autograders on the Kod chess platform. we are going to be using once Sarah starts presenting. feel free to ask any questions in the Q&A portion. we probably won't be checking the chat for questions as much. So again, if you have a question once Sarah starts presenting, feel free to use the Q&A. Lindsay and I will get it answered in there. But yeah, please feel free to drop all questions in there. if you are not a code teacher and you would like to make a code account, codes.comsignup will be where you can make that. I will put that link in the chat and we will get the slides link fixed as well too so you all can have access. But here is a link to sign up for a teacher account if you have not. Oh, looks like it's fixed. Perfect. Kim, thanks for letting us know. So, yeah, if you've not if you don't have a code teacher account, feel free to sign up. If you want to log into your codes teacher account and explore some of the features Sarah's going to be showing you, that might be great as well too. but you also are welcome kind of watch and follow along and check out the autograder features later for yourself as well. and if you are not familiar with codes or this is maybe your first time using it, we're super excited you're here and learning more about our autograder interface and our platform. we are a comprehensive platform for teaching computer science. We have elementary, middle, and high school course offerings. So, we have an entire K to2 web-based curriculum. It's super awesome that our our our curriculum requires no plugins. It's all browser based. Everything you need and your students need is right on the website. We offer a lot of online and offline professional development including this webinar that you all are in right now and we will go over some of the upcoming webinars at the end of the today as well too if you'd like to sign up for any more. and again we have a lot of awesome tools on our platform including what we're going to be talking about today those extensive grading and tracking tools for teachers. Students can code with our curriculum. They can do our activities but we also give you so many resources for grading and progress monitoring. And one of our favorites and our newest features is the autograder updates, which we're excited to be sharing with you today. before I pass it over to Sarah, it seems like the links are good to go and fix, right? So, the slides link we were able to get. Awesome. and again, feel free to put any questions in the Q&A. I am going to pass it over to Sarah from this point on to talk to you about our autograder platform. I'll let you share your screen too, Sarah.
Okay, great. so like Steve just said, we're talking about autograders. if you don't know anything about autograders yet, when the students are completing their code, they'll have a little test cases tab in their editor where they'll be able to easily check whether their code completes the assignment. And generally, they will not be able to submit their code before submitting. So, this is kind of what it looks like. pro users are now able to set some more configur configuration settings on the autograder. So all users are able to edit their autograders but some of the settings like turning the autograders on and off, changing the visibility, changing the feedback level, these are pro features. So when you're on your assignments page, there is an edit autograder button that you can click on to view and edit all of the autograders. So, this little window will pop up and it will show you whether or not the assignment has an autograder. If it does have an autograder, you'll see this little edit autograder button. And if it doesn't have an autograder, then you'll have a button to start creating your autograders from scratch. so, this is the autograder editor. so, you'll see on that little panel on the left there, there are several different autograders. You've got autograder 1, autograder 2, comments, contain spec specific content. and you can turn these off and on and edit the code in order to change your autograders. so if you like you'll see here some of our autograders like the custom autograders are just code. you are writing the code which you can use to analyze the students code and see if it completes the assignment. Now most of our codes created assignments will have some code written for you but we do have other options to configure this code or you could just add autograders yourself. So we have two settings for configuration. We have visibility and we have feedback level. So visibility is whether the details of the autograder are given to your students. So we've heard for some of our assignments that students are very smart, very tricky. They will before writing any code just go in and run the autograders on the starter code. All of them will fail. They'll see exactly what they need to do to pass the autograders and they'll write their code based on the autograders, not based on what the assignment says to do. And this is a way for them to put minimal effort into actually completing the assignment. So if there's a specific assignment that you know that they could only complete half the assignment and complete the autograders, you might want to set the visibility to hidden so that they won't have as much information on what exactly the autograders are looking for. and I would recommend this only if the assignment is very clear in the instructions of what they're actually supposed to do. so you'll see on the left here, this is what the students will usually see. They'll see they need an area of a circle with radius 5. They need an area of a circle with radius 10. but with if you set the visibility to hidden, then they'll just see hidden test. and they'll only be able to see if they passed or failed the test, not necessarily which test they passed or failed or what exactly the autograders are looking for. The second option is feedback level. so with this setting, I think it's very important on to communicate to your students what you expect them to do for an assignment. So the default for all of the autograders is error. That means that if the student does not pass all of the autograders, they are not allowed to submit the assignment. You can also set it to warning or info. Now, warning and info are similar in the fact that the student does not have to pass these autograders in order to submit the assignment. So, for example, if you are setting your assignments to be teacher graded, meaning that you're going to look at their assignments after they've submitted the assignment, they will not actually have to complete the warning and info and you can go in later and set whether or not these are requirements for them to get full credit. so for example, if you want to show your students that they should include comments or make sure they have semicolons after every line of their JavaScript code, maybe you want to set that to warning so that the student will have an indication that this is what's expected of them. But if they miss a semicolon somewhere or they don't include a comment, then they can still submit the assignment and not have to worry about going through and fixing everything before submitting it. So, it's really just what you want your students to have the freedom to do or not do before they submit the assignment. So, we can see here this is what an error feedback level looks like to if they didn't do it, they'll see an X and if they try to go and submit the assignment, it will just say you have errors. You can't submit the assignment with warning. they'll get a little orange icon here that says this is something you should fix, but you can still submit your work. and then the only difference between warning and info is the color of the icon here. With both of these, they'll still be able to submit their work. but you as a teacher, you know, you have the freedom to say like warnings, definitely do these before you submit it or you'll lose points. whereas info is kind of a like nudge like hey you should do this but it's not the end of the world if you don't submit. okay so we're going to talk a little bit about the code review app. So if you go to this autograders tab in the code review app you will see all of your assignments that have autograders or can have autograders. So we can see these first several assignments here have an autograder. and you'll see like these last two, the Etch a Sketch project and the horizontal circles, maybe they're a little bit more open-ended and they don't currently have an autograder, but you have the option here to add an autograder. Okay, so we're going to talk about preset configs. So, there's two different types of autograders. There's the custom autograders which is the code that you write and the preset config autograders and these are different for each language. these are the JavaScript autograder options. You'll see here JavaScript has the most options. but then Java and Python and HTML also have a few and we are working on adding more. If you have a preset config that you want us to make, definitely write in. Maybe we'll make it for you. I'd love to make that for you. but basically this is a way that you can write autograders without having to actually write code. So here you can see the example of the for loop autograder. You just have to switch this on and the students will be required to include a for loop in their code in order for their assignment to pass. And you can see you can even set like the start and the end value like the for loop has to start at zero and end at 10 or whatever is needed for that assignment. The custom autograders have a lot more freedom. you'll see there's this all of them look different for each programming language. Like for JavaScript you're going to be writing JavaScript to write the autograders. For Python you're going to be writing Python to write the autograders. But you'll see like here we have this before run function and then there's an afterrun function. And you can write specific code for like okay this is what the students code looks like before it runs. This is what you know the graphics looks like before it runs if you're working on a graphics assignment. and you can just write very specific things. So this would be the like oh it has to have a circle that is a radius of five. It has to have a function called this very specific name that's defined in the assignment. So generally for a lot of the code HS assignments there will be a custom autograder already there and you'll have the option to add your own or you can duplicate it and edit it and you know add or remove or update the autograders. We do have a lot of resources that could help you to write these autograders. like I said you are writing code. It is very specific to each language. and we have extensive documentation that walks you through how to do that. All right. Do we have any questions? I can also go through the autograders a little bit and kind of show what I'm talking about. so for example, we have this assignment that is for drawing a rainbow on the canvas. and there actually are multiple ways to get to the autogers page. Like even if you just go to the test cases here, there's a little pencil where you can go to edit the autograder settings. So let's say I this is a solution code here. If a student were to miss a semicolon here, they wouldn't be able to submit their code. And maybe you don't really care if they include a semicolon on every single line because for JavaScript you don't have to have semicolons. So it's still going to work. so I could come into the autograder editor and find my semicolons check here and I could say, okay, I want this to be info. Like I want the student to know that they missed a semicolon. but they can still submit it. I don't plan on taking points off. So then they'll just see the info here and they'll still be able to submit their work and continue and they'll get another heads up here that they're missing semicolon. Another thing is for this assignment specifically, the autograders don't check whether the colors are in the right order. It doesn't even check that all of the colors are different. So, I could change this to be orange and I could make all of the stripes orange and it would still pass all of the test cases. So, maybe I want to add a custom check to make sure that my students are actually doing the full assignment and not just making the minimum effort. so I can come here and I can I can see that in this code here. there's some like helper functions that might be useful to me. So, I'm going to duplicate it instead of adding a whole new one. And then once I add this new autograder, you'll see like this one here, this top one, it says the autograder files are not editable. So, that means that this is a code created autograder. And we don't really let you edit those autograders directly. I could just turn this off and then edit the autograder myself if I only wanted it to be my custom autograders, but I'm just going to add an autograder here. So, I'll just get rid of some of this code that I don't want. and then I'll say, okay, this one's kind of similar. An element exists with parameters. I'm going to say I want the color to be red. So, for the first item here, I want the color to be red. And I can change the description. The first stripe should be red. and then I'll save this. And if I go and refresh now, I should see that I have this additional check here, and I won't be able to submit my code because I don't have the first stripe to be read. But now maybe you're thinking, "Okay, well what if the student put purple on the left side and red on the right side?" Like maybe I still want them to be able to submit it. then I can come into the autograder editor and say, "Okay, I'm going to make this a warning." like I want them to make sure they're actually having all of the colors. but not necessarily you know requiring the first stripe to be read. Or I could say, okay, there's a picture in the assignment. I know they need to do it this exact way. I'm going to make this test case hidden. because the instructions give them a very clear explanation of what they should do. and if they run this and they don't pass this, they're just trying to put in minimal effort, make all the stripes the same color, then they have to go back and they know, oh, I'm not doing this correctly. Let me just do what the assignment says and then I'll be able to pass all the test cases. I can also show that code review app right here. You can go to autograders and this will show all of the assignments. it does take a second to load, but then if I have a specific assignment that I'm looking for, I can search for it by name or by like unit number. So, I can see, okay, this assignment has an autograder. This assignment does not have an autograder. you'll also notice here that we have feedback level mixed. so a lot of our autograders have several different sections here. and if I don't want to go in and look at the details of what autograder does what, I just know I want all of the autograders to be warning so the students can submit their best effort. Then here I can just say, okay, I want all of them to be warning. I want all of them to be visible. and then I can see here that they should all be I guess here will show it will just set all of the autograders to warning invisible so that I don't have to go in and read each autograder specifically. All right. Did we have any questions? We had one and I answered it but you can speak more to too about the if you are editing the autograder the code the language that you're editing in the languages for the autograder are typically the same language as the exercise correct
yes correct
so yeah if you want to edit a Python one you'll be if you're editing a Python autograder it'd be Python coding and those links that we put in the chat with the Autograder guide in the knowledge base have all sorts of steps for you to be able to add your own code if you'd like to and kind of play around with custom autograders. And if you have any other questions while we have an autograder expert here on the call, feel free to ask them. or if there's anything else you'd like to see or features you you anything you were wondering about using autograders in your classroom or editing the autograders. feel free to ask any questions you may have Sarah, we had a question about if the autograder doesn't require them to pass all tests if it's teacher graded. so if it's teacher graded, they will still be required to pass all of the autograders before they submit it, but they don't have to submit it in order for you to grade it, if that makes sense. So yeah, the teacher grading section doesn't really affect what the student sees in their editor. It's really just it affects the score. So, even if a student was able to submit it, that doesn't mean that they're going to get a five out of five per perfect score when they click submit and continue. it just means that if it's autograded, they'll get that score immediately. And then if it's teacher graded, then you have to go in and give them that score afterwards. Awesome. Thanks, Sarah. if you have any other questions, feel free to ask or anything else you'd like to see. while we're talking autograder updates. and one thing that I like about the updates that Sarah has up on her screen right now is all those preset configurations I think are super cool because a lot of the stuff that you may think you may oh I'm going to have to code this myself as Sarah pointed out there are so many awesome things that you can turn on. I like the function just being able to have them make a function and have it you can you can even say the name of the function that they have to put. I think it's a super awesome way to just automatically level up your autograders without needing to know how to do a lot of the code.
Yeah, definitely. Like if you want it to have a specific function name, you can type in the name. You can say if there's any specific content that needs to be inside of the function, you can say, "Oh, after they define the function, they need to call it three times." and really for most of these, there is a lot of configuration that you can do without writing any code. awesome. If you have any other questions about the autograde interface, any of the features, I see the one question that in the Q&A now, which I'll I'll type an answer out to, but if there's any other questions on general interface before we wrap up Here. Awesome. yeah, we can go to the wrap-up slides then and Paul, I can we can try to answer your question, too. but yeah, I'll share for the wrap-up slides and then if you have any other questions, feel free to put in the Q&A and we'll we'll get those answered before we depart. so again, thank you Sarah for walking us through all the awesome features. Did want to highlight, again, just in case you missed it, that if you want to learn how to do, and I wanted to click into one and show you, this is just what the autograder guides look like. So, if you are interested in learning how to write your own autograders, the Java ones, here's an example. It shows you kind of how to set it up and it'll just kind of walk you through some code you can try adding. So, it's pretty it's pretty straightforward and like it might feel like a lot of coding at first, but once you get the hang of it, it's something you might be able to it's kind of fun to play around with. So, if you wanted to check these out, we again those links are in the chat and in the slides for whatever language you're looking to kind of add an autograder for. again, we have the autograders here for you to check out. of course, we always have a lot of other resources and how you can get involved in codes. becoming a cod certified educator, earning codes micro credentials, we have a cod Facebook educators group and of course we have a lot of social media accounts for you to follow as well. [snorts] before we get to the feedback survey, I did want to put we have a link in the chat. I'm going to drop it in the same way you found this link today. codhs.comfreepd would be where you can find really all of our other upcoming free PDs as well. So, if you want to check that out, again, that link should should bring you to codehs.comfreed. We have a lot of great upcoming webinars as well. I will add the link and again, we'll answer the outstanding questions in the Q&A, too, but we do have a link to a webinar survey. first of all, thank you to Sarah for hopping on and walking us through the awesome autograder features that are there. feel free to fill out the webinar survey. Let us know any lingering questions. again, you can add those to the Q&A and we'll we'll answer them before we log off officially. but again, if you feel free to let us know any feedback on how we did today and I will also drop in a link that you can click to get your certificate of completion before we wrap up. That link should, if you click that link, as long as you're logged into CodeHS, you'll get an email with a link to a certificate of completion for your your attendance today. So again, feel free to fill out the survey, let us know how we did. Click that link to get into the autograders. And again, these are our big upcoming webinars that I'll highlight. We have a lot of for CSED week. We have a lot of Hour of Code and Hour of AI activities. We have webinars coming up for both CodeHub, elementary as well as high school, middle school on Hour of AI and Hour of Code. If you would like to bring any of those activities to your school, feel free to attend those webinars. you can also sign up and get the recording. Again, you can sign up at codehs codes.comfrepd. but those are the upcoming ones next week. Kind of our our big hour of code and hour of AI webinars for you as we look for some activities to do during computer science education week. And as always, if you're interested in learning more or bringing us for a visit, codes.com learn more. You can get in touch. And again, thank you so much for attending. We'll answer the questions that are still in the Q&A. and otherwise, we will wrap up here. And thank you so much for attending. And so if you have any questions, feel free to put them into the Q&A. I'm going to get to the one that's in there, but if there's any other questions, feel free to add them to the Q&A. And if not, have a great day and thank you so much for joining us.
Happy to have you all here in our autograder updates webinar.
And as you're coming in, I will add to the chat. I'm going to add the slides if you want to kind of follow along. that link should drop you should take you to the slides. but also feel free to put in the chat, kind of introduce yourself, let us know who you are, where you're attending from today.
And, we'll go ahead and get started here in about a minute or so, with the content of it. But yeah, feel free to introduce yourself in the chat. And we're excited you logged in today to learn about the autograder updates with us.
And again, yeah, while y'all are introducing yourself in the chat, I will introduce myself. I met a few of you. My name is Steve. I'm a PD specialist with CodeHS. I am based in Houston, Texas. I taught computer science for 11 years. seven of which I use the code chess platform and super excited about anytime I get to be involved in an autograder PD. I think these updates are amazing. I think they are a lot of super cool capabilities and with me presenting them today is one of our software engineers, Sarah. I'll let Sarah introduce herself briefly before we because she'll be you'll be hearing from her a lot later, but Sarah, if you want to introduce herself before we keep moving on. Hi, I'm Sarah Heden. I'm a software engineer and I've been working at CodeHS for about three years.
awesome Sarah. Yeah, Kim, I will fix the slides link here just in a moment. sorry it did not show up as a link for you, but yeah, we'll get the link fixed, but in the meantime, keep introduce yourself in the chat. we will keep going over kind of our agenda. So, basically what we're going to talk about today and then I will get the slide link to you all in a bit. But we are going to go over what the code's autograders are. We're going to learn how to access the autograders. We'll be talking about the different visibility settings. Lindsay just dropped it in as a link as well. So hopefully that that link should work. U, we'll go over the visibility settings and feedback levels. Those are two of the new big changes we've made to the autograder interface. some cool new features that you can do just to control how your students interact with the autograders. and then the last thing we'll go over is the preset configurations. Those were there before, but we've added a lot of features depending on your language. And we'll also talk a little bit about giving you resources for how you can make your own custom autograders as well. So action-packed 45 minutes here today. that's what we'll be covering. Sarah is going to walk you through all of those. And there again, they're all some super cool updates to features or new features that we've rolled out that just will allow you to have more control over how your students interact with the different autograders on the Kod chess platform. we are going to be using once Sarah starts presenting. feel free to ask any questions in the Q&A portion. we probably won't be checking the chat for questions as much. So again, if you have a question once Sarah starts presenting, feel free to use the Q&A. Lindsay and I will get it answered in there. But yeah, please feel free to drop all questions in there. if you are not a code teacher and you would like to make a code account, codes.comsignup will be where you can make that. I will put that link in the chat and we will get the slides link fixed as well too so you all can have access. But here is a link to sign up for a teacher account if you have not. Oh, looks like it's fixed. Perfect. Kim, thanks for letting us know. So, yeah, if you've not if you don't have a code teacher account, feel free to sign up. If you want to log into your codes teacher account and explore some of the features Sarah's going to be showing you, that might be great as well too. but you also are welcome kind of watch and follow along and check out the autograder features later for yourself as well. and if you are not familiar with codes or this is maybe your first time using it, we're super excited you're here and learning more about our autograder interface and our platform. we are a comprehensive platform for teaching computer science. We have elementary, middle, and high school course offerings. So, we have an entire K to2 web-based curriculum. It's super awesome that our our our curriculum requires no plugins. It's all browser based. Everything you need and your students need is right on the website. We offer a lot of online and offline professional development including this webinar that you all are in right now and we will go over some of the upcoming webinars at the end of the today as well too if you'd like to sign up for any more. and again we have a lot of awesome tools on our platform including what we're going to be talking about today those extensive grading and tracking tools for teachers. Students can code with our curriculum. They can do our activities but we also give you so many resources for grading and progress monitoring. And one of our favorites and our newest features is the autograder updates, which we're excited to be sharing with you today. before I pass it over to Sarah, it seems like the links are good to go and fix, right? So, the slides link we were able to get. Awesome. and again, feel free to put any questions in the Q&A. I am going to pass it over to Sarah from this point on to talk to you about our autograder platform. I'll let you share your screen too, Sarah.
Okay, great. so like Steve just said, we're talking about autograders. if you don't know anything about autograders yet, when the students are completing their code, they'll have a little test cases tab in their editor where they'll be able to easily check whether their code completes the assignment. And generally, they will not be able to submit their code before submitting. So, this is kind of what it looks like. pro users are now able to set some more configur configuration settings on the autograder. So all users are able to edit their autograders but some of the settings like turning the autograders on and off, changing the visibility, changing the feedback level, these are pro features. So when you're on your assignments page, there is an edit autograder button that you can click on to view and edit all of the autograders. So, this little window will pop up and it will show you whether or not the assignment has an autograder. If it does have an autograder, you'll see this little edit autograder button. And if it doesn't have an autograder, then you'll have a button to start creating your autograders from scratch. so, this is the autograder editor. so, you'll see on that little panel on the left there, there are several different autograders. You've got autograder 1, autograder 2, comments, contain spec specific content. and you can turn these off and on and edit the code in order to change your autograders. so if you like you'll see here some of our autograders like the custom autograders are just code. you are writing the code which you can use to analyze the students code and see if it completes the assignment. Now most of our codes created assignments will have some code written for you but we do have other options to configure this code or you could just add autograders yourself. So we have two settings for configuration. We have visibility and we have feedback level. So visibility is whether the details of the autograder are given to your students. So we've heard for some of our assignments that students are very smart, very tricky. They will before writing any code just go in and run the autograders on the starter code. All of them will fail. They'll see exactly what they need to do to pass the autograders and they'll write their code based on the autograders, not based on what the assignment says to do. And this is a way for them to put minimal effort into actually completing the assignment. So if there's a specific assignment that you know that they could only complete half the assignment and complete the autograders, you might want to set the visibility to hidden so that they won't have as much information on what exactly the autograders are looking for. and I would recommend this only if the assignment is very clear in the instructions of what they're actually supposed to do. so you'll see on the left here, this is what the students will usually see. They'll see they need an area of a circle with radius 5. They need an area of a circle with radius 10. but with if you set the visibility to hidden, then they'll just see hidden test. and they'll only be able to see if they passed or failed the test, not necessarily which test they passed or failed or what exactly the autograders are looking for. The second option is feedback level. so with this setting, I think it's very important on to communicate to your students what you expect them to do for an assignment. So the default for all of the autograders is error. That means that if the student does not pass all of the autograders, they are not allowed to submit the assignment. You can also set it to warning or info. Now, warning and info are similar in the fact that the student does not have to pass these autograders in order to submit the assignment. So, for example, if you are setting your assignments to be teacher graded, meaning that you're going to look at their assignments after they've submitted the assignment, they will not actually have to complete the warning and info and you can go in later and set whether or not these are requirements for them to get full credit. so for example, if you want to show your students that they should include comments or make sure they have semicolons after every line of their JavaScript code, maybe you want to set that to warning so that the student will have an indication that this is what's expected of them. But if they miss a semicolon somewhere or they don't include a comment, then they can still submit the assignment and not have to worry about going through and fixing everything before submitting it. So, it's really just what you want your students to have the freedom to do or not do before they submit the assignment. So, we can see here this is what an error feedback level looks like to if they didn't do it, they'll see an X and if they try to go and submit the assignment, it will just say you have errors. You can't submit the assignment with warning. they'll get a little orange icon here that says this is something you should fix, but you can still submit your work. and then the only difference between warning and info is the color of the icon here. With both of these, they'll still be able to submit their work. but you as a teacher, you know, you have the freedom to say like warnings, definitely do these before you submit it or you'll lose points. whereas info is kind of a like nudge like hey you should do this but it's not the end of the world if you don't submit. okay so we're going to talk a little bit about the code review app. So if you go to this autograders tab in the code review app you will see all of your assignments that have autograders or can have autograders. So we can see these first several assignments here have an autograder. and you'll see like these last two, the Etch a Sketch project and the horizontal circles, maybe they're a little bit more open-ended and they don't currently have an autograder, but you have the option here to add an autograder. Okay, so we're going to talk about preset configs. So, there's two different types of autograders. There's the custom autograders which is the code that you write and the preset config autograders and these are different for each language. these are the JavaScript autograder options. You'll see here JavaScript has the most options. but then Java and Python and HTML also have a few and we are working on adding more. If you have a preset config that you want us to make, definitely write in. Maybe we'll make it for you. I'd love to make that for you. but basically this is a way that you can write autograders without having to actually write code. So here you can see the example of the for loop autograder. You just have to switch this on and the students will be required to include a for loop in their code in order for their assignment to pass. And you can see you can even set like the start and the end value like the for loop has to start at zero and end at 10 or whatever is needed for that assignment. The custom autograders have a lot more freedom. you'll see there's this all of them look different for each programming language. Like for JavaScript you're going to be writing JavaScript to write the autograders. For Python you're going to be writing Python to write the autograders. But you'll see like here we have this before run function and then there's an afterrun function. And you can write specific code for like okay this is what the students code looks like before it runs. This is what you know the graphics looks like before it runs if you're working on a graphics assignment. and you can just write very specific things. So this would be the like oh it has to have a circle that is a radius of five. It has to have a function called this very specific name that's defined in the assignment. So generally for a lot of the code HS assignments there will be a custom autograder already there and you'll have the option to add your own or you can duplicate it and edit it and you know add or remove or update the autograders. We do have a lot of resources that could help you to write these autograders. like I said you are writing code. It is very specific to each language. and we have extensive documentation that walks you through how to do that. All right. Do we have any questions? I can also go through the autograders a little bit and kind of show what I'm talking about. so for example, we have this assignment that is for drawing a rainbow on the canvas. and there actually are multiple ways to get to the autogers page. Like even if you just go to the test cases here, there's a little pencil where you can go to edit the autograder settings. So let's say I this is a solution code here. If a student were to miss a semicolon here, they wouldn't be able to submit their code. And maybe you don't really care if they include a semicolon on every single line because for JavaScript you don't have to have semicolons. So it's still going to work. so I could come into the autograder editor and find my semicolons check here and I could say, okay, I want this to be info. Like I want the student to know that they missed a semicolon. but they can still submit it. I don't plan on taking points off. So then they'll just see the info here and they'll still be able to submit their work and continue and they'll get another heads up here that they're missing semicolon. Another thing is for this assignment specifically, the autograders don't check whether the colors are in the right order. It doesn't even check that all of the colors are different. So, I could change this to be orange and I could make all of the stripes orange and it would still pass all of the test cases. So, maybe I want to add a custom check to make sure that my students are actually doing the full assignment and not just making the minimum effort. so I can come here and I can I can see that in this code here. there's some like helper functions that might be useful to me. So, I'm going to duplicate it instead of adding a whole new one. And then once I add this new autograder, you'll see like this one here, this top one, it says the autograder files are not editable. So, that means that this is a code created autograder. And we don't really let you edit those autograders directly. I could just turn this off and then edit the autograder myself if I only wanted it to be my custom autograders, but I'm just going to add an autograder here. So, I'll just get rid of some of this code that I don't want. and then I'll say, okay, this one's kind of similar. An element exists with parameters. I'm going to say I want the color to be red. So, for the first item here, I want the color to be red. And I can change the description. The first stripe should be red. and then I'll save this. And if I go and refresh now, I should see that I have this additional check here, and I won't be able to submit my code because I don't have the first stripe to be read. But now maybe you're thinking, "Okay, well what if the student put purple on the left side and red on the right side?" Like maybe I still want them to be able to submit it. then I can come into the autograder editor and say, "Okay, I'm going to make this a warning." like I want them to make sure they're actually having all of the colors. but not necessarily you know requiring the first stripe to be read. Or I could say, okay, there's a picture in the assignment. I know they need to do it this exact way. I'm going to make this test case hidden. because the instructions give them a very clear explanation of what they should do. and if they run this and they don't pass this, they're just trying to put in minimal effort, make all the stripes the same color, then they have to go back and they know, oh, I'm not doing this correctly. Let me just do what the assignment says and then I'll be able to pass all the test cases. I can also show that code review app right here. You can go to autograders and this will show all of the assignments. it does take a second to load, but then if I have a specific assignment that I'm looking for, I can search for it by name or by like unit number. So, I can see, okay, this assignment has an autograder. This assignment does not have an autograder. you'll also notice here that we have feedback level mixed. so a lot of our autograders have several different sections here. and if I don't want to go in and look at the details of what autograder does what, I just know I want all of the autograders to be warning so the students can submit their best effort. Then here I can just say, okay, I want all of them to be warning. I want all of them to be visible. and then I can see here that they should all be I guess here will show it will just set all of the autograders to warning invisible so that I don't have to go in and read each autograder specifically. All right. Did we have any questions? We had one and I answered it but you can speak more to too about the if you are editing the autograder the code the language that you're editing in the languages for the autograder are typically the same language as the exercise correct
yes correct
so yeah if you want to edit a Python one you'll be if you're editing a Python autograder it'd be Python coding and those links that we put in the chat with the Autograder guide in the knowledge base have all sorts of steps for you to be able to add your own code if you'd like to and kind of play around with custom autograders. And if you have any other questions while we have an autograder expert here on the call, feel free to ask them. or if there's anything else you'd like to see or features you you anything you were wondering about using autograders in your classroom or editing the autograders. feel free to ask any questions you may have Sarah, we had a question about if the autograder doesn't require them to pass all tests if it's teacher graded. so if it's teacher graded, they will still be required to pass all of the autograders before they submit it, but they don't have to submit it in order for you to grade it, if that makes sense. So yeah, the teacher grading section doesn't really affect what the student sees in their editor. It's really just it affects the score. So, even if a student was able to submit it, that doesn't mean that they're going to get a five out of five per perfect score when they click submit and continue. it just means that if it's autograded, they'll get that score immediately. And then if it's teacher graded, then you have to go in and give them that score afterwards. Awesome. Thanks, Sarah. if you have any other questions, feel free to ask or anything else you'd like to see. while we're talking autograder updates. and one thing that I like about the updates that Sarah has up on her screen right now is all those preset configurations I think are super cool because a lot of the stuff that you may think you may oh I'm going to have to code this myself as Sarah pointed out there are so many awesome things that you can turn on. I like the function just being able to have them make a function and have it you can you can even say the name of the function that they have to put. I think it's a super awesome way to just automatically level up your autograders without needing to know how to do a lot of the code.
Yeah, definitely. Like if you want it to have a specific function name, you can type in the name. You can say if there's any specific content that needs to be inside of the function, you can say, "Oh, after they define the function, they need to call it three times." and really for most of these, there is a lot of configuration that you can do without writing any code. awesome. If you have any other questions about the autograde interface, any of the features, I see the one question that in the Q&A now, which I'll I'll type an answer out to, but if there's any other questions on general interface before we wrap up Here. Awesome. yeah, we can go to the wrap-up slides then and Paul, I can we can try to answer your question, too. but yeah, I'll share for the wrap-up slides and then if you have any other questions, feel free to put in the Q&A and we'll we'll get those answered before we depart. so again, thank you Sarah for walking us through all the awesome features. Did want to highlight, again, just in case you missed it, that if you want to learn how to do, and I wanted to click into one and show you, this is just what the autograder guides look like. So, if you are interested in learning how to write your own autograders, the Java ones, here's an example. It shows you kind of how to set it up and it'll just kind of walk you through some code you can try adding. So, it's pretty it's pretty straightforward and like it might feel like a lot of coding at first, but once you get the hang of it, it's something you might be able to it's kind of fun to play around with. So, if you wanted to check these out, we again those links are in the chat and in the slides for whatever language you're looking to kind of add an autograder for. again, we have the autograders here for you to check out. of course, we always have a lot of other resources and how you can get involved in codes. becoming a cod certified educator, earning codes micro credentials, we have a cod Facebook educators group and of course we have a lot of social media accounts for you to follow as well. [snorts] before we get to the feedback survey, I did want to put we have a link in the chat. I'm going to drop it in the same way you found this link today. codhs.comfreepd would be where you can find really all of our other upcoming free PDs as well. So, if you want to check that out, again, that link should should bring you to codehs.comfreed. We have a lot of great upcoming webinars as well. I will add the link and again, we'll answer the outstanding questions in the Q&A, too, but we do have a link to a webinar survey. first of all, thank you to Sarah for hopping on and walking us through the awesome autograder features that are there. feel free to fill out the webinar survey. Let us know any lingering questions. again, you can add those to the Q&A and we'll we'll answer them before we log off officially. but again, if you feel free to let us know any feedback on how we did today and I will also drop in a link that you can click to get your certificate of completion before we wrap up. That link should, if you click that link, as long as you're logged into CodeHS, you'll get an email with a link to a certificate of completion for your your attendance today. So again, feel free to fill out the survey, let us know how we did. Click that link to get into the autograders. And again, these are our big upcoming webinars that I'll highlight. We have a lot of for CSED week. We have a lot of Hour of Code and Hour of AI activities. We have webinars coming up for both CodeHub, elementary as well as high school, middle school on Hour of AI and Hour of Code. If you would like to bring any of those activities to your school, feel free to attend those webinars. you can also sign up and get the recording. Again, you can sign up at codehs codes.comfrepd. but those are the upcoming ones next week. Kind of our our big hour of code and hour of AI webinars for you as we look for some activities to do during computer science education week. And as always, if you're interested in learning more or bringing us for a visit, codes.com learn more. You can get in touch. And again, thank you so much for attending. We'll answer the questions that are still in the Q&A. and otherwise, we will wrap up here. And thank you so much for attending. And so if you have any questions, feel free to put them into the Q&A. I'm going to get to the one that's in there, but if there's any other questions, feel free to add them to the Q&A. And if not, have a great day and thank you so much for joining us.
