In this demo, we showcase how the Autograder System, named Autograding, leverages a self-hosted Local Language Model (LLM) based on Llama 3.2, specifically modified and curated to provide feedback on student code submissions. This system is designed to integrate seamlessly with GitHub, automatically grading students’ code and generating personalized feedback.

Key steps highlighted in the demo:

1. Code Submission: Students submit their code for automatic grading through GitHub. The autograder is triggered on push events to any branch, excluding certain paths like feedback.md.
2. Autograding Process: Upon submission, the autograder system runs predefined tests using the education/autograding action. The output is logged in autograder_output.txt, and the working directory is cloned for persistent storage in the logs directory.
3. AI-Powered Feedback Generation: The self-hosted Llama 3.2 model evaluates the code, analyzing structure, syntax, and logic. It then generates detailed feedback that points out specific areas of improvement, highlights coding errors, and offers suggestions to optimize the code this is all based from the autograding output.
4. Feedback Push: After the Python script completes, the autograder stages and pushes the feedback to the GitHub repository in the feedback.md file, providing students with AI-driven feedback alongside their test results.
5. Results Display: The system presents the grading results, showing both the outcomes of the autograding tests and the Llama 3.2 feedback, allowing students to review their work, learn from their mistakes, and refine their coding techniques.

свернуть