Evaluating AI Chatbots in RL and RLC Circuit Problems: Benefits, Pitfalls, and Student Perceptions

Background: This full research-to-practice paper examines how undergraduate engineering students interact with generative AI chatbots while solving RL and RLC circuit problems at two institutions during Spring and Summer 2025. As large language models become integrated into engineering coursework, understanding patterns of student-AI interaction is critical for effective pedagogy. The intended outcome was to identify how prompt quality and circuit complexity influence the educational value of AI-supported problem solving.Research-to-Practice: The implementation draws on self-regulated learning and metacognitive monitoring frameworks, which emphasize planning, evaluation, and verification during problem solving. These principles informed a multi-part activity requiring students to (a) solve circuit problems manually, (b) interact with an AI chatbot of their choice, and (c) reflect critically on AI-generated solutions. An Interaction Parameter (IP) framework was developed to characterize student–AI engagement, ranging from IP = 1 (unstructured prompts and superficial interaction) to IP = 4 (well-structured prompts with critical identification and correction of chatbot errors).Method of Assessment: Three course sections (N = 79) examined RL and RLC circuits with varying complexity: G1 and G2 analyzed AC circuits while G3 addressed a DC circuit. Student submissions including manual solutions, prompts, chatbot outputs, and reflections were evaluated using a structured rubric assessing conceptual depth, prompt quality, and AI solution accuracy. Submissions were coded according to the IP framework. A perception questionnaire was administered to G3 students.Findings: Students exhibited varied engagement patterns: some demonstrated over-reliance on AI outputs with minimal verification (IP = 1), while others engaged critically by identifying and correcting chatbot errors (IP = 4). AI chatbots supported procedural organization but sometimes produced incomplete or misleading guidance for complex analyses, particularly with low-quality prompts. For simpler problems (G3), chatbot accuracy was consistently high. Survey results indicated that 95% reported perceived improvement in understanding, while 82% reported regular AI tool use. The IP framework proved transferable across both institutional contexts.Implications: The educational value of AI chatbots depends critically on how students are guided to engage with and evaluate AI responses. The IP framework provides a practical lens for instructors to distinguish superficial from reflective engagement. Results suggest explicit instructional scaffolding on prompt engineering and critical evaluation is essential for developing higher-order conceptual understanding. Future work may incorporate pre-test/post-test designs to examine longitudinal impacts on student reasoning.