CLOUD

A data-mining and analysis software program called Signals developed by Purdue University has increased six-year graduation rates by 21.48 percent, according to a recent review of data from the 2007 cohort of students.

The increased graduation rates were for students who were in two or more Signals-enabled classes compared to students who had not taken any Signals-enabled courses. Students who had taken just one Signals-enabled course graduated 20.87 percent higher than those who had taken none.

Five-year graduation data from the 2008 cohort showed that students who had taken two or more Signals-enabled courses graduated from Purdue at a rate that was 24.36 percent higher than students who did not take a Signals-enabled course.

The Signals software looks at students' online academic behaviors, such as whether they opened a reading assignment or completed a set of math exercises. It combines this information with demographic information about the student, such as his or her standardized test scores, high school GPA, and current grades. In all the system uses more than 20 reference datapoints.

Signals then displays a simple and intuitive red, yellow or green signal on the student's course website to let them know how they are doing in that course.

In addition to the visual warning, students also receive messages prepared by the instructors. The messages may tell students in the green to keep up the good work, for example, or remind those with a yellow signal that evening tutoring sessions are available, or ask a student who has recently changed from having a yellow signal to one that is red to make an appointment to visit the instructor's office.

Purdue recently released a web application called PassNote that helps instructors write these messages to students using text that research has shown are most effective.

Gerry McCartney, Purdue University's CIO, vice president for information technology and Olga Oesterle England Professor of Technology, says one strength of Signals is that it gives students feedback on how they are doing soon after they arrive on campus.

"In some courses students might not receive an actual grade until they take their midterm exam halfway through the semester," McCartney says. "By that time they have already set most of their study behaviors. Depending on the course assignments, Signals can begin offering feedback as early as the second week of classes so students have plenty of time to make changes."

Signals launched as a pilot program in 2007 and rolled out to general campus use in 2008. It has been released as a commercial product by Ellucian called CourseSignals.

Each instructor or faculty member can decide whether to incorporate the Signals system into their course. Currently more than 125 courses (many of which have multiple sessions) have incorporated Signals at Purdue.

Matt Pistilli, research scientist in the Teaching and Learning area of Information Technology at Purdue, says Signals provides benefits to instructors, too.

"The application of analytics can transform how a student learns, but it can also transform how an instructor teaches," Pistilli says. "Faculty who teach may see an unusually high number of students move to the red warning and realize students weren't getting a concept or successfully completing assignments. They can quickly address that in the course and make changes for when they teach the course the next year. Ultimately, the students get better over time, but so do the courses."

During the six years Purdue has been using the Signals system, the primary surprise to the developers has been that taking more than one Signals-enabled course significantly increases both retention and graduation for students.

"For some reason, two courses is the magic number. Taking just one course with Course Signals is beneficial for students, but we've seen significant improvement when students are enrolled in at least two courses using the system," Pistilli says.

"We need to continue to do more research on why this is significant," Pistilli continues. "We think it is because if a student gets feedback from two different instructors, they can make better links between their performance and their behaviors. For example, a student may be taking a course in biology and and another in communications, which are very different courses, but if they get the same feedback in both courses about study habits and needing to spend more time on task - and they hear suggestions on how to improve from two different professors - they appear to make the change. What's notable is that this improvement stays with them in their other courses and for the rest of their academic careers."

Staff at Purdue continue to incorporate "Big Data" techniques in order to boost student success. Work has begun on the next generation of course analytics systems, which will provide more targeted information.

For example, Signals looks at a student's mathematics aptitude, but that skill may be more significant in a chemistry class than in a writing course.

"In the next generation system we want to have multiple algorithms for different departments and eventually for each course at Purdue," Pistilli says. "Eventually by using artificial intelligence we hope to develop a custom algorithm for each student."

The next generation academic analytics system at Purdue will also pull in data from a separate project currently in development at Purdue called "Students Like Me."

This data analytics program, which is being developed with EMC Corp.'s Pivotal, will help students achieve academic goals. Students Like Me will then look at similar students who were successful and recommend specific courses, student organizations, or other activities for the student to consider.

"Signals was just the beginning of how were are planning to use data mining and analytics to improve student success," McCartney says. "We have seen what a great benefit Signals is already, but we know there is much more we can do to provide our students information that will make our students as successful as they can be."