Computer Based Essay Grading Text

Jonathan Friesen - Writing Coach

But look out, teachers: a new study finds that software designed to automatically read and grade essays can do as good a job as humans maybe even better. The study, conducted at the university of akron, ran more than 16,0 essays from both middle school and high school tests through automated systems developed by nine companies. In a piece in the new york times, education columnist michael winerip described the outcome: computer scoring produced virtually identical levels of accuracy, with the software in some cases proving to be more reliable, according to a university of akron news release. In terms of consistency, the automated readers might have done a little better even, winerip tells all things considered host melissa block. The automated systems look for a number of things in order to grade, or rate, an essay, winerip says.

Among them are sentence structure, syntax, word usage and subject verb agreements. What the automated readers aren't good at, he says, is comprehension and whether a sentence is factually true or not. Les perelman, a director of writing at the massachusetts institute of technology, was allowed to test e rater. A 716 word essay perelman wrote that was padded with more than a dozen nonsensical sentences received a top score of 6 a well argued, well written essay of 567 words was scored a 5. There are all kinds of things you could say that have little or nothing to do in reality that could receive a high score.

How to End a Persuasive Essay on Abortion

An average teacher might spend an entire weekend grading 150 essays, he says, and that efficiency is what drives more education companies to create automated systems. Virtually every education company has a model, and there's lots of money to be made on this stuff, he says. A greater focus on standardized testing and homogenized education only serves to increase the development of automated readers to keep up with demand, winerip says. Winerip says that what worries him is that if automated readers become the standard way of grading essays, then teachers will begin teaching to them, removing a lot of the juice of the english language. The intelligent essay assessor iea is a set of software tools for scoring the quality of essay content. The iea uses latent semantic analysis lsa , which is both a computational model of human knowledge representation and a method for extracting semantic similarity of words and passages from text.

Simulations of psycholinguistic phenomena show that lsa reflects similarities of human meaning effectively. Then student essays are characterized by lsa representations of the meaning of their contained words and compared with essays of known quality on degree of conceptual relevance and amount of relevant content. Over many diverse topics, the iea scores agreed with human experts as accurately as expert scores agreed with each other. Implications are discussed for incorporating automatic essay scoring in more general forms of educational technology. While writing is an essential part of the educational process, many instructors find it difficult to incorporate large numbers of writing assignments in their courses due to the effort required to evaluate them.

However, the ability to convey information verbally is an important educational achievement in its own right, and one that is not sufficiently well assessed by other kinds of tests. In addition, essay based testing is thought to encourage a better conceptual understanding of the material on the part of students and to reflect a deeper, more useful level of knowledge and application by students. Thus grading and criticizing written products is important not only as an assessment method, but also as a feedback device to help students better learn both content and the skills of thinking and writing. Nevertheless, essays have been neglected in many computer based assessment applications since there exist few techniques to score essays directly by computer. In this paper we describe a method for performing automated essay scoring of the conceptual content of essays. Based on a statistical approach to analyzing the essays and content information from the domain, the technique can provide scores that prove to be an accurate measure of the quality of essays.

Honor System Essay

The text analysis underlying the essay grading schemes is based on latent semantic analysis lsa. Detailed treatments of lsa, both as a theory of aspects of human knowledge acquisition and representation, and as a method for the extraction of semantic content of text are beyond the scope of this article. They are fully presented elsewhere deerwester, dumais, furnas, landauer, amp harshman, 1990 landauer amp dumais, 1997 landauer, foltz amp laham, 1998 , as are a number of simulations of cognitive and psycholinguistic phenomena that show that lsa captures a great deal of the similarity of meanings expressed in discourse rehder, schreiner, wolfe, laham, landauer, amp kintsch, 1998 wolfe, schreiner, rehder, laham, foltz, kintsch, amp landauer, 1998. The lsa similarity between words and passages is measured by the cosine of their contained angle in a 300 dimensional semantic space.

The lsa measured similarities have shown to closely mimic human judgments of meaning similarity and human performance based on such similarity in a variety of ways. For example, after training on about 2,0 pages of english text, it scored as well as average test takers on the synonym portion of toefl 150 the ets test of english as a foreign language landauer amp dumais, 1997. After training on an introductory psychology textbook, it achieved passing scores on two different multiple choice exams used in introductory psychology courses landauer, foltz amp laham, in preparation. This similarity comparison made by lsa is the basis for performing automated scoring of essays through comparing the similarity of meaning between essays.

Dissertation Topics Property Development