Tutorial 2 Activities#

Automated essay grading is the use of software to automatically assign (part of) a grade for a written essay.

According to a 2019 research article, the following factors are used to assign grades:

Dimension Description
Grammar Use of grammar
Usage Use of prepositions, word usage
Mechanics Spelling, punctuation, capitalisation
Style Word choice, sentence structure
Relevance Relevance of the content to the prompt
Organisation How well the essay is structured
Development Development of ideas with examples
Cohesion Appropriate use of transition phrases
Coherence Appropriate transitions between ideas
Thesis clarity Clarify of the thesis/argument
Persuasiveness Convincingness of the major argument

Imagine that the COMP90087 subject coordinators decide to use an automated essay grading tool that uses state-of-the-art in natural language processing technology. The tool uses the factors listed above to assign a grade.

Discuss in your group:

  1. How would you react if you were told about the COMP90087 coordinators’ decision?

  2. Using the model of contractual trust, list three of the factors from the table in the notes (reproduced below) that you think an automated essay grading tool would be unlikely to uphold?

  3. What are some of the ethical risks that could occur if these factors/contracts are not upheld? Use the model from lecture of how power and trust intersect.

  4. For these risks, analyse these against the concepts of use, misuse, abuse, and disuse of machines. Are the risks caused by misuse, abuse, or disuse?

Note

Nominate a note taker to record your answers. Put your answers in the text of the forum discussion post, rather than as an attachment, to encourage others to read your notes.

✨ secrets ✨

Note

This is a “holdout” task that we are not giving out in advance, as it will affect your answers to Task 1.

Contracts for Ethical Trust#

../../_images/contracts-in-trust.png

Fig. 4 The set of contracts that a AI should uphold to be considered trustworthy according to European guidelines. Source: Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI#