Tutorial 11 Activities
Contents
Tutorial 11 Activities#
Activities#
Consider the following news article about Ofqual: an algorithmic approach that predict final grades for secondary school students in England in 2020, who were unable to sit exams due to the COVID-19 pandemic.
Ofqual is/was a statistical model based on past performance that predicted final grades, which are used for university entrance, etc. At the time, after the predicted grades were released, there were protests from students and their families, leading to Ofqual not being used.
Exercise
In your group, discuss what information an explainability technique should give for Ofqual. Remember to think about relevant stakeholders in this scenario: who will want to know information about how Ofqual works.
Document at least three stakeholders and list what you think their primary goal should be for explainability of Ofqual.
In financial institutions such as banks, credit scoring models have been used for decades to try to predict the risk of someone applying for a loan. These models use information such as: the loan amount requested, interest rate, term (length) of loan, age, previous loan information, current amount of debt, number of credit cards, occupation, education level, salary, and investments.
Consider the three local explanations below for a business owner, Peta, who applied for a $12500 loan on a 24-month term at an interest rate of 12.5%. Peta has a credit rating of E (on a scale of A to F, where A is an excellent rating and F is a poor rating). Peta has run a successful business for five years and earns an annual salary of approximately $130,000.
Peta’s primary goal in these explanations is recourse: to find out how to get a loan by either: (a) updating information that is incorrect; or (b) changing circumstances so that the next time, the loan is approved.
Exercise
In your group:
Compare the three types of local explanations below and rank them for suitability based on whether they help Peta to achieve the goal of recourse.
Do you think the rankings would be the same for a data scientist building the credit scoring model?
Document your outcomes and justify your rankings.
Your loan application has been rejected in this instance. Here are the five most important reasons for this decision and their weights:
| Attribute | Value | Weight |
|---|---|---|
| Salary | $130,000 | 2.9 |
| Interest rate | 12.5% | -2.1 |
| Credit rating | E | 2.0 |
| Loan term | 24 months | 1.9 |
| Occupation | Business owner | 1.1 |
Here, weight of the interest rate is negative because a higher interest rate makes the loan harder to pay off.
Your loan application has been rejected in this instance. The following lists three examples of customers who have also had their loan rejected:
| Attribute | Example customer 1 | Example customer 2 | Example customer 3 |
|---|---|---|---|
| Salary | $120,000 | $80,000 | none |
| Interest rate | 12.5% | 12.5% | 17.5% |
| Credit rating | E | A | D |
| Loan term | 12 months | 60 months | 12 months |
| Occupation | Accountant | Car restorer | Author |
Your loan application has been rejected in this instance. The following list four reasons why:
If your income had been $180,000 or above, you would have received the loan; OR
If you increased your loan term to 48 months, you would have received the loan; OR
If you applied for a small business loan, which has an interest rate of 9.5%, AND your income had been above $150,000, you would have received the loan; OR
If your occupation had been Chief Executive Officer, you would have received the loan.
Note
As per usual, please remember to collate your answers and discussion points to these questions and post them in the discussion board! :)