Tutorial 3 Activities#

Pre-mortems#

In this tutorial, we will apply a simple and well-established technique in risk analysis called a pre-mortem. The method of a pre-mortem was devised by Gary Klein and his colleagues. Gary is the founder of a field called Naturalistic Decision Making, which studies how experts make decisions in the ‘field’. He uses insights from his research to train experts in improving their decision making.

Research that shows that by spending time at the start of a project to imagine that the project has failed can increase insight into real reasons why it will fail. That is, rather than asking: “What could go wrong?” at the start of a project, we imagine that the project has already failed and ask “What went wrong?”. This makes people are more creative and more likely to put ideas forward to their team rather than keep them to themselves.

Klein and his associates took this idea and turned it into the pre-mortem method. A pre-mortem is the opposite of a post-mortem. The idea of a post-mortem comes from medicine, where after a patient dies, medical specialists discuss what went wrong to help improve patient outcomes in the future. This concept is generalised to business, government, etc., where people analyse why projects fail. A pre-mortem though, does this analysis before the project starts, by proposing a hypothetical future in which the project has been a complete failure.

The steps for a pre-mortem are:

  1. Assemble your project team into a meeting at the start of the project.

  2. Ask the team to imagine to some point in the future (e.g. 1 year, 2 years) and to imagine that the project was a complete failure and everyone now agrees it was a bad idea.

  3. Ask each team member to spend five minutes independently writing down reasons why the project could have failed. This step must be done independently to avoid ‘groupthink’.

  4. Then ask each team member to present one reason from their list that hasn’t been presented already. Continue going around until all team members have presented their ideas.

At this point, either the team or the project leaders can strengthen the plan to avoid the most serious of these issues.

You can read more here

As a team, select one of the following two applications:

Criminal Identification via Facial Recognition

In early 2020, London police started using facial recognition technology to identify wanted criminals on the streets of London. The use case is simple: a facial recognition application is used to match people coming out of major shopping and travel hubs against a database of wanted criminals. If a person is matched, the police arrest the person. Faces of people not in the database are not stored. Faces of people not in the database are blurred on police screens. So, if you are not a wanted criminal, you have “nothing to worry about”, right?

Sensors for Pedestrians

Recently, a research team at MIT has developed small sensors that can be worn by pedestrians so that self-driving cars can detect people better. This is because contemporary self-driving car technology is poor at detecting passengers in busy environments, such as cities, and in the rain or the dark, which are major safety concerns. The sensor does not need to have identifying information, so cannot be used to track an individual. It is effectively a digital version of a high-visibility vest to make pedestrians more ‘visible’. The group who developed the tech propose that laws should be changed to mandate that pedestrians wear such a sensor at all times to allow self-driving cars to be deployed safely.

Assume that you are on the project team for your project when it is first conceived. Assume that the application has been deployed for two years and it has been a complete disaster due to ethical reasons. It is no longer deployed. Conduct a pre-mortem as outlined above.

Document your findings and vote on what you think would be the primary reason for failure. Link back your reasons to the models of contractual trust in the previous module.

In your group, discuss whether you think conducting this as a pre-mortem had a different effect from the way that you analysed the automated grading system in last weeks tutorial. If you think it did, what was it about the pre-mortem that had this effect? If not, why do you think it did not?

Note

We’ll discuss this as a class, but remember to also take notes and post them on this week’s forum discussion.