Adzuna CEO Raife Watson will be joined by special guest Dr. Guglielmo Briscese from the Behavioural Insights Team to discuss how Behavioural Economics can be applied to recruitment, with particular focus on:
Recruitment Bias: An example from academia.
Recruitment Bias: Applied technique that you can implement tomorrow.
Raife Watson: Today I’m joined by Doctor Guglielmo Briscese from the Behavioral Insights Team. So look, I think it’s been proven time and time again that diverse teams of different cultural backgrounds, ethnicities, gender, etc., produce more effective, creative results than teams of one or no diversity. I’m sure all of you out there have got diversity targets you’re working to, and this is a huge topic right now in recruitment, which is why we’re very lucky to have been joined by Doctor Guglielmo. Now, just before I hand over to him, I just wanted to talk to you a little bit about Adzuna of course, about why we’re putting this on.
Raife Watson: So, for those of you who don’t know, we’re a job search engine or aggregator. We’re a Google for jobs and we’ve got about 130,000 jobs on our website, but what we have uniquely in Australia is a partnership with Fairfax Media. This enables us to not only recruit active talent that exists on our site, but we also can then amplify your ads all across 150 to 200 other sites to get the active user again, but more importantly probably the passive user in any regional or metro area. So I’d really urge you after this to try something different. Try us. Contact us and give us a go as such. It’s 2018, the year of digital experimentation.
Raife Watson: Okay, without further ado then, I shall hand over to the expert who can talk to us about why diversity etc., is so important.
Dr Guglielmo: Great. Thank you. Thanks for having me here today. So what I’d like to talk to you today about is behavioral economics. Probably some of you have heard what behavioral economics is all about, but for those of you who have not heard the term before, basically behavioral economics is this growing field which is at the intersection of economics and psychology. The underpinning principle behind behavioral economics is that if we take a more realistic model of human behavior, we can design better policies and programs both in government as well as in private sector organizations. So human beings are not this rational individuals, utility maximizers that traditional economics want us to believe. We are subject to heuristics and biases, and by better understanding how we fall victim of these biases we can actually design systems and processes and policies that can help us make better decisions for ourselves and for others.
Dr Guglielmo: So we’ve been applying at The Behavioral Insights Team behavioral economics, behavioral insights across a broad range of policy areas, from education to employment, transport, crime, health and so on. Today I’d just like to talk to you about how we’ve been applying this approach to recruitment, specifically about human resources.
Dr Guglielmo: Before I talk about some examples, I’d like to start this presentation with a couple of quizzes. So if you have a piece of paper and a pen in front of you, just write down the answers to the following questions. The very first question is, was Mahatma Gandhi younger or older than 100 years old when he died? Just write down your answer. The second question is, how old was Mahatma Gandhi when he died? Don’t Google it, just write down your best guesstimate, okay?
Raife Watson: And while we’re asking questions, please feel free to ask questions in the chat and we can answer them as we go through. Great.
Dr Guglielmo: So, same quiz. Was Marlon Brando older or younger than 50 years old when he died? And the last question is how old was Marlon Brando when he died? So just write down younger or older, and then how old was Marlon Brando when he died? Now, when we ask these questions in workshops and presentations, and then we ask people to raise their hand to guess whether Gandhi or Marlon Brando died at a certain age, the first answer that people give is that Mahatma Gandhi probably died younger than 100 years old, simply because 100 is a big number, and when we ask them to provide their best guesstimate, people come up with answers such as 90-something or 85, 89 and so on. Same thing with Marlon Brando, people think, ‘Surely he must have died older than 50 years old, but if I had to guess, maybe 60, 65, or 70.’
Dr Guglielmo: Now, when we look at the actual numbers, we see that Marlon Brando died at 80 whilst Mahatma Gandhi died at 78. Now what happened there? What happened is that I gave you a couple of cues, right? A couple of signals. The very first signal was a number. In the first question for Gandhi, I gave you the number 100, and for Marlon Brando I gave you the number 50, and people started basically providing guesstimates based on that number that I gave you. So from 100, people go down. From 50, people go a little bit up, but not too high up.
Raife Watson: Does Marlon Brando have cut through these days with a younger audience?
Dr Guglielmo: Probably yeah. Well, I hope so. If you don’t know who Marlon Brando is, I strongly recommend-
Raife Watson: Look it up.
Dr Guglielmo: Yeah, look it up and watch a couple of great movies of Marlon Brando. The other cue, the other signal that we gave you was the picture, right? So I gave you a picture of an old Gandhi and a picture of a young Marlon Brando, and you can see how easy it is to make wrong assumptions based on some couple of information that you probably thought were not even important when I asked you to guess the age of these two characters. Now, what happens is that this is an effect that is known in psychology as Anchoring Effect, Anchoring or Signaling Effect where basically you make certain decisions based on some numbers or some information or some visual cues that someone else might have given you.
Dr Guglielmo: If you have to think about where does that apply in the real world as opposed to just a funny quiz, well it applies in recruitment and that’s probably something quite scary when you think about these effects when it comes to recruitment, specifically when we think about CVs. CVs in traditional economics are actually studied as a signaling game. So I have a piece of paper and I write down a bunch of information that I want to send you where I can provide signals about my qualities, my experience and my skills to you and you can think about how worth it that information is. You will probably choose also whether you want to hire me or not, and also how much you think you want to pay me based on that information, but there is so many aspects of the CVs that actually distract you from making the best decision as an employer. There are cues such as the picture or the date of birth that a lot of people still put on CVs. That can lead to some age discrimination. Am I too old or too young for this specific job, right?
Dr Guglielmo: You can also infer, for instance, my socioeconomic status by looking up the area where I live if I provide my residential address. You can think about my background and even my political view sometimes based on where I live and where I’m based. These are all information … The same applies with typos or if I have any gap in my employment history and so on. Now, these are all information that are actually not that important for you to choose whether or not you want to hire me or not, but somehow they distract you, whether deliberately or not, from making the best decision when it comes to recruitment.
Raife Watson: So you’re saying before we even read about their skill set, you may already have sort of made a pre-judgment?
Dr Guglielmo: Immediately, absolutely, and you might not even know. You might not even be aware that you are making that judgment, but somehow in the back of your head you are doing that, right? And that applies to age, to gender, to status and so on. Now, I want to show you an example of a study that was done by Harvard academics where they actually measured how important this type of information is, and what they did in this experiment in the US, they applied to vacancies that were posted online and they applied by sending CVs of fictional candidates basically. They prepared two identical CVs. The only thing that they changed in these two CVs was the name of the applicant. For a group of vacancies they applied using a white name, like a white sounding name, and for another group they applied using a name of a candidate that sounded like an African-American name.
Dr Guglielmo: What you see here from this slide is that for those that had a white person name, they received on average about 50% more callbacks from employers. So that means that a white name on a CV was equivalent to about eight years of experience, right?
Raife Watson: Right.
Dr Guglielmo: So these CVs were virtually identical. Again, you probably are not hiring the best person if you are a victim of these biases. Now, that is quite interesting because we like to think when it comes to recruitment or HR it’s like investments in initiatives and strategies to diversify our workforce or making sure that it’s a more inclusive company and so on and so forth, but sometimes it’s these small changes that can have a huge effect. It can be as simple as asking people to submit a CV without their name on it, and at the end of the day, what’s in a name? You know? So it’s Shakespeare. So it’s not that important, right? Like, information that you need is about experience and the skillset.
Dr Guglielmo: So we’ve applied these insights from academia in real world policy at BIT, and we ran an experiment with the Australian Department of Employment. So the Australian Department of Employment has a program that is called wage subsidy, and it consists in financial incentives for companies that hire disadvantaged job seekers.
Raife Watson: Right.
Dr Guglielmo: So if you’re an employer, you get contacted usually by a job service provider and they say, “Look, I’ve got a candidate that I think is quite right for your vacancy, and by the way, there’s also up to $10,000 for you to provide that training or induction to this employee because they just happen to be a little bit more disadvantaged than others,” right?
Raife Watson: Right.
Dr Guglielmo: Now, traditional economics says that financial incentives always work. If you want people to do something, pay them and they will do it, right? In this case, something was going wrong and the department realized that there was quite a low uptake of these financial incentives. So they asked BIT to spend a bit of time in these job centers and we listened to some of these conversations that staff with employers. What we realized is that most of the times that these wage subsidies were being mentioned, the very first reaction from employers was, “What’s wrong with this person that you have to pay me to hire them?” Right?
Dr Guglielmo: Now, that makes sense, and that’s exactly where behavioral economics comes into place. So we ran an experiment and we said, “Okay, how about we just change the way we frame and present these financial incentives?” So, we changed the title. We called it the Unemployment Bonus, to have a more attractive effect and an attractive name, and we also changed the way the payment installments worked. So instead of just paying you everything upfront or everything at the end, and we’ll just pay you little by little, and that sort of gave the idea that this was like a longterm relationship between the employee and the employer. What we found is that lead to an increase in hiring of disadvantaged job seekers for these wage subsidies.
Dr Guglielmo: Now, that is an example of how we applied these behavioral economics insights to change the behavior of employers. So we try to understand a little bit better how they really behave and make these incentives a little bit more attractive to them. We also did the same thing on changing behavior of job seekers. What we realized is that job seekers, when they apply for jobs and they want to write down a CV, they usually just Google CV templates and there’s a million things that you can find out there, and unfortunately, a lot of these templates still have this information that you actually should not have in these CVs. The same applies to cover letters.
Dr Guglielmo: So what we did was created a website that was as user friendly and simple as possible, and in this website we provided some tips on how to apply and look for jobs, and also we provided a template for a CV and a template for a cover letter. Again, we reviewed these templates and made sure that they were as free from potential biases as possible.
Raife Watson: Okay.
Dr Guglielmo: We then emailed thousands of job seekers using our randomized controlled trial, and what we found is that those job seekers that had access to our website and access to our templates were actually 5% on average more likely to find a job. So we increased-
Raife Watson: Wow, how can you be so precise about that impact?
Dr Guglielmo: Yeah, so most of our studies are randomized controlled trials, and this, we think, is the gold standard of evaluation, especially in policy. It consists in basically randomly allocating a group of individual to business as usual, nothing changes, and a group of individuals to your intervention, right? So your trial, your experiment. This case was an email directing you to this website to download this template. So in this particular case, we partnered with a job service provider. They had about 20 offices around Sydney region, and we randomly allocated these offices to control or treatment, so business as usual versus the website treatment. Then we compared the outcomes by controlling for job seeker characteristics. So we can be quite sure that the characteristics of job seekers and job service providers, job sites in control and treatment were as similar as possible. So in this case we can be confident that in 95%, 95 out of 100 cases, our effect is not due to chance.
Raife Watson: I mean, 5% maybe doesn’t sound like a lot, but I think we all in recruitment know that it can just take one good candidate to change everything, right?
Dr Guglielmo: Absolutely.
Raife Watson: So in fact it is significant.
Dr Guglielmo: And if you think about how much money is invested in fixing unemployment, right? When you switch on the TV or read the newspaper, it seems like everything is about jobs. So every politician will talk about jobs, everyone in politics will talk about jobs. They are considered to be the key priority and a top item on anyone’s agenda. When you fix unemployment by simply creating a user friendly website with a couple of good templates and you increase employment by 5%, 5% is a huge number when you [inaudible 00:14:52] it up. We’re talking about thousands of people that would otherwise have been unemployed for much longer, and obviously that generates substantial savings as well.
Dr Guglielmo: Now, we’ve been applying these principles also in our own recruitment process. So BIT, just because we are behavioral scientists doesn’t mean that we’re not victims of the same biases as everyone else.
Raife Watson: I’ve just got a quick … I don’t know if it’s a question or more of a comment, so I’ll let you hear. It’s saying basically, from Catherine, “I remember reading bias actually works against women in the gender bias studies. It’s better to know that a woman wrote the application to get fair distribution of callbacks.” So what do you …
Dr Guglielmo: I’m not sure-
Raife Watson: Is that a question or a statement?
Dr Guglielmo: I’m not sure which study they are referring to. I know there has been a study that has been done recently that sort of contradicts a little bit … Seems to contradict this thing that blinding applications can help, but my understanding is that by re-looking at the analysis, actually it confirms. So it supports previous findings. So what we know is that there is a number of areas where blinding CVs is important for gender equality as well, and there is a number of other initiatives that we can take also to address that gender inequality in the recruitment process, but sometimes it really depends on the context as well. So sometimes if an employer has a specific statement, for instance, where they really encourage female applicants, then maybe the recruiters know that’s important and they’re really looking to hire more female applicants. Then in that case, you might not want to remove that information, right?
Dr Guglielmo: So it’s case by case, and I think that was the case of one experiment that was done in the public sector here in Australia, but there’s another study done by Professor [Liberon 00:16:48] from Monash University that shows how the gender pay gap is also due to basically the job ads. In this study, if I remember correctly, what they did was basically tested the impact of affirmative action or not, as well as testing the impact of having the line that says the salary is negotiable or not. Turns out that from a gender point of view, if the salary is not negotiable, or if it’s not stated explicitly, most men will still negotiate the salary during the job interview process, and most women won’t do that, but the moment you add that line and say salary is negotiable, that actually increases substantially and significantly the proportion of female applicants that will negotiate their salary.
Raife Watson: Great.
Dr Guglielmo: And when you think about-
Raife Watson: That’s a great thing to know.
Dr Guglielmo: Yeah, when you think about the gender pay gap, it starts from the very beginning. Obviously if you have a good start with the salary then the chances are that you are slightly more likely to be able to progress in your career salary wise.
Raife Watson: Okay, thanks.
Dr Guglielmo: So as I said, we’ve been applying these principles also internally at BIT, and we know that we are victims of the same biases as anyone else. It doesn’t matter how careful we are, we still get this misleading information. So we said, “You know what? Maybe CVs are not the right thing to do for recruitment, maybe we should think about something else.” So, normally when we recruit at BIT, we have a multistage process of recruitment. At the beginning candidates have to take part in a multiple choice test, then those who pass that test in the old version they would get screened through a CV. So we would check their CVs and see if they were suitable or not for the job, and then from there you were invited to an assessment center where you had group tasks and so on, group presentations. Then, after the people that passed that stage would be invited for a final interview and then there would be a reference check and the usual things that most companies do.
Dr Guglielmo: Now, what we wanted to do is to see what happens if we remove that CV screening step in the recruitment process and we replaced it with what we now call the applied platform, but back then was basically this system where we asked candidates to take part in further tests. These tests were anonymous, so we didn’t know who answered what, and the answers were assessed by everyone in the company, so not just HR or the line managers. So everyone took part in this assessment. Everyone had a specific scoring matrix that they had to use to evaluate the answers, and these answers were also evaluated in chunks. So I wouldn’t just evaluate all your answers, but I would evaluate everyone’s answers to Question A, then Question B, and then Question C, and so on, so that you compare horizontally rather than vertically.
Raife Watson: Yep, okay.
Dr Guglielmo: So that you wouldn’t be affected by potentially the fact that you might have answered very well one question but not so well the second one. So I just evaluate everyone in the most objective way as possible. Then what we did at the very beginning was basically randomly allocating some candidates between the business as usual CV versus this new system, and we later on looked at the data and we compared the scores of candidates in the assessment center, in the final interview, versus the scores they would have got in the CV versus applied.
Dr Guglielmo: Now, what we see here from these graphs is that applied scores actually were quite good indicators of how well people would have performed later on in the interviews in the assessment center. If we look instead at the scores that people got from their CVs, they’re basically as good as random. So we might as well have hired people by tossing a coin, and the same thing is with the final interview. Now what that tells us basically is that there is some value for companies as well to remove this recruitment. It’s not just about the social responsibility of a corporation by saying, “We want to improve diversity and social inclusion,” and so on and so forth, it’s really more about, “I really want to hire the best people,” and it turns out their names, their gender, their age, surprisingly are not that important, right? There is one simple way to do that basically, to make better decisions for your company, is to remove these things that you know lead you to make bad decisions for the company. Based on these tests, Applied became actually a platform and it span out of BIT so now it’s an independent company that provides this very user friendly platform that hundreds, and I think by now thousands, of companies all around the world are using, so it’s great.
Raife Watson: So what’s the name of the platform?
Dr Guglielmo: Applied.
Raife Watson: Applied.
Dr Guglielmo: Yeah. So this is about [inaudible 00:21:55]. So basically the two principles behind what we do at the Behavioral Insights Team is the application of insights from behavioral insights, behavioral economics and robust testing. Again, as I mentioned before, we do this testing through randomized controlled trials where possible so that we are able to see what works, what doesn’t, and why. The mantra of BIT is that sometimes small changes can have a big effect. So sometimes you don’t have to think about investing hundreds of thousands of dollars in government programs or corporate policies to fix something when it comes to recruitment or HR or staff motivation, and so on, but it’s just about the small cues.
Raife Watson: Sorry, I’ve just seen a question on here. Now, it’s referring to the IT sector, saying, “That seems all very well and good, I guess. I mean, the results sound interesting or effective, but how long does this process take?” So the question’s relating to IT, saying, “Look, you’ve got to have a really fast process because it’s so competitive in that industry that if it takes too long you’ve lost that candidate in the second round because they’ve been snapped up by someone else, so how effectively or fast can you implement something like this?”
Dr Guglielmo: Yeah, so in the case of Applied, you basically receive an email if you are an employee that has been selected as an assessor. I think it’s up to you how many questions you want the candidates to take part in and answer, but you might receive an email as an assessor as a member of the company and sometimes it takes as quick as half an hour, right?
Raife Watson: So it’s an automatic type process, you mean?
Dr Guglielmo: It’s as automated as possible.
Raife Watson: [Crosstalk 00:23:30] doesn’t have to be everybody in the company, I suppose.
Dr Guglielmo: It doesn’t have to be everybody.
Raife Watson: [Crosstalk 00:23:34] your panel, or-
Dr Guglielmo: The more the better because you have what’s called the wisdom of the crowds, but obviously it’s up to you, right?
Raife Watson: Yep.
Dr Guglielmo: And this platform basically allows you to refine and customize this process as much as you want. Obviously the slide that I showed before is the process that takes place at BIT, but different companies might use the platform in different ways.
Raife Watson: So they can still travel at speed, but hopefully get the better results, right? I mean, again, you want to hire the right candidate.
Dr Guglielmo: Exactly. Yeah, it’s one of those things where you might have a little bit of a higher waiting time, a longer waiting time and higher costs at the beginning, but the value adds down the track. It’s enormous, as you can see from quick tests and experiments. So this is about it basically, and this also why it’s important to test because you never really know what is the value of taking part in these processes. We get similar questions from government departments or companies we’ve worked with where they say, “This is great. I think this is okay, let’s not waste time testing it. Let’s not spend time collecting new data,” right?
Raife Watson: Yeah. Can you scale it up fast? I know that some of our participants are large companies, so is it practical that you can scale, if it works … I mean, have you seen it scale fast in these government departments or somewhere?
Dr Guglielmo: Yes. Absolutely, yeah. So that is a key component of what we do. We will actually only advise to test solutions that can then be scaled up. If we know that something cannot be scaled up, there’s no point in testing it. When you think about solutions that can be scaled up, you always need to have a little bit of a cost-benefit analysis, right? Where you say, “Is this worth it? Am I going to generate enough savings or enough revenue that actually makes it worth it to invest in a little bit of testing and data collection sometimes?”
Raife Watson: It’s a complex area and everybody wants to be in it or … How do you start?
Dr Guglielmo: Yeah. So the best place to start is basically to read whatever is out there. There’s a couple of books that are the typical pop psych type of books that you can find in any app. One is called, Thinking, Fast and Slow.
Raife Watson: Thinking, Fast and Slow? Okay.
Dr Guglielmo: Yeah, from Daniel Kahneman who has a Nobel Prize in economics. He basically … But he’s a psychologist, and he says that our brains are almost like two brains into one. We have a System One, which is the one we activate when we have to think fast, and the System Two is the one that takes a little bit more energy to activate. This book is great because it tells you how you are a victim of these biases, and this System One-
Raife Watson: So like a CV is a think fast-
Dr Guglielmo: Pretty much, pretty much.
Raife Watson: You’ve got to-
Dr Guglielmo: Without even realizing, yes. So the System One is the one you activate when you just have to cross the road, for instance, or come to work every day, things that you do without even thinking about it. The System Two is the one you activate when you have to plan a holiday or plan a company strategy and so on. Now, unfortunately, most of the times we should apply System Two to make better decisions, but our brains are hardwired to make fast and efficient decisions with the least amount of energy consumption as possible. A lot of times, unfortunately, we make bad decisions because of that, because we just activate System One instead of System Two. So that’s a great book, I think, as an intro. The other great book is Nudge, which is a book that is written by Professor Thaler, Richard Thaler who is a Professor of Economics at Chicago University who won this year’s Nobel Prize in Economics, and he basically says, “If we know that people are not fully rational, how about we design policies and programs to make sure that we help people to make better decisions?” That’s the-
Raife Watson: Is Nudge a hard book to read?
Dr Guglielmo: Very easy, very super accessible.
Raife Watson: Okay.
Dr Guglielmo: Non-academic textbooks, and if you want to apply behavioral economics in your company, you do not need to have a PHD in psychology or economics. You can learn about it and obviously there’s so many opportunities to engage with experts in the field as well. BIT obviously has an office in Australia, but there is multiple government departments now that have their own behavioral insights teams.
Raife Watson: Sure.
Dr Guglielmo: So there’s so many opportunities to engage with practitioners as well.
Raife Watson: Great. Okay, so for our participants out there, they can either try one of those two books, they could contact you of course.
Dr Guglielmo: Sure.
Raife Watson: But I guess triggers them to start somewhere. Anything else from you today?
Dr Guglielmo: No, I think that’s about it.
Raife Watson: That’s it?
Dr Guglielmo: Yeah, thanks for having me.
Raife Watson: Okay, so think slow. That may seem counterintuitive, but for example, use that sooner, do something a bit different. Look, I hope you’ve enjoyed today’s session, and we’ll have another one in a couple of month’s time too. Let me first just check some questions, see if there’s another question. No, I think that is about it, but feel free to shoot any other questions to us and we’ll steer them to Dr Guglielmo. Thank you very much for your time today, I really appreciate it.
Dr Guglielmo: Thank you for having me.
Raife Watson: And thank you all.
Dr Guglielmo: Thank you.