S2Ep1 - Cognitive bias in forensic pathology decisions with Dr. Itiel Dror

S2EP1_Dr Itiel Dror

Angela: good morning

[00:00:00] Itiel: Good morning

[00:00:02] Angela: or afternoon, technically early evening for you.

[00:00:05] Itiel: Yeah. No, not technically. It's actually four thirty afternoon on the Sunday. 

[00:00:11] Angela: Today we have with us Dr. Itiel Dror, and I'm going to let you go ahead and give your, brief introduction of your background and your area of expertise. And then we're gonna talk about some rather controversial studies that you've done on cognitive bias in forensic pathology decision making, which has caused quite a stir and a lot of letters to the editor of different journals, being less than constructively critical of the work.

[00:00:43] Itiel: My name is Itiel and I'm a cognitive neuroscientist. So I study the brain and cognitive mechanisms in perception, judgment, and decision making. And I'm especially interested in expert decision making and why experts make mistakes, why smart people make mistakes.

So I'm not interested when experts make a mistake because they're negligent or incompetent, but hardworking, dedicated experts, whether it's forensic examiner, police officer, medical doctor, pilot, banker, a whole range of domains that they work to study why experts make mistakes. And once we have a better understanding of the information processing of the cognitive brain, reasons that cause mistakes, we can minimize them by taking certain actions. So I develop and provide training how to minimize mistakes by competent experts.

[00:01:48] Angela: Awesome. I've been through several of your trainings and found them to be very useful. I think I said to you earlier that when you first learn about cognitive bias and you think about it in your field, you're like, Oh, no, that, no, it's not like that at all. And then as you get further into it, you realize actually it is, and you can look back on your own work or your organization's work and realize we probably could have done that better. and a lot of this stuff comes back to, there, there's those fallacies that people think, about what bias means. That,Well, they're fallacies, so they're not actually true. and one of them is that, I guess when I spoke to my students about this last week, I asked them, when you say the word bias, what does it make you feel? And all of them said that it has a negative connotation to it, but what you do isn't a negative at all. It's just bringing awareness to how your brain works. Is that correct?

first of all, I think it's important to distinguish between what I'm talking about with cognitive bias versus the way people usually talk about bias. So the usual term bias refers to intentional discrimination like racism and sexism and antisemitism, where it's intentional bias and discriminatory.

[00:03:13] Itiel: I'm talking about the way the brain actually works, which affects all of us. Whether it's good or bad is something we can discuss, but generally it's a good mechanism. It developed because of the architecture of the brain, but sometimes it can lead us to make mistakes, and this is where we want to minimize the way it goes off track, and the cognitive bias gives us a mistake, and it's unconscious implicit bias, so people are not aware of it.

If it were the intentional bias, I wouldn't be researching it. First of all, most people are not biased in the terms of racism in sexism whether police officers or medical doctor, or forensic examiners. So it's a small problem affecting only a few B. It's relatively easy to find, right? All you have to do is to have to the KKK database and to see who is a member of the KKK.

And it's easy to deal with those people. What I'm interested in is a much bigger problem. It's implicit cognitive bias that affects everyone. It's harder to detect and it's harder to deal with. From a scientific point of view, it's interesting why some people are intentionally biased. It's not interesting. What's interesting is why, hard working competent, why everyone have those biases that impact us,

so it's not your fault, it's just how your brain works,

fault is a big issue. No, it's not your fault. It's not. When you say it's not to your fault, I will accept it in the sense it's under your control. And it's not badly intended. There's no bad intention or malice behind it. It's a way your brain works. For example, if you go to meet someone at the airport and you're waiting in the arrival halls and expecting someone and waiting for them to come out, you may mistake somebody coming out to be your friend when actually it's not your friend because you are expecting it. Or if you don't want to see someone for one reason or the other, you see them everywhere, right? Because you don't want to see them. So it's how you state of mind and your expectation and past experience and your hope and the whole range of cognitive and psychological phenomena affects what you seein your judgments, 

[00:05:31] Angela: and this is something that's of particular interest when you look at people who are experts in their fields, whether they're, forensic scientists like myself, or they're forensic pathologists or like medical doctors, like you were saying, it's all walks of life. And it's important when you talk about bias for experts because you want to ensure that any kind of implicit bias they have doesn't impact their decision making in such a way that it would ultimately lead to some type of miscarriage of justice or errors in their work.

[00:06:07] Itiel: Yes, it again, it gets, again, it gets a bit, complicated. Because we have known for some time that people are not rational, right? Daniel Kahneman got a noble prize and he had the book Sinking Fast, Sinking Slow, published about 10 years ago, where we know people are not rational, but they're not random. They're systematically irrational. What I've contributed is that it also affects experts. It's not only normal lay people, but even experts doing scientific work are not rational and are susceptible to bias. Indeed, by the way, Daniel Kahneman had a new book just came out last year, Noise, A Flaw in Human Judgment about that, and yet whole chapter on forensic science chapters also on other, domains.

In a nutshell, I always like to say the human mind is not a camera. People think that the human mind is a camera. We look at the world, we hear the world, and it comes into our brain. And we see things and judge things the way they are when actually the human mind is not a camera. And experts are susceptible to that too.

That's what I've shown. And they get very defensive because they're expert doing supposedly objective scientific work. Some domains are more modest than others, and forensic science is one of the most problematic domains. Why? Because experts in other domains, they're aware of their mistakes. When the military pilot shoots and they shoot their friends instead of their enemy, they know they made a mistake.

When the medical doctor amputates the wrong leg, they find out about it. When the banker tells you which stock to buy, you can open the Dow Jones and see a week later if the stock went up or down. So if and when they make a mistake, they know about it. In the forensic domain, very rarely do we know about the mistake.

So if and when a mistake happened, they don't know because they go and say, This is the match. the DNA belongs to this person, the fingerprint match or the different forensic domain, and the person is convicted in court, in fact, all around the world. But let's talk about the United States. 95%, sometimes more than 95% are plea bargain. When you're going to a suspect and telling them we have fingerprint or firearms or whatever forensic evidence, they will confess cause they offer the good, plea bargain. So the forensic examiners who don't know if they make a mistake, so they think they don't make mistakes, somebody comes and tells them,you are doing a very good job, but you're not perfect. You can make a mistake, you can be biased. They get very offended because for decades they claimed they have zero error rates. They claim to be infallible. This is, what the FBI had in the fingerprint manual and testified in code. We are infallible. And you know,when I met him in the beginning, I said, What do you mean you infallible?

it just, was mindboggling at an outsider, specializes in human and expert errors that they sought that and the court accepted that. if you want to add more fuel to the fire, the forensic examiner live in the adversarial system. So in the other expert domains, first of all, they know about their mistakes and we are all on the same side. All the medical doctors want the patients to heal and get better. All the bankers want you and themselves to make more money. We are all on the same side. But when it comes to the criminal justice system, the defense and this prosecution, and there are two different sides, and many time you hear the forensic examiner ask me, are you a supporter of forensic science, or you a critic of forensic science? I'm not a supporter, I'm not a critic in this war. And this war is not only not scientific, it's anti-scientific. Science is misused and abused in the court by the prosecution to convict someone and by the defense to try to get someone innocent when it gets to court. They just want to use it for their goal of I've, having someone convicted or having someone found not guilty.

[00:10:30] Itiel: All of this, put all of that together. Then the forensic scientists, experts generally, but the forensic scientists especially do not want to hear about mistakes, do want to hear about error rate, and definitely do not want to hear about bias.

[00:10:44] Angela: yeah, I think there is some in one of the, one of your articles or papers that I read that you were talking about the, my side bias, So it's the same information, but it's provided to you by the prosecution side or the defense side, whomever it was, that hired the expert to do the work and that their findings were biased towards the person that hired them to do the work. They didn't, Not necessarily, It wasn't intentional necessarily. 

[00:11:09] Itiel: Absolutely not intentional. We call it allegiance bias. We gave the same exact evidence to over a hundred, examiners. Half of them were told the work, by the prosecution. Half of them were told the defense. It's a research by Dan Murrie from UVA and others, and showed a very strong,allegiancy effect. And when you talk to them, they, we have what's called the bias blind spot. It's very easy to see bias in other people, very hard to see bias in yourself. So we have this allegiancy effect that you work, and this is part of the problem in forensic science in laboratory that work very closely with the police, they see themself as being there to support the police. In fact, some forensic laboratories in the United States don't only work as part of the police, they're part of the prosecution, they're part of the DA office rather than being independent scientific entity. And when you are part of a team and you are there to help and you get reward and recognition when you solve the case i e when someone is convicted, then you get some of those biases that we're talking about.

And that's something, I think it was the 2009 National Academy of Science Report recommended that forensic science laboratories be completely separate from anything to do with, the prosecution or law enforcement agencies. 

[00:12:33] Itiel: Yes, you would be, you'd be surprised because I wrote a paper in 2009 criticizing that recommendation because I thought it's quite naive to take the forensic laboratory out of police control and law enforcement, where are you going to put them? They're going to be biases in somewhere else. You're gonna put them in the commercial world. There are biases and pressures there. Even if you put them in university in the ivory tower, if you think the universities do not have politics and pressure and biases they have too. So in my paper in 2009, I said, Where are you gonna put in the labs on Alpha Centauri or Gamma? What star you go, What planet? So by, it's out of the fire into the fry pan. And the question is, and they say back in 2009 is not whether the forensic lab in the police or where they are, but to give the forensic examiner independence of mind, whether they're not interfere. So they can be theoretically administratively under police, but isolated. So the chief of police, she cannot tell them what to do. She cannot put pressure, the detectives cannot call and put pressure on them. So if they're in the police and they're isolated and left to do the work, they can. under the police, and they can be in the business world, in academia and not be given the peace of mind. You need to let the forensic examiner do their work and not be exposed to irrelevant information or pressure. For example, we did a survey on fingerprint examiner and found that in the United States, in many cases, the fingerprint examiner, she knows whether the suspect had a criminal record. Why do they need to know if the suspect had the criminal record when you're comparing fingerprints? That is totally irrelevant. And the research showed that information changed with the way they perceive the fingerprints, the minutia, how they do the comparison. So this is an example why we want to give them independence of mind, regardless where they're situated. And any organization has organizational culture and pressure and motivation and ideology. We just need to make sure they do the scientific work based on the scientific evidence and data and will be much better off than it is today.

[00:14:56] Angela: Yeah, so that was the thing , there's this idea that if you remove them from law enforcement, then there won't be any bias. But then again, like you said, in a private industry, you just have different pressures and things that could cause cognitive bias. And then you have the same thing when you're working in academia. It's all these organizational pressures, are definitely a factor in a scientist's ability to do their work and isolate themselves from all these external influences and really focus in on the data and the data analysis and interpreting that in as unbiased a way as possible. 

[00:15:34] Itiel: And going back to the fallacy that you mentioned, so the six fallacy about bias and experts, which I published in analytical chemistry, it's a paper freely available to the public. They can, click on the link and see the paper. Some experts believe that they can by mere willpower, ignore the contextual information. So yes, I know the criminal had a past record. Someone else identified them and eyewitness or whatever the evidence is, and they can ignore it and block it out and just focus on the evidence. They believe it, but scientifically, that is incorrect. You cannot do that. And that's why awareness of the bias is not sufficient to minimize bias. It's a necessary but not sufficient in order to combat and minimize bias. We don't control, I wish we could control our brain processes, you know, where many different things going on and we can't. Right?If something is going wrong with your life, you can't not think about it because you don't want to think about it. We don't control our thought processes, so this is why it's very important that they're not exposed to irrelevant information that can bias them even though they believe their immune from bias or they can control it by mere willpower. That is scientifically incorrect.

[00:17:00] Angela: I will link the paper that you just mentioned. I'll put a link to it in the show notes so everyone can have an opportunity to go and read that. I highly recommend it because if you're just trying to understand the subject, the basis for the subject and all these conversations that you're trying to have in the field of forensic science or any fields, really, it's a good foundational document for everyone to take a look at.

So getting into this, study that you did on forensic, bias in forensic pathology. So the question, the initial question that you asked was, is it there, is that it in a nutshell or is there a little bit more to it? 

[00:17:35] Itiel: Yeah, so there little a bit more to it. So given the bias blind spot and that people, and especially experts and especially for forensic scientists, don't want to hear about it. When I came out initially on fingerprint and showed bias in fingerprint, the fingerprint community was very defensive. But after a few years, they came around. But the other forensic domain said, Well, you showed it in fingerprints, so didn't show it in DNA. So I showed it in DNA. So then the forensic anthropology said, You showed it in DNA fingerprint, but forensic anthropology is different. So we showed it in forensic anthropology, and then we showed it in digital forensic, domain after domain. Everyone, it affects them and not me. And slowly we move domain after domain, and the last domain remaining was the forensic pathologist. And they have somehow managed to be exempt in all the committee, in all the inquiry, in Texas example, right? They're exempt, right? if you look at, all the Forensic Science Commission and whatever forensic pathologists have managed to escape it and to avoid scrutiny and you know their turn has come. They were the last one. So they believed that they are different from other forensic sciences because they're medical doctors and every forensic science is different. I don't deny they're different. But fingerprint is different from DNA and that's different from digital forensic. And I work a lot with medical doctors and medical errors. I'm very familiar. And doctors, what is funny, medical doctors acknowledge bias. We know these bias against black people. There's bias against women. A whole long list of biases that's for that medical doctor are susceptible to. So it was time to do the forensic pathologist who have resisted for a long time and successfully somehow managed to avoid the scrutiny of other forensic sciences. So we did a study on forensic pathology. the first ever to dare examine bias, how dare you even examine the bias on that. So that's was the first study on bias in forensic pathology. Yes.

[00:19:52] Angela: And forensic pathology, so for people who aren't as familiar with it, probably people watch things on tv, Bones and all those other TV shows that I never actually watch, because they're not very realistic. Pathologists determine cause of death. And then they also determine manner of death. The cause of death is obviously how somebody was, how somebody died. And the manner would be more like something like, was it suicide or was it homicide or undetermined? There's certain categories, depending on the jurisdiction that they're in, that they're allowed to say judgment or conclusions that they're allowed to draw. 

In most, in most jurisdiction. So cause of death is almost purely medical manner of this. In most jurisdictions there are five options. A person died naturally, homicide, suicide, or accident, or undetermined. And I should say in most jurisdiction in the United States, the manner of death is the United States invention. It's not done in other countries. In other countries, they understand that the manner of death is a legal circumstantial decision to a large extent and not a medical one. So the medical examiner, the forensic examiner, the forensic pathologist will give the cause of death and then they will testify by the coroner hearing, for example. And then the coroner will say this consistent, with suicide, it is consistent with homicide or whatever. But in the United States, in almost all jurisdictions, they decide one of the five categories on the manner of death, which is a contextual and a medical decision combined.

[00:21:37] Angela: So it's interesting because one of the thoughts that I had after reading through the papers and my personal experience on the level of that I've been involved in autopsies,and investigations on that side, from the DNA aspect, the only involvement that I have is take a section of the long bone or give me one of those molars and I'll get a DNA identification, or attempt to identify the deceased. What I wondered was, you have the actual medical determination cause of death. Should they be delving into the manner of death determinations, especially in light of what your findings were in these preliminary studies that you've done. 

[00:22:15] Itiel: Even before we go to the study, I think it's very important to emphasize, I'm not here if you ask me do I think they shouldn't decide the manner of death. I'm not sure what I'm going to say, but I am sure that I want a transparency, so I know it wanting to be very transparent when they make the manner of death determination. What parts, if any, are based on medical findings? What parts, if any, are based on contextual information? So the fact finder the judge or the jurors or the DA or the police detective understand what the is under this decision and also they and the public and the judge and everyone needs to understand that they're not qualified or have training in some of it. For example, they have a dead body and they have to decide if the person committed suicide or they died from homicide and they will take, when they decide the manner of this contextual information, the letter of suicide. So they will get a copy of the letter of suicide. But they're not handwriting examiners. They're not experts. If you wanna murder someone and you want it to look like homicide, you can fake a suicide letter. And then a medical doctor, let's say forensic pathologist, who is a medical doctor, they may use that letter of suicide to decide the factor that decides in their view that it was suicide and not homicide, but actually the letter was faked by the murderer. But they're not experts, so at least I want them to very clearly to send the report based on the medical information. I cannot decide if it's suicide or homicide, but because there was a letter, a suicide letter, I'm determining the manner of death is suicide. Then the other side can bring a document examiner and say, This is all very nice, but the letter is fake. It's forged. I brought an expert in handwriting and she compared and said, This letter was not written by the deceased, It's written by somebody else. But the problem is when they say suicide or homicide or whatever, the manner of death, and it seems as underpinned by a medical decision when actually is not and there's no transparency. That is an example of if they keep, and I'm not against them deciding manner of death, but I want transparency what their decision is based on and to be very careful what they're qualified and have training to decide and to use and what they don't know about and they use it and may bring them to the wrong decision.

[00:25:01] Angela: So it's interesting because there's two things that I just took from what you said is context is important. at the correct time and the context that you use to draw your conclusions should be disclosed. And we've gotten into the habit of doing that in the other forensic disciplines that have already been made aware of the issues that they have in their fields or they're progressing towards the proper documentation. And we'll talk about the methodology that you're recommending or have recommended over time later on, I'm sure. But...

[00:25:35] Itiel: You hit, by the way, the nail on the head said it. Exactly correct. And I would take it one more step, that forensic pathologist to decide on the manner of death, the manner of death based on contextual information, they need contextual information. The question is what contextual information and when they get it and how it's used. I can give, for example, if you have a dead person and they have blue, marks all over their body, you want to know if that person is a boxer or a cage fighter. And a week earlier they were in a fight and they have this blue marks because this is how they fight and it may not be related to the, manner of death. On contrast, if a person died in police custody or not, it's not a context that I want them to change their decision. So if someone died like George Floyd in police custody, I don't want them to decide it's not homicide, it's an accident or undetermined. But if the same exact autopsy medical information happened, In some gang fight where a bunch of people caught someone and restrained them and beat the shit out of them or blocked their neck or whatever and they died, they would decide on that context it's homicide, but if it happened under police control, they decide, no, it's not homicide, then it's a problem.

So you see what context they used and how they use context is very problematic. And all I'm saying as you said very well, we need to open that box. Until now, the forensic pathologist have resisted discussing what context they need, how they use it, when it's needed. They said, we are medical doctor, we need everything leave us alone. And they got away with it. And my research and what I'm pushing it, I acknowledge that they need context, but we need, they need to discuss openly what context they use, what context they should not use, transparency, how they used it. And we have method to optimize the sequencing and how they use information.

This discussion has not happened and they refused to have it. And when I raised it research and I said in a British way, they were not very polite. They were very nasty and defensive. Yes, you're laughing, but it's, it was very, it wasn't, is very

[00:27:59] Angela: Yeah. That's my coping mechanism for things. so let's maybe tell people a little bit about what did you do in this study? The preliminary study, just to say, okay, let's ask this question, 

[00:28:09] Itiel: The first study we have a, was published a year ago, I believe in February, 2021. We have a new study that came out just a few weeks ago and we have a third study coming out, relatively soon.

So the first study. Had two parts. The first part is death certificates. We were able to get tens of thousands of death certificates of people who died in the United States. And we looked at children under the age of six and looked at the death certificates, the ones who died at a result of natural death. we were not interested in children cannot die from homicide when they're, two months old or two years old, obviously. And we were not interested in undetermined. We were interested in accidental versus homicide. And then we looked at white children versus black children. And when we compare white and black children, we see that relative to one another, black children are determined homicide relative to accident much more than white children. White children.are much more accidents than homicide? So this was the first piece of interesting data. Now, this data is a real cases, that is a strength of it. It's real cases, but each case is different. And it may be, as we say very clearly in the paper, it may be that black children are murdered more than white children. That may be what happened. We don't know that or not. Even here we have a problem of a base rate, expectation bias. Let's say that it is true that black children are murdered more than white kids, and I'm a forensic pathologist and I do autopsy, and I do them for five years, 10 years, 20 years. And every time a black child is more likely to be determined in homicide relative to a white kid, I get into this habit. And then it may change maybe because of a change in socioeconomical and education and society changes. Now, Black Childrens are not murdered more than white children, but I already have a bias based on my expectation to associate black children, to homicide, white children more to accident. It's like a self-fulfilling prophecy. The same thing I ask police officers, Why do you stop young black men more than white young men? They say, because the black young men carry more guns and have more drugs on them than white kids. I said, How do you know? He said, look at the people in jail. Look at the number of people who are convicted. There are many more black men than white men. But I said, did it occur to you that maybe they're more in jail and more convicted because you stop them more? And the more you stop the black kids, the more you find guns, the more you find guns, the more they're convicted. And you get in this vicious circle. So this is the first part of the data based on death certificates. So then we the, we have a second part, which is experimental data where we can give the same exact case to many different experts who determine the manner of death, whether it's homicide, suicide, accident, undetermined, or natural, as we said earlier.

And what we did, we gave exactly the same medical, the same autopsy information, but half of them, we said the child was black and brought to the hospital by the mother's boyfriend, the other half who said your child was white and brought to the hospital by the grandmother. From our perspective, whether the child is black or white, and who brought them to the hospital is irrelevant in determining if the child died from an accident. If the child, the toddler hit their head on the table or they were actually murdered, it's homicide, it doesn't matter. And we found a huge effect when the child was black and bought to the hospital by the mother's boyfriend. They was a huge decision that the story, that it was an accident, they didn't believe it, and they determined that homicide in contrast with the same medical information, same autopsy for the child, was white and brought to the hospital by the grandmother, they believed the story, many of them, and said this was an accident. Remember that those people were very confident because if they're not sure, you can always say undetermined. Indeed, many of them said undetermined, but also many said homicide or accident, depending on the irrelevant information.

Was there a correlation between, the experience level of the pathologist that made undetermined determinations rather than

[00:33:11] Itiel: This is the first study to just establish that irrelevant non-medical information can bias manner of death determination. There's many, many follow up, whether it's their experience, their training, where they are in the United States. Are black forensic pathologist also biased or not? There's many interesting questions. This was the first step, generally speaking. So every research we do answers one question, brings new questions on board, and especially a first study ever on bias in forensic pathology would just brings up the information basically that there is a bias. So this is the preliminary study, a first step. Specifically about your question. We have research in domain after domain showing that experts, and the more experience you have doesn't immune you from bias. In fact, it increases the bias because a lot of the bias comes from being an expert because the brain of experts work differently. They use more heuristics and selective attention and chunking. And I can sew a lot of technical terms, which I don't want to do. So more experience, this is another one of the fallacies, going back to the paper on fallacy where experts testify in court and believe, I've been doing it for 10 years or 20 years. The more years you do it, you increase your expertise, but bias does not decrease and often increase and we have research comparing novice with an experts showing that time and time again, including in forensic science.

It's interesting too because when you're in the courtroom testifying to your conclusions, the amount of experience you have is something that is very important to the court. They want to know. So if you're that scientist or that only has two years of experience, but the expert on the other side has 20 years of experience, somehow your findings as the new person on the block has less weight or bearing or meaning in the courtroom, which is a different problem.

[00:35:21] Itiel: Yes. But it, it is amazing. But if you want, we can start talking about mistakes of jurors. So they're much bigger because we've done experiment where we had the fingerprint examiner testify, in an experiment. And one time he was wearing a suit and a tie and he was shaven. Another time he came with a t-shirt and unshaven, same testimony. And the jurors took the testimony much stronger and more weight when he was shaved and had a suit. And we compare also when it's a man or a woman testifying young or old, white or black, all of this influences the jurors, how they take the credibility and the weight of the expert and years of experience it's very impressive. And again, the jurors bring a lot of biases, but the jurors at least don't pretend to be scientists and don't pretend to make objective decisions. When experts come to the court, they misrepresent themselves many times as being infallible, impartial. And their experience, as you said, really is impressive and I can tell you an anecdote if you want me. I'll tell you, I appeared in court. I appeared in court. I don't appear in court a lot, but I appeared in court and my side, that,commissioned me for this case, said, Can you present yourself? I said, My name is Itiel Dror. He said, Can you tell us about your degrees? I said, Yes, I have a PhD a doctorate. Now they have my cv. So they know him and said, where did you get your cv? I said, At Harvard University. So he said, I can't hear you. He said, Harvard University. So he said, You mean the Harvard in the United States? So I couldn't take it anymore. And I said, Yes, I got my PhD at Harvard. But I know many people who have PhD in Harvard who say stupid things and they're totally wrong. I want the jurors to listen to me. And here you see the lawyer trying to bias the jurors. It was in London to really be impressed before I say anything that they think, oh, because the had a PhD in Harvard, it carried a lot of weight and the other hundred years experience and I wanted the jurors to listen to me and not to be impressed. And he was pushing and pushing it this point until I had too much and he was quite angry at me and I said, Yes, have a PhD in Harvard, but it means nothing because people have a PhD in Harvard and can say things that are wrong and make mistakes and can be stupid. I want the jurors to listen to what I say, but this is exactly how they go. They know your CV and ask you about your experience and how many To bias the jurors rather than the jewels listening to your testimony. 

[00:38:05] Angela: exactly. Yeah. So I've been in a similar situation, but not quite to that extent, because you know, I don't have a phD from harvard

[00:38:12] Itiel: but you're still smarter than me.

yeah. I've also thought about maybe I should stop putting crazy colors in my hair and let my gray hair show maybe people take me more seriously then.

this is very interesting. For example, I, where I go, to high level policing and military and medical doctor, I'm the only man who have an earring. And in fact they have two. And I don't take it out when I testify in court because I think it helps me connect to the jurors, they see. I'm a human being, right? Not this Harvard guy with glasses living in the ivory tower. All of this played the role and they shave and put a suit and tie. So I play the game. Another domain that they work on, and we can talk about it another time, is branding and advertising. What is branding and advertising is to bias people to buy the product. They don't change the quality or the price. They change the packaging, the color of the packaging or the shape of the packaging or the logo. And people buy or don't buy the product based on that. You say, Oh my God, people are so stupid. Yes, we are all stupid because we are irrational and it affects experts, too. So all of these things really impact those issues. And going back to what we were talking about, so the years of experience. It's very impressive to the jurors and to the fact finder, but not necessarily say anything, especially in the forensic domain where you could have been an expert for 20 years and testified in 500 cases and got many of them wrong, but the person was convicted. It's not like the banker and the medical doctor where you know you've made a mistake and you can say, I've been a medical doctor, or I've been a banker for 20 years and my clients on average made X amount of money and only this percentage of the client lost. You have data to say how good you are and how bad you are, how many mistakes you've done. Medical doctors also, and I'm not saying it's good, but many heart surgeons will have statistics. How many patients died when they did the heart transplant when they operated, And it gets more messy because it depends how complicated the case is. But in forensic science, we don't know if and when the forensic scientist make a mistake. So how many years been doing. Is nice for some things, but doesn't say anything about the mistakes, and definitely is not a protection from bias.

And then of course, the only way that we have to try to measure and figure out if we're doing things correctly in forensic science is to do proficiency testing or inter laboratory comparisons, but these only closely resemble what you would see in a forensic case, but they can't actually truly represent what you actually see, because what we do in forensic science, we never know the true answer. We can only come up with some possible explanations for the data that we're observing, and we have to consider all these different alternatives and possibilities. When we're doing it, but we don't ever really know the true answer. And I think that's the same for the pathologists. They don't, You don't know the true answer unless the whole thing was caught on video and technicolor sound and movie theatre quality. 

[00:41:26] Itiel: Absolutely and with the error rate studies, there are many more problem. We can do a number of podcasts on that. First of all, the error rate studies are done by forensic examiners. Or the organizations like the FBI pays companies to do it. It's almost like the tobacco industry doing research on smoking if it's bad for you or not. It's people who've been doing it for decades now doing research to validate their techniques.

Beyond that, they have a lot of problems. For example, how they treat inconclusive. So when they calculate mistakes, you do a test. I'm gonna give you a hundred cases to do a hundred test items, and we will see how many you get wrong. And you can say inconclusive. I don't know the answer. If you say, I don't know the answer, they never count it to the mistake.

So if you say, I don't know. I don't know everything, you have zero errors. Imagine having to do a driving test and before that you do a theory test and you have a hundred questions, multiple choice, and they say If you make more than five mistakes, you fail, but you don't have to answer. If you have a question and you're not sure, you can say, I don't know, and we don't count it to the mistake.

So that have that issue of inconclusive. They don't count inconclusive. Any examiner in error rate studies have been dozens of them who said, I don't know, it's too difficult. I'm not sure, and skipped the test item. They don't count it as a mistake, so they only count mistakes the test item. The examiner was confident to make a decision.

[00:43:00] Angela: But does that mean that inconclusive is an incorrect answer? sometimes you can make that you don't know. 

[00:43:06] Itiel: Absolutely. Some inconclusive is correct and some inconclusive are incorrect. Absolutely. And as any forensic examiner that will tell you in any lab, sometimes they get evidence that if inconclusive, there's not enough quantity and quality of evidence to determine. And the correct answer is inconclusive. And if you have inconclusive evidence, Inconclusive is the correct answer. The error rate studies do not include inconclusive evidence. They don't include inconclusive items. And sometimes it's clearly a match or clearly not a match. And if you say inconclusive, that's an error. And we have a paper on that, that stir a lot of trouble in another domain where we said, deciding not to decide. When you say, I cannot decide, it is a decision, it's a decision not to decide. And that decision can be correct and that decision can be incorrect. Something it's correct, something it's incorrect, but you can't say every time, every inconclusive decision is never a mistake. Some of them are absolutely correct and some are not.

[00:44:16] Angela: Any determination that you come to you, you should be saying that based on the information I've been given, based on the data, this is my conclusion. You have to be, like you said before, you need to be transparent about it. Why did you come to that decision? what, what was involved in your decision making process. And one of the things that you were looking at .In the, these pathology, studies. Oh... 

[00:44:37] Itiel: Just more on the inconclusive and error weights to put one more nail in the coffin. In my view, and you may disagree or not on the viewers, but in my view, if you have two examiners looking at the same evidence and one said it's a match, I'm sure this is a match that's in firearms or in fingerprint.

And once this is inconclusive, they cannot both be right either there's enough evidence in the bullet cartridge on the fingerprint to determine a match, and then the person who said it's inconclusive, she's wrong because it's a match. They can determine a match. Or the person who said it's a match is wrong because it's not enough information.

But you can't have to examiners give you different answer, and both of them are correct. One is correct and one is wrong. Now I cannot say which one, but I can say it can be both of them are correct and both of them in error rate studies are not considered an error.

[00:45:31] Angela: So the mutually exclusive hypotheses cannot both be true.

[00:45:35] Itiel: Yes. But error rate studies, they can count both of them as correct response or will not count both of them as an error. So it's a bit messy how they calculate. And again, people can go to my webpage or other places we argue with them, and they continue to do the study, then present really low unrealistic error weights because of the way they conduct the studies.

Well, that's true for any study, right? If you didn't construct your study, right.But it's, a legitimate thing to criticize how someone constructed the study. Like how they did it and have recommendations or thoughts and how it could be improved. But some of the reactions that you've gotten are completely emotional. They're unprofessional, to say the least.

[00:46:22] Itiel: So look, first of all, every study has limitations. Every study needs to say what the limitations are. And every study they said earlier answers one question, then bring new question. So in all my studies, and especially in the forensic pathology paper, they can, the audience is more than welcome to read it. It's a, you can put a link on the podcast, it's publicly available, can see that in the study, we are a bit repetitive saying, this study show this, but it doesn't show this. And we need more data and here are the limitations. So every study has limitations. that is true. But not every study has, flaws. But in this specific study, we published it, and again, a first. They're looking at and showing a bias in forensic pathology decision making in determining a manner of death, and the response was very defensive, aggressive, nasty, big pushback. But not only from personal, I understand some person got offended and wrote a nasty email or something. All other we're talking about the leadership, the professional organization that is supposed to provide leadership and increasing the professional, expertise, played into it like a professional union, defending its members, whether they are right or wrong. So the National Association of Medical Examiner, NAME. led, they put fuel into the fire rather than putting the fire down and said, Let's investigate it. Let's consider it, Let's have a discussion. And they filed a whole series of complaints to my university, to the journal, and a whole range of complaints. There was a total, I believe, of eight complaints out there. And the complaints also are very nasty and personal. And they wrote a whole bunch of letters to the editor and to such a level that the editor of the Journal of Forensic Science, he himself wrote a letter to the editor before he published like 22 letters, saying I've never seen, there's never been anything like this in the history of the journal of such unprofessional and emotional responses. Let me give you an example, The statistical analysis in the paper. So first of all, the paper was peer reviewed by three experts who looked at the statistical analysis. However, NAME decided the statistical analysis they believed would incorrect, nevertheless. So if I would complain against someone just out of modesty, I would say to the journal, to the university in the complaint, we've read the paper and I'm very concerned that statistical analysis was incorrect. Can you look at it? So they complained to the journal in a very nasty way, and the journal, even though it was peer reviewed by three experts, decided to send it to a statistical expert after it was published because of the NAME complaint, and a statistical expert- now it's reviewer number four, looked at it and said, The statistic is perfect. There's no flaw. The statistics is fine. Then they complained to my university about the statistics and what do they say? They don't say, the statistics may be wrong. We are concerned. They said the statistics is flawed. There are terrible statistical mistakes. And then they said even a junior professor, junior academic would not make his trivial mistakes. Dr. Dror is an embarrassment to your university, he's an embarrassment to science. And they said, there's no statistical mistakes, so this is very nasty to say you embarrassed. And then they say that the flaws I made in the paper were intentional. You know, A), they are no flaws and B) even the flaws, you know, I make mistakes, I'm human, but they're mind readers. The attribute, intentional deception and bad intention. And it's not a private email. And what they write on the chat boards behind my back, this is what they complain to the university against me and not an examiner, this is their professional organization, their leadership, the president, an executive president and vice president. I don't remember all the signatures on the formal complaints to the university accusing me of intentional deception of being an embarrassment to the university cause I am making statistical, trivial statistical mistakes that a junior professor wouldn't make.

This is very nasty. Now, people tell me that's a real good sign that they're going after you personally and they're nasty because they can't go after the science going after you personally. But still it was, I have to say I've been there before. I had it with DNA, I had it fingerprint, I had it with forensic anthropologists, but all of them have come around and none of them are even close to the forensic pathologist.

When this all came out, I'll give you one more example, then I'll shut up, but I can give you example after example. I wrote to the president of NAME, I said to him, Look, we disagree. Why don't you bring me to your conference? They have an annual conference. Bring me to your conference. You don't have to pay me.

You don't even have to reimburse my flight. I'll fly from London. I'll pay it out of my own pocket and the expense it, and we'll have a panel to discuss it and you can put whoever you want on the panel. I'll present my side and they can say whether why I'm stupid and we are having this open discussion.

They were not interested. So we are talking here about supposed a scientific professional organization and people who testify in court. Now, it's been over year and a half, and I hope that things have calmed down. I've tried to be optimistic, I hope I'm not naive, and I hope likethe fingerprint, like the forensic anthropology, like digital for when they like, DNA, they've all come around. Let's hope that the forensic pathologist will also come around even though they had this very strong emotional, unprofessional, nasty personal response. Let's hope that after the emotions calm down a bit, they will look, they don't have to disagree with me and we can discuss. The issues that need to be discussed are quite simple.

Two issues. One is what is relevant and what is not relevant. What information they need as forensic pathologists, as coroners, as medical examiners as people who determine the manner of death, what information they need and what is relevant or irrelevant. That's number one. And transparency. This is still under number one, transparency, what information they used and how they used it.

Number two, the relevant information. What's irrelevant, what they don't need. They shouldn't know if it's irrelevant. And the information they need, what sequence did they see it? This is where we invented linear, sequential unmasking because you can't see all the information at once and what you see first and second has a lot of impact on you because your brain, even if you try to keep an open mind. You can't, the brain processes information. So you see the first piece of information, it gives you a hypothesis and it affects how you look at subsequent information. So what order should you see the information? We want them to consider? And not only them, every forensic examiner, what order to see the relevant information.

And we are very practical suggestions. I don't wanna go into too much details. People can read the paper on linear sequential unmasking, but for example, some evidence is more objective. Some lev evidence is less objective, right? If you have a murder scene and you have a camera, you have a video camera, and you have an eyewitness, you need both if you need that.

But the camera is more objective than human memory. So start with the camera and then look at, the eyewitness. Or if there were two eyewitnesses, one was sober and one was intoxicated, start with the sober one. Because sober eyewitnesses are more objective than people who are intoxicated, so they're more objective.

Also more relevant if you have to decide if someone committed suicide, then you have evidence like an empty prescription bottle next to the bed of the dead person. That is very relevant cause you can see the medication that they may have taken to commit suicide, and you can run a toxicological exam.

Another piece of information may be the neighbor said they were very depressed lately. So that's relevant maybe to decide if there's suicide, but the bottle of pills is much more relevant than the gossip of the neighbor who thinks a person was depressed or not. So start with the more objective and the more relevant.

And the last criteria very quickly is the bias. Some information is very biasing by its very nature, and some is more technical and doesn't include so much bias and emotions start with the less biasing information. And then you sequence the information and you optimize. You have to see it in a certain order.

You can't get around it. So I want them to consider with cognitive input, what sequence you want to look at the information to optimize it. So that's in a natural, what they need to do. It seems very simple to me. I feel like the kid saying the Emperor has no clothes on and they don't wanna hear it because when I say it every time, you know what they say, It's not so smart, not so insightful. It's quite clear and obvious, but they have not heard it for decades. And when I came out with it, they were not very receptive. I've never received it. Thank you very much. It's very important we think about it, but rather nasty complaints. And I'm not even talking about personal hate mail. I get hate email. People talk behind my back. really going after this personally and emotionally, rather than having a professional scientific discussion cognitive bias. That's quite sad. Hopefully it'll change.

I think maybe there's a level of people who are just being defensive. that emotional, defensive, knee jerk reaction to something that's like a, it would be a major paradigm shift in their field because it is brand new to them. They haven't considered it. And their medical doctors is by and large, and a lot of them are trained to be, to think a certain way, and have a very high degree of confidence in everything they do. They can confidently go wrong.

[00:57:02] Itiel: But don't you, even on a personal basis, let alone if you are a scientist, when somebody really gets me angry and I write a real response email, I don't send it, cause I know I'm emotional. I write the email, but I save it in a draft and say, let's sleep on it for a day or two and come back to it when I'm a bit more relaxed and not so emotional. Don't make big decisions when you are emotional and don't give a knee jerk reaction. This is on a trivial personal level.

If you are a scientist and doing science and doing important forensic science, that a child may have been murdered and you may classify it as an accident and let a child murderer go free. Or alternatively, a child died of an accident and you say homicide, and not only someone can be falsely convicted, even if they don't convict you, leave a tragedy of a dead baby. Family knows he was murdered, the child was murdered. Even if you never convict someone, it's terrible mistakes. You wanna minimize it. You owe for fairness and justice to say, Look, this is very emotional, it's very upsetting, but let's calm down the leadership and every forensic pathologist, let's calm down. Let's relax, come back, read the paper again. Let's bring him, let's discuss, let's try to understand and not have this emotional, defensive, knee jerk reaction.

[00:58:24] Angela: Yeah, you're right. Because you, if you need to learn to recognize when you're emotionally compromised. And that's something I think you develop as you get older and in a professional environment, it's not okay to have an emotional outburst in the workplace. You should take 10 seconds and go outside for a while. but it, and it does impact your work in forensics cuz when you're having a bad day in your personal life, that's a bad time to go into the lab and be looking at your. It could color your judgment. Just like,you mentioned emotional responses and you're dealing with a dead child that is, in and of itself, the case that you presented to them is very emotionally triggering. Or it could be emotionally triggering to some people 'cause nobody wants to deal with, be confronted with the possibility that somebody murdered a child. It's hard enough when they pass away naturally. So you would think you would wanna be clearheaded and not take into co in consideration information that could be irrelevant to the actual determination of what's going on.

And it's sometimes difficult when you've always had all of that information given to you to step back and realize, okay, actually that information about the child's race and who the caretaker was that brought the child in. is that really relevant to my, medical determination that I'm making? Or should that be set aside for later? Let me look at the injuries. Let me document everything that happened and then I can come back and look at that. And I know you were saying with the linear sequential unmasking, one of the recommendations that you have is that there's a case manager, right? that kind of, I don't know if they, they're like the gatekeeper of the context that you need as you progress through your analysis.

[01:00:10] Itiel: And Yes, before we go to technically case manager and how you deal it, I just want to say, to try to understand the forensic pathologists, even though I think the response was not great, I'll say it in a British way. First of all, you have to understand, Or, I need to understand and I try to understand. This is the first paper ever to bring out the rabbit out of the hat. So they got caught with their pants down. No one has ever done research on that. They've never been called bias got away with it. So the first time it comes, that earth is not flat it's round. Or the first time someone says, the universe doesn't turn around earth. We go around the sun or whatever. It's a shock. And I understand the initial shock. And also going back to the beginning of when we chatted about the adversary legal system.

They're worried that this will be used against them in court. Until now they come to court and say, I'm a medical doctor, I'm an expert. I've been in it for 20 years, the coroner, or medical examiner, whatever. And the court loves them and listens to them, and they're not given a hard time. Suddenly this paper, they're afraid will [?]. Are you biased? Are you making a decision unbiased? So the response was so emotional, and I'm trying to think in those shoes, not that I agree with their response, but understand and have some sympathy and some empathy to their initial response. And what will determine in long run, whether they take it on board, or not. Going back to the paper and to your question about linear sequential unmasking. So there are two questions here. A, what is relevant or not, and B, when they need to be exposed to it. So in the paper I think they agreed and everyone agreed that it's not relevant whether the child is black and white, but what they were trying to defend, which I think was very wrong, and indefensible, that it is relevant, who brought the child to the hospital, and with exactly the same medical autopsy information, identical medical, they will say it's a homicide or an accident based only who brought the child to the hospital saying, they believe that statistically speaking, boyfriend of the mother is more likely to murder a toddler than the grandmother. Even if I accept the statistics, which I don't can discuss why for hours later, even if when accepted, and let's say 80% of the cases the children who are murdered are going to be murdered by the mother's boyfriend and not the grandmother. Still, that is not enough in a specific case to say that, right?

Generally, men are taller than women and if I have a certain height, quite a tall height, it's more likely to be a man, but it can be a woman and if someone is a bit short, it can still be a man to take general statistics, even if they're true. And I have doubts about how they do the statistics and apply to a specific case with no medical. To say homicide based on that, I think is very bad. It's a bias and no transparency. Cause I dare, a forensic pathologist to go to court and say, This child was murdered, it's homicide, and I base it only because the mother's boyfriend brought him to the hospital. If the grandmother brought him to the hospital, then I would say it's an accident.

No. What they do, they give the impression and also convince themselves that also medical information said it's homicide because they're biased, knowing that it's a child's mother's boyfriend that brought him to the hospital, and then they're biased in how they interpret the autopsy and the medical finding. And then it's hidden under the pretence of medical information. That's why we want the linear sequential unmasking. We want initially for them, even if it's relevant. Let's play the game and say it's relevant. Who brought him to the hospital? Say to them if they would listen to me, the forensic pathologist. Okay. You have that information, but I don't want you to start with that information before you do the autopsy, before you have the medical information. First, get the medical information, you get the autopsy, and make your decision. And after you make your initial decision based on the medical information, then get the information that you consider relevant.

I don't think it's relevant, but you consider it. Have it. I'm not going to deny or deprive you of it, but get it later. And then you get the information that the child was brought to the hospital by the mother's boyfriend. Initially you said it's an accident or undetermined and now I say, Oh eureka!, it's a mother's boyfriend who brought him to the hospital. It must be a homicide. So now we have transparency. Before they knew it's a mother's boyfriend, they said it's undetermined or accident. Now you need to classify and to sequence the information. One way to do it is the case manager, as you suggested, where someone looks at all the information and sequences it to the person who makes a decision.

Often you don't need to do that because if the forensic pathologist will discuss and sit down and agree to talk about it and is okay to talk, I think we'll agree. If they just listen and get in a discourse rather than personal attacks, then you can up your in, not case by case, have a list of how you order the information.

They're very repetitive. You can clearly. Say, what is the normal sequence? And then you don't need, in every case to have a case manager and decide the same in fingerprint, in firearm, in DNA. And I can give example from any one of those domains where certain things are sequenced that are in every case, part of best cognitive practices or standard operating procedure, what you see in what order, and you don't need the case manager.

Sometimes in complex, unusual cases, you may have a case manager, but most cases it's very clear the order you need to see the information. The problem is not the solution. The problem with acknowledging that there is bias and you need to do something about it. Anyone does that. They're halfway to solving and minimizing the bias.

The problem is a bias blind spot where you deny the bias, believe you are an expert, believe you have many years of experience, and that immunes you from bias. So you're not willing to listen or to do anything about it. That is a problem.

[01:06:50] Angela: Yeah. so that, are they, from what you were looking at, it's the part of the issue is not only are they not recognizing the bias and they're apparently opposed to having a conversation about it, but in the meantime, what they're doing is they're determining this is how someone died, and then muddling it together with that contextual information. So not recog, not letting the people who are on the receiving end of that information understand that here is medical determination, here is taking into account this particular contextual information and not considering that perhaps that contextual information is incorrect or incomplete and that it has an effect on their overall determination.

[01:07:33] Itiel: Absolutely. But worse than that, it's not only that they're muddling it together to the fact finder, to the judge and the jury in the police in their own mind, they're muddling it because they misinterpret the medical information based on context and they're not aware of it that their exposure to the irrelevant information changes how they see and how they interpret the medical information.

So they believe the medical information supports a certain interpretation. But it's not based on medical information, It's based on the irrelevant contextual information. So they're muddling their own cognitive decision making processes without awareness, without being aware of how the context cause them to see the medical information in a different way. And the only way to deal with it is not to see the contextual information before the medical and to see the medical to make a decision, see the context.

[01:08:32] Angela: Okay. In the, the second study that you got, that you just published a few weeks ago, and it'll be interesting to see whether or not you get more demands to have that one retracted, So that was one of the things that you looked at was the order of the information that you gave them. Some of them received the contextual information first. Some of them received it after they'd made their initial determinations. And what kind of effect did the or that order of information have on bias?

[01:08:58] Itiel: What we found that the order of information where you get the medical information first and then the context or vice versa, the context or the medical, depends how strong the medical information is. When the medical information is very, very limited, it doesn't say very much. Then it doesn't matter when you get the context. The context is going to take over cause the medical information didn't say very much. On the other extreme when the medical information is very clear, the context doesn't matter because the medical information is so loud and open and clear. Then it also, it doesn't matter when you get the context. The order, the context is in the middle area. When the medical information is open to different interpretations, then the order is very important. So it depends on the level of the medical information and the context and the order of the context seem to be most powerful. When the medical information gives you some information, but it's ambiguous and open to different interpretation, then it's very critical,

So when you have somebody who's very clearly had a knife that bisected their aorta, it's pretty clear that someone, they were stabbed in the chest . And especially if you've got other information, someone else has injuries, this contextual information can be helpful. But you've already decided that person was stabbed in the chest, 

[01:10:28] Itiel: Yeah. If you get a person, 

[01:10:30] Angela: stabbed 79 times times. 

[01:10:32] Itiel: Yeah. The example I give is, someone had five bullet holes in the head. Well that's not suicide. but if they have one bullet, like in an experiment, then it can be suicide or homicide and then the medical information is not very helpful. so it depends, exactly. And the slippery slope, what they call the bias danger zone is exactly in the cases where these different interpretation. The same with fingerprints and firearm and DNA. When the fingerprints match perfectly or when the fingerprints are totally different, the context doesn't matter. But when the fingerprints are similar but not kind of in the middle, and it depends on your interpretation, then the context has the greatest power to influence interpretation of the evidence and then the order is very important to the evidence first a context second.

[01:11:27] Angela: Yeah, I think it, it's interesting to me because I'm thinking back to how things have changed in forensic science just in the 20 years that I've been involved in it and I remember... reports come out and studies are done and everybody's Oh no, that's not me. I'm not like that. We're perfect. We didn't do anything incorrectly. And then slowly over time you're like,actually we could have done that better. We weren't being as objective as we thought we were. And so we developed tools. like you were saying, that the linear sequential unmasking has been very helpful in forensic DNA, for example, and I know I use it in my blood stain pattern analysis as well. I can't, I don't want to know any of that contextual information. I just want to look at the data. I just want to look at the spatter patterns. And then later on, if I need contextual information, I will use that. I mean, you need something up front, right? You need something to know, Okay, I'm looking for blood, or I'm looking for this other kind of body fluid. You gotta have some kind of indication. But we don't necessarily need, we don't need to know what somebody's race was. We don't need to know how old they were. We don't need to know what their names were because you know, you get certain types of names, is that going to unconsciously bias people in a certain direction? And the more information that we, some of the information you need to strip away some of the information you need to have, but you only need just enough to get started and then you can add in the other information later on.

[01:12:53] Itiel: Yes. I'll give you, I'll say two things. One, all I'm saying, and when I said it's so trivial, I totally agree. Some context you may need upfront, some you may need later, some you may never need. All I want the forensic scientists, and now they're doing it except the pathologist and I hope they will join the rest, is to consider, to be able to say we need to discuss it and consider what context we need in the beginning, what is later, what we don't need, rather than I need everything in advance. So to enter and to think about it. 

The example you gave about names, just an example to show you how powerful it is. So there's an experiment in the United States where people applying for a job, sending the same CV, same GPA, same letter of recommendation, same employment history- would just change the name. One time we give a black name, the guy's name is Jamal, or a white name David. If it's a woman, sometimes we call her Laquisha and someone, we call her Elizabeth. You think that doesn't have an effect? If you had the white name on the same application for a job, you 50%, 50% more likely to get a phone call to invite you for an interview. Not five, 10%, 50% more if you have a white name on the same CV, same employment history same identical application. A white, just a name, a white name, you're more likely to get a call and inviting you for an interview. That's how powerful even a name is, as you said. And criminal record and again, more and more stuffs. 

[01:14:30] Angela: Information that is prejudicial, but people don't recognize it, that it is prejudicial. 

[01:14:36] Itiel: And whether, and we don't even have to argue, if it is or not, if you don't need it, don't have it. Rather than getting a discussion, someone, the name doesn't impact me. It does. You don't know about it. I'm not accusing you of intentional racism or whatever, but it does. But why even get in a discussion if a name is important or not. You agree that you should not change your DNA or fingerprint or forensic pathology decision based on the name. I hope everybody agreed. Then why have it? Let's get into an argument or debate if it biases you or not. If it's irrelevant and it shouldn't impact your forensic decision, don't have it. That's quite simple, right? Simple to me, but not to them. They will not even discuss it. I did everything and hopefully again, I want to end, or try to end on a good optimistic note, the fingerprint examiners initially also said they're not biased. And if any fingerprint examiner is biased, they should seek employment in Disneyland. You know, this is what they the response, and now we are laughing, but this is our official response of the Chairman of the Fingerprint Society wrote in response to our research, youknow, when we came out with fingerprint bias and they came around the DNA and everyone, including the leaders, of DNA like Bruce Budowle had the paper with me on using Linear Sequential Unmasking and DNA so on. So all the forensic domains have taken steps forward and let's hope that the forensic pathology, after the emotions come down, they will agree. I don't want them to even agree with me to listen, to have a discussion. I hear many of them even give lectures. Attacking linear sequential unmasking, they don't even understand what it is, and they're attacking and going in lectures, why it's so bad. Let's have a discourse, let's have a discussion so you understand what the cognitive behavior scientists say, and then you can take it or dismiss it out of understanding and after a professional scientific debate rather than emotional. And I encourage the readers to go and read the letters, that they wrote and the response and judge for themself about all this issue. 

[01:16:49] Angela: Yeah. I, I have all, a lot of those exchanges. I'm happy to share them on in the show notes so people can have access to them and see them for themselves and make their own determination of how they thought this- 

[01:16:59] Itiel: Excellent. 

[01:17:00] Angela: Discussion Or lack thereof went. if that's okay with you? 

[01:17:04] Itiel: Yeah, abso- absolutely. It's Open Access and as you said, the new paper just came out and meanwhile, I haven't seen any complaints and I hope there won't be any complaints and there will be more discussion of the topics themself and in the next paper. And I have to say that in all my paper, I have the forensic experts as part of the research team. I'm not doing this academic without understanding. The first paper had seven, also four of them were forensic pathologists. The new paper has, again, multiple forensic pathology experts as authors on the paper. So there are pathologists that are open and in the United States, I have to say, I wanna give the other side. There are forensic collaborator who do forensic pathologists that have been open, invited me to give training and workshops I'll consider procedures. So it's not all the forensic pathologist, it's the initial response. And slowly, hopefully it'll change. We will see.

Well, I think that if you're, if you look at what happened in the past with all the other forensic disciplines slowly coming around, I think they will. And especially because even some of the criticism that the initial study received, it wasn't all negative. It wasn't all attacking. Some of it was, Okay, did you consider this? And it was constructive in nature. Maybe it was a little overshadowed by some of the more emotional ones, but it wasn't entirely bad. And you haven't gotten hate mail yet on the new paper, so... that's a bonus. 

[01:18:39] Itiel: Not yet,but to put fuel into the fire, this topic is a very hot topic because of Black Lives Matter and people dying in police custody. So in the United States, there are many cases of people who die in police custody. And there's a big question about the forensic pathology decisions of manner of death. And for example, the state of Maryland, the Attorney General is looking into the cases of people who died in police custody, were the forensic pathologist making impartial decision about manner of death or they had the bias because it was in police custody or because the person who died is black. So this is an ongoing issue of, that's going on and especially in the United States and especially it relates to people who died in police custody and how, and what the forensic pathologists decide on manner of death when people died in police custody. This is going on and will happen in many, not only in Maryland, probably across the US and hopefully there can be a constructive discussion regardless of what all these different, examinations will show. What is the best way to move forward.

hopefully there's enough documentation for them to actually make a objective determination in, those in, those studies to determine it. Because if you didn't record it in your notes... 

[01:20:08] Itiel: Studies that is a, if I cannot say a lot on that because I've been involved in this, effort in Maryland. All I say, you have no idea how correct you are., what you said , you have no idea. And I say no more how correct you are.

yeah. Document things. That's one of the things you learned in all the other forensic disciplines. If you didn't document it it didn't happen, and it's so important 

[01:20:31] Itiel: in all sciences, Any person that does science, not in university, in high school when you do chemistry or physics or biology

[01:20:38] Angela: yeah. 

[01:20:38] Itiel: and you do a lab report, is to document your lab report what you've done. That's science 1 0 1.

and the important thing about forensic science being accredited, I'm not saying accreditation is a perfect metric for determining the quality of the laboratory results, but some of the things that you do in an accredited discipline where you put your date on your paperwork and document when things happened and when you made a particular decision that's so important to these retrospective looks at all of the cases. It's so important in cold case investigations and post-conviction testing that happens, and I, It's okay if you know the science has moved on. We've learned more. We understand more, and maybe you are going to back and look at something that you did 20 plus years ago and it was perfectly valid and accepted at that time. But because we know more now than we did, maybe we want to go back and take another look at it and say, Okay, maybe this wasn't the correct determination. We could have done this better. Maybe we should add some statistical analyses in on our determinations , rather than just saying it's a hundred percent.

[01:21:48] Itiel: Yeah. And but not only going back 20 years, even now, one of the things we are really pushing is documenting your report, what contextual information you had. We want transparency. What did the police, detective, what did she tell you? What did you know about the case? So there needs to be documentation, even if it's a year later, and you can ask them in court, they don't remember what they knew, who told them when they knew.

And in the report, if you work on the case, if the police detective called you and gives you certain information, you write it in your logbook. So we want to know at least if you were exposed to irrelevant information, what it was, and when you were exposed. Basic documentation, as you say, it's very important so we know what happened. They'll try to understand it. 

[01:22:35] Angela: And an extension on that is that the, in the US adversarial system, the other side needs to know to ask for that information, they need to go and look at that information. You can't just accept, here's this forensic report and we had these, this forensic evidence and this expert made this determination. Therefore, you should just confess to the crime and go and serve your jail time. Because a lot of times that information is misrepresented or misinterpreted, misunderstood, and there's so many instances of wrongful convictions, not because the science was bad, but because the science was misunderstood because people didn't take the time to look at it and understand what it meant and what it doesn't mean. 

[01:23:19] Itiel: Precisely when you look at wrongful convictions. for example, the Innocence Project in New York who exonerated hundreds of people based on DNA technology. They looked at the case why people were wrongfully convicted, and I was hoping they're going to find incompetent lawyers. Nondisclosure of evidence to the defense. More than half the cases of wrongful convictions involved flawed or exaggerated or misunderstood forensic science. So forensic science contributed to the, wrongful conviction in many of the cases. Then we want forensic science to convict people who are guilty and to exonerate people who are innocent, and forensic science has the power to do that. If it's practiced scientifically and if we minimize the cognitive biases and this is the way forward and many forensic disciplines are taking it, and hopefully it would continue.

Well, I hope that this second study that you've done and the third study that you're doing on forensic pathology, has a more positive outcome and that it does gain some more traction and and acceptance because I know that it's been, at least for the realms where I'm an expert, it's been, essential to my work and understanding that, and for the forensic evidence to actually have meaning and proper meaning. So like you said, we exonerate the innocent and only make and make sure that, if someone's going to be punished for something, that it's the right someone.

[01:24:51] Itiel: Absolutely. 

 so did you have any other closing thoughts that you wanted to share?

I do, but we've already talked for a very long time and it's Sunday after Sunday evening. We started in say, Sunday afternoon. So I think that, the readers who wanna read more, they can go, you can put links on the podcast website and they can read more and follow the discussions. And I would like, not to be naive, but, hope, things will get better. At least we're going to try, even if it doesn't get better, we're going to try to make it better. So if it doesn't get better, at least we know we did our best to move things forward.

And of course I'm would love for all the listeners. if you have any questions that you want me to pass on or any comments or thoughts that you had on this episode or about understanding more about cognitive bias, please email those to me. and I will be sure to share those with Itiel. Because I know you, you, appreciate feedback a lot. 

[01:25:50] Itiel: Absolutely. a lot, and I'm happy to hear criticism, that's how we learn. And I'm not perfect and I don't, I'm not error free and make mistakes and I learn. And every paper hopefully it gets better by learning. And this is the process.

[01:26:03] Angela: Yeah, the feedback is a gift.

[01:26:05] Itiel: Absolutely. 

[01:26:05] Angela: Thank you very much for your time on a Sunday afternoon. I really do appreciate, especially since we had some scheduling difficulties early on trying to get this, but it's good because we got more information because we had your second study came out. So thank you again. It was great talking to you. 

[01:26:20] Itiel: Thank you for having me and giving me the opportunity to 

[01:26:21] Angela: My pleasure.