AI Can Predict People’s Race From X-Ray Images, And Scientists Are Concerned

A new study found that computers can figure out what race a person is just from looking at their X-rays. This is something that would be impossible for a human doctor. The findings raise some troubling questions about the role of AI in medical diagnosis, assessment, and treatment. Could racial bias be unintentionally applied by computer software when studying images like these?


Key Points:

  • A new study has found that deep learning models based on artificial intelligence can identify someone’s race just from their X-rays, something that would be impossible for a human doctor looking at the same images.
  • The findings raise some troubling questions about the role of AI in medical diagnosis, assessment, and treatment: could racial bias be unintentionally applied by computer software when studying images like these?
  • Having trained their AI using hundreds of thousands of existing X-ray images labeled with details of the patient’s race, an international team of health researchers from the US, Canada, and Taiwan tested their system on X-ray images that the computer software hadn’t seen before (and had no additional information about).
  • The AI could predict the reported racial identity of the patient on these images with surprising accuracy.
  • Right now scientists aren’t sure why the AI system is so good at identifying race from images that don’t contain such information. It’s possible that the system is finding signs of melanin that are as yet unknown to science.
  • The research adds to a growing pile of evidence that AI systems can often reflect the biases and prejudices of human beings.

Personal Insights:

  • I think we’ll soon discover some amazing new capabilities that we’ve never had. But at the same time, learn how the training data has historical biases or inaccuracies that have downstream implications that we haven’t yet thought of.
  • This will be a tricky new world as we celebrate with amazement our new superpowers, but at the same time discover frustrating biases and unpredictable outcomes.

Article Highlights:

  • “Our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging,” write the researchers.
  • “We aimed to conduct a comprehensive evaluation of the ability of AI to recognize a patient’s racial identity from medical images,” write the researchers in their published paper.
  • Scientists aren’t sure why the AI system is so good at identifying race from images that don’t contain such information, at least not on the surface. Even when limited information is provided, by removing clues on bone density for instance or focussing on a small part of the body, the models still performed surprisingly well at guessing the race reported in the file.

Source Article:

Paul Helmick

Paul Helmick

Technology CEO

@PaulHelmick
@HealthVentures

An Experienced Entrepreneur.

I love helping people use technology to grow their business.

I live Charleston, West Virginia with my amazing wife and two wonderful children.

 

About This Site

Artificial Intelligence (AI) will dramatically transform business and industries in ways we can only imagine.

This site shares the resources and references I’ve found in my own journey to learn more about Artificial Intelligence.