Artificial Intelligence in Health Care: 9 Hard Truths
Artificial intelligence in health care is having a moment. Not a “cute demo” moment—an “it’s slipping into real hospital workflows” moment. And once AI gets embedded into documentation, triage, patient messaging, decision support, and scheduling, it stops being optional. It becomes infrastructure.
That’s why a Toronto researcher getting funding to study the consequences matters more than yet another flashy AI announcement. Rahul G. Krishnan, a University of Toronto assistant professor focused on computational medicine, received an Amazon Research Award to support work that aims to build machine learning into the health-care system—while also confronting the risks that come with it.
Here’s the blunt truth: artificial intelligence in health care can save time, reduce errors, and improve outcomes—but it can also scale bias, normalize bad assumptions, and create “silent failures” that are hard to detect until people get harmed. So let’s talk about the parts that actually matter: safety, fairness, regulation, accountability, and what hospitals should demand before they deploy anything at scale.
🏆 Why this Toronto award matters (and why you should care)
The Amazon Research Awards program funds academic research that aligns with Amazon’s scientific priorities. In April 2023, Amazon announced 79 awardees from universities across multiple countries, including a proposal from the University of Toronto focused on “towards a learning healthcare system.” That’s the sort of phrase that sounds harmless—until you realize it implies AI woven into the everyday logic of care delivery.
Krishnan’s work sits right in the blast radius of the biggest question in artificial intelligence in health care: How do we get the benefits without turning patients into collateral damage of automation?
If you’re a patient, you care because these tools can influence what gets documented, what gets flagged, and how quickly a clinician gets to your case. If you’re a clinician, you care because AI can either reduce burnout—or create a new layer of “admin chaos” where you spend your day correcting machine output. If you’re a policymaker, you care because medicine is not a playground for “move fast and break things.”
🧑🔬 Who is Rahul G. Krishnan (in plain English)
Rahul G. Krishnan is a University of Toronto assistant professor working at the intersection of machine learning and medicine. The North Star of his research is building tools that can support clinical decision-making and create a “learning” health-care system—where digitized clinical and biological data helps improve care over time.
The practical angle is obvious: better interfaces for patient records, better clinician support tools, and faster translation from data to useful clinical insight. The risk angle is equally obvious: when you build a system that “learns,” you must be deadly serious about what it learns, who it learns from, and who gets left out.
🩺 Hard truth #1: Medicine isn’t static, so AI can’t be static either
In artificial intelligence in health care, people love to talk about “accuracy.” Fine. Accuracy matters. But medicine changes constantly: new guidelines, new drugs, new protocols, new population patterns, new outbreaks, and new standards of care.
So even if your model performs well today, you need a plan for next month and next year. Otherwise, you’re deploying yesterday’s assumptions into tomorrow’s clinic. That’s not innovation. That’s a liability with a user interface.
What to do about it: treat every clinical AI tool like it has an expiry date unless proven otherwise. Build a re-validation schedule. Monitor drift. Set rollback triggers. If you can’t turn it off quickly, you don’t control it.
🧾 Hard truth #2: EHR integration is where AI stops being “a tool”
The biggest shift in artificial intelligence in health care is not the model—it’s where the model lives. When AI is integrated into electronic health records, it rides along with nearly everything clinicians do: messages, charting, summaries, routing, quality reporting, and more.
In April 2023, Microsoft and Epic announced a collaboration to integrate generative AI into healthcare by combining Azure OpenAI Service with Epic’s EHR ecosystem. That’s a big deal because it’s not “a standalone app.” It’s closer to “AI inside the operating system of hospital workflows.”
This is where small errors can scale. A slightly biased summary. A slightly wrong suggestion. A slightly misleading auto-generated message. Multiply that across thousands of encounters and you don’t just have isolated mistakes—you have a pattern.
What to do about it: require that AI features in the EHR come with guardrails, logging, audit trails, and clinician override signals that are easy to use and culturally encouraged. If staff are punished for overriding AI, your safety plan is fake.
⚡ Hard truth #3: AI can “feel” helpful while quietly being wrong
Artificial intelligence in health care has a nasty failure mode: it can sound confident and still be wrong. In generative tools, that’s the well-known “hallucination” problem. But even non-generative models can fail silently if they’re trained on the wrong populations, outdated practices, or incomplete data.
Here’s what makes it dangerous: clinicians are busy, and when an output is fluent, clean, and formatted like a professional note, it earns trust fast. People don’t trust the AI because they’re gullible—they trust it because the workflow pressures them to move.
What to do about it: design friction on purpose. For high-risk tasks (medication decisions, triage priority, critical summaries), force a review step. Make the tool show its sources (where possible). Make uncertainty visible. Don’t hide it behind pretty language.
⚖️ Hard truth #4: Bias isn’t a bug you patch later—it’s a system problem
If you only remember one thing about artificial intelligence in health care, make it this: bias usually enters before the model even exists. It shows up in who gets diagnosed, who gets referred, who gets believed, who gets follow-ups, and who gets access.
A real example often discussed in health equity research is kidney failure and transplant access disparities in the U.S. Peer-reviewed work has reported that African Americans are significantly more likely to develop end-stage renal disease but less likely to receive a kidney transplant—disparities that appear across stages of the transplant process.
Now imagine training a model on a system with those patterns. The model doesn’t “discover truth.” It learns what happened. And if “what happened” includes unequal access and unequal referral patterns, the model can reproduce that inequality as if it’s normal.
What to do about it: don’t accept average performance. In artificial intelligence in health care, “overall accuracy” can hide subgroup harm. Require subgroup evaluation. Require fairness and performance reporting across identity factors. If a vendor refuses, that’s your sign.
🇨🇦 Hard truth #5: Canada’s data gap can hide inequity (and fool AI)
One Canadian challenge is that health systems often lack consistent, high-quality subgroup identity data. That creates a dangerous illusion: if you don’t see disparities in the dataset, you can pretend they don’t exist.
But Canadian health data organizations have been clear that race-based and Indigenous identity data is essential for measuring health inequities and identifying inequities that stem from racism, bias, and discrimination. Without measurement, “equity” becomes a slogan instead of an outcome.
What to do about it: if you’re deploying artificial intelligence in health care in Canada, you must pair it with serious data governance. That includes consent, transparency, and safe collection standards—because the alternative is blind AI making confident decisions.
🛡️ Hard truth #6: Regulation is catching up, and the direction is clear
For years, tech has operated on “ship first, apologize later.” In artificial intelligence in health care, that attitude should get you escorted out of the building.
Health Canada has published pre-market guidance for machine learning–enabled medical devices (MLMD), emphasizing safety and effectiveness evidence and highlighting that data should be representative of the intended population, including identity-based factors (for example, skin pigmentation and biological differences between sexes). That’s not a minor detail. That’s a direct warning against lazy datasets and one-size-fits-all validation.
Globally, the World Health Organization has also released guidance focused on the ethics and governance of AI for health—specifically addressing large multi-modal models (LMMs) and the need for strong governance because the upside is big and the risks are real.
What to do about it: treat governance as part of the product. If you can’t explain how the model was validated, monitored, and governed, you’re not “innovating.” You’re gambling with patients.
🔐 Hard truth #7: Privacy “gotchas” multiply when AI enters the room
Artificial intelligence in health care increases the temptation to reuse data for secondary purposes: training, fine-tuning, evaluation, analytics, and product development. And every new use is another chance for trust to break.
The practical privacy questions you should insist on answering are simple:
- What data is used for training vs. real-time prediction?
- Are prompts, summaries, or outputs stored—and for how long?
- Who can access logs and model outputs?
- Can patients opt out without losing quality of care?
- What happens when the vendor updates the model?
If your vendor answers these with hand-wavy marketing language, you don’t have a privacy plan. You have vibes.
🧠 Hard truth #8: “Human-in-the-loop” is not enough by itself
People love saying “don’t worry, a human reviews it.” Great—unless the human is rushed, undertrained on the tool, or pressured to follow AI suggestions because it’s “policy.” Then your human-in-the-loop becomes a rubber stamp.
What to do about it: in artificial intelligence in health care, you need more than a human. You need:
- Clear responsibility (who owns failures?)
- Training that includes how the AI fails
- Easy overrides (no guilt trips, no extra paperwork)
- Monitoring that checks whether staff are overriding appropriately
📉 Hard truth #9: If you don’t measure harm, you won’t find it
The scariest failures in artificial intelligence in health care are the ones that look like “normal variance.” A slightly higher miss rate for one subgroup. A slightly longer wait time after an AI routing change. A slightly lower referral rate after an AI summary tool is introduced.
That’s why evaluation must be ongoing, not a one-time pilot. You need post-deployment surveillance—just like medication monitoring—because real-world conditions are not lab conditions.
✅ A practical deployment checklist for artificial intelligence in health care
If you’re a hospital, clinic, or health startup rolling out artificial intelligence in health care, here’s the checklist that separates professionals from people playing with matches:
- Define the intended use: what it does, what it does not do, and what “success” looks like.
- Validate locally: test on local data and workflows, not just vendor benchmarks.
- Measure subgroup performance: don’t let “overall accuracy” hide inequity.
- Design guardrails: friction for high-risk actions, confirmations, second reviews.
- Log everything that matters: outputs, overrides, corrections, user roles.
- Monitor drift: schedule re-validation and watch for performance decay.
- Build a rollback plan: know exactly how you turn it off safely.
- Assign accountability: name the owner of outcomes and incidents.
And yes, this is extra work. That’s the price of using powerful tools in a life-and-death environment. If someone tells you otherwise, they’re selling you something.
📊 Risk table
| Risk | What it looks like in real life | Practical mitigation |
|---|---|---|
| Automation bias | Staff defer to AI suggestions without checking | Require confirm steps for high-risk actions + train “challenge the tool” habits |
| Hallucinations / misleading text | Fluent summaries that contain incorrect clinical claims | Human review for clinical content + show uncertainty + limit scope |
| Subgroup underperformance | Model works “on average” but fails specific populations | Track outcomes by subgroup + require representative evaluation data |
| Workflow drift | Care adapts to the tool, not the patient | Audit overrides + periodic workflow reviews + staff feedback loops |
| Privacy & retention | Prompts/outputs stored longer than expected | Minimize retention + tighten access + document vendor data handling |
| Model drift over time | Performance degrades as populations/practices change | Re-validate on schedule + set rollback triggers |
❓ FAQs about artificial intelligence in health care
✅ 1) Is artificial intelligence in health care already being used in hospitals?
Yes. It’s used for documentation support, imaging support, risk prediction, routing, and administrative tasks. The big trend is embedding it into existing systems like EHRs.
✅ 2) What’s the biggest danger with AI in clinical settings?
Silent failure: outputs that look reasonable but are wrong, biased, or outdated—especially when workflow pressure pushes people to trust the tool.
✅ 3) Can artificial intelligence in health care reduce clinician burnout?
It can, especially in messaging and documentation, but only if it’s accurate and doesn’t add extra steps for correction, review, and auditing.
✅ 4) Why is bias so hard to fix?
Because bias often comes from the system: who gets diagnosed, referred, followed, and documented. AI can learn those patterns and scale them.
✅ 5) What should a hospital demand from an AI vendor?
Local validation support, subgroup performance reporting, audit logs, monitoring tools, clear data retention policies, and an easy rollback plan.
✅ 6) Does Canada regulate AI medical tools?
Health Canada regulates certain machine learning–enabled medical devices and has issued pre-market guidance describing evidence expectations for safety and effectiveness.
✅ 7) What’s “model drift” in artificial intelligence in health care?
It’s when a model’s performance degrades as patient populations, clinical practices, or data inputs change over time. If you don’t monitor drift, you won’t notice risk rising.
✅ 8) How can patients protect themselves?
Ask if AI is involved, request human review when something feels off, and keep copies of key records, test results, and care plans.
✅ 9) Should AI ever make final clinical decisions?
For high-risk care decisions, no. It can support, summarize, and flag—but final responsibility must stay with trained clinicians using transparent processes.
✅ 10) What’s the “right” way to think about artificial intelligence in health care?
Think of it like a powerful assistant with failure modes—not a replacement for clinical judgment. Build guardrails, measure harm, and keep accountability human.
✅ Conclusion: Use artificial intelligence in health care—just don’t be reckless
Artificial intelligence in health care has enormous upside. It can reduce delays, help clinicians focus, and improve decision-making. But it also has the power to scale bias, automate bad assumptions, and normalize mistakes inside systems people already trust.
Rahul G. Krishnan’s award is a reminder that the adult work isn’t “building AI.” The adult work is governing it: measuring outcomes, identifying disparities, validating responsibly, and making sure safety is a feature—not a press release.
🔗 Sources
- CTV News Toronto — Toronto scientist gets big award to study artificial intelligence in health care
- University of Toronto (Computer Science) — Rahul G. Krishnan receives Amazon Research Award
- Amazon Science — Amazon Research Awards: 79 recipients announced (April 17, 2023)
- Microsoft Source — Microsoft and Epic expand collaboration with Azure OpenAI integration (April 17, 2023)
- Epic — Microsoft and Epic expand collaboration (April 17, 2023)
- Health Canada — Pre-market guidance for machine learning-enabled medical devices
- CIHI — Race-based and Indigenous identity data
- World Health Organization — Ethics and governance of AI for health: large multi-modal models
- PubMed Central — Racial disparity in kidney transplant wait-listing (2020)
- Class Central — Reliable Machine Learning in Healthcare (links to the YouTube talk + duration)
Toronto scientist Rahul Krishnan gets big award to study artificial intelligence in health care
Rahul G. Krishnan
Assistant Professor
Dept. of Computer Science
Dept. of Laboratory Medicine and Pathobiology
The University of Toronto
CIFAR AI Chair at Vector Institute
Bio
Rahul G. Krishnan is an Assistant Professor of Computer Science and Medicine (Laboratory Medicine and Pathobiology).
He is a CIFAR AI Chair at the Vector Institute and a member of the Temerty Center for Artificial Intelligence in Medicine.
He works on leveraging tools from developing algorithms for probabilistic inference, and applied machine learning to problems
in healthcare such as modeling disease progression and risk stratification. Previously, he was a Senior Researcher at
Microsoft Research New England. He received his MS from New York University and his PhD in Electrical Engineering and
Computer Science from MIT in 2020.
Research – Toronto scientist Rahul Krishnan gets big award to study artificial intelligence in health care
My goal is to develop machine learning algorithms to create a learning healthcare system, where digitized clinical and
biological data are used to improve clinical care while improving our understanding of human & disease biology.
My research interests lie in the following topics:
- Deep learning: Unsupervised and self-supervised learning algorithms for extracting predictive patterns from noisy, high-dimensional data.
- Causal inference: Developing methods for estimating causal effects to identify good interventional policies from high-dimensional, time-varying observational data.
- Reliable machine learning: Developing guardrails for the reliable deployment of machine learning models.
Updates – Toronto scientist Rahul Krishnan gets big award to study artificial intelligence in health care
Publications
Neural Pharmacodynamic State Space Modeling
(Data)
Z. Hussain*, R. Krishnan*, D. Sontag
International Conference on Machine Learning (ICML), 2021
(*: equal contribution)
Preprints – Toronto scientist Rahul Krishnan gets big award to study artificial intelligence in health care
Peer-reviewed workshop papers
Teaching
Fall 2022: CSC2541H – Topics in Machine Learning: Introduction to Causality
Fall 2022: CSC311H1 – Introduction to Machine Learning
Fall 2021: CSC2541H – Topics in Machine Learning: Machine Learning for Health
Fall 2021: CSC311H1 – Introduction to Machine Learning
Graduate Student Researchers
- Vahid Belazadeh Meresht — PhD Student, CS
- Michael Cooper — PhD Student, CS, Co-supervised with Michael Brudno
- Jerry Ji — PhD Student, CS, Co-supervised with Anna Goldenberg
- Ian Shi — PhD Student, CS, Co-supervised with Quaid Morris
- Tom Ginsberg — MS Student, CS
- Asic Chen — MS Student, CS
- Aslesha Pokhrel — MScAC Student, CS
- Vahid Zehtab — MScAC Student, CS
- Keerat Guilani — MScAC Student, CS
Undergraduate Student Researchers
- Xiang Gao, CS
- Shivang Mistry, EngSci
- Zhongyuan Liang, CS
- Yingke Wang, CS
- Jinyu Hou, CS
- Wendy Cheng, CS
- Ethan Choi, CS
- Aryan Dhar, ECE
- Anindro Bhattacharya, CS and Microbiology
- Yukti Makhija, IIT Delhi
Alumni
- Edward de Brouwer — Visiting PhD student from KU Leuven, Belgium
- Jacob Si — BSc in CS → MSc at UCLA
- Taewoo Kim — MScAC in CS → Layer6
Related Videos:
Related Posts:
City of Toronto data at risk of cyber attack: report
How Many Weeks Are There in a Year?
Contextualising Legal Research: Practical Methods Guide
Coase Social Cost: 17 Practical Insights for Law + Econ
Spur Industries v Del E Webb: Indemnity and Urban Growth
Rawls Theory of Justice Explained: Justice as Fairness
Modern AI Concepts Explained: 5 Pillars Shaping Our Future
Autumn’s Splendor: Embracing the Beauty and Transition of Fall
Unveiling the Espionage: The Termination of Two Scientists at Canada’s Top Lab
Why video game developer, New World Interactive, picked Calgary over Toronto for new studio
Security experts say health care industry is prized target for cyber criminals
Facebook’s chief AI scientist: Deep learning may need a new programming language
Edmonton doctors, AI scientists team up with drug giant in health software project
AI Tech and medical Healthcare
The Amazing Ways The Ford Motor Company Uses Artificial Intelligence And Machine Learning
Cybersecurity Analyst (CSA+) Exam Study Guide