Artificial Intelligence in Health Care: 9 Hard Truths

Artificial intelligence in health care is having a moment. Not a “cute demo” moment—an “it’s slipping into real hospital workflows” moment. And once AI gets embedded into documentation, triage, patient messaging, decision support, and scheduling, it stops being optional. It becomes infrastructure.

That’s why a Toronto researcher getting funding to study the consequences matters more than yet another flashy AI announcement. Rahul G. Krishnan, a University of Toronto assistant professor focused on computational medicine, received an Amazon Research Award to support work that aims to build machine learning into the health-care system—while also confronting the risks that come with it.

Here’s the blunt truth: artificial intelligence in health care can save time, reduce errors, and improve outcomes—but it can also scale bias, normalize bad assumptions, and create “silent failures” that are hard to detect until people get harmed. So let’s talk about the parts that actually matter: safety, fairness, regulation, accountability, and what hospitals should demand before they deploy anything at scale.

🏆 Why this Toronto award matters (and why you should care)

The Amazon Research Awards program funds academic research that aligns with Amazon’s scientific priorities. In April 2023, Amazon announced 79 awardees from universities across multiple countries, including a proposal from the University of Toronto focused on “towards a learning healthcare system.” That’s the sort of phrase that sounds harmless—until you realize it implies AI woven into the everyday logic of care delivery.

Krishnan’s work sits right in the blast radius of the biggest question in artificial intelligence in health care: How do we get the benefits without turning patients into collateral damage of automation?

If you’re a patient, you care because these tools can influence what gets documented, what gets flagged, and how quickly a clinician gets to your case. If you’re a clinician, you care because AI can either reduce burnout—or create a new layer of “admin chaos” where you spend your day correcting machine output. If you’re a policymaker, you care because medicine is not a playground for “move fast and break things.”

🧑‍🔬 Who is Rahul G. Krishnan (in plain English)

Rahul G. Krishnan is a University of Toronto assistant professor working at the intersection of machine learning and medicine. The North Star of his research is building tools that can support clinical decision-making and create a “learning” health-care system—where digitized clinical and biological data helps improve care over time.

The practical angle is obvious: better interfaces for patient records, better clinician support tools, and faster translation from data to useful clinical insight. The risk angle is equally obvious: when you build a system that “learns,” you must be deadly serious about what it learns, who it learns from, and who gets left out.

🩺 Hard truth #1: Medicine isn’t static, so AI can’t be static either

In artificial intelligence in health care, people love to talk about “accuracy.” Fine. Accuracy matters. But medicine changes constantly: new guidelines, new drugs, new protocols, new population patterns, new outbreaks, and new standards of care.

So even if your model performs well today, you need a plan for next month and next year. Otherwise, you’re deploying yesterday’s assumptions into tomorrow’s clinic. That’s not innovation. That’s a liability with a user interface.

What to do about it: treat every clinical AI tool like it has an expiry date unless proven otherwise. Build a re-validation schedule. Monitor drift. Set rollback triggers. If you can’t turn it off quickly, you don’t control it.

🧾 Hard truth #2: EHR integration is where AI stops being “a tool”

The biggest shift in artificial intelligence in health care is not the model—it’s where the model lives. When AI is integrated into electronic health records, it rides along with nearly everything clinicians do: messages, charting, summaries, routing, quality reporting, and more.

In April 2023, Microsoft and Epic announced a collaboration to integrate generative AI into healthcare by combining Azure OpenAI Service with Epic’s EHR ecosystem. That’s a big deal because it’s not “a standalone app.” It’s closer to “AI inside the operating system of hospital workflows.”

This is where small errors can scale. A slightly biased summary. A slightly wrong suggestion. A slightly misleading auto-generated message. Multiply that across thousands of encounters and you don’t just have isolated mistakes—you have a pattern.

What to do about it: require that AI features in the EHR come with guardrails, logging, audit trails, and clinician override signals that are easy to use and culturally encouraged. If staff are punished for overriding AI, your safety plan is fake.

⚡ Hard truth #3: AI can “feel” helpful while quietly being wrong

Artificial intelligence in health care has a nasty failure mode: it can sound confident and still be wrong. In generative tools, that’s the well-known “hallucination” problem. But even non-generative models can fail silently if they’re trained on the wrong populations, outdated practices, or incomplete data.

Here’s what makes it dangerous: clinicians are busy, and when an output is fluent, clean, and formatted like a professional note, it earns trust fast. People don’t trust the AI because they’re gullible—they trust it because the workflow pressures them to move.

What to do about it: design friction on purpose. For high-risk tasks (medication decisions, triage priority, critical summaries), force a review step. Make the tool show its sources (where possible). Make uncertainty visible. Don’t hide it behind pretty language.

⚖️ Hard truth #4: Bias isn’t a bug you patch later—it’s a system problem

If you only remember one thing about artificial intelligence in health care, make it this: bias usually enters before the model even exists. It shows up in who gets diagnosed, who gets referred, who gets believed, who gets follow-ups, and who gets access.

A real example often discussed in health equity research is kidney failure and transplant access disparities in the U.S. Peer-reviewed work has reported that African Americans are significantly more likely to develop end-stage renal disease but less likely to receive a kidney transplant—disparities that appear across stages of the transplant process.

Now imagine training a model on a system with those patterns. The model doesn’t “discover truth.” It learns what happened. And if “what happened” includes unequal access and unequal referral patterns, the model can reproduce that inequality as if it’s normal.

What to do about it: don’t accept average performance. In artificial intelligence in health care, “overall accuracy” can hide subgroup harm. Require subgroup evaluation. Require fairness and performance reporting across identity factors. If a vendor refuses, that’s your sign.

🇨🇦 Hard truth #5: Canada’s data gap can hide inequity (and fool AI)

One Canadian challenge is that health systems often lack consistent, high-quality subgroup identity data. That creates a dangerous illusion: if you don’t see disparities in the dataset, you can pretend they don’t exist.

But Canadian health data organizations have been clear that race-based and Indigenous identity data is essential for measuring health inequities and identifying inequities that stem from racism, bias, and discrimination. Without measurement, “equity” becomes a slogan instead of an outcome.

What to do about it: if you’re deploying artificial intelligence in health care in Canada, you must pair it with serious data governance. That includes consent, transparency, and safe collection standards—because the alternative is blind AI making confident decisions.

🛡️ Hard truth #6: Regulation is catching up, and the direction is clear

For years, tech has operated on “ship first, apologize later.” In artificial intelligence in health care, that attitude should get you escorted out of the building.

Health Canada has published pre-market guidance for machine learning–enabled medical devices (MLMD), emphasizing safety and effectiveness evidence and highlighting that data should be representative of the intended population, including identity-based factors (for example, skin pigmentation and biological differences between sexes). That’s not a minor detail. That’s a direct warning against lazy datasets and one-size-fits-all validation.

Globally, the World Health Organization has also released guidance focused on the ethics and governance of AI for health—specifically addressing large multi-modal models (LMMs) and the need for strong governance because the upside is big and the risks are real.

What to do about it: treat governance as part of the product. If you can’t explain how the model was validated, monitored, and governed, you’re not “innovating.” You’re gambling with patients.

🔐 Hard truth #7: Privacy “gotchas” multiply when AI enters the room

Artificial intelligence in health care increases the temptation to reuse data for secondary purposes: training, fine-tuning, evaluation, analytics, and product development. And every new use is another chance for trust to break.

The practical privacy questions you should insist on answering are simple:

• What data is used for training vs. real-time prediction?
• Are prompts, summaries, or outputs stored—and for how long?
• Who can access logs and model outputs?
• Can patients opt out without losing quality of care?
• What happens when the vendor updates the model?

If your vendor answers these with hand-wavy marketing language, you don’t have a privacy plan. You have vibes.

🧠 Hard truth #8: “Human-in-the-loop” is not enough by itself

People love saying “don’t worry, a human reviews it.” Great—unless the human is rushed, undertrained on the tool, or pressured to follow AI suggestions because it’s “policy.” Then your human-in-the-loop becomes a rubber stamp.

What to do about it: in artificial intelligence in health care, you need more than a human. You need:

• Clear responsibility (who owns failures?)
• Training that includes how the AI fails
• Easy overrides (no guilt trips, no extra paperwork)
• Monitoring that checks whether staff are overriding appropriately

📉 Hard truth #9: If you don’t measure harm, you won’t find it

The scariest failures in artificial intelligence in health care are the ones that look like “normal variance.” A slightly higher miss rate for one subgroup. A slightly longer wait time after an AI routing change. A slightly lower referral rate after an AI summary tool is introduced.

That’s why evaluation must be ongoing, not a one-time pilot. You need post-deployment surveillance—just like medication monitoring—because real-world conditions are not lab conditions.

✅ A practical deployment checklist for artificial intelligence in health care

If you’re a hospital, clinic, or health startup rolling out artificial intelligence in health care, here’s the checklist that separates professionals from people playing with matches:

• Define the intended use: what it does, what it does not do, and what “success” looks like.
• Validate locally: test on local data and workflows, not just vendor benchmarks.
• Measure subgroup performance: don’t let “overall accuracy” hide inequity.
• Design guardrails: friction for high-risk actions, confirmations, second reviews.
• Log everything that matters: outputs, overrides, corrections, user roles.
• Monitor drift: schedule re-validation and watch for performance decay.
• Build a rollback plan: know exactly how you turn it off safely.
• Assign accountability: name the owner of outcomes and incidents.

And yes, this is extra work. That’s the price of using powerful tools in a life-and-death environment. If someone tells you otherwise, they’re selling you something.

📊 Risk table

❓ FAQs about artificial intelligence in health care

✅ 1) Is artificial intelligence in health care already being used in hospitals?

Yes. It’s used for documentation support, imaging support, risk prediction, routing, and administrative tasks. The big trend is embedding it into existing systems like EHRs.

✅ 2) What’s the biggest danger with AI in clinical settings?

Silent failure: outputs that look reasonable but are wrong, biased, or outdated—especially when workflow pressure pushes people to trust the tool.

✅ 3) Can artificial intelligence in health care reduce clinician burnout?

It can, especially in messaging and documentation, but only if it’s accurate and doesn’t add extra steps for correction, review, and auditing.

✅ 4) Why is bias so hard to fix?

Because bias often comes from the system: who gets diagnosed, referred, followed, and documented. AI can learn those patterns and scale them.

✅ 5) What should a hospital demand from an AI vendor?

Local validation support, subgroup performance reporting, audit logs, monitoring tools, clear data retention policies, and an easy rollback plan.

✅ 6) Does Canada regulate AI medical tools?

Health Canada regulates certain machine learning–enabled medical devices and has issued pre-market guidance describing evidence expectations for safety and effectiveness.

✅ 7) What’s “model drift” in artificial intelligence in health care?

It’s when a model’s performance degrades as patient populations, clinical practices, or data inputs change over time. If you don’t monitor drift, you won’t notice risk rising.

✅ 8) How can patients protect themselves?

Ask if AI is involved, request human review when something feels off, and keep copies of key records, test results, and care plans.

✅ 9) Should AI ever make final clinical decisions?

For high-risk care decisions, no. It can support, summarize, and flag—but final responsibility must stay with trained clinicians using transparent processes.

✅ 10) What’s the “right” way to think about artificial intelligence in health care?

Think of it like a powerful assistant with failure modes—not a replacement for clinical judgment. Build guardrails, measure harm, and keep accountability human.

✅ Conclusion: Use artificial intelligence in health care—just don’t be reckless

Artificial intelligence in health care has enormous upside. It can reduce delays, help clinicians focus, and improve decision-making. But it also has the power to scale bias, automate bad assumptions, and normalize mistakes inside systems people already trust.

Rahul G. Krishnan’s award is a reminder that the adult work isn’t “building AI.” The adult work is governing it: measuring outcomes, identifying disparities, validating responsibly, and making sure safety is a feature—not a press release.

🔗 Sources

• CTV News Toronto — Toronto scientist gets big award to study artificial intelligence in health care
• University of Toronto (Computer Science) — Rahul G. Krishnan receives Amazon Research Award
• Amazon Science — Amazon Research Awards: 79 recipients announced (April 17, 2023)
• Microsoft Source — Microsoft and Epic expand collaboration with Azure OpenAI integration (April 17, 2023)
• Epic — Microsoft and Epic expand collaboration (April 17, 2023)
• Health Canada — Pre-market guidance for machine learning-enabled medical devices
• CIHI — Race-based and Indigenous identity data
• World Health Organization — Ethics and governance of AI for health: large multi-modal models
• PubMed Central — Racial disparity in kidney transplant wait-listing (2020)
• Class Central — Reliable Machine Learning in Healthcare (links to the YouTube talk + duration)

Related Videos:

Related Posts:

City of Toronto data at risk of cyber attack: report

How Many Weeks Are There in a Year?

Contextualising Legal Research: Practical Methods Guide

Coase Social Cost: 17 Practical Insights for Law + Econ

Spur Industries v Del E Webb: Indemnity and Urban Growth

Rawls Theory of Justice Explained: Justice as Fairness

Modern AI Concepts Explained: 5 Pillars Shaping Our Future

Autumn’s Splendor: Embracing the Beauty and Transition of Fall

Unveiling the Espionage: The Termination of Two Scientists at Canada’s Top Lab

Why video game developer, New World Interactive, picked Calgary over Toronto for new studio

Security experts say health care industry is prized target for cyber criminals

Facebook’s chief AI scientist: Deep learning may need a new programming language

Edmonton doctors, AI scientists team up with drug giant in health software project

AI Tech and medical Healthcare

The Amazing Ways The Ford Motor Company Uses Artificial Intelligence And Machine Learning

Cybersecurity Analyst (CSA+) Exam Study Guide