Stay informed
When Evidence Can Be Deepfaked, How Do Courts Decide What’s Real?
For years, there was a box on a back shelf of P’s house in Edmonton labelled “insurance.” (For reasons of privacy, we are using this initial only.) Her husband at the time thought it contained paperwork, but P had filled it with a different kind of insurance: a physical record of smashed phones, broken eyeglasses, and photographs of the bruises his violent episodes had left on her body. “I had a circle of blue bruises around my mouth because he would cover my mouth and smother me,” she told me.
Photographs, videos, and audio recordings are highly persuasive to judges and juries. When a crime occurs in private, with no witnesses, a court contest is a tussle in which two stories compete to offer the most plausible explanation of the same facts. Photographs and audio recordings join seemingly unimpeachable objectivity with emotional impact: one study says combining visual and oral testimony can increase information retention among jurors by 650 percent.
Criminal defence lawyer Emily Dixon told me that, if a client shows her an exonerating photo or video, she isn’t expected to run analytic tests before submitting it into evidence. It’s reasonable to assume that a photo is real—for now. Yet we are fast approaching a world in which we can no longer believe our eyes or ears. The onset of artificial intelligence, in the justice system as elsewhere, is poised to overturn existing practices.
Specialists can still spot the anomalies that distinguish AI-generated images from real photos. “But in a year,” digital forensics expert Simon Lavallée told me, “we won’t.” Maura R. Grossman is a lawyer who has long worked to promote the use of advanced technologies for legal tasks, such as document review. When it comes to the threat of deepfakes, however, Grossman believes Canada’s evidence laws will require an overhaul. “Before, if I wanted to fake your signature, I had to have some talent,” she told me. “In this day and age, you could make a deepfake of my voice in two minutes.” Juries, Grossman has written, may increasingly be skeptical of all evidence.
For complainants like P, the erosion of this trust carries a high cost: left with only doubt, we may be tempted to rely on our instincts—indistinguishable from our desires, fears, and prejudices. In cases of domestic violence in particular, audio and visual evidence is a weighty counter to the social impulse to minimize. At work and with their friends, P says, her husband was funny and kind, but “he’s a very different person at home, to the point that most people wouldn’t believe you.”
Recently, the Law Commission of Ontario (LCO), an independent body that pushes for law reform, concluded a nationwide study of what is shaping up to be a head-on collision between two epistemic systems: the criminal justice system, which rests on the elimination of reasonable doubt, and artificial intelligence, a factory for doubt dissemination. “The risk of impacts on people’s lives is catastrophic,” Ryan Fritsch, the lawyer who headed the initiative, told me. The project brought together police officers, defence attorneys, prosecutors, judges, and human rights advocates, with the aim of formulating a set of recommendations to deliver to the Ontario government later this year.
The black mirror of deepfake evidence is not the only challenge the advent of artificial intelligence poses to the criminal justice system. Whether through the adoption of advanced analytics for risk prediction, the rise of predictive policing, or the use of large language models to summarize depositions or draft decisions, Canadian courts are about to grapple with a form of intelligence the mysterious workings of which chip away at the law’s foundation of transparency.
Internationally, Canadian lawmakers are already far behind in developing governance for the use of artificial intelligence within the justice system. Few guardrails are in place, and we are faced with clashing examples: Europe has taken a skeptical stance, sometimes outright banning the use of certain technologies, while the United States has leaned into decision making driven by complex algorithms and big data. Canada’s legal rules and procedures are designed for a pre-AI era. How will the public retain confidence in a system that rests on the painstaking articulation of reasoned logic as more and more of what happens in our courtrooms starts in a black box?
The attorney Rupert Ross used to work as a fishing guide, and in his book Dancing with a Ghost, he wrote about learning to predict lake conditions before leaving the shore. Standing at the edge of the dock, he would layer impressions of wind pattern, cloud cover, and temperature over his mental images of different spots on the water. It was as if each image were “a transparency of sorts, with all the variables sketched opaquely on its surface. Then similar images of past days at the same spot are slid under it.” If the feeling of the day contained a Proustian twinge of a previous day when he had found pickerel at a certain cove, he brought his angler clients to those waters.
Ross’s description reads like a paean to the subtleties of embodied memory, one of the quintessential characteristics that separates humans from machines. To the empirically minded, however, this is just another way of saying that Ross made predictions based on comprehensive historical data. And aren’t machines simply better at this?
That’s why, P tells me, she’s excited about the prospect of criminal risk assessments being performed by artificial intelligence. It seemed to P that justice system workers had been overly swayed by her husband’s charm. “These people are trying to predict outcomes, going off their imagination, their feelings, their gut instinct,” she said. “And AI takes all of that out.”
Since the 1990s, statistical risk assessments have been conducted within the prison systems of many jurisdictions in Canada, and elsewhere, to sort offenders by the level of security needed and as evidence in parole board hearings. By correlating today’s data with outcomes from past cases—did prisoners who answered questions similarly go on to commit further crimes?—prisons had a data-driven system for deciding who was eligible for laxer security or early parole. The accuracy of computerized algorithmic models quickly rivalled the careful assessments of expert psychologists. But, according to correctional lawyer Simon Borys, prisons and parole boards don’t have to explain their decisions in the same granular detail that courts do. It was in the 2010s, when algorithmic risk assessment tools were being incorporated into sentencing in the United States, that the higher standard for legal reasoning brought to light the concerning qualities of these tools.
In 2016, the American media outlet ProPublica published a landmark study finding racial bias in COMPAS, a widely used algorithmic risk assessment tool. Studies of similar tools used in Canada, such as the Level of Service Inventory (LSI), have found comparable results. For a kaleidoscope of reasons—including that models rely on historical arrest data and racialized communities are more heavily policed than white neighbourhoods—algorithmic predictions tend to overstate the likelihood of racialized people committing future crimes and understate these risks for white offenders.
If a bigger mirror only shows more of a distorted reality, artificial intelligence powered by big data exaggerates the problems of actuarial risk prediction: because we live in an unfair world in which people who are poor, mentally ill, or racialized are disproportionately arrested, simply belonging to one of these categories can make you look more likely to commit crimes. (Equivant, the company that currently owns COMPAS, disputes that the tool is racially biased, and some subsequent studies undertaken by independent bodies also found ProPublica’s results overblown.)
If I am accused of a crime in Canada, I have a right to know what evidence is being used against me. I have a right to test that evidence, to hear the reasons why a judge or jury decides to deprive me of my liberty, and to appeal their decision if these reasons are insufficient. In one of the papers commissioned by the LCO, Gideon Christian, the University of Calgary’s research chair of AI and law, and lawyer Armando D’Andrea propose a novel problem: How do you cross-examine an algorithm? If a risk assessment is to be classified as expert opinion, usually the expert proffering it would be expected to make themselves available to the court. In the case of an AI tool, however, even if a software engineer who had designed the program could be persuaded to testify, the nature of machine learning would limit their ability to vouch for what the program was actually doing.
“You should be able to explain why a person got the score and what the score means,” Tim Brennan, who originally designed COMPAS, told me. “An AI technique like random forests”—a statistical sampling method (not used in COMPAS) that involves hundreds of decision trees branching into further decision trees—“is a great way to do prediction. But you cannot, for your life, explain it to a judge.”
Even if courts banned the use of AI by lawyers, they would be unable to prevent reports or assessments used in other spheres—by social workers, for example, or addictions counsellors—from being compiled with the use of AI. But the use of AI could mean the reports of these professionals, currently a routine part of court proceedings, would no longer pass tests making them admissible evidence.
Fritsch, the lawyer for the LCO, told me that AI is weakest in an area the justice system prizes highly: meeting standards of evidence and reasoning. AI systems occlude information or processes, which means they may not provide the full disclosure the law requires. For defendants hoping to appeal their sentences, this missing information could be key to overturning decisions that rob them of their liberty or even their lives. The inner workings of AI software are proprietary information owned by private companies, and in the United States, courts have shown a disturbing tendency to side with corporations that argue they can’t be compelled to disclose trade secrets—even when their products are being used to send people to death row.
In a case that came before British Columbia’s Civil Resolution Tribunal in 2025, a tribunal member, Eric Regehr, posited in his decision that both parties must have used AI to write their submissions. His reasoning was that “there is no way a human being” could have made some of the errors these submissions contained. Furthermore, he did not consider himself obliged to respond to every one of the specious arguments the software had thrown his way. “I accept that artificial intelligence can be a useful tool to help people find the right language to present their arguments, if used properly,” he wrote. “However, people who blindly use artificial intelligence often end up bombarding the CRT with endless legal arguments.”
The promise of AI for self-representation is tantalizing. To state the obvious: lawyers are expensive. Hourly rates for a criminal defence attorney can hit $800 an hour, and many people who apply for legal aid are denied. Accordingly, the number of people representing themselves in court is growing. The increase has been most marked in family law cases, but some Canadians accused of a criminal offence find themselves with no option but to mount their own defence.
Depending on the nature of the case, people representing themselves in court using Microsoft CoPilot or ChatGPT may find themselves up against opposing counsel using even more sophisticated tools. An inventory compiled in June of 2025 found 638 generative AI tools available in the “legaltech” field, for uses such as trawling through legal databases, reviewing contracts, searching for patents, and drafting documents. Companies are developing specialized artificial intelligence tools for the use of big firms, who can pay for the most advanced tech. For a self-represented litigant, ChatGPT may be better than nothing, but it’s the kiddie-pool version of the paid products available to a well-resourced lawyer on the opposing side.
Within Canada, some non-profits are attempting to level the playing field. Beagle+ is a free legal chatbot trained on British Columbia law, built by People’s Law School, and powered by ChatGPT. And the Conflict Analytics Lab at Queen’s University has built OpenJustice, an open-access AI platform drawing from Canadian, US, Swiss, and French law. Originally, says Samuel Dahan, the law professor who primarily oversees the project, the idea was to build a Canadian large language model from scratch. “The sad reality of universities right now is that there’s no university in the world that has the resources to build a language model,” Dahan told me. At a Legal AI Hackathon co-hosted with Stanford and held in Toronto in February 2025, the Conflict Analytics Lab saw teams devising programs that use OpenJustice to help pro bono organizations streamline client intake and to identify trends in jurisprudence that might help direct a class-action lawsuit.
While artificial intelligence seems poised to widen, rather than narrow, the divide in access to justice, Grossman did suggest a key way that AI could serve the average person in the civil context. When a pair of earrings she bought online arrived, one post was bent. She contacted the seller, sent a photo, and asked if a replacement earring could be sent. “He basically responded, if you weren’t a moron, you would be able to fix this.” So Grossman made her case in the dialogue box of the website’s online dispute resolution system. It scanned her photo, looked at the value of the item (the earrings had cost Grossman about $12, including shipping), tabulated comparable transactions, and made a determination: it awarded her $2.38.
Grossman asked me to imagine that, instead of a customer dissatisfied with an earring, she was a tenant, and the other party a landlord illegally withholding her security deposit. “I’m a janitor, and if I take a day off from work, that means my kids don’t eat, and I don’t know anything about the court system, and maybe I don’t speak English well, and maybe I’m not here legitimately,” she said. The online adjudication system might not be perfect; it could award her less than the full amount. But the case might be over in a week, with no requirement to miss work or bring attention to her immigration status. “It’s not perfect,” she acknowledged, “but rough justice is better than no justice.”
So far, Canada’s attempts to pass comprehensive legislation on artificial intelligence have failed. AIDA, the Artificial Intelligence and Data Act, died on the table when Parliament was prorogued in January of 2025, and the legislation was controversial: Amnesty International objected that the act overlooked potential harms to vulnerable groups, and the Assembly of First Nations threatened to sue over the lack of consultation. Indigenous groups such as the Quebec-Labrador regional body of the AFN have proactively published guides to regulating and developing artificial intelligence with respect for Indigenous data sovereignty, but at present, these are statements of moral principle rather than enforceable law. With the appointment of Evan Solomon as the inaugural minister of artificial intelligence and digital innovation this past spring, it’s certain that another legislative effort is coming soon.
The federal court has proclaimed that it will not use AI or any automated decision-making tools without first engaging in public consultations. And when it comes to the criminal system, Fritsch told me, “The public needs to be giving social license to these kinds of things.”
But what would meaningful consultation on AI look like when experts can’t even agree on what artificial intelligence is? The LCO materials discuss COMPAS as an artificially intelligent tool, but Equivant denies that COMPAS makes use of AI. “It does not ‘learn’ or change over time,” the company says on its website. Brennan told me a tool similar to COMPAS, designed for female offenders, has incorporated AI for the past eight years. He later amended this statement, saying that the tool uses a “machine-learning-derived classifier” but is not itself an AI tool. The category of artificial intelligence seems to be simultaneously narrowing and widening. “When it’s like a spam filter, and we get used to it, we don’t call it AI; we just call it software,” Grossman remarked. “When it’s new and magical and we don’t understand it, we call it AI.”
This past summer, Karine Gentelet, a sociologist at the Université du Québec en Outaouais, piloted an alternative form of consultation on AI. “It’s not a consultation about AI,” Gentelet told me. “It’s a consultation about how people want to be consulted.” Rather than the large-scale forums that governments tend to convene, often dominated by industry, these sessions were small and took place in a Montreal community centre. Thirty to forty participants per session proposed definitions of basic concepts like “trust” and “transparency.” Gentelet’s team put up a poster showing an Excel spreadsheet of 168 artificial intelligence programs in development or in use by the province’s agencies, available on the government’s website. Descriptions were brief and often vague: “Service to citizens.” Clearly, this was someone’s idea of transparency, but it didn’t match what workshop participants had written down. They wanted to know exactly how information would be collected and used and how the government planned to guarantee ethical guardrails.
When I asked P what she would tell a government commission that asked her what AI should and shouldn’t do, she didn’t need to think twice. “I would ask if there’s a way for a lot of those computer engineering students to come up with a system to take some of the burden off shelter workers,” she said. Harried workers were constantly on the phone, calling around to different facilities to find spots for women in need of a safe place to stay. Couldn’t it be more like the aggregators that find airline tickets? You could key in your location, any special needs (not all shelters are safe places for transgender women, P noted), and an algorithm could take care of the rest.
Fundamentally, citizens want the criminal justice system to deliver two conflicting outcomes: to keep them safe, and to respect individual liberties. Dial public safety all the way to ten, and we are all pre-emptively kept in solitary confinement to prevent us from causing each other harm. Dial liberty to the max, and there is no protection for the weak. Fairness is one name for the balance between these concerns. A cornerstone of our legal tradition is the 1769 doctrine known as Blackstone’s ratio: “It is better that ten guilty persons escape than that one innocent suffer.” This is a moral, rather than a mathematical, maxim.
But deep within the coded instructions that govern algorithmic tools, fairness does find mathematical expression—or rather, expressions. The balance struck in risk predictions is between the likelihood of a formula returning false positives or false negatives: either imprisoning people unnecessarily, or returning to the community people who go on to hurt others.
When engineers attempt to tinker with the variables that produce racist or sexist results, they apply different formulas that are fair by different standards. “Fairness through unawareness” means using training data for machine learning that excludes variables like race or gender. “Counterfactual fairness” means a prediction would remain constant even if the demographic group to which an individual belongs were modified. At last count, there are about twenty different definitions of fairness circulating in the risk-assessment field, offering different trade-offs that are statistically impossible to reconcile.
In a 2024 paper in the journal Artificial Intelligence and Law, a group of researchers in Germany warned that by designing models encoded with particular trade-offs, software engineers and companies are usurping what should be the choices of citizens. The research team devised a code that shifted the balance struck in COMPAS between victims’ and offenders’ rights. The researchers acknowledged that legislators might not agree the new balance reflected the state’s vision of fairness, but at least the choices being made were now out in the open, where citizens could debate them.
Most of us, as Gentelet anticipated, don’t have the technical knowledge to speak with sophistication about artificial intelligence. The participants in her workshop were not there to parse how random forests differ from bootstrap classification methods, or to wax eloquent on OOB (out-of-bag) samples. But they knew what scared them, including the environmental costs of the technology. The advent of artificial intelligence, they told Gentelet, was increasingly presented as a kind of destiny: just one more ineluctable step on a path humanity must follow. The majority of people in the room had one pressing question: Was it an option to say no?
This article was supported by the Law Commission of Canada–Canadian Bar Association Journalism Fellowship for 2025.
The post When Evidence Can Be Deepfaked, How Do Courts Decide What’s Real? first appeared on The Walrus.
Unpublished Newswire
Comments
Be the first to comment