Smoke & Mirrors: Protecting Children or Building the Architecture of Digital Control?

Don't have time to read the article? View as a short video storyboard or listen to it whilst jogging.

There are few subjects more emotionally powerful than child protection. When a government says it is acting to protect children from harm, most decent people instinctively want to support it. Children should not be exploited. Children should not be groomed. Children should not be pressured into taking or sharing sexual images. On that point, there should be little disagreement.

But the question is not whether children should be protected.

The question is how.

The UK government has announced plans to push technology companies toward device-level systems that can stop children from taking, sharing or viewing nude images on smartphones and tablets. The proposal has been presented as a child-safety measure, and on the surface that sounds reasonable. Yet beneath the reassuring language lies a much larger issue: once software is placed at operating-system level to inspect private content, the boundary between protection and surveillance becomes dangerously thin.

This is not an article attacking one political party or one government. Governments change. Ministers change. Prime ministers change. But technological infrastructure remains. The real concern is not simply what this government says it wants to do today. The concern is what any future government, regulator, court, corporation, or compromised system could do tomorrow once this capability exists.

The Problem With “It Does Not Collect Data”

One of the claims surrounding device-level scanning is that it can happen without collecting data. Technically, there is a difference between uploading an image to a server and analysing it locally on a device. On-device analysis may indeed be more private than cloud scanning.

But that does not mean no data is processed.

For software to block an image, it must inspect the image. It must classify the image. It must decide whether the image is acceptable or unacceptable. That is data processing, even if the image never leaves the phone.

This distinction matters because public debate often collapses into comforting slogans. “Nothing is collected” sounds reassuring. “Your phone will analyse your private images and decide whether they are allowed” sounds very different.

Both can describe the same system.

The Context Problem

A photograph does not explain itself.

A child in a swimming costume by a pool may be a normal holiday photograph. A family beach picture may be innocent. A step-parent, grandparent, uncle, aunt, teacher, youth leader or family friend may have photographs of children who are not biologically theirs. Life is complicated. Families are complicated. Human relationships do not always fit neatly into government databases.

Software may see only:

child
skin
swimwear
pool
nudity probability
risk score

A human being sees context.

That gap matters.

Imagine a man who helped raise his partner’s children for years. He was not their biological father. He was not married to their mother. Yet he went on family holidays, attended birthdays, took photographs by the pool, and kept memories like millions of ordinary people do.

Years later, the relationship ends. The photographs remain.

To a human being, those images may represent a chapter of family life. To a machine, stripped of context, they could become suspicious signals. And once suspicion enters a database, innocent people can find themselves trapped in a system they do not understand and cannot easily challenge.

False positives are not a small technical inconvenience. They can destroy reputations, families, jobs and lives.

The “If You Have Done Nothing Wrong” Trap

The lazy answer to privacy concerns is always the same:

“If you have done nothing wrong, you have nothing to fear.”

But that argument assumes the system is always correct.

It assumes databases are accurate.
It assumes algorithms understand context.
It assumes human reviewers are fair.
It assumes future governments will be restrained.
It assumes mission creep will not happen.
It assumes the innocent will be protected from error.

History gives us no reason to be that trusting.

Most people are not afraid of being caught doing something wrong. They are afraid of being wrongly interpreted as having done something wrong.

There is a profound difference.

Why Not Start With Less Intrusive Alternatives?

If the sole objective is protecting children, then there are other possible approaches.

Children under a certain age do not need full smartphones with unrestricted cameras, social media access, encrypted messaging, livestreaming, adult content, app stores and algorithmic feeds. A basic child-friendly phone with calls, text messaging, GPS, emergency contacts and long battery life would meet many safety needs without requiring surveillance software inside every smartphone ecosystem.

That would place the restriction where the risk is: on children’s devices.

Instead, the proposed direction risks altering the architecture of everyone’s devices. Adults may still need age verification. Platforms may need proof of age. Operating systems may need content controls. VPNs and workarounds may become the next target because once restrictions exist, bypassing those restrictions becomes the next policy problem.

This is how digital control expands. Not necessarily through one dramatic authoritarian act, but through a chain of individually reasonable steps.

Each step is justified.
Each step is narrow.
Each step is for safety.
Each step creates the need for the next.

The Amazon Example: When Context Disappears

We already see the problem in everyday moderation systems.

A harmless product description might say:

“Perfect for travellers, homeowners and technology enthusiasts.”

A human reader understands that “travellers” means people who travel. But an automated moderation system may flag the word because, in another context, it could refer to a protected group or be used insultingly.

The system is not offended. It does not understand offence. It is simply trying to avoid the possibility that someone, somewhere, might interpret the word badly.

That is the danger of automated censorship. It moves from:

“This is harmful”

to:

“This might possibly be interpreted as harmful.”

Once that logic enters public policy, free expression begins to shrink. People stop asking, “Is this true?” and start asking, “Could this be misread?”

That is not a healthy society.

Who Decides What Is Harmful?

This question becomes even more serious when the same logic is applied to misinformation, unrest or public disorder.

Who decides what causes unrest?
Who decides what is fake news?
Who decides when public safety overrides open communication?
Who decides whether criticism is dangerous or merely uncomfortable?

Most people agree that direct incitement to violence should be dealt with. But the difficulty begins when governments, regulators or platforms move into interpretation, opinion, satire, political anger, cultural criticism or controversial reporting.

During times of tension, the temptation to control the narrative becomes stronger. Governments may genuinely believe they are preventing harm. But citizens are entitled to ask whether systems designed to suppress “dangerous misinformation” could also suppress inconvenient truths, legitimate dissent or early reporting that later proves correct.

The issue is not whether moderation should ever exist. The issue is whether the public can trust vague powers applied through opaque systems.

Powers Are Temporary. Capabilities Are Permanent.

This is the heart of the matter.

A government may introduce a power for one reason. A future government may use the same power for another. A regulator may expand guidance. A court may reinterpret obligations. A company may over-comply to avoid fines. A future crisis may justify emergency measures.

The original promise may have been child protection.

The inherited capability may be content control.

That does not require a conspiracy. It only requires the normal behaviour of institutions: expand, regulate, standardise, enforce, and protect themselves from blame.

Once the architecture exists, the debate changes. The question is no longer “Should we build this?” It becomes “What else should it scan for?”

Nudity today.
Extremism tomorrow.
Misinformation after that.
Copyrighted material.
Political imagery.
Protest organisation.
Unapproved news.

The technical principle is the same: inspect private content, classify it, then decide whether it should be allowed.

A Better Test: Proportionality

The proper test for any child-safety measure should be proportionality.

Does it target the actual risk?
Is it transparent?
Is it independently audited?
Can people appeal mistakes?
Does it affect adults unnecessarily?
Does it create new security risks?
Does it require identity checks?
Could it be expanded later?
Were less intrusive alternatives considered first?

If those questions are not answered clearly, the public should be cautious.

Child safety must not become a blank cheque for permanent digital monitoring.

Conclusion: The Smoke and the Mirror

The smoke is child protection.

The mirror is privacy.

Behind both is a much larger question: are we building a society where private devices remain under the control of their owners, or one where every image, message and interaction becomes subject to automated permission?

Protecting children is necessary. But protecting children should not require treating every citizen as a potential suspect, every device as a checkpoint, and every private photograph as something waiting to be judged by unseen software.

The danger is not only what this technology is designed to do today.

The danger is what it makes possible tomorrow.

And once that door is opened, history suggests it rarely closes by itself.

Latest Smoke & Mirrors Articles