Your keyboard learned from what a million people typed and nobody read any of it. The awkward part is that a model can still leak

Your phone's keyboard gets better at predicting the words you use, including words no dictionary contains. Somebody had to learn that from what people actually type. Shipping everyone's messages to a server to do it would be indefensible, and not learning at all would mean a worse product. The industry's answer is a set of techniques worth understanding, because they split cleanly into two halves — and almost everyone who talks about privacy in machine learning only knows the first half.

Send the model to the data

The first idea is simple once stated. Instead of collecting data centrally, send the current model out to the devices that hold the data.

Each phone trains the model a little on whatever it has locally — your messages, your corrections, your habits — and then sends back not the data but the update: the adjustment its own training suggested. The server collects many such updates, averages them, applies the average to the shared model, and sends the improved version out again. No example ever leaves any device. This is federated learning, and it is not a proposal; mobile keyboards and on-device speech features have run this way for years.

It is harder than it sounds, for reasons that are mostly not about privacy at all.

The data is unevenly distributed in the worst way. Ordinary training assumes each batch of examples is a fair sample of the whole. A single person's phone is a wildly unrepresentative sample — one language, one vocabulary, one set of habits — so averaging updates computed on radically different distributions does not behave like normal training. Models drift, and a great deal of the research literature is about stopping that.

Devices are not servers. They go offline, run out of battery, sit on metered connections. You train with whoever happens to be available, plugged in and on a good network, which is itself a biased sample of your users.

Updates are large. A model's worth of numbers is expensive to send from a phone, repeatedly, which is why update compression is a standard part of the design rather than an optimisation.

The update is not innocent

Here is the turn that matters, and the part usually skipped.

An update is computed from the data. It is a function of those specific examples, so it carries information about them. This is not theoretical: researchers have reconstructed recognisable training inputs from gradients under some conditions, and separately shown that trained models memorise rare examples and can be induced to repeat them. If a person's unusual password, address or medical detail appeared once in training, a sufficiently determined query can sometimes pull it back out.

So "we only send updates, never data" is a genuine reduction in exposure and not a privacy guarantee. It is the difference between not handing someone your diary and not telling them anything about your week.

Hiding the data is not the same as hiding what the data did. Anything computed from a person's records carries some information about that person, and a model is something computed from records.

Making individual contributions invisible

Two further mechanisms close different parts of the gap.

Secure aggregation uses cryptography so that the server can see the sum of thousands of updates without seeing any single one. Devices agree on masks that cancel out when added together: each participant's update is obscured by random values, and only when enough of them are combined do the masks vanish and the total emerge. The server gets what it needs and is structurally incapable of inspecting one contributor — which matters because the server operator is one of the parties you might want to be protected from.

Differential privacy attacks the remaining problem, and it is the idea worth learning properly because it is so often invoked loosely.

Its definition is a promise about indistinguishability: an algorithm is differentially private if its output is almost exactly as likely whether or not any one individual's data was included. If your record cannot detectably change the result, the result cannot betray you. You get this by adding carefully calibrated random noise — to gradients during training, or to statistics before release.

Three consequences follow, and all three are routinely misrepresented.

It is a budget. The strength of the guarantee is a parameter, and every query or training run spends some of it. Ask enough questions of the same dataset and the guarantee erodes to nothing, because the noise averages out across repeated answers. Privacy accounting is bookkeeping you cannot skip.

It costs accuracy, explicitly and unavoidably. Noise added for privacy is noise the model must work around. There is no configuration in which you get the same result for free; the honest engineering question is how much utility you are buying with how much exposure.

And it is a property of the procedure, not of a dataset. You cannot look at a released table and determine whether it is differentially private. You have to know what was done to produce it.

Why "we anonymised it" is not a claim

This is the practical part, and it is the mistake most organisations make.

Removing names, phone numbers and identity numbers does not anonymise a dataset. What remains are combinations of ordinary attributes that are close to unique. A well-known analysis found that the large majority of a national population could be pinned down from just postcode, date of birth and sex. Datasets released as anonymous — film ratings, taxi journeys, mobility traces — have been re-identified repeatedly by matching against public information, sometimes years after release, at which point nothing can be undone.

Differential privacy exists precisely because removal-based anonymisation does not work. The distinction is between a claim you can state mathematically and defend, and a claim you have merely asserted about a file you are about to share.

Why it matters for students and researchers

In India this has stopped being an ethics seminar topic. The Digital Personal Data Protection Act creates obligations around handling personal data, which means the question "can we train on this?" now has a legal answer as well as a technical one — and the technical answer determines what you are permitted to build. Health records, credit and payments data, telecom traces and education data are all large, sensitive and exactly the datasets where useful models would help most.

The standard expensive mistake is to design a system on the assumption that data can be collected now and anonymised later. Anonymisation-by-deletion is not a step you can add at the end; the architecture either keeps raw data out of the centre or it does not, and retrofitting that decision means rebuilding.

The general skill worth developing is narrower than it sounds: learning to tell the difference between a privacy property that has been proved about a procedure and one that has been asserted about an outcome. Most published claims are the second kind. Being the person in the room who can tell them apart is unusually valuable, and it requires understanding these mechanisms rather than their names.

Frequently asked questions

What is federated learning?

It is training a shared model by sending it to the devices that hold the data, training locally on each, and returning only the model updates to be averaged centrally. The raw data never leaves the device.

If the data never leaves my phone, is my data private?

Not entirely. The update sent back is computed from your data and carries information about it, and trained models can memorise rare examples. Federated learning reduces exposure substantially but is not a guarantee on its own.

What is differential privacy?

It is a mathematical guarantee that an algorithm's output would be almost equally likely whether or not any single person's data were included, achieved by adding calibrated random noise. It is a property of the procedure, not of a dataset.

Why does differential privacy reduce accuracy?

Because the protection comes from deliberately added noise, which the model or statistic must then work around. Stronger privacy means more noise and less precision, so the setting is always an explicit trade-off.

Isn't removing names enough to anonymise data?

No. Combinations of ordinary attributes such as postcode, date of birth and sex identify most people uniquely, and datasets released as anonymous have repeatedly been re-identified by matching against other available information.