For most of the last decade, “ethical data use” was something that happened in a different building. The lawyers wrote the privacy policy, the data protection officer ran the impact assessment, and the engineers built whatever the ticket said. The ethics lived in a PDF, and the pipeline lived in the warehouse, and the two rarely met. In 2026 that separation has quietly collapsed. The reason is not that engineers suddenly became more principled - it is that the decisions which determine whether data is used ethically are now made at the schema, the table, and the access-control layer, and those are the engineer’s decisions. Consent, deletion, minimisation, provenance, bias: every one of them is now something you either build into the pipeline or fail to. This is a practical look at what that means.

TL;DR

  • Ethical data use (EDU) is now an engineering concern, not a compliance afterthought. The choices that decide whether data is handled ethically - what you collect, how long you keep it, who can see it, whether you can delete it - are made in the pipeline, not in the privacy policy
  • Consent and purpose limitation have become schema-level metadata. The mature pattern in 2026 is to carry consent and allowed-purpose tags alongside the data itself, so enforcement happens at query time rather than in a spreadsheet nobody reads
  • The right to be forgotten collides head-on with the immutable lakehouse. Object storage does not let you edit a file in place, so honouring a GDPR Article 17 erasure request is a real engineering problem solved with merge-on-read deletes and deletion vectors, not a checkbox
  • Training-data ethics is the new frontier. Provenance, licensing, and consent for data used to train models are now first-class pipeline requirements, driven partly by the EU AI Act whose transparency obligations bite in August 2026
  • Privacy-preserving techniques have matured. De-identification, differential privacy, and synthetic data are no longer research curiosities - but each has sharp edges, and “anonymised” is a claim you have to be able to defend
  • Documentation is part of the deliverable. Datasheets for datasets and data cards turn “what is in this table and where did it come from” into an artefact, not tribal knowledge
  • The regulatory backdrop has hard deadlines now. GDPR, CCPA, the EU AI Act, the NIST AI Risk Management Framework, and the OECD AI Principles form a stack you build against, not a thing you read once

EDU is an engineering problem now, and that is the whole point

The phrase “ethical data use” sounds like it belongs in a values statement, and historically it did. The useful shift to internalise in 2026 is that ethics, for a data engineer, is not an abstract disposition. It is a set of concrete properties of a system: can it forget a person, can it prove where a column came from, can it stop the marketing team from joining medical data onto a campaign list, can it tell you whether the rows it is training a model on were collected with consent. Those are testable. They are the kind of thing you can write an assertion for, fail a build on, and monitor in production.

That reframing matters because it changes who is responsible. When ethics lives in a policy document, the engineer’s job is to not actively break the rules. When ethics lives in the data platform, the engineer’s job is to build the rules into the platform so that breaking them is hard by default. The second is a much higher bar, and it is the bar that regulators, customers, and increasingly courts now expect data teams to clear. The good news is that most of the building blocks already exist in the modern stack. The work is wiring them together with intent rather than bolting them on after an incident.

The oldest principle in data protection is also the one engineers used to handle worst. Article 5 of the GDPR lists purpose limitation and data minimisation as core requirements: data collected for one specified purpose should not be quietly repurposed for another, and you should only collect what you actually need. For years the standard implementation of “purpose limitation” was a sentence in a privacy policy and an honour system in the analytics team. That does not survive contact with a modern lakehouse where any table is one join away from any other.

The pattern that has matured in 2026 is to treat consent and allowed purpose as metadata that travels with the data. A row, or more often a column or a dataset, carries tags describing the legal basis under which it was collected and the purposes for which it may be used. Enforcement then happens at the catalog and query layer: a query that tries to use marketing-consented data for a credit decision is blocked or masked, not because someone remembered the policy, but because the platform checked the tag. Catalog-driven governance has made this practical - tags, row filters, and column masks defined once and enforced everywhere a query runs. The shift is from “we trust people to remember the rules” to “the rules are data, and the engine enforces them.” That is the single highest-leverage thing a data team can do for EDU, because it converts a human-discipline problem into an infrastructure problem, and infrastructure problems are the ones engineers are good at.

Data minimisation versus the “collect everything” reflex

Every data engineer has been trained, culturally, to hoard. Storage is cheap, you might need it later, and the most painful conversations are the ones where someone asks for a metric you did not start collecting two years ago. So we collect everything, keep it forever, and call it a data lake. Data minimisation asks for the opposite discipline, and in 2026 it has teeth: regulators increasingly treat “we collected it because we could” as a liability, and every extra field of personal data you hold is extra blast radius when - not if - there is a breach.

The practical move is to make retention and minimisation explicit, automated decisions rather than passive defaults. That means time-to-live policies on personal data, partition expiry that actually deletes rather than just archiving, and a real answer to the question “why do we hold this column” for every column that touches a person. The storage-limitation principle in Article 5 is not asking you to throw away your business data. It is asking you to stop holding identifiable personal data for longer than the purpose requires, and to be able to demonstrate that you do not. The teams doing this well in 2026 have made deletion a scheduled, monitored pipeline rather than a thing that happens when someone files a ticket.

The right to be forgotten meets the immutable lakehouse

This is where EDU stops being philosophy and becomes a genuinely hard engineering problem, and it is my favourite example because it is so concrete. Article 17 of the GDPR - the right to erasure, popularly the “right to be forgotten” - gives an individual the right, under certain conditions, to have their personal data deleted without undue delay. Now consider how a modern lakehouse stores data. Object storage like S3, GCS, and ADLS is fundamentally immutable: you cannot modify a file in place. A Parquet file containing one person’s data alongside ten million others cannot be edited to surgically remove that one person. So how do you honour an erasure request?

The answer that the open table formats have converged on is row-level deletes, and it is worth understanding the mechanics because they have direct cost and correctness implications. With copy-on-write, deleting a person’s rows means rewriting every data file that contained them - correct, but expensive if those rows are scattered across thousands of files. With merge-on-read, you instead write a small delete file that records which rows to exclude, and the engine merges those deletes at read time, with the actual data removed later during compaction. Newer table-format versions add deletion vectors as a more efficient hybrid. The engineering choices here are real: merge-on-read makes the deletion request cheap but adds read-time cost and, crucially, means the data is not physically gone until compaction runs. For an erasure obligation, “logically deleted but physically present until the next compaction” may or may not satisfy your regulator, and that is a conversation worth having before the request arrives, not after. The point is that “delete this person” is a system-design question with tradeoffs, and EDU means owning those tradeoffs deliberately.

The fastest-growing EDU concern in 2026 is not about analytics at all - it is about what data goes into models. As pipelines increasingly feed machine learning and large language models rather than dashboards, the data engineer inherits a set of questions that used to belong to researchers: where did this training data come from, who owns it, was it collected with consent, and can we prove any of that. The legal environment has made these questions unavoidable. The EU AI Act, the first comprehensive legal framework for AI, brings transparency obligations into force in August 2026, and its broader regime - documented on the European Commission’s official AI Act pages - pushes provenance and documentation of training data from good practice toward legal requirement.

For the pipeline, this means provenance is no longer optional metadata. You need to know, and be able to show, the source and licence of the data you use to train or fine-tune a model. It means filtering out data that was scraped without a clear basis, respecting the licensing terms of datasets you ingest, and keeping the lineage that lets you answer “what was this model trained on” months or years later. The uncomfortable reality is that a lot of the data sitting in lakes today was collected for analytics, under consent terms that say nothing about training models, and quietly repurposing it for that is exactly the purpose-limitation violation discussed earlier. The ethical and the legal point in the same direction here: if you cannot account for where your training data came from, you should not be training on it.

Bias, fairness, and representativeness start in the pipeline

It is tempting to think of model bias as a data-science problem that happens downstream of engineering. In practice, most bias is baked in long before a model sees the data, in the choices about what gets collected, how it is sampled, how missing values are handled, and which populations are over- or under-represented in the tables. A model trained on a dataset that systematically under-samples a group will be worse for that group, and no amount of clever modelling fully fixes data that was skewed at the source. That makes representativeness an engineering concern.

The practical EDU work here is unglamorous and valuable: understanding the population a dataset actually represents versus the one it claims to, documenting known gaps, being honest about sampling and selection effects, and surfacing the demographic composition of data where that is relevant and lawful to assess. This connects directly to the OECD’s framing in its AI Principles, which centre fairness and human rights, and to the NIST AI Risk Management Framework, which treats bias as a risk to be managed across the lifecycle rather than a property of the final model. The engineer’s contribution is to make the data’s limitations visible rather than letting them be silently inherited. You cannot manage a bias you have not measured, and you cannot measure one in data you have not documented.

Privacy-preserving techniques have grown up

The good news is that the toolkit for using data without exposing the people in it has matured considerably. The bad news is that the easy version - “we removed the names, so it is anonymous” - has never been weaker. Re-identification research has repeatedly shown that supposedly anonymised datasets can be linked back to individuals using a handful of quasi-identifiers, so “anonymised” is now a claim you have to be able to defend, not assert.

Three techniques are worth knowing well in 2026. De-identification - removing or masking direct identifiers - is necessary but rarely sufficient on its own, because of the linkage problem above. Differential privacy is the strongest formal guarantee available: by adding carefully calibrated noise, it bounds how much any single individual’s presence can affect the output, which is why it has been adopted for things as consequential as census releases. The tradeoff is a privacy-versus-utility dial that you have to set deliberately, and aggregate statistics survive it better than row-level detail. Synthetic data - generating artificial records that preserve statistical properties without corresponding to real people - has become genuinely useful for testing and development, though it carries its own risk that a poorly generated synthetic dataset can leak the very records it was meant to protect. The EDU discipline is matching the technique to the use case and being honest about the residual risk, rather than reaching for whichever one sounds most reassuring in a meeting.

Documentation is part of the deliverable

One of the quietest but most important shifts in EDU practice is the idea that a dataset is not finished until it is documented. The reference point here is Datasheets for Datasets, the proposal by Timnit Gebru and colleagues that every dataset should ship with a standardised document covering its motivation, composition, collection process, recommended uses, and known limitations - by analogy with the datasheets that accompany electronic components. The same idea appears as “data cards” in various forms across the industry.

For a data engineer this turns a vague responsibility into a concrete artefact. A datasheet forces you to write down where the data came from, what it contains, who is in it, what it should and should not be used for, and what is known to be wrong with it. That document is what lets the next person - or the next model - use the data responsibly instead of guessing. It is also, not incidentally, what auditors and regulators increasingly expect to see. The discipline is to produce it as part of building the dataset, while the knowledge is fresh, rather than reconstructing it from memory two years later when someone asks an uncomfortable question. Treat documentation as code: versioned, reviewed, and shipped alongside the data it describes.

Access control and least privilege

None of the above matters if everyone can see everything. The most common ethical data failure is not exotic - it is over-broad access, where data collected for a narrow purpose is readable by half the company because that was the path of least resistance when the table was created. The principle of least privilege - people and services get access to exactly what they need and nothing more - is the unglamorous backbone of ethical data use, and in 2026 the catalog-and-governance layer of the modern stack makes it genuinely enforceable: role-based access, attribute-based policies, row filters, and column masks defined centrally and applied wherever a query runs.

The EDU work is to use those capabilities with intent rather than defaulting to broad grants. Sensitive columns should be masked or tokenised for users who do not need the raw values. Access to personal data should be logged, reviewable, and time-bound where appropriate. And the answer to “who can see this” should be a property you can query, not a thing you have to reconstruct from a tangle of grants. This is where governance investment - in catalogs, in tagging, in policy-as-code - pays off directly as ethics, because least privilege enforced by the platform is far more reliable than least privilege enforced by good intentions.

The regulatory backdrop has real deadlines

It is worth being concrete about the regime data engineers are now building against, because in 2026 it has moved from principles to dated obligations. The GDPR remains the most influential data protection law in the world and the de facto baseline for anyone touching European personal data. The California Consumer Privacy Act and its successors anchor the US side, with rights to know, delete, and opt out. The EU AI Act adds an AI-specific layer with a risk-based structure and transparency obligations landing in August 2026, with high-risk system requirements phasing in through 2027 and beyond. And two voluntary-but-influential frameworks shape expectations even where they are not law: the NIST AI Risk Management Framework, which is becoming the common vocabulary for AI risk, and the OECD AI Principles, the first intergovernmental AI standard, adhered to by dozens of countries.

The right way to read this stack is not as a compliance burden to be minimised but as a reasonably coherent specification for what responsible data handling looks like. Strip away the legal language and these frameworks are asking for the same things good engineering already wants: know what data you have, know where it came from, only keep what you need, let people see and delete their own data, control who can access it, and document your decisions. If you build for those properties, compliance with most of the regime falls out as a byproduct. If you build against them and try to bolt compliance on at the end, you get neither good ethics nor good engineering.

Observability closes the loop

The final piece is that none of this is a one-time setup. Consent tags drift, access grants accumulate, retention jobs silently fail, and a schema change quietly starts collecting a field nobody approved. EDU in 2026 is something you monitor, not something you configure once. Data observability practices - the monitoring, testing, and alerting that data teams already use for quality - extend naturally to ethical properties: alert when personal data lands in a table that is not tagged for it, when an access pattern looks anomalous, when a retention deletion does not run, when lineage breaks so you can no longer trace where data came from. Treating ethical properties as things to be continuously verified, the same way you verify freshness and volume, is what turns EDU from a project into a practice.

A practical EDU checklist for 2026

If you want to turn all of this into something actionable, here is the short version to take into your next design review:

  • Tag consent and allowed purpose as metadata that travels with the data, and enforce it at the catalog and query layer
  • Set explicit retention and minimisation policies, with deletion as a scheduled, monitored pipeline rather than an ad-hoc ticket
  • Have a real, tested answer to erasure requests that accounts for the immutable-storage tradeoffs of your table format
  • Track provenance and licensing for all training data, and never repurpose analytics data for model training without checking the original consent basis
  • Document datasets as you build them, using a datasheet or data card that records source, composition, limitations, and intended use
  • Default to least privilege, with masking for sensitive columns and access that is logged, reviewable, and queryable
  • Choose privacy techniques deliberately - de-identification, differential privacy, or synthetic data - and be honest about the residual risk of each
  • Monitor ethical properties continuously, the same way you monitor data quality, so drift and failures surface before an incident does
  • Build against the regulatory stack as a spec, not a burden - GDPR, CCPA, the EU AI Act, NIST AI RMF, and the OECD principles mostly want the same things good engineering wants

Closing thought

The thing I keep coming back to is that ethical data use stopped being a values question and became a design question, and that is genuinely good news for engineers. Values are hard to argue about and easy to ignore. Designs are the thing we are paid to get right. When “can this system forget a person” and “can we prove where this column came from” become acceptance criteria rather than aspirations, ethics becomes tractable - it becomes the kind of problem we are equipped to solve. The teams that will look good in a few years are not the ones with the most carefully worded privacy policy. They are the ones who built the policy into the platform, so that doing the right thing with data was the path of least resistance rather than an act of heroism. In 2026, that is well within reach, and it is increasingly the job.