Beyond Turing
Introduction
A Customer Relationship Management system does not record customers: it produces profiles. The distinction is not rhetorical, it is architectural. In the entity-relationship model that underpins any CRM, a customer is a row in a master table - an identifier, a few attributes, a set of linked transactions. A profile is something different: it is the output of a processing chain that aggregates that master record with behavioural tracking, enriches it with derived attributes, applies statistical models to it and derives a predictive representation of the subject. The record is what the customer declared; the profile is what the system inferred about them.
This series has repeatedly identified a threshold: the point at which a technical artefact stops being a neutral instrument and becomes a decision about a person. In a CRM that threshold has a precise location in the data-processing graph. It coincides neither with collection nor with storage: it coincides with the moment an inferred attribute is fed back to the customer in the form of an automated action - a price, an offer, a refusal. Identifying that threshold, instrumenting it and making it observable is, before being a legal obligation, a problem in the engineering of information systems.
Regulation (EU) 2016/679 (GDPR) and Regulation (EU) 2024/1689 (the AI Act) are not, in this view, external constraints added to an already designed system. They are non-functional requirements to be embedded in the architecture from the design phase - compliance by design, in the same sense in which one designs for security or for scalability. The five stages that follow reconstruct the profiling pipeline as a sequence of components, and for each one identify the corresponding design requirement.
1. Identity: resolving the subject
Before it can profile a customer, a system must establish that two records refer to the same subject. This is the problem of entity resolution: reconciling fragmented identities across channels, devices and heterogeneous data sources. The computer-science literature treats it as a problem of probabilistic record linkage (Fellegi-Sunter, 1969), addressed today with blocking techniques, string similarity and supervised classifiers that estimate the probability that two records constitute a match.
The desired engineering outcome is the golden record of Master Data Management: a single, authoritative representation of the customer, produced by survivorship rules that resolve conflicts between sources. Here the first design requirement arises. Probabilistic resolution is, by definition, fallible: it produces false matches (records of two people merged into one profile) and false non-matches (one person split into two profiles). Every entity-resolution error propagates downstream and contaminates every subsequent inference.
The accuracy principle laid down in Article 5(1)(d) of the GDPR - personal data must be "accurate and, where necessary, kept up to date" - thus becomes a measurable data-quality specification: precision and recall of the matching procedure, rate of contested golden records, latency of correction propagation. A correct architecture exposes matching confidence as a first-class metadatum and maintains the provenance (data lineage) of every attribute of the golden record, a necessary condition for a rectification under Article 16 GDPR to be technically executable.
2. Category: segmenting behaviour
Once identity is established, the system places the subject into classes. Segmentation is the canonical operation of analytical marketing: unsupervised clustering (k-means, Gaussian mixture models), RFM scoring (Recency, Frequency, Monetary), construction of look-alike audiences by statistical extension from a seed of known customers. The product is a categorisation: each customer inherits the expected properties of the class to which they are assigned.
On the engineering side two risks are known and documented. The first is bias: a clustering model trained on historical data reproduces and amplifies the asymmetries present in the data; the composition of the classes is not neutral with respect to protected variables, even when these are not used explicitly, because they are reconstructed by correlation from apparently innocuous variables (postal code, device, browsing time). The second is opacity: the boundaries between segments, and the reasons for an assignment, are often non-inspectable, in tension with any explainability requirement.
The AI Act introduces a sharp boundary here. Article 5 prohibits certain practices regardless of sector: systems that exploit the vulnerabilities of specific groups or that employ manipulative techniques below the threshold of awareness to distort their behaviour are impermissible. Categorisation, in itself lawful and useful, crosses the threshold of the prohibition when it stops describing a segment and begins to exploit its fragilities. The difference, for the designer, is between a system that optimises the relevance of a communication and a system that optimises the reduction of the recipient's resistance.
3. Prediction: from segment to score
Segmentation describes the present; prediction anticipates the future. It is the stage in which the contemporary CRM concentrates its value: propensity models (probability of buying a product), churn models (probability of leaving), Customer Lifetime Value estimation, recommender systems that rank the offering according to expected behaviour. These are supervised-learning and collaborative-filtering problems, evaluated with standard metrics - AUC-ROC, precision@k, lift - which measure accuracy but not, in themselves, the legitimacy of the use.
It is here that the profile acquires normative force, because the predictive score becomes the premise of a decision. Article 22 of the GDPR grants the data subject the right not to be subject to a decision based solely on automated processing which produces legal effects or significantly affects them. The Court of Justice of the European Union, in case C-634/21 (SCHUFA, 7 December 2023), clarified that the very generation of a credit score, where the score determines to a significant degree the decision of the third party, constitutes an "automated decision" within the meaning of Article 22, even if the final act is formally adopted by another party.
The design consequence is precise: a scoring engine cannot be treated as a black box whose output is consumed downstream. It must be built for model governance - versioning of the model and of the training data, recording of the features used, the ability to produce an intelligible explanation of the individual prediction (the underlying logic, under Articles 13-15 GDPR). When the application domain falls among the high-risk ones of Annex III of the AI Act - notably access to essential services and creditworthiness assessment - the transparency requirement is compounded by the risk-management, data-quality and technical-documentation obligations of Articles 9-15.
4. Action: closing the loop
A profile that generates no action is a cost without return. The action stage closes the feedback loop: next-best-offer, dynamic pricing, interface personalisation, continuous experimentation through A/B tests that optimise the conversion rate in real time. It is applied behavioural engineering: the system does not merely predict the customer, it acts on them and measures the response in order to recalibrate the next action.
Precisely because it acts, this stage is the most exposed. Regulation (EU) 2022/2065 (the Digital Services Act), in Article 25, prohibits providers of online interfaces from designing them in a way that deceives or manipulates recipients, or that distorts or impairs their ability to make decisions - so-called dark patterns. EDPB Guidelines 03/2022 offer an operational taxonomy of these, directly readable as interaction-design anti-patterns: artificial scarcity, obstacles to unsubscribing, visual hierarchies that pre-orient the choice. For the systems designer they are documented defects, not aesthetic choices.
The dominant engineering requirement here becomes auditability. Every automated action delivered to the customer - which offer, at what price, on the basis of which segment and which score - must leave an immutable, reconstructible trace. Without structured logging of the action, none of the guarantees of the preceding stages is verifiable after the fact: the transparency declared upstream remains undemonstrable downstream, and the organisation loses the technical capacity to respond to a complaint or an access request.
5. The threshold: automated and assisted decision
The five stages converge on a distinction that is at once technical and regulatory: between an automated decision and an assisted one. A decision is automated when the model's output translates into an effect on the customer without substantive human intervention. It is assisted when an operator evaluates, can override and assumes responsibility for the outcome. The distinction is not declarative: it depends on the architecture of the control point.
Article 14 of the AI Act, on human oversight, and Article 22(3) of the GDPR, on the right to obtain human intervention, identify the same system component: a human-in-the-loop that is effective and not ceremonial. The difference between the two forms is entirely one of design. A system that presents the operator with only the model's recommendation, a confirm button and no information on the margin of uncertainty, produces an automated endorsement disguised as supervision. A system designed for substantive oversight exposes the prediction's confidence, makes visible the factors that determined it, sets a confidence threshold below which escalation to the operator is mandatory, and provides a safe fallback path when the model operates outside its domain of validity.
The confidence threshold is the exact engineering translation of the invariant running through this series: a system must know when it does not know, and stop. In a CRM this means that the profile is not a verdict but a signal: an estimate with an associated uncertainty which, beyond a certain degree of ambiguity, does not authorise the automated action but subordinates it to human judgement. Designing that threshold - choosing its value, instrumenting it, monitoring its drift over time (model drift) - is the point at which the engineering of information-processing systems meets the law and turns it into a measurable requirement. The customer remains a person, and not their profile, exactly to the extent that this threshold is designed to remind the system of it.