Imagine an e-commerce business that wants to use artificial intelligence to better understand its customers. The AI monitors what a person views, how long they hesitate before making a purchase, which advertisements they respond to, and at what price point they leave the website. It then adjusts the offer to make it as appealing as possible.
Using artificial intelligence in this way is not necessarily a problem in itself.
But what if the system goes further? Based on the available data, it determines that a particular customer is in financial distress or is especially susceptible to influence at a certain moment. It then deliberately exploits that vulnerability to push the person into making a purchase or entering into a contract that they would not normally have concluded.
At that point, it is no longer enough to review the e-shop’s terms and conditions, GDPR requirements, data protection rules or the contract with the software provider. The European Union’s AI Act also recognises certain uses of artificial intelligence that are simply prohibited.
And that is a fundamental distinction. With many AI systems, the question is what conditions a company must meet in order to use them lawfully.
The EU AI Act Also Establishes Absolute Red Lines
The AI Act, Regulation (EU) 2024/1689, categorises the use of artificial intelligence according to risk. However, for certain uses, the European Union has concluded that merely adding documentation or introducing additional safeguards is not enough.
Article 5 therefore establishes prohibited AI practices. These rules are not aimed solely at technology giants. They may also apply to an ordinary limited liability company, an insurance company, a bank, a marketing agency, an online retailer, or a startup developing a new AI product.
The AI Act distinguishes, for example, between a “provider” — broadly speaking, an entity that develops an AI system or places it on the market under its own name — and a “deployer”, meaning a company or other organisation that uses AI in the course of its activities.
This distinction is important not only from the perspective of the AI Act, but also in the broader context of commercial law and the liability of the company, its management or its managing director.
When Personalised Advertising Stops Being Just Advertising
Advertising naturally aims to persuade customers. An e-shop recommends products based on previous purchases, a bank offers a client a product based on their needs, and an online service adapts content according to what the user has searched for in the past.
The AI Act does not automatically prohibit any of this.
The line may be crossed where AI uses subliminal, deliberately manipulative or deceptive techniques that materially impair a person’s ability to make an informed decision. The result must be a decision that the person would otherwise not have made, and the conduct must cause, or be reasonably likely to cause, significant harm.
A subliminal technique can broadly be understood as influencing a person outside their conscious awareness. Imagine an application offering financial products. Based on the user’s behaviour, it is able to determine that the person is under severe financial pressure. Instead of simply displaying an offer, it deliberately exploits that situation. It creates an artificial sense of urgency, presents the offer as though it were the only available solution, and guides the person towards taking out an unfavourable consumer loan.
In such a case, the issue may no longer concern only consumer protection law, a consumer contract, general terms and conditions, or whether the conduct constitutes misleading advertising or another unfair commercial practice. A prohibition under the AI Act may also become relevant.
However, it remains true that not every aggressive advertisement constitutes prohibited AI manipulation. It is not enough that an advertisement persuaded someone. The AI Act requires additional conditions to be met, including a material impairment of decision-making and significant harm.
It is therefore necessary to distinguish between ordinary marketing, issues such as misleading advertising under commercial law, unfair competition in practice or consumer protection, and the specific prohibition on manipulation using AI.
Social Scoring Does Not Exist Only in the Public Sector
At first glance, the term “social scoring” may sound somewhat remote.
The principle is relatively simple. AI collects information about a person, assesses their behaviour or inferred characteristics, and uses that information to create an overall score. That score is then used to determine how the person will be treated.
The AI Act prohibits this type of assessment in certain circumstances.
Imagine an insurance company that, when entering into an insurance contract, does not rely solely on information relevant to the insured risk. The AI also begins analysing the customer’s public social media posts, purchasing habits, leisure activities and other information from various areas of their life.
It then creates a general “trustworthiness score”. A person with a low score is subsequently denied a service or offered significantly worse terms by the insurance company.
That is very different from an ordinary assessment of a specific insurance or credit risk. What matters in particular is which data the AI combines, for what purpose the person is being assessed, and what consequences result from the score.
And where personal data is being processed, the AI Act does not, of course, displace the GDPR. Regulation (EU) 2016/679 and, where relevant, Slovak data protection legislation must still be taken into account.
The AI Act and the GDPR are simply two different layers of regulation. The fact that a particular use of AI is not prohibited under the AI Act does not automatically mean that it is lawful from a data protection perspective.
A Supplier Contract or Terms and Conditions Cannot Turn Prohibited AI Into Permitted AI
In business, we are accustomed to addressing many risks contractually.
A contract may include provisions on supplier liability, representations of legal compliance, rules on data use, or trade secrets. If necessary, an amendment can be signed, a licence modified, or general terms and conditions updated.
When it comes to prohibited practices under the AI Act, however, one simple rule applies: a contract cannot turn a legally prohibited use of AI into a permitted one.
For example, if a company uses a system for prohibited biometric categorisation, it does not help that the supplier promised compliance with the law in the contract. Likewise, it cannot automatically be assumed that obtaining the consent of the person concerned resolves everything.
In some legal regimes, consent is highly important. The GDPR recognises consent, and the AI Act itself expressly works with consent in connection with the new prohibition concerning intimate content.
For companies using AI products, it is therefore important to look beyond the validity of the contract, the terms and conditions or the licence.
The AI Act Does Not Operate in Isolation: GDPR, Consumer Law, Copyright and Other Rules Still Apply
One common mistake when conducting a legal assessment of artificial intelligence is to look for a single “AI law” that resolves every issue.
European regulation does not work that way.
The AI Act expressly does not replace the GDPR or existing rules on data protection and consumer protection.
A single AI product may therefore require simultaneous consideration of, for example:
- the AI Act, in relation to how the AI may be used;
- the GDPR and Regulation (EU) 2016/679, where the system processes personal data;
- national data protection legislation;
- consumer protection law, where AI is used in relation to consumers;
- rules against unfair commercial practices, where the system influences customers’ commercial decisions;
- unfair competition or misleading advertising rules;
- copyright law, where AI works with photographs, texts, software code or other protected works;
- licensing agreements and intellectual property rights, where the company uses a third-party model, database or software;
- and, of course, traditional commercial law, contractual obligations and liability for damage.
Fines May Reach Millions of Euros
Prohibited practices are among the infringements for which the AI Act provides the most severe penalties.
Failure to comply with the prohibitions under Article 5 may result in an administrative fine of up to EUR 35 million or 7% of the undertaking’s total worldwide annual turnover for the preceding financial year, in accordance with the rules of the AI Act.
This does not mean that every small business or startup will automatically receive a EUR 35 million fine. The specific circumstances are taken into account when determining a penalty, and the AI Act also contains specific rules applicable to smaller businesses.
The maximum level of the penalty nevertheless illustrates how seriously the European Union views these practices.
For the managing director of a limited liability company, a startup or any other business, the legal assessment of an AI product should therefore not be something that begins only after the service has already been launched.