T 1952/21 - Is AI / machine learning as such technical?

Key points

• Inventive step is at issue for a claim directed to a "machine learning system" comprising generic hardware and some sotware.
• The Board: "The application relates to reinforcement learning. In reinforcement learning, an agent explores the environment according to a policy, determining which action the agent takes (e.g. move right) at every juncture as a function of its current state (e.g. its position in the environment). The agent receives rewards, positive or negative. In this way it can "learn" the value of the various actions and states. The goal of training is to maximize a value function which reflects the expected sum of rewards given a certain action. The application builds upon the method of D1, called A3C (asynchronous advantage actor-critic). That method separately approximates the policy and value models as neural networks. The raw input (describing the environment) is preprocessed in sequence by a feed forward network. The result is fed to the value and the policy networks. Developments of that method [i.e. the distinguishing feature of claim 1, as I understand it], termed NoisyNet A3C in the current application, inject randomness into the training by using stochastic weights (e.g. by adding random noise or using stochastic models) in the policy and value networks. This allows for further exploration of the parameter space )." [initernal citations omitted]
• The decision does not seem to explicitly identify the distinguishing features of claim 1 over D1. Claim 1 recites: "wherein said feed-forward neural network comprises stochastic units". This may be the relevant feature.
• The Board extensively summarizes the applicant's arguments, but I'm more interested in the Board's reasoning.
• The Board: "The Board remarks that pseudorandom number generators were known to the person skilled in the art. Their use, in general or in the more specific context of "stochastic units" [] does not change in substance the computer, which remains as "deterministic" as any conventional computer. So the Board cannot see a contribution on this level."
• I'm not sure if "well known" and "non-technical" are the same under the EPO approach to inventive step (cf. the US approach in Alice, 573 U.S. 208)
  
• "The system for reinforcement learning as claimed is a neural network, comprising various sub-networks, implemented on a computer. The network, as a whole, defines a mathematical function mapping inputs into outputs. Effectively, the claim is to a mathematical method implemented on a computer."
• I guess this observation is true for any type of software?
• "the Board holds that the Enlarged Board decision G 1/19, addressing the patentability of computer-implemented mathematical models for simulation, should be the starting point when assessing the technical character of reinforcement learning. It is commonly accepted that a large part of the findings in G 1/19 apply to any computer implemented inventions.
• " In G 1/19, the Enlarged Board of Appeal stated (reasons 137) that (simulation) models by themselves are not technical but that "they may contribute to technicality if, for example, they are a reason for adapting the computer or its functioning, or if they form the basis for a further technical use of the outcomes of the simulation". However, "such further use has to be at least implicitly specified in the claim"."
• The Board finds there is no implicit further technical use in the case at hand.
• " The Board has already explained above that the functioning of the computer, or the computer itself, are not adapted. "
• Regrettably, I can't find the 'above' quickly.
• "This conclusion is consistent with that in the case T 702/20, which is in many ways similar to the present one, where this Board (in a different composition) decided, also following G 1/19, that a trained machine learning model, namely a neural network, can "only be considered for the assessment of inventive step when used to solve a technical problem, e.g. when trained with specific data for a specific technical task" []."
• "The Appellant also argued that reinforcement learning was technical based on an analogy with the case law regarding cryptography, in particular RSA [T 1326/06]
• From that case: "Nach Meinung der Kammer handelt es sich beim sicheren Austausch von elektronischen Nachrichten um eine technische Wirkung, die zu erzielen als eine technische Aufgabe angesehen werden muss. RSA löst diese Aufgabe mit mathematischen Mitteln. Mit RSA gelang ein Durchbruch in der Entwicklung der Kryptografie: RSA wird als das erste praktikable, konkret implementierbare asymmetrische Kryptosystem angesehen und ist heute in zahlreichen kryptografischen Sicherheitssystemen eine zentrale Komponente. Die RSA zugrundeliegende Mathematik dient somit unmittelbar der Lösung eines konkreten technischen Problems."
•  "The Board notes that [...] RSA and reinforcement learning are different and serve different purposes. In particular, RSA and other cryptographic methods have a specific, and at least implied, purpose, namely data security. This is not the case for reinforcement learning. So the findings regarding RSA cannot directly be transferred to reinforcement learning."
• This raises the question: why is 'data security' technical and 'machine learning' not?
• To me, as a simple chemist, ChatGPT feels just like database management technology, turbo code interleavers for wireless communication, and copy/paste (clipboard) on the computer: I have (basically) no idea how it works, but it definitely makes my computer more useful, even though they are perhaps not all prize-winning.
• Whether introducing randomness for "further exploration of the parameter space " in the given context is obvious or not, I don't know. 
• "The Board accepts that the use of the term technical in the case law of the Boards of Appeal may differ from its use elsewhere in society, especially from its colloquial use. However, this does not mean that the Boards of Appeal interpret the law incorrectly: it is common place that the legal interpretation of a term may differ from its colloquial meaning. In particular, the Boards use the term "non-technical" to denote matter excluded under Article 52(2) and (3) EPC. Any alternative interpretation of the terms "technical" and "non-technical" can only be used to justify the patentability of subject-matter to the extent that it does not contradict the law, in particular the exclusion of mathematical methods."

The link to the decision and an extract of it can be found after the jump.

https://www.epo.org/en/boards-of-appeal/decisions/t211952eu1

The Board's opinion

16. The Appellant's allegation that stochastic units over come the limitations of the "deterministic" computer, goes beyond reinforcement learning and relates to a computer in general.

17. The Board remarks that pseudorandom number generators were known to the person skilled in the art. Their use, in general or in the more specific context of "stochastic units" (which the Appellant acknowledged to be known in the art, see the statement of grounds of appeal, page 4), does not change in substance the computer, which remains as "deterministic" as any conventional computer. So the Board cannot see a contribution on this level.

18. On the more narrow level of reinforcement learning, the non-deterministic behaviour of the claimed system is considered below.

19. The Board agrees with the Appellant that the skilled person would understand the claimed system to be one "for", i.e. meant to be used in, "reinforcement learning". The Appellant submission is, in a nutshell, that this field is technical and that the claimed invention makes improvements in this field.

20. The system for reinforcement learning as claimed is a neural network, comprising various sub-networks, implemented on a computer. The network, as a whole, defines a mathematical function mapping inputs into outputs. Effectively, the claim is to a mathematical method implemented on a computer.

21. Considering this, the Board holds that the Enlarged Board decision G 1/19, addressing the patentability of computer-implemented mathematical models for simula tion, should be the starting point when assessing the technical character of reinforcement learning. It is commonly accepted (also by the Appellant, see statement of grounds of appeal page 10, bottom) that a large part of the findings in G 1/19 apply to any computer implemented inventions.

22. In G 1/19, the Enlarged Board of Appeal stated (reasons 137) that (simulation) models by themselves are not technical but that "they may contribute to technicality if, for example, they are a reason for adapting the computer or its functioning, or if they form the basis for a further technical use of the outcomes of the simulation". However, "such further use has to be at least implicitly specified in the claim".

23. The implied use of the system in reinforcement learning requires, as the Appellant argued, an agent acting in an environment (see point 12.1 above). However, the agent and its environment need not exist in the real world, and can be completely virtual, e.g. part of a simulation model (a simulated agent acting within a simulated environment) or even a completely imaginary video game. The Board notes that both the prior art (see D1, section 5.1) and the scientific paper corresponding to the application referred to by the Appellant present results on video games. The concept of reinforcement learning in general does not imply a technical context.

24. The Board has already explained above that the functioning of the computer, or the computer itself, are not adapted. A further technical use is also not implied by the claim. So, even if the advantages in reinforcement learning brought forward by the Appellant were to be acknowledged (which is not the case, see below from point 32 on), the Board must conclude, on the basis of G 1/19, that the claimed system does not solve a technical problem.

24.1 This conclusion is consistent with that in the case T 702/20, which is in many ways similar to the present one, where this Board (in a different composition) decided, also following G 1/19, that a trained machine learning model, namely a neural network, can "only be considered for the assessment of inventive step when used to solve a technical problem, e.g. when trained with specific data for a specific technical task" (T 702/20, Catchword; see also reasons 12 and 17 to 19).

25. The Appellant also argued that reinforcement learning was technical based on an analogy with the case law regarding cryptography, in particular RSA (see 12 above).

25.1 The Board notes that, as the Appellant also acknowledged, notwithstanding certain similarities, RSA and reinforcement learning are different and serve different purposes. In particular, RSA and other cryptographic methods have a specific, and at least implied, purpose, namely data security. This is not the case for reinforcement learning. So the findings regarding RSA cannot directly be transferred to reinforcement learning.

25.2 It is therefore immaterial for the present decision whether individual Board of Appeal decisions relating to RSA are still applicable after G 1/19 or whether, as the Appellant seemed to imply, they are now wrong, i.e. "bad law".

26. The Appellant's opinion that decision G 1/19 has narrowed the scope of patentable subject matter and that this is in conflict with the evolution of technology and with a teleological interpretation of the EPC is noted. However, before the Board can deviate from the interpretations or explanations of the EPC given in G 1/19 it has to refer a question to the Enlarged Board (Article 21 RPBA). The Appellant did not propose a question to be referred, nor did it request that a suitable question be referred.

27. The Board itself sees no reason to deviate from G 1/19 in the present case.

27.1 The Appellant's argument that it should be possible to patent mostly abstract, mathematical inventions without a limitation to a specific technical application if they are generally applicable and have practical utility for a wide range of new products, may, from a business perspective, be a legitimate one. Although it may be assumed that the Appellant would find substantially less desirable an equally broad patent when held by a competitor.

27.2 But it was the lawmaker's choice to exclude from patentability, albeit only "as such", mathematical methods and programs for computers (see Articles 52(2) and (3) EPC).

27.3 Mathematical methods have always been generally appli cable (e.g. Pythagoras' theorem used to calculate dis tances) and been applied in many new - and undoubtedly technical - inventions. This did not prevent the legislator to list mathematical methods amongst the things which, as such, are not to be considered inventions. The fundamental nature of mathematical methods and their wide applicability may in fact have been a reason for excluding them from patentability.

27.4 The Board accepts that the use of the term technical in the case law of the Boards of Appeal may differ from its use elsewhere in society, especially from its colloquial use. However, this does not mean that the Boards of Appeal interpret the law incorrectly: it is common place that the legal interpretation of a term may differ from its colloquial meaning. In particular, the Boards use the term "non-technical" to denote matter excluded under Article 52(2) and (3) EPC. Any alternative interpretation of the terms "technical" and "non-technical" can only be used to justify the patentability of subject-matter to the extent that it does not contradict the law, in particular the exclusion of mathematical methods.