Providing a trained classification model for a classification task in a production process

The present disclosure relates to a computer-implemented method for providing a trained classification model for a classification task in a production process for manufacturing at least one workpiece and/or product. Further, the disclo- sure relates to a corresponding computer-implemented method for controlling a production process and a computer program product.

Artificial Intelligence ("Al") systems, such as machine learning models, are known from the prior art. The machine learning models are software programs whose behavior is learned from data instead of being explicitly programmed. The learning process is called "training" which requires plenty of data and significant computational resources. Thereby, the trained machine learning model solves a specific task for which it was trained, such as prediction of machine or pro- cess properties. Unlike conventional software, the learned behavior of the machine learning models highly depends on the data including parameters used during training, namely train- ing data.

In some industrial manufacturing environment, only small da- tasets of annotated data are available, which could be due to expensive data collection processes, e.g., high material costs for experiments in additive manufacturing, and time- consuming manual data labelling, which often requires domain experts. Using insufficient data typically leads to a low performance of the data-driven approaches developed for a certain application.

In industrial manufacturing there exist a high diversity of manufacturing scenarios. For example, for a production pro- cess there can be used different machine types from many ma- chine manufacturers. The production process may be performed with different settings of the machines or different sensor settings for collecting data from the machine. As an example, a camera is positioned in different angles to the monitored workpiece. Similarly, the wear of the tooling of the differ- ent machines as well as other production-related factors, e.g., changes in the manufacturing process, influence the distribution of the collected data from different clients. In result, the distribution of data recorded from the different manufacturing / production process scenarios differ. Such shifts in the data distributions typically lead to a degrada- tion of the performance of the machine learning model devel- oped for a certain production process. Traditional federated learning approach was conceived to securely aggregate knowledge between clients that have the same data distribu- tion and fails for clients with different data distribution.

Traditional training methods of a machine learning models, especially a federated learning approach perform well on in- domain data, i.e., data similar to the dataset they were trained with, they fail to generalize to out-of-domain data incurred by strong distribution shift. The distribution shift problem is also addressed by collecting data from the new distribution, e.g., from the new manufacturing machine, and labelling it, before training a new ML model or finetuning an existing ML model on it. This method has two drawbacks.

First, in most cases only few labelled data are available, which are sampled randomly. The few data samples of a new un- seen manufacturing scenario used to customize the adaptive learning model or even more used to customize an untrained learning model leads to a moderate performance of the model.

WO 2021/177879 Al discloses a method of generating a synthet- ic training dataset for training a machine learning model us- ing an original training dataset including a plurality of features. The synthetic training data is generated by select- ing one feature of the orig-inal training dataset as a target vector, selecting the remaining features of the original training dataset as a set of training input vectors and training a prediction model.

MA SHIYAO ET AL: "Privacy-Preserving Anomaly Detection in Cloud Manufacturing Via Federated Transformer", IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, IEEE SERVICE CENTER, NEW YORK, NY, US, vol. 18, no. 12, 13 April 2022 (2022-04- 13), pages 8977-8987, XP011921627, ISSN: 1551-3203, DOI:

10.1109/T11.2022.3167478 discloses a federated learning (FL) framework that allows edge devices to train an anomaly detec- tion model in collaboration with the cloud without compromis- ing privacy.

YI LIU ET AL: "A Systematic Literature Review on Federated Learning: From A Model Quality Perspective", D3 ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 1 December 2020 (2020-12-01),

XP081828970 reviews and objectively analyses the approaches to improving the quality of federated models.

One approach to generalize a machine learning model to unseen out-of-domain data is to train the ML model on data from dif- ferent manufacturing conditions, e.g., different machine types. This would allow to directly use this ML model on un- seen data collected by unknown machine types without requir- ing any labelling or retraining. The problem here is, that data owners, e.g., manufacturing companies are unwilling to share their data in order to preserve data-privacy and know- how, and possibly prevent reverse engineering. Such an ap- proach accelerates the development of ML models, but usually requires high volume of data transferred to the different da- ta sources.

Therefore, it is the object of the present application to train a ML model for at least one manufacturing scenario of different manufacturing scenarios with a small dataset of la- belled data and of differing data distribution in a fast and low memory consuming way while preserving the data-privacy. This object is solved by the features of the independent claims. The dependent claims contain further developments of the invention.

A first aspect concerns a computer-implemented method for providing a trained classification model for a classification task in a production process for manufacturing at least one workpiece and/or product, comprising: a. generating a central classification model and a central generative model on a central server, b. transferring a copy of the central classification model and a copy of the central generative model to several local servers, wherein each local server stores a process-specific dataset collected at a local production process, and the pro- cess-specific datasets having different data distributions, c. at the local server, optimizing the copy of the central classification model de- pending on a subset of the process-specific dataset and a synthetic local dataset generated by the copy of the central generative model resulting in a current local classification model, optimizing the copy of the central generative model depending on the synthetic local dataset resulting in a current local generative model, d. updating, at the central server, the central classifica- tion model and the central generative model by aggregating all current local classification models and by aggregating all current local generative models received from the local servers resulting in an aggregated central classification model and an aggregated central generative model, e. iterating step b-d by applying the aggregated central classification model and aggregated central generative model as central classification model and central generative model until a terminating criterion is met, f. providing the current local classification model of the last iteration as output and using it for the classification task in the production process of the respective local serv- er. Data-privacy is preserved even for process-specific data with different data distribution in the various local produc- tion processes.

The several local production processes are production pro- cesses which are different in one or few of the following process conditions: using different machine types, using the same machine type but from different machine manufacturers, different wear of the tooling of the machines, environmental differences in different production sites, manufacturing a different workpiece or changes in the manufacturing process. The small, insufficient amount data in each of the process- specific datasets is enhanced by additional synthetic local datasets generated by the generative model. This leads to a higher performance of the classification model in the optimi- zation step. By aggregating all the local generative models, the aggregated central generative model is trained to provide domain/process-agnostic synthetic data, which trains the classification model to better adapt to changing collected dataset during the production process. The generative model is optimized in a way, that its synthetic data will be recog- nized by all local production processes. Further, each pro- cess-specific dataset contributes to the training of all lo- cal classification models but remains in its local server. In total, the method improves the efficiency of training and the performance of the central classification and all the local classification models while protecting the data-privacy of each local server.

In an embodiment the optimizing step is performed iteratively applying the current local classification model and the cur- rent local generative model as copy of the central classifi- cation model and as copy of the central generative model.

Thus, the local classification model and local generative model are adapted and optimized for the local production pro- cess, i.e., to provide process-specific classification re- suits and to generate synthetic datasets similar to the

"original" process-specific dataset.

In a further embodiment the copy of the central generative models and local generative models are optimized to generate images or feature maps of images from a randomly initialized vector.

This increases the diversity of the generated images or fea- ture maps.

In an embodiment each classification model consists of a fea- ture extractor and a classifier, and wherein the feature ex- tractor inputs an image and outputs a feature map, and where- in the classifier inputs a feature map and outputs a class probability.

In an embodiment the process-specific dataset of each local production process are private data, and the owner of the lo- cal production process stores the private data on its local server, and the private data are not transferred to the cen- tral server.

This secures data-privacy.

In an embodiment the original classification model and/or the original generative model are deep learning models.

In an embodiment the copy of the central generative model is optimized by minimizing the error of the classification model when applying the synthetic local dataset.

Such, the local generative model is trained to generate syn- thetic data for the local production process. Using the opti- mized synthetic local dataset during the optimization of the local classification model accelerates the performance and consequently also the training of the local classification model. According to the invention the central and local generative model generate feature embeddings and output them as synthet- ic local dataset for optimizing the copy of the central clas- sification model.

The Feature embedding is data with reduced dimension compared to the feature which is originally encoded in the datapoint. The feature embedding provides the most relevant information of the feature. Therefore, the optimizing the local classifi- cation model with feature embeddings enhances the performance in on optimization step and less optimization steps are re- quired to meet the termination criterion. The training of the local classification model is reached faster.

In a further embodiment the aggregating in step d. is per- formed by averaging parameters of all local classification models and averaging parameters of all the current local gen- erative models into the aggregated central classification model and central generative model.

This provides an aggregated central classification model and central generative model which is equally balanced with re- spect to the contributing local production processes. The ag- gregated central classification model provides a good perfor- mance also applied in a new production process different to all of the local production processes.

In an embodiment the feature of the datapoint comprises in- formation of a feature map of an image of the production pro- cess.

The feature of a datapoint comprises a set of parameters measured on at least one machine performing the production process or a set of parameters measured on at least one work- piece and/or product manufactured by the production process. The feature can also be an image. In this case, the feature map of an image provides the information which is most dis- tinguishing for object on the image. Datapoints comprising only the information of a feature map is of reduced volume compared to a dataset encoding the complete image.

In an embodiment the process-specific dataset of each local production process is private data, and the private data is not transferred to the central server.

In an embodiment the production processes are one of an addi- tive manufacturing process or a milling process, and the classification task is one of anomaly detection, failure classification, condition monitoring of a machine in the pro- duction process, or a quality control, product sorting of the workpiece and/or product manufactured in the production pro- cesses.

A second aspect concerns a Computer-Implemented method for performing a classification task in at least one of several local production process for manufacturing at least one work- piece and/or product, comprising:

- collecting sensor data measured by sensors monitoring the production process while operation and manufacturing at least one workpiece and/or product,

- inputting the collected sensor data into a current local classification model of a last iteration trained as described above,

- outputting a classification result by the current local classification model indicating a classification result of the classification task of the local production.

The current local classification model shows high performance without significant adaption and training due to the method of its training.

A third aspect concerns a system for providing a trained classification model for a classification task in a produc- tion process for manufacturing at least one workpiece and/or product, comprising a central server and several local serv- ers configured to perform: a. generating a central classification model and a central generative model on a central server, b. transferring a copy of the central classification model and a copy of the central generative model to several local servers, wherein each local server stores a process-specific dataset collected at a local production process, and the pro- cess-specific datasets having different data distributions, c. at the local server, optimizing the copy of the central classification model de- pending on a subset of the process-specific dataset and a synthetic local dataset generated by the copy of the central generative model resulting in a current local classification model, optimizing the copy of the central generative model depending on the synthetic local dataset resulting in a cur- rent local generative model, d. updating, at the central server, the central classifica- tion model and the central generative model by aggregating all current local classification models and by aggregating all current local generative models received from the local servers resulting in an aggregated central classification model and an aggregated central generative model, e. iterating step b-d by applying the aggregated central classification model and aggregated central generative model as central classification model and central generative model until a terminating criterion is met, f. providing the current local classification model of the last iteration as output and using it for the classification task in the production process of the respective local serv- er, wherein the central generative model Gi and local genera- tive model Gk generate feature embeddings and output them as synthetic local dataset for optimizing the copy of the cen- tral classification model.

A fourth aspect concerns a computer program product directly loadable into the internal memory of a digital computer, com- prising software code portions for performing the steps as described before, when said product is run on said digital computer.

The computer program product can be provided by a computer- readable memory medium having computer program instructions stored thereon, which perform the steps of the method accord- ing to the preceding claims when executed by a processor.

The invention will be explained in more detail by reference to accompanying figures. Similar objects will be marked by the same reference signs.

Figure 1 schematically illustrates an embodiment of the in- ventive system.

Figure 2 shows an embodiment of the inventive method for providing a trained classification model illustrat- ed by a schematic information flow diagram.

Figure 3 shows an embodiment of the inventive method for controlling a production process illustrated by a schematic information flow diagram.

It is noted that in the following detailed description of em- bodiments, the accompanying drawings are only schematic, and the illustrated elements are not necessarily shown to scale. Rather, the drawings are intended to illustrate functions and the co-operation of components. Here, it is to be understood that any connection or coupling of functional blocks, devic- es, components or other physical or functional elements could also be implemented by an indirect connection or coupling, e.g., via one or more intermediate elements. A connection or a coupling of elements or components or nodes can for example be implemented by a wire-based, a wireless connection and/or a combination of a wire-based and a wireless connection. Functional units can be implemented by dedicated hardware, e.g., processor or firmware, and/or by a combination of dedi- cated hardware and firmware and software. It is further noted that each functional step of the method can be performed at a functional unit on the related system.

In industrial manufacturing, there exists a high diversity of manufacturing scenarios. E.g., in production processes pro- ducing the same product, there exist different machine types from different manufacturers that can be used, or different tools of the machine are used, or a variation of products are produced, to name a few. For each of these production pro- cesses only a small set of data is available, which is col- lected during operation of the local production process and annotated with respect to a certain classification. To train- ing a ML model for performing a classification task, e.g., for anomaly detection, failure classification of a machine or of the product manufactured by the process a much larger vol- ume of annotated data is required. Even more, there are dif- ferent stakeholders, like a product process owner, a machine manufacturer or a service provider which are not are unwill- ing to share their data.

Fig. 1 shows such a scenario with more than one local produc- tion processes 13.1, 13.2, 13.k and an embodiment of a system 10 for providing a trained classification model for a classi- fication task in at least one of the local production pro- cesses. The local production processes 13.1, 13.2, 13.k manu- facture at least one workpiece and/or product.

The system 10 comprises one central server 11 and several lo- cal servers 12.1, 12.2, 12.k. Each of the local servers 12.1, 12.2, 12.k comprises a data interface to communicate with the central server 10 and a storage unit that stores a process- specific dataset PD1, PD2, PDk collected at the respective local production process 13.1, 13.2, 13.k. The process- specific dataset PD1, PD2, PDk of each local production pro- cess 13.1, 13.2, 13.k is private data, i.e., owned by differ- ent customers, and the private data as such is not trans- ferred to the central server 11 throughout the method steps performed by the system.

The process-specific dataset PD1, PD2, PDk are sensor data recorded over time by sensors measuring physical parameters, e.g., torque, rotation speed, temperature, pressure, voltage or image data captured by a camera from the work piece or a machine during the local production process 13.1, 13.2, 13.k. A datapoint of the dataset comprises a feature and a label assigned to the feature. The feature comprises a set of dif- ferent sensor data measured at a certain point in time in the local production process 13.1, 13.2, 13.k. The label indi- cates a class characterizing the production process, e.g., a machine in the production process, or a workpiece or product manufactured by the production process. The classification task which shall be performed by the machine learning method can be a dual-class classification task, like anomaly detec- tion with two classes indicating either for normal mode or failure mode, or a multi-class classification tasks like failure classification or condition monitoring. Labelling or labelled data is used as synonym for annotating or annotated data in this description.

The local production processes 13.1, 13.2, 13.k as well as a production process 13.n are preferably one of an additive manufacturing process or a milling process. The classifica- tion task is one of anomaly detection, failure classifica- tion, condition monitoring of a machine in the production process, or a quality control, product sorting of the work- piece and/or product manufactured in the production process- es.

The system 10 is configured to provide a trained classifica- tion model for at least one up to all of the local production processes 13.1, 13.2, 13.k according to following embodiment of proposed method. The method illustrated in Figure 2 and described below. In a first step SI, a central classification model H and a central generative model G are generated and randomly ini- tialized on a central server 11. Preferably, the classifica- tion model H and the central generative model Gare deep learning models. The classification model H can be split into a feature extractor F_k, which e.g., inputs an image and out- puts a feature map, and a classifier C_k, which inputs a fea- ture map and outputs a class probability.

In the next step S2, a copy Hi of the central classification model H and a copy Gi of the central generative model G is transferred to several local servers indicated by the arrows in Fig.2. In the system 10 shown in Fig.l, each of the local servers 12.1, 12.2, 12.k received such copies indicated by G1,H1, G2,H2 and Gk,Hk.

In step S3, at each of the local servers 12.1, 12.2, 12.k the copy Hi of the central classification model is optimized de- pending on a subset of the respective process-specific da- taset PD1, PD2, PDk and a synthetic local dataset generated by the copy Gi of the central generative model G resulting in a current local classification model Hkr. The copy Gi of the central generative model is optimized depending on the syn- thetic local dataset resulting in a current local generative model Gkr. Fig.2 exemplarily shows the steps performed at lo- cal server 12.k. In an embodiment the copy Gk of the central generative model is optimized by minimizing the error of the classification model Hk when applying the synthetic local da- taset. Preferably, the optimizing step S3 is performed sever- al times, i.e., in rounds applying the resulting current lo- cal classification model Hkr and the resulting current local generative model Gkr as starting point. The central Gi and local generative model Gk generate feature embeddings and output them as synthetic local dataset for optimizing the copy of the central classification model. In an embodiment the current generative model Gk is optimized to generate im- ages or feature maps of images from a randomly initialized vector. In step 4, the central server 11 receives a copy of the cur- rent local classification models Hlr, H2r, Hkr from each lo- cal server 12.1, 12.2, 12.k and aggregates them securely into an aggregated central classification model Hi+1 and an aggre- gated central generative model Gi+1. In an embodiment the ag- gregating is performed by averaging parameters of all local classification models Hlr, H2r, Hkr and averaging parameters of all the current local generative models Glr, G2r, Gkr into the aggregated central classification model Hi+1 and central generative model Gi+1.

In step S5 the aggregated central classification model Hi+1 and the aggregated central generative model Gi+1 is checked with respect to a termination criterion, e.g., a predefined number of iterations, a predefined duration, or a predefined convergence criterion. If the termination criterion is not met, the aggregated central classification model Hi+1 and ag- gregated central generative model Gi+1 is applied as central classification model Hi and central generative model Gi and steps S2 to S5 are repeatedly performed.

If the termination criterion is matched, the current local classification model Hki+1 of the last iteration is output and used for the classification task in the production pro- cess 13.k of the respective local server 12.k.

The current local classification model Hki+1 of the last it- eration is output and applied as trained model in multiple manufacturing scenarios, e.g., defects detection on the prod- uct surface of 3D printing/additive manufacturing. Here, the types of the defect are normally predefined and distributed among local production processes. However, the collected im- ages may vary in appearance due to the production setups, e.g., different camera configurations or different material types, leading to image appearance deviation. Also, the de- scribed method is applicable for non-vision tasks, e.g., time-series temperature monitoring with sensors, where the installation position of sensor in different clients could cause input data characteristic deviation.

Below the method is provided in pseudo code.

Applying the trained classification model, i.e., performing a classification task in at least one of several local produc- tion process for manufacturing at least one workpiece and/or product is performed as shown in Fig.3.

In a first step Sil, at each of the several production pro- cesses 12.1, 12.2, 12.k sensor data are collected in the pro- duction process while operation. The collected sensor data are input into a current local classification model Hki+1 of the last iteration trained according to above-described meth- od, see step 12. The current local classification model Hki+1 outputs a classification result, see step S13, by indicating the classification task of a local production. In last step S14, the local production process is controlled depending on the classification result. As example, the settings of a ma- chine in the local production machine are adapted depending on the classification result or a product manufactured by the local production process is sorted depending on the classifi- cation result.

The trained classification model can be applied also to a new production process if the sensor data measured at the new production process contains the same features as the dataset the classification model was trained with.

It is to be understood that the above description of examples is intended to be illustrative and that the illustrated com- ponents are susceptible to various modifications. For exam- ple, the illustrated concepts could be applied for different technical systems and especially for different sub-types of the respective technical system with only minor adaptions.