Title:
METHOD AND APPARATUS FOR TRAINING NEURAL NETWORK MODEL, AND DEVICE AND SYSTEM
Document Type and Number:
WIPO Patent Application WO/2024/060727
Kind Code:
A1
Abstract:
A method and apparatus for training a neural network model, and a device and a system, which are applied to a computing device for training a neural network model. The method comprises: during the process of performing quantization training on a neural network model, for the problem of an inaccurate gradient caused by quantization, a computing device changing a gradient compensation strategy according to a fluctuation value of a quantization error of a parameter, using an applicable gradient compensation strategy to correct the gradient, and updating a parameter of the neural network model on the basis of the gradient determined by the gradient compensation strategy, so as to obtain an optimized neural network model. Therefore, the accuracy of a gradient of a parameter of a neural network model is improved, and the precision of model training is ensured according to the precision of a parameter determined by the gradient.
Inventors:
PAN YIRONG (CN)
YAO YIWU (CN)
WANG BING (CN)
YAO YIWU (CN)
WANG BING (CN)
Application Number:
PCT/CN2023/101170
Publication Date:
March 28, 2024
Filing Date:
June 19, 2023
Export Citation:
Assignee:
HUAWEI TECH CO LTD (CN)
International Classes:
G06N3/082
Foreign References:
| CN112884146A | 2021-06-01 | |||
| CN111429142A | 2020-07-17 | |||
| CN112085074A | 2020-12-15 | |||
| US20200202213A1 | 2020-06-25 | |||
| KR102389910B1 | 2022-04-22 |
Attorney, Agent or Firm:
BEIJING ZBSD PATENT & TRADEMARK AGENT LTD. (CN)
Download PDF: