Training Neural Networks with Manifold Constraints via PyTorch

In the following several examples, we mainly aim to illustrate that it is easy to describe neural networks with manifold constraints by the build-in layers in CDOpt based on PyTorch package. Therefore, training these networks can be greatly facilitated by the advanced features from PyTorch and various optimizers provided by PyTorch and torch-optimizer packages.

• Training LeNet with Constrained Convolution Kernels
• Training Single-Layer RNN with Constrained Weights
• Training Multi-Layer RNN with Constrained Weights
• Training LSTM with Constrained Weights
• Time Sequence Prediction with Orthogonality Constrained LSTM
• Distributed Training for RNN with Constrained Weights
• Distributed Training for A Simple Network by Distributed RPC Framework