Parameters:#

• in_features – size of each input sample

• out_features – size of each output sample

• bias – If set to False, the layer will not learn an additive bias. Default: True

• manifold_class – The manifold class for the weight matrix. Default: cdopt.manifold_torch.euclidean_torch

• penalty_param – The penalty parameter for the quadratic penalty terms in constraint dissolving function

• manifold_args - The additional key-word arguments that helps to define the manifold constraints.

Shape:#

• Input: $(\ast ,H_{in})$ where $\ast$ means any number of dimensions including none and $H_{in}=\mathrm{in}\mathrm{\_}\mathrm{features}$.

• Output: $(\ast ,H_{out})$ where all but the last dimension are the same shape as the input and $H_{out}=\mathrm{out}\mathrm{\_}\mathrm{features}$.

Attributes:#

• manifold (cdopt manifold class) – the manifold that defines the constraints. The shape of the variables in manifold is set as $(\mathrm{out}\mathrm{\_}\mathrm{features},\mathrm{in}\mathrm{\_}\mathrm{features})$ if $\mathrm{out}\mathrm{\_}\mathrm{features}\ge \mathrm{in}\mathrm{\_}\mathrm{features}$. Otherwise, it is set as $(\mathrm{in}\mathrm{\_}\mathrm{features},\mathrm{out}\mathrm{\_}\mathrm{features})$.

• weight (torch.Tensor) – the learnable weights of the module of shape $(\mathrm{out}\mathrm{\_}\mathrm{features},\mathrm{in}\mathrm{\_}\mathrm{features})$. The values are initialized from manifold.Init_point(Xinit), where $\mathrm{Xinit}\sim \mathcal{U}(-\sqrt{k},\sqrt{k})$ with $k=\frac{1}{\mathrm{in}\mathrm{\_}\mathrm{features}}$.

• bias – the learnable bias of the module of shape $(\mathrm{out}\mathrm{\_}\mathrm{features})$. If bias is True, the values are initialized from $\mathcal{U}(-\sqrt{k},\sqrt{k})$ where $k=\frac{1}{\mathrm{in}\mathrm{\_}\mathrm{features}}$.

• quad_penalty (callable) – the function that returns the quadratic penalty terms of the weights. Its return value equals to $||\mathrm{manifold}.\mathrm{C}(\mathrm{weight})|{|}^2$.

Example:#

my_layer = cdopt.nn.Linear_cdopt(20, 30, manifold_class = cdopt.manifold_torch.symp_stiefel_torch)
input = torch.randn(128, 20)
output = my_layer(input)
print(output.size())
# expected to print torch.Size([128, 30])