Parameters:#

• in1_features – size of each first input sample

• in2_features – size of each second input sample

• out_features – size of each output sample

• bias – If set to False, the layer will not learn an additive bias. Default: True

• manifold_class – The manifold class for the weight matrix. Default: cdopt.manifold_torch.euclidean_torch

• penalty_param – The penalty parameter for the quadratic penalty terms in constraint dissolving function

• manifold_args - The additional key-word arguments that helps to define the manifold constraints.

• weight_var_transfer (callable) – The function that transfer the weights (3D-tensor) to the shape of the variables of the manifold.

  • The weight_var_transfer is called by
    weight_to_var, var_to_weight, var_shape = weight_var_transfer( (out_features, in1_features, in2_features) )

  • The inputs of weight_var_transfer should be the size of the weights. As for the outputs, weight_to_var is the callable function that transfer the weights to the variables of the manifold. var_to_weight is the callable function that transfers the variables of the manifold to the weights. var_shape is a tuple of ints that refers to the shape of the variables of the manifolds.

  • Default:

    • var_shape = (in1_features * in2_features, out_feature_total)

    • weight_to_var = lambda X_tensor: torch.reshape(X_tensor, var_shape)

    • var_to_weight = lambda X_var: torch.reshape(X_var, tensor_shape)

Shapes:#

• Input1: $(\ast ,H_{in1})$ where $\ast$ means any number of dimensions including none and $H_{in1}=\mathrm{in}1\mathrm{\_}\mathrm{features}$.

• Input2: $(\ast ,H_{in2})$ where $\ast$ means any number of dimensions including none and $H_{in2}=\mathrm{in}2\mathrm{\_}\mathrm{features}$.

• Output: $(\ast ,H_{out})$ where all but the last dimension are the same shape as the input and $H_{out}=\mathrm{out}\mathrm{\_}\mathrm{features}$.

Attributes:#

• manifold (cdopt manifold class) – the manifold that defines the constraints. The shape of the variables in manifold is set as var_shape.

• weight (torch.Tensor) – the learnable weights of the module of shape $(\mathrm{out}\mathrm{\_}\mathrm{features},\mathrm{in}1\mathrm{\_}\mathrm{features},\mathrm{in}2\mathrm{\_}\mathrm{features})$. The values are initialized from var_to_weight(manifold.Init_point(weight_to_var(Xinit))), where $\mathrm{Xinit}\sim \mathcal{U}(-\sqrt{k},\sqrt{k})$ with $k=\frac{1}{\mathrm{in}1\mathrm{\_}\mathrm{features}}$.

• bias – the learnable bias of the module of shape $(\mathrm{out}\mathrm{\_}\mathrm{features})$. If bias is True, the values are initialized from $\mathcal{U}(-\sqrt{k},\sqrt{k})$ where $k=\frac{1}{\mathrm{in}1\mathrm{\_}\mathrm{features}}$.

• quad_penalty (callable) – the function that returns the quadratic penalty terms of the weights. Its return value equals to $||\mathrm{manifold}.\mathrm{C}(\mathrm{weight})|{|}^2$.

Example:#

my_layer = cdopt.nn.Bilinear_cdopt(20, 30, 40, manifold_class=cdopt.manifold_torch.sphere_torch)
input1 = torch.randn(128, 20)
input2 = torch.randn(128, 30)
output = my_layer(input1, input2)
print(output.size())