[FIXED] Calling pytorch neural network forward() gives error "mat1 and mat2 shapes cannot be multiplied"

Issue

I have following code which defines simple neural network:

class MlpNN(nn.Module):

    def __init__(self, in_dim=301, hidden_dims=[500,500,500,500,500,30]):

        super(MlpNN, self).__init__()

        self.net = nn.Sequential()
        self.net.add_module("lin_0", nn.Linear(in_dim, hidden_dims[0]))
        self.net.add_module("relu_0", nn.ReLU())

        layer_id = 1
        for hidden_dim in hidden_dims[1:]:
            self.net.add_module("lin_"+str(layer_id), nn.Linear(hidden_dim, hidden_dim))
            self.net.add_module("relu_"+str(layer_id), nn.ReLU())
            layer_id += 1
    
    def forward(self, x):
        return self.net(x)

The network formed is:

MlpNN(
  (net): Sequential(
    (lin_0): Linear(in_features=301, out_features=500, bias=True)
    (relu_0): ReLU()
    (lin_1): Linear(in_features=500, out_features=500, bias=True)
    (relu_1): ReLU()
    (lin_2): Linear(in_features=500, out_features=500, bias=True)
    (relu_2): ReLU()
    (lin_3): Linear(in_features=500, out_features=500, bias=True)
    (relu_3): ReLU()
    (lin_4): Linear(in_features=500, out_features=500, bias=True)
    (relu_4): ReLU()
    (lin_5): Linear(in_features=30, out_features=30, bias=True)
    (relu_5): ReLU()
  )
)

while doing forward(), it gives me following error:

mat1 and mat2 shapes cannot be multiplied (1x500 and 30x30)

Whats wrong I am doing here? I know it must be stupid basic given that I am quite new machine learning.

Solution

In your network, your lin_4 layer has 500 output features, while your lin_5 layer has 30 input features. This causes a mismatch in the shapes. To fix this, you should ensure that the output features of a layer and the input features of the layer after it are the same. In your code, you could do that as:

class MlpNN(nn.Module):

    def __init__(self, in_dim=301, hidden_dims=[500,500,500,500,500,30]):

        super(MlpNN, self).__init__()

        self.net = nn.Sequential()
        self.net.add_module("lin_0", nn.Linear(in_dim, hidden_dims[0]))
        self.net.add_module("relu_0", nn.ReLU())

        for layer_id in range(1, len(hidden_dims)):
            self.net.add_module("lin_"+str(layer_id), nn.Linear(hidden_dims[layer_id-1], hidden_dims[layer_id]))
            self.net.add_module("relu_"+str(layer_id), nn.ReLU())
    
    def forward(self, x):
        return self.net(x)

Answered By - GoodDeeds