[FIXED] RuntimeError: mat1 and mat2 shapes cannot be multiplied (28x28 and 784x64)

Issue

import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):

    def __init__(self):
        super().__init__()
        self.fc1=nn.Linear(28*28,64)
        self.fc2=nn.Linear(64,64)
        self.fc3=nn.Linear(64,64)
        self.fc4=nn.Linear(64,10)

    def forward(self,x):
        x=F.relu(self.fc1(x))
        x=F.relu(self.fc2(x))
        x=F.relu(self.fc3(x))
        x=self.fc4(x)
        return F.log_softmax(x,dim=1)

net=Net()
print(net)

X=torch.rand((28,28))
X=X.unsqueeze(0)
output=net(X)
print(output)

RuntimeError: mat1 and mat2 shapes cannot be multiplied (28x28 and 784x64)

When i use X=X.view(-1,28*28) in place of X=X.unsqueeze(0), it gives me the desired result. I cannot completely understand the differnce between using unsqueeze() and view() here.

Solution

You are currently working with two dimensions only: axis=0 is your batch and axis=1 is your features.

Reshaping with X.view(-1, 28*28) will flatten all axes but the last leaving your with a shape of the form (batch_size, feature_size).

While X.unsqueeze(0) will just add an additional axis not affecting the overall form of your tensor X, it is not a reshape. By that I mean if X is shaped (batch_size, channel, height, width) which is likely for an input image. Then X.unsqueeze(0) will be shaped (1, batch_size, channel, height, width).

Since you are using a fully-connected network, you are and should be using the former, that is to reshape X into a 2-dimensional tensor.

Answered By - Ivan