[FIXED] Confusion in setting shape and input shape of a sequential Keras model

Issue

I have a dataset whose scheme is like:

X1 ... X20 C

where the first 20 columns are input data, and the last column is the target one. The dataset includes 2000 record. I want to design a sequential Keras model to classify those target labels (which vary from 1 to 10, thereby being multi-label classification problem). Assuming that I have saved those input data and labels in X_train_1 and y_train_1, Here is my model:

def build_model_1(n_hidden = 1, n_neurons = 30, learning_rate = 3e-3, input_shape = X_train_1.shape):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.InputLayer(input_shape=input_shape))
    model.add(tf.keras.layers.BatchNormalization(momentum=0.999))
    for layer in range(n_hidden):
        model.add(tf.keras.layers.Dense(n_neurons, tf.keras.activations.selu,
                                        kernel_initializer="lecun_normal",
                                        kernel_regularizer= tf.keras.regularizers.l2(0.01)))
        model.add(tf.keras.layers.BatchNormalization(momentum=0.999))
    model.add(tf.keras.layers.Dense(10, tf.keras.activations.softmax, kernel_initializer="lecun_normal"))
    loss = tf.keras.losses.categorical_crossentropy
    optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate, beta_1=0.9, beta_2=0.999)
    metric = [tf.keras.metrics.Accuracy()]
    model.compile(loss = loss, optimizer=optimizer, metrics=[metric])
    return model

I thought the shape of the input should be that of my training dataset, however when I compile and fit my model, I get the following error:

ValueError: Input 0 of layer sequential_12 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (32, 20)

What am I doing wrong here?

Solution

Your input shape is simply 20, since you have 20 features and 2000 samples. You do not have to provide the batch size. Here is a working example:

import tensorflow as tf
import numpy as np

def build_model_1(n_hidden = 1, n_neurons = 30, learning_rate = 3e-3, input_shape = (20,)):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.InputLayer(input_shape=input_shape))
    model.add(tf.keras.layers.BatchNormalization(momentum=0.999))
    for layer in range(n_hidden):
        model.add(tf.keras.layers.Dense(n_neurons, tf.keras.activations.selu,
                                        kernel_initializer="lecun_normal",
                                        kernel_regularizer= tf.keras.regularizers.l2(0.01)))
        model.add(tf.keras.layers.BatchNormalization(momentum=0.999))
    model.add(tf.keras.layers.Dense(10, tf.keras.activations.softmax, kernel_initializer="lecun_normal"))
    loss = tf.keras.losses.categorical_crossentropy
    optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate, beta_1=0.9, beta_2=0.999)
    metric = [tf.keras.metrics.Accuracy()]
    model.compile(loss = loss, optimizer=optimizer, metrics=[metric])
    return model

train_data = np.random.random((2000, 20))
model = build_model_1()
y = model(train_data)

Also, ask yourself if you are really dealing with a multi-label classification problem. Can a sample from your dataset belong to more than one class, or are the classes mutually exclusive? If the classes are not mutually exclusive, I would recommend changing the activation function for the output layer to sigmoid and changing the loss function to binary_crossentropy. The intuition behind this can be found here.

Answered By - AloneTogether