[FIXED] Why does the result from evaluate() differ from last epoch result?

Issue

I have a very simple neural-network which works in 250 epochs and in the last epoch it shows the mae = 0.1397 , however, if i try to get the model.evaluate((m * test_x + b), predict_y)) then the mae is about 44009.296875.

Why there is such big difference ?

Here is my code:

import tensorflow as tf
from tensorflow.keras import Input
from tensorflow.keras.layers import Dense
from tensorflow.keras.utils import plot_model

import numpy as np
import matplotlib.pyplot as plt

train_x = np.arange(2000)
m = 5
b = 4

train_y = m * train_x + b

# -----------------------------------------------------
# Create a Sequential Nerual Network
model = tf.keras.Sequential()
model.add(Input(shape=(1,), name="input_layer"))
model.add(Dense(10, activation="relu"))
model.add(Dense(1, activation=None, name="output_layer"))

# -----------------------------------------------------
# Compile the model
model.compile(loss=tf.keras.losses.mae,
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
              metrics=["mae"])

# -----------------------------------------------------
# Train the model
model.fit(train_x, train_y, epochs=250)

# -----------------------------------------------------
# Test the model

test_x = np.arange(2000, 2400)
predict_y = model.predict([test_x])
# ------------------------------------------------------
# Evaluation
print("Evaluate Testing : ", model.evaluate((m * test_x + b), predict_y))

Solution

I am not too sure if you are using the model.evaluate method correctly. Similar to the model.fit method, when evaluating your model you should provide x and y values. I get quite similar results when running this code snippet:

import tensorflow as tf
from tensorflow.keras import Input
from tensorflow.keras.layers import Dense
from tensorflow.keras.utils import plot_model

import numpy as np
import matplotlib.pyplot as plt

train_x = np.arange(2000)
m = 5
b = 4
train_y = m * train_x + b

# -----------------------------------------------------
# Create a Sequential Nerual Network
model = tf.keras.Sequential()
model.add(Input(shape=(1,), name="input_layer"))
model.add(Dense(10, activation="relu"))
model.add(Dense(1, activation=None, name="output_layer"))

# -----------------------------------------------------
# Compile the model
model.compile(loss=tf.keras.losses.mae,
              optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
              metrics=["mae"])

# -----------------------------------------------------
# Train the model
model.fit(train_x, train_y, epochs=5, batch_size=32)

# -----------------------------------------------------
# Test the model
test_x = np.arange(2000)
predict_y = model.predict([test_x])
# ------------------------------------------------------
# Evaluation
print("Evaluate Testing : ", model.evaluate(test_x,  m * test_x + b, batch_size=32))

Epoch 1/5
63/63 [==============================] - 1s 3ms/step - loss: 4978.4922 - mae: 4978.4922
Epoch 2/5
63/63 [==============================] - 0s 3ms/step - loss: 4954.3252 - mae: 4954.3252
Epoch 3/5
63/63 [==============================] - 0s 3ms/step - loss: 4929.9980 - mae: 4929.9980
Epoch 4/5
63/63 [==============================] - 0s 3ms/step - loss: 4905.5146 - mae: 4905.5146
Epoch 5/5
63/63 [==============================] - 0s 3ms/step - loss: 4880.8120 - mae: 4880.8120
63/63 [==============================] - 0s 2ms/step - loss: 4868.2192 - mae: 4868.2192
Evaluate Testing :  [4868.21923828125, 4868.21923828125]

Due to the stochastic nature of the whole process, the results will naturally vary a little.

Answered By - AloneTogether