• Tensorflow Neural Network Regression
  • Create a Dataset
  • Input and Output Shapes
  • Building the Model
    • Making Predictions
  • Improving the Model
    • Making Predictions

Github Repository

See also:

• Fun, fun, tensors: Tensor Constants, Variables and Attributes, Tensor Indexing, Expanding and Manipulations, Matrix multiplications, Squeeze, One-hot and Numpy
• Tensorflow 2 - Neural Network Regression: Building a Regression Model, Model Evaluation, Model Optimization, Working with a "Real" Dataset, Feature Scaling
• Tensorflow 2 - Neural Network Classification: Non-linear Data and Activation Functions, Model Evaluation and Performance Improvement, Multiclass Classification Problems
• Tensorflow 2 - Convolutional Neural Networks: Binary Image Classification, Multiclass Image Classification
• Tensorflow 2 - Transfer Learning: Feature Extraction, Fine-Tuning, Scaling
• Tensorflow 2 - Unsupervised Learning: Autoencoder Feature Detection, Autoencoder Super-Resolution, Generative Adverserial Networks

Tensorflow Neural Network Regression

Using Tensorflow to predict a numerical variable based on other combination of variables.

import tensorflow as tf
# print(tf.__version__)
# 2.11.0
from tensorflow.keras import Sequential, layers, optimizers

import numpy as np
import matplotlib.pyplot as plt

Create a Dataset

## create features
X = np.array([-140., -110., -80., -50., -20., 10., 40., 70., 100., 130., 160., 190., 220., 250,])
## create labels
y = np.array([1., 4., 7., 10., 13., 16., 19., 22., 25., 28., 31., 34., 37., 40,])

## visualize dataset
plt.scatter(X, y)
plt.show

Input and Output Shapes

# create tensors from np arrays
X = tf.cast(tf.constant(X), dtype=tf.float32)
y = tf.cast(tf.constant(y), dtype=tf.float32)

# test shape
input_shape = X.shape
output_shape = y.shape
input_shape, output_shape
## (TensorShape([8]), TensorShape([8]))

Building the Model

# set random seed
tf.random.set_seed(42)

# build a sequential model with Keras
model = Sequential([
    layers.Dense(1)
])

# compile the model
model.compile(
    loss="mae",
    optimizer="sgd",
    metrics=["mae"])

# model training
## error message: "Input 0 of layer "dense_3" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (None,)"
# => needed to expand input shape
model.fit(tf.expand_dims(X, axis=-1), y, epochs=99)
## Epoch 99/99
## 1/1 [==============================] - 0s 4ms/step - loss: 14.8669 - mae: 14.8669

Making Predictions

model.predict([70.])
# a X value of 100 would correspond to a y value of 25.0
# with an mae: 17.4613 we can expect to be around 17.5 points of with our prediction:
## array([[29.203451]], dtype=float32)
# we get 29.2 instead of the expected 25.0

Improving the Model

1. Creation: add more layers / change activation function / prevent overfitting with dropouts
2. Compilation: change optimization function / adjusting the learning rate
3. Fitting: add more epochs / add more data / training-testing split validation

# train / test split
X_Split = np.array_split(X,2)
X_train = tf.cast(tf.constant(X_Split[0]), dtype=tf.float32)
x_test = tf.cast(tf.constant(X_Split[1]), dtype=tf.float32)

Y_Split = np.array_split(y,2)
Y_train = tf.cast(tf.constant(Y_Split[0]), dtype=tf.float32)
y_test = tf.cast(tf.constant(Y_Split[1]), dtype=tf.float32)

X_train.shape, x_test.shape

# set random seed
tf.random.set_seed(42)

# build a sequential model with Keras
model_improved = Sequential([
    layers.Dense(128, activation="relu"),
    layers.Dense(128, activation="relu"),
    layers.Dropout(.2),
    layers.Dense(64, activation="relu"),
    layers.Dense(1)
])

# compile the model
model_improved.compile(
    loss="mae",
    optimizer=optimizers.Adam(learning_rate=0.01),
    metrics=["mae"])

# model training
model_improved.fit(tf.expand_dims(X_train, axis=-1), Y_Split, validation_data=(x_test, y_test), epochs=99)
# Epoch 99/99
# 1/1 [==============================] - 0s 27ms/step - loss: 4.0467 - mae: 4.0467 - val_loss: 2.8862 - val_mae: 2.8862

Making Predictions

model_improved.predict([160.])
# expected y=31 for X value of 160 and got:
## array([[28.041351]], dtype=float32)