Task 1: Regression Model

Essentially, a linear regression model can also be seen as a simple neural network model. In this exercise, we will use Keras to estimate a basic linear regression model. Through this exercise, you will not only understand the Keras 4-step method I mentioned earlier but also gain some extremely simple and concrete examples to help you grasp the concepts discussed in my lecture notes.

Step 1: Prepare Data

For this problem, we will use simulated data. Therefore you just use the following code to get the data into your space.

set.seed(2025)
N = 30 # sample size
x = runif(N, 0, 5) # regressor from uniform distribution
e = rnorm(N) # noise is from normal
y = 0.5 + 1.5*x + e

Step 2: Draw the model

The following chunk of code demonstrates the basic pattern for modeling with Keras. We can use the keras_model_sequential() function to define a layer-by-layer neural network model. This function provides us with a large container, and then, as needed, we insert various layers and modules into it. Since this is a simple linear regression model, we only need a single Dense layer unit. Here, you need to specify the input_shape, which determines the number of input variables for the model. Then, you can use the pipe (%>%) operator to connect it to a neural network layer defined with the layer_dense() function. Similarly, you need to specify the number of neurons in this layer and choose an appropriate activation function. Note: The question marks in the code require your judgment and modification.

Reg_Mod = keras_model_sequential(input_shape = ?) %>% 
  layer_dense(units = ?, activation = ?)

Once the model is ready, you can use function summary to visualize the model

summary(Reg_Mod)

In fact, at this point, you can already start using the model for predictions because all model parameters have been initialized. Keras acts like a “shopping assistant” who presets model parameters based on experience or certain methods. You can use the get_weights() function to check the initialized model parameters. We can also try using the predict() function to compute the model’s predictions on our generated x values.

get_weights(Reg_Mod)
y_pre = predict(Reg_Mod, x)
# Are you satisfied with this prediction?

Step 3: Compile the model

You can use the following chunk of code to compile the model. Our model Reg_mod will be piped into compile function. Within the function, we need to specify the loss function and choose a ‘shopping assistant’ (optimizer). Here, we will simply apply stochastic gradient descent algorithm with learning_rate = 0.05.

Reg_Mod %>% 
  compile(loss = "?",
          optimizer = optimizer_sgd(learning_rate = 0.05))

Step 4: Train the model

In the last step, you just pipe the model Reg_Mod into the fit function. Specify the input feature values x and the target information y. epochs represent the maximum number of iterations you want to train the model. The batch_size determines how many times the data will be fed into the model within a single epoch.

Reg_Mod %>% fit(x, y, epochs = 20, batch_size = N)

Now, you can check the trained model and compare the results with the estimation by OLS method

# check the weights estimation
get_weights(Reg_Mod)
# compare it with the outputs of 'lm'
summary(lm(y~x))

y_pre = predict(Reg_Mod, x)
mean((y_pre - y)^2)
plot(x,y,pch=20,cex=2)
points(x, y_pre, col = "blue")

Previous page | Lecture 3 Homepage | Next page