Node / Express with Typescript - (Re)Introduction 2023

TST, Hongkong

Getting Started
Webserver
Database
- SQL Database
- MySQL Connector
Authentication
Products
Orders
- Pre-Seeding Orders
- Order Controller
  - Exporting Order to CSV
  - Charting Order Data
Route Permissions
- Update Role Seed
- Permission Middleware

Getting Started

Node.js

Install the latest version of Node.js and create a new Node project using npm:

mkdir hello-next && cd hello-next
npm init -y

Test your Node.js installation by creating a hello world js file:

nano ./index.js

with the following content:

console.log('Hello from Node.js')

Add a start script to ./package.json

 "scripts": {
    "start": "node index.js"
  },

And execute it using npm:

 npm start

> hello-next@1.0.0 start
> node index.js

Hello from Node.js

Typescript

sudo npm install -g typescript 
npm install -D typescript ts-node nodemon

tsc --init

Created a new tsconfig.json with:

TS
  target: es2016
  module: commonjs
  strict: true
  esModuleInterop: true
  skipLibCheck: true
  forceConsistentCasingInFileNames: true

Move the index file into a src sub-directory and rename the file to index.ts and change the npm script to "start": "ts-node src/index.ts". Verify that everything still works:

npm start

> hello-next@1.0.0 start
> ts-node index.ts

Hello from Node.js

Nodemon

For development we can use "dev": "nodemon src/index.ts" to get live reloading with the following configuration:

nano nodemon.json

{
    "ignore": [
        ".git",
        "node_modules/",
        "dist/",
        "coverage/"
    ],
    "watch": [
        "src/*"
    ],
    "ext": "js, json, ts"
}

npm run dev

> hello-next@1.0.0 dev
> nodemon src/index.ts

[nodemon] 2.0.22
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): src/**/*
[nodemon] watching extensions: js,json,ts
[nodemon] starting `ts-node src/index.ts`
Hello from Node.js
[nodemon] clean exit - waiting for changes before restart

Webserver

Express.js

npm install express cors
npm install -D @types/express @types/cors

Clean out the index file and replace it with the Express boilerplate:

import express, {Request, Response} from 'express'
import cors from 'cors'

const expressPort = 8888;
const app = express();

// use json for API routes
app.use(express.json());
// cors for api address/port
app.use(cors({
    credentials: true,
    origin: ["http://localhost:3000"]
}));

app.get('/', (req: Request, res: Response) => {
    res.send('INFO :: Root route called');
});

app.listen(expressPort, () => {
    console.log('INFO :: Webserver started on port ' + expressPort)
});

Test that the root route is working:

curl localhost:8888
INFO :: Root route called

Express Router

Above we created a route inside the index file. But Express allows us to export routes as well as controller functions - e.g. authentication - from separate files to keep things tidy:

nano src/routes.ts

import { Router } from 'express';
import { Register } from './controller/auth.controller'

export const routes = (router: Router) => {
    router.post('/api/register', Register)
}

nano src/controller/auth.controller.ts

import { Request, Response } from 'express';

export const Register = (req: Request, res: Response) => {
    res.send(req.body);
};

The routes can now be imported into your index file:

import { routes } from './routes';

// import routes from router
routes(app)

We can now test the registration URL:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"username":"itsme","password":"secret"}' \
  http://localhost:8888/api/register

This returns the JSON body {"username":"itsme","password":"secret"} as configured.

Request Validation

Use the express request-validation inside your authentication controller to verify that an incoming request is valid:

npm install express-validation

nano ./src/validation/register.validation.ts

import { Joi } from 'express-validation';

export const registerValidation = Joi.object({
    firstName: Joi.string().required(),
    lastName: Joi.string().required(),
    email: Joi.string().email().required(),
    password: Joi.string().required(),
    passwordConfirm: Joi.string().required(),
})

The validation can now be imported into the auth controller to return an error message if the validation fails:

src/controller/auth.controller.ts

import { Request, Response } from 'express';
import { registerValidation } from '../validation/register.validation';

export const Register = (req: Request, res: Response) => {
    const body = req.body;

    const { error } = registerValidation.validate(body);

    if (error) {
        return res.status(400).send(error.details);
    }

    if (body.password !== body.passwordConfirm){
        return res.status(400).send({
            message: 'ERROR :: Passwords do not match!'
        });
    }

Test the validation with missing data and a valid request:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"firstName":"its","lastName":"me","email":"me@email.com","password":"secret"}' \
  http://localhost:8888/api/register

[{"message":"\"passwordConfirm\" is required","path":["passwordConfirm"],"type":"any.required","context":{"label":"passwordConfirm","key":"passwordConfirm"}}]

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"firstName":"its","lastName":"me","email":"me@email.com","password":"secret","passwordConfirm":"secret"}' \
  http://localhost:8888/api/register

{"firstName":"its","lastName":"me","email":"me@email.com","password":"secret","passwordConfirm":"secret"}

Database

SQL Database

Bring up an SQL database to handle your app data, e.g. with Docker-Compose:

./src/docker/docker-compose.yml

version: '3'

services:

  db:
    image: mariadb
    restart: always
    ports:
      - 3306:3306
    environment:
      MARIADB_ROOT_PASSWORD: secretpassword
      MARIADB_DATABASE: react-app
      MARIADB_USER: app-user
      MARIADB_PASSWORD: secretpassword

  adminer:
    image: adminer
    restart: always
    ports:
      - 8080:8080

You can login on port 8080 to verify that MariaDB is up and the app database was created:

Node.js 2023

MySQL Connector

Now we need to connect the app to freshly created database:

npm install typeorm reflect-metadata @types/node mysql2 bcryptjs @types/bcryptjs

To keep things simple we can move the entire DB logic into it's own file:

./src/db-connector.ts

import { DataSource } from "typeorm";
const dataSourceOptions = require('../dataSourceOptions.json')

export const dataSource = new DataSource(dataSourceOptions);

export const Manager = dataSource.manager
// export const UserRepository = dataSource.getRepository(User)

dataSource
    .initialize()
    .then(() => {
        console.log('INFO :: Data Source has been initialized');
    })
    .catch((err) => {
        console.error('ERROR :: Data Source initialization error', err);
    })

export default dataSource;

The database configuration is exported into a separate JSON file for convenience:

./dataSourceOptions.json

{
    "type": "mysql",
    "host": "localhost",
    "port": 3306,
    "username": "app-user",
    "password": "secretpassword",
    "database": "react-app",
    "entities": [
        "src/entities/*.ts"
    ],
    "logging": false,
    "synchronize": true
}

NOTE that the configuration here has to match up with the configuration set inside the Docker-Compose file you used to start your instance of MariaDB!

The connector can be imported into the index file before calling the Express app - to make sure that the DB connection is establish before the app starts:

./src/index.ts

// Import DB config
import dataSource from "./db-connector";
// Create connection with DB
dataSource

The connector now only connects us to our database. To have actually do something we have to add "entities" in the configured entity folder src/entities/*.ts:

./src/entities/user.entity.ts

import { Column, PrimaryGeneratedColumn, Entity } from "typeorm";

@Entity()
export class User {
    @PrimaryGeneratedColumn()
    id: number;

    @Column()
    firstName: string;

    @Column()
    lastName: string;

    @Column({
        unique: true
    })
    email: string;

    @Column()
    password: string;

}

This will make sure that the user table is generated as soon as the app is restarted:

Node.js 2023

Authentication

Registering Users

To be able to populate the User table with user logins we now have to add a function to the auth controller:

src/controller/auth.controller.ts

// REGISTER USER
export const Register = async (req: Request, res: Response) => {
    const body = req.body;

    // check if all infos were send
    const { error } = registerValidation.validate(body);
    // break if something is missing
    if (error) {
        return res.status(400).send(error.details);
    }
    // verify that password is confirmed
    if (body.password !== body.passwordConfirm){
        return res.status(400).send({
            message: 'ERROR :: Passwords do not match!'
        });
    }
    // save password to database
    const { password, ...user } = await repository.save({
        firstName: body.firstName,
        lastName: body.lastName,
        email: body.email,
        password: await bcryptjs.hash(body.password, 10)
    })

    res.send(user);
};

Here I am using bcryptjs to make sure that the password is not stored in plain text. The user registration can be tested using curl:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"firstName":"Mike","lastName":"Polinowski","email":"me@email.com","password":"secret","passwordConfirm":"secret"}' \
  http://localhost:8888/api/register

Note that the password in the returned body is hashed and also received an ID - the first user registered in the User table:

{"firstName":"Mike","lastName":"Polinowski","email":"me@email.com","password":"$2a$10$6LKB6iIhOjzK7TK3kv1qeOOYkds6mvh1JENfzJDFqtnY3wR999Ue2","id":1}

Node.js 2023

Create a login route that points to a function called Login in the auth controller:

./src/routes.ts

import { Router } from 'express';
import { Register, Login } from './controller/auth.controller'

export const routes = (router: Router) => {
    router.post('/api/register', Register)
    router.post('/api/login', Login)
}

And create the login function:

./src/controller/auth.controller.ts

// LOGIN USER
export const Login = async (req: Request, res: Response) => {
    // check if user exists in db
    const user = await repository.findOneBy(
            {
                email: req.body.email
            }
        )
    
    // if does not exists break
    if(!user){
        return res.status(404).send({
            message: 'ERROR :: User does not exists!'
        })
    }

    // if exists but password is wrong break
    if(!await bcryptjs.compare(req.body.password, user.password)) {
        return res.status(404).send({
            message: 'ERROR :: Invalid credentials!'
        })
    }

    // don't return password after successful login
    const { password, ...data } = user;
    res.send(user);
}

Again, we can test the user login with curl:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"email":"me@email.com","password":"secret"}' \
  http://localhost:8888/api/login

{"id":1,"firstName":"Mike","lastName":"Polinowski","email":"me@email.com"}

JSON Web Tokens

Return a JWT to keep users authenticated after a successful login:

npm install jsonwebtoken @types/jsonwebtoken cookie-parser @types/cookie-parser

Instead of returning the user we now need to return a JWT in Login function:

./src/controller/auth.controller.ts

// return JWT to authenticated user
const payload = { id: user.id }
const token = sign(payload, 'secretkey')

// don't return password after successful login
// const { password, ...data } = user;
res.send(token);

Test that the login now returns the token:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"email":"me@email.com","password":"secret"}' \
  http://localhost:8888/api/login

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MSwiaWF0IjoxNjg1ODkwNjIxfQ.W8D81og0GuWZ8Q8Nvp1XDpqK0XwDFXy0qYgjXxP8uno

Write token to HTTP cookie instead of returning it directly:

// return JWT to authenticated user
const token = sign(
    {
        id: user.id
    }, 'secretkey'
)

res.cookie('jwt', token, {
    // keep cookie in node.js backend
    httpOnly: true,
    maxAge: 24*60*60*1000 //1day
})

res.send({
    message: 'INFO :: Successfully logged in.'
})

Try the login again:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"email":"me@email.com","password":"secret"}' \
  http://localhost:8888/api/login

{"message":"INFO :: Successfully logged in."}

Authenticate Users

Start by creating a route for returning the authenticated user:

./src/routes.ts

import { Router } from 'express';
import { Register, Login, AuthenticatedUser } from './controller/auth.controller'

export const routes = (router: Router) => {
    // register new user
    router.post('/api/register', Register)
    // login known user
    router.post('/api/login', Login)
    // get authenticated user from jwt
    router.get('/api/user', AuthenticatedUser)
}

Then create the referenced function for this route:

./src/controller/auth.controller

export const AuthenticatedUser = async (req: Request, res: Response) => {
    // get cookie from authenticated user
    const jwt = req.cookies['jwt'];
    // get user id from jwt
    const payload: any = verify(jwt, secret)

    if(!payload) {
        return res.status(401).send({
            message: 'ERROR :: User unauthenticated!'
        })
    }
    // return user info  for user id
    const {password, ...user} = await repository.findOneBy(payload.id)

    res.send(user)
}

We can test the function in Postman by first logging in:

Node.js 2023

And then calling the /api/user route with the received web token:

Node.js 2023

Our web token contained our user id that the backend could use to identify the authenticated user and return the user information.

Deauthenticate Users

Once logged in we need to provide a function that allows us to remove the web token and log out the user. Start by providing a route:

import { Router } from 'express';
import { Register, Login, AuthenticatedUser, Logout } from './controller/auth.controller'

export const routes = (router: Router) => {
    // register new user
    router.post('/api/register', Register)
    // login known user
    router.post('/api/login', Login)
    // get authenticated user from jwt
    router.get('/api/user', AuthenticatedUser)
    // force expire jwt to log out
    router.post('/api/logout', Logout)
}

To logout we need to expire the active JWT:

./src/controller/auth.controller.ts

export const Logout = async (req: Request, res: Response) => {
    res.cookie('jwt', '',  {maxAge: 0})

    res.send({
        message: 'INFO :: Successfully logged out.'
    })
}

The function can be verified with Postman:

Node.js 2023

Dotenv

Using dot environment variables to pass in global variables like the JWT secretkey:

npm install dotenv @types/dotenv

Collect all global variables in a dotenv file:

./.env

# Express webfrontend port
WEB_PORT = 8888
# JWT secret key 
SECRET_KEY = 'secretkey'
# API url for CORS
API_URL = 'http://localhost:3000'
# Datasource options for TypeORM
DS_HOST = 'localhost'
DS_USER = 'app-user'
DS_PASS = 'secretpassword'
DS_DB = 'react-app'
DS_ENTITIES = 'src/entities/*.ts'

And require the configuration file ON TOP of your index file:

require('dotenv').config();

Now you are able to use these variables in your entire app:

./src/index.ts

const apiUrl = process.env.API_URL;
const expressPort = process.env.WEB_PORT || 8080;

./src/db-connector.ts

export const dataSource = new DataSource({
    type: 'mariadb',
    host: process.env.DS_HOST,
    port: 3306,
    username: process.env.DS_USER,
    password: process.env.DS_PASS,
    database: process.env.DS_DB,
    entities: [process.env.DS_ENTITIES],
    logging: false,
    synchronize: true
  });

Verify Auth Status Middleware

Move the auth status check into middleware to simplify the AuthenticatedUser function that should only return the user details of authenticated users:

src/middleware/auth.middleware.ts

import { Request, Response } from 'express';
import { verify } from 'jsonwebtoken';

import Manager from '../db-connector';
import { User } from '../entities/user.entity';
const secret = process.env.SECRET_KEY

const repository = Manager.getRepository(User);

export const CheckAuthState = async (req: Request, res: Response, next: Function) => {
    
        try {
            // get cookie from authenticated user
            const jwt = req.cookies['jwt'];
            // get user id from jwt
            const payload: any = verify(jwt, secret)

            if(!payload) {
                return res.status(401).send({
                    message: 'ERROR :: User unauthenticated!'
                })
            }
            // return user info  for user id
            req['user'] = await repository.findOneBy(payload.id)

            next();

        } catch (e) {
            return res.status(401).send({
                message: 'ERROR :: User unauthenticated!'
        })
    }
}

The entire verification logic now moved into the middleware. By adding it to our user and logout route we can now directly request the user details inside the auth controller:

src/routes.ts

import { Router } from 'express';
import { Register, Login, AuthenticatedUser, Logout } from './controller/auth.controller'
import { CheckAuthState } from './middleware/auth.middleware'

export const routes = (router: Router) => {
    // register new user
    router.post('/api/register', Register)
    // login known user
    router.post('/api/login', Login)
    // get authenticated user from jwt
    router.get('/api/user', CheckAuthState, AuthenticatedUser)
    // force expire jwt to log out
    router.post('/api/logout', CheckAuthState, Logout)
}

src/controller/auth.controller.ts

// RETURN USER DATA BY JWT ID
export const AuthenticatedUser = async (req: Request, res: Response) => {
    const {password, ...user} = req['user']
    res.send(user);
}

Update User Info

Add two more routes to update the user name and password:

src/routes.ts

import { Router } from 'express';
import { Register, Login, AuthenticatedUser, Logout, UpdateUserInfo, UpdateUserPass } from './controller/auth.controller'
import { CheckAuthState } from './middleware/auth.middleware'

export const routes = (router: Router) => {
    // register new user
    router.post('/api/register', Register)
    // login known user
    router.post('/api/login', Login)
    // get authenticated user from jwt
    router.get('/api/user', CheckAuthState, AuthenticatedUser)
    // force expire jwt to log out
    router.post('/api/logout', CheckAuthState, Logout)
    // update user info
    router.put('/api/users/info', CheckAuthState, UpdateUserInfo)
    // update user password
    router.put('/api/users/pass', CheckAuthState, UpdateUserPass)
}

And add the corresponding controller functions:

src/controller/auth.controller.ts

// UPDATE USER INFO
export const UpdateUserInfo = async (req: Request, res: Response) => {

    const user = req['user'];

    await repository.update(user.id, req.body)

    const { password, ...data } = await repository.findOneBy(user.id)

    res.send({data})
}

// UPDATE USER PASSWORD
export const UpdateUserPass = async (req: Request, res: Response) => {

    const user = req['user'];

    // verify that password is confirmed
    if (req.body.password !== req.body.passwordConfirm){
        return res.status(400).send({
            message: 'ERROR :: Passwords do not match!'
        });
    }

    await repository.update(user.id, {
        password: await bcryptjs.hash(req.body.password, 10)
    })

    const { password, ...data } = await repository.findOneBy(user.id)

    res.send({data})
}

Use the /api/users/info route to update the user name and email:

Node.js 2023

And /api/users/pass to update the user password:

Node.js 2023

User Administration

The admin user has to be able to administer other users. Start by creating routes that:

fetch all registered users
fetch user by ID
create new users
update user
delete user

src/routes.ts

export const routes = (router: Router) => {

    // register new user
    router.post('/api/register', Register)
    // login known user
    router.post('/api/login', Login)
    // get authenticated user from jwt
    router.get('/api/user', CheckAuthState, AuthenticatedUser)
    // force expire jwt to log out
    router.post('/api/logout', CheckAuthState, Logout)
    // update user info
    router.put('/api/users/info', CheckAuthState, UpdateUserInfo)
    // update user password
    router.put('/api/users/pass', CheckAuthState, UpdateUserPass)

    // user administration - get all users
    router.get('/api/users', CheckAuthState, GetUsers)
    // user administration - get user by ID
    router.get('/api/users/:id', CheckAuthState, GetUser)
    // user administration - create new user
    router.put('/api/users/:id', CheckAuthState, UpdateUser)
    // user administration - create new user
    router.post('/api/users', CheckAuthState, CreateUser)
    // user administration - delete user
    router.delete('/api/users/:id', CheckAuthState, DeleteUser)
}

Get all Users

The administration functions are collected in the user controller. The GetUsers function selects the user table and returns everything: