Skip to main content

NGINX Websocket Proxy

Shen Zhen, China

See also:

Basic Setup​

This example uses ws, a WebSocket implementation built on Node.js. NGINX acts as a reverse proxy for this simple WebSocket application.

Minimal WS Server (Node.js)​

npm init
npm install ws

server1.js

// import the ws library
var WebSocketServer = require('ws').Server;

// start websocket service on port 8010
wss = new WebSocketServer({port: 8010});

// tell me when you are ready
console.log("Server started");

// handle connection, send msg to console and
// confirm reception to client
wss.on('connection', function(ws) {
        ws.on('message', function(message) {
        console.log('Received from client: %s', message);
        ws.send('Server received from client: ' + message);
    });
});
node server1.js
Server started

NGINX WS Proxy​

To have NGINX proxy these requests, create the following configuration using a map block so that the Connection header is correctly set to close when the Upgrade header in the request is set to '':

http {
    map $http_upgrade $connection_upgrade {
        default upgrade;
        '' close;
    }
 
    upstream websocket {
        server 127.0.0.1:8010;
    }
 
    server {
        listen 8020;
        location / {
            proxy_pass http://websocket;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
            proxy_set_header Host $host;
        }
    }
}

Start the container with:

docker run --rm --network host -v /path/to/docker_ws_proxy:/etc/nginx --name proxy nginx:alpine

Testing the Service​

To test the service we can use wscat:

npm install -g wscat

Run the WS client to listen on port 8020 - the NGINX proxy port - and send a text message:

wscat --connect ws://127.0.0.1:8020
Connected (press CTRL+C to quit)
> Echo
< Server received from client: Echo

Check your console running the WS server and you should see your message there as well:

node server1.js
Server started
Received from client: Echo

Secure Websocket Proxy​

Now we have a websocket server and a proxy that can be used as an ingress to forward traffic to our ws server. The next step is to use the proxy to terminate incoming encrypted traffic and direct the unencrypted backend service:

# WebSocketSecure SSL Endpoint

upstream websocket {
    server 127.0.0.1:8010;
}

server {
    listen 8020 ssl;

    # host name to respond to
    server_name 127.0.0.1;

    # your SSL configuration
    # ssl_certificate /etc/letsencrypt/live/my.domain.com/fullchain.pem;
    # ssl_certificate_key /etc/letsencrypt/live/my.domain.com/privkey.pem;
    ssl_certificate /etc/nginx/certs/nginx-selfsigned.crt; # Replace with the 2 lines above when using CA Cert
    ssl_certificate_key /etc/nginx/certs/nginx-selfsigned.key;

    location / {
        # switch off logging
        access_log off;

        # redirect all HTTP traffic to 127.0.0.1:8010
        proxy_pass http://websocket;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # WebSocket support (nginx 1.4)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Here I am including a self-signed TLS certificate and key that still needs to be created:

cd docker_ws_proxy/certs
openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out nginx-selfsigned.crt -keyout nginx-selfsigned.key

Here you should set the Common Name to your server address or domain. I will use localhost for this test-run:

Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:127.0.0.1
Email Address []:

When trying to connect with a self-signed certificate using wscat you run into error: self-signed certificate. You can get around it with -n (--no-check) flag:

wscat --connect wss://127.0.0.1:8020 -n

Secure Websocket Proxy with Path Re-Writing​

To be able to add our WSS backend into a frontend service we usually have to add routes different from what the backend provides - e.g. the backend provides the service on / but our frontend sends API calls to /api/ws. NGINX allows us to re-write these calls according to our backend requirements:

upstream websocket {
    server 127.0.0.1:8010;
}

server {
    listen 8020 ssl;

    # host name to respond to
    server_name 127.0.0.1;

    # your SSL configuration
    # ssl_certificate /etc/letsencrypt/live/my.domain.com/fullchain.pem;
    # ssl_certificate_key /etc/letsencrypt/live/my.domain.com/privkey.pem;
    ssl_certificate /etc/nginx/certs/nginx-selfsigned.crt; # Replace with the 2 lines above when using CA Cert
    ssl_certificate_key /etc/nginx/certs/nginx-selfsigned.key;

    location /api/ws {
        # switch off logging
        access_log off;

        # redirect all HTTP traffic to localhost:8010
        proxy_pass http://websocket;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # WebSocket support (nginx 1.4)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        # Path rewriting
        rewrite /api/ws/(.*) /$1 break;
        proxy_redirect off;
    }
}

The service can be tested with:

wscat --connect wss://127.0.0.1:8020/api/ws -n

Secure Websocket Proxy with Load-Balancing​

When traffic increases we might need to expand our backend and load-balance the incoming request. From the NGINX side we only need to ensure that every incoming request sticks to the backend server it initially negotiated the TLS connection with - this can be done with Session persistence:

# WebSocket Proxy with Load Balancing

upstream websocket {
    # Clients with the same IP are redirected to the same backend
    ip_hash;

    # Available backend servers
    server 127.0.0.1:8010;
    server 127.0.0.1:8030;
    server 127.0.0.1:8040;
}


server {
    listen 8020 ssl;

    # host name to respond to
    server_name 127.0.0.1;

    # your SSL configuration
    # ssl_certificate /etc/letsencrypt/live/my.domain.com/fullchain.pem;
    # ssl_certificate_key /etc/letsencrypt/live/my.domain.com/privkey.pem;
    ssl_certificate /etc/nginx/certs/nginx-selfsigned.crt; # Replace with the 2 lines above when using CA Cert
    ssl_certificate_key /etc/nginx/certs/nginx-selfsigned.key;

    location /api/ws {
        # switch off logging
        access_log off;

        # redirect all HTTP traffic to websocket backend
        proxy_pass http://websocket;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # WebSocket support (nginx 1.4)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        # Path rewriting
        rewrite /api/ws/(.*) /$1 break;
        proxy_redirect off;
    }
}

From the client side we can just make copies of the the initial server1.js and replace the port where they are providing their service. Start all of them and re-run the test:

wscat --connect wss://127.0.0.1:8020/api/ws -n