Application Development and Server Operation
I just want to build something useful
Coming from a conservative country I was raised with a strong sensibility for the separation of concerns - put your inline CSS into a style sheet, let the computer scientist handle the shopping list for new office computers and never even think about putting the salt shaker into the spice rack! Salt belongs to MSG and sugar rack on the right.
So is DevOps really the monster it is made out to be? Is a backend developer able to design a usable user interface and will a frontend dev bring down the whole service by ignoring best practices of harding the server before uploading his application?
How does a Jack-of-all-Trades handle a specific task compared to a specialist?
The Curse
When you start out as a backend server admin it can be frustrating to get started on your own. I was always the guy who did not dive into the details but wanted to learn skills that allowed me to build something. As a backend engineer this lead you down the road of setting up a webserver, maybe even a cluster of servers that exchange information through micro services. You might even add hardened military grade security so that not even that Nigerian prince, everyone is talking about, can hack his way into your Cloud Servers. Amazing! But what now?
Or you start our at the frontend of things. You saw this beautiful website - all those smooth animations, beautiful colour palets and responsiveness. That is what you want to learn how to create yourself. And after months of hard work you figured out your stack, you put it all together and your page looks amazing! But what now?
Backend and frontend belong together but they - literally - do not even speak the same language...
The Cure
When I started out there was still this divide and the internet was full of memes about sysadmins being unable to grasp the necessities of modern web applications - there must have been memes coming in from the backend as well... I just did not get the IRC invite.
But there was also a new shining light on the horizon:
Why don't you do it in Javascript?
I fell in love with React frontends right away. Then learning that you can use a Node.js backend to drive your application - that sounded just like the solution I was looking for.
The first full-stack web application I ever wrote for my company was a Node.js/Express.js knowledge base. Just a bunch of EJS pages served by a Javascript backend. I never looked back to the old times since.
The Future
I have - so far - never worked on a web project that reached the limitations of a Javascript backend. I am aware that they are there. But breaking down my applications into small, single purpose services has prevented me from bumping into them.
There is one trend that is slowly creeping it's way into my work, though. The backend itself is slowly abstracted away from me. First I started to wrap my code into Containers - because that was convenient. And now I am supposed to com up with solutions to keep an eye on my swarm of little virtual minions. The big K arrived. Written in the Go language and complex to the degree of a Google Megacorp server center. And all controllable by the magic word kubectl.
So what do I really need to know?
Server Security
In a true sysadmin fashion - let's start with fighting the dark minions of Mordor.
I remember back in the days of Windows 95 - when I set up a new PC at my workplace and plugged it into the university network... It died. The network was so overloaded with virtual pathogens that you first had to find a way to update your Windows offline to the newest version, add a firewall and a solid virus scanner. And then you could plug it into the network.
Welcome to the Internet
Securing your server is mandatory. Even if it is just your personal playground. It is valuable training to keep your system updated (which sometimes kills your application and has you hunt down insects in your code) and maintain a sensible degree of security:
- Randomly generate a long SSH login: There are ways to use those without memorizing them - so make them hard... so to speak.
- Change your default ports: When an evildoer decides to attack your server, it does not matter where you put your SSH port - she will find it. But most attacks that I witnessed were executed by bots. Those bots just rattle your cage and see if they can find a weakness. If they can't find one fast enough they move on. If they can't find your SSH port automatically, they don't bother to return.
- A Mould with Crocodiles and a Wall of Fire: Your server is your castle. Make sure that the only ports that are opened are the ones that your application needs to communicate with the world. Those entry point can be hardened against brute force and denial-of-service attacks.
- Keep your Software Up2Date: There are many reasons why your server still runs a system that is 5 years old and was never updated or restarted. Such servers are kept caged in inside your local network. Every server that communicates with the world is kept fresh.
Server Monitoring
When you have your first 1, 2 or 3 servers to maintain it is exciting! The first thing you do in the morning is to check all of them, run the commands to see that everything is still running and happy and check the log files for anomalies. Once you have more than 10 under your control you do not do that anymore. You need a server dashboard that gives you the all-green and you are good to go.
I did try out a few Kubernetes based monitoring tools and they are fantastic. You get everything you need at a glance. And since it is Kubernetes you usually just see that there was an issue and it was fixed 20ms later.
But I consider myself lucky to have worked for companies that still maintained their own servers - things still had to be done by hand. Giving you an inside into what tools like Kubernetes present to you as a magic blackbox.
There isn't a shortage of tools that install agents on your server that report to a master dashboard. They can be configured to monitor your server hardware and web services. Making sure that everything is running smoothly.
Continous Integration
The time of major releases is over!
There is no longer a new Windows every 2 years - now it is Windows 10 and it stays that way. Continous Integration (CI) means that you commit your edited source code today. It will be tested, automatically over night and be deployed before you return the next day - or if that sounds scary to you, there might be a blinking button waiting for you to start the deployment whenever you are ready.
What do you need to achieve this?
- A source control system that keeps track of your changes
- A tool that receives your source code and runs a build for you
- A testing suite that knows what the release should be able to do
- A containerized solution to swap in your new code without interrupting your service
State Management
Just like Server Management, State Management frees up your hands when you start to become overwhelmed with the amount of servers that you have to maintain. There will be the point where you are no longer able to be logged into every server, see drop-outs and react in second notice.
Again, Kubernetes handles most of that for you. But since we are dealing with a classical setup here - we do it ourself. State management tools are often configured by something like a YAML file (ever wondered where Kubernetes got all those ideas from?). And the state that is configured could be, e.g. install a webserver, copy a configuration file to the conf folder and some HTML content to the serve folder, when done run the server as a system service, if a new configuration file becomes available upload it to the server and restart the service, etc...
State management often overlaps with CI/CD pipelines. Or it can be the Continous Deployment (CD) part of that pipeline. The selection of tools available can be a bit overwhelming ~ similar to Server Management tools. But you will quickly find out that all of the do the same thing with different naming conventions and slightly different looking config files. Just make sure that the program you choose integrates well into the rest of your pipeline and that the rest of the team is comfortable with it.
Web Application
Most modern "website" is now applications. Gone are the days where all you needed to set up a site was HTML, CSS and maybe some jQuery for a fancy accordion animation inside your navigation menu.
An web application connects your user to databases, services that he can use e.g. to authenticate himself and delivers a rich interface that sometimes looks and behaves more like a desktop application than a website.
How to set up a web application broadly differs depending on what kind of services you want to provide. But the basic ingredients of every application are:
- Database: To store your page data as well as data generated by user interactions
- Webserver: It doesn't matter if you prefer NGINX, Apache or Express.js, your interface needs to be served
- Frontend: If your backend speaks Javascript it makes sense to have flavour of JS in your frontend as well - React, Angular, Vue