I want backup sysadmins to have login access to some systems, with said access rarely (if ever) used. To prevent abuse I’d like strong audit logging, logging that stands out from the rest of all the logging, logging that cannot be tampered with, and that can easily be followed up in case of abuse. I’ve been fixing a working setup through PAM, sending emails or instant messaging through third-parties. Jump down to the last section (“Specific complete examples”) to find ... [continue reading]
OpenShift is currently en vogue in the company. The ease of use and scalability found in a container based system allows us to automate the build and deployment steps of containers through software like Kubernetes/OpenShift.
We have visited Jekyll in several previous blog posts. Our techblog and the SysAdvent calendar (decomissioned as per 2023) utilize the Jekyll software to produce a static site from markdown content. The content and templates is stored in Git repositories on an ... [continue reading]
Thruk comes natively with Naemon, and is a free and open source full drop in replacement web interface for Nagios, Icinga and Shinken. These are flexible tools for alerting us when something goes horribly wrong, and Thruk adds a few tricks for even better monitoring.
With this blog entry I will highlight some of Thruk’s nicer features, and show how easy it is to get get up and running in a couple of steps.
Thruk is ... [continue reading]
Lately I have been working a bit with the monitoring platform Zabbix, and the instance in question is backed by the PostgreSQL RDBMS.
Apart from data regarding such as hosts, services and checks, a significant amount of the data in the RDMS is historical time-series data aggregated for over a year back in time.
There are several data-stores optimised for time-series data to choose from, as SQL is not the best choice for this, ... [continue reading]
fail2ban is one of several tools designed to protect other services by blocking unwanted and possibly repeating activities. Its most common use case is probably protecting the SSH server from brute-force attacks, where repeatedly failed login attempts will be generously rewarded with an iptables firewall ban or some other variant of blocking or null routing.
By monitoring the correct set of log files and applying regular expression patterns to the observations, fail2ban will extract and remember offending IP addresses. After ... [continue reading]
OpenShift Container Platform (OCP) builds on Docker for container-technology and Kubernetes for orchestration of those containers. OpenShift solves the network annoyances in Kubernetes and adds features like authentication and authorization, multi-tenancy, source-to-image (S2I) and templating of applications.
To easily get started with OpenShift development, the OpenShift client (oc) includes an all-in-one cluster that provides a seamless way to get up-and-running with a local OpenShift installation. The prerequisites are:
Everything was ready. The deploy should have been clean and fast. But then, the developers had added just another language module. Not a big thing, just something you could have pulled down, and stashed somewhere below /usr/local. But then, there is this policy that was added early in the project process: All software should be packaged as rpm files. Sounded a dream for the ops people, this time we should get it done right. But for this single library, there ... [continue reading]
The first time I successfully fired up a container I was pretty excited with the potential this tool had to make a lot of everyday tasks much easier. For example when I had a colleague ask for package xyz from EPEL/PPA made available from our internal mirrors, I could just fire up a clean CentOS/Ubuntu/Debian container and download the packages much faster. This seemed much better than having a CentOS7 virtual machine that I needed to fire up or browsing ... [continue reading]
Sometimes you come across problems with websites that normal configuration does not address usefully. A case in point was a PHP-based application that from time to time returned a 302 to a login page instead of the front page, which is not optimal when you serve news articles.
Our solution was to add a simple rule to Varnish, so we serve old cached content, using “grace”, instead of the redirect. Grace allows Varnish to serve expired content in case there ... [continue reading]
While dropping root account passwords completely in favour of sudo is an option in many cases, we prefer keeping root passwords around for when we need direct console access. We keep these passwords in an encrypted password-store (we will write about this in a later blog post this season), and change them when someone should no longer have access or the passwords approach three months in age.
We prefer “horse” passwords for ease of communicating verbally, and use “diceware” ... [continue reading]
