Amazon Clouddrive, Encfs and Rsync – Part II

Welcome Back!

So now that you set up your own syncmachine that will push all your data encrypted to your Amazon Clouddrive, it’s time to up the ante. A quick recap: At this point you can mount and decrypt your Drive into a local folder and read files with ‘ok’ speeds. Writing actually takes ages because if you copy/move a file into your clouddrive decrypted folder the whole process of encrypting and uploading takes place. This is annoying for large files and some programs have serious problems with that.

So if you are a CouchPotato like me, that has some issues coming up on your Sonarr, err, Radar you want those issues to be gone. So like magic you raise your wand and begin to cast the “Abra-ca sabnzbd” spell.

– Anonymous post on usenet.

If that sentence made no sense to you, well no worries. Here is an analogy:

  1. You download you favorite Linux ISO with your favorite command line tool.
  2. You save said ISO in a temporary folder somewhere.
  3. You wrote or got a neat program that scans and sorts your temp folder into your CloudDrive.
  4. The program starts a copy/move from the temporary directory into the Clouddrive.
  5. Encfs and acd_cli start working on encrypting and uploading.
  6. Before the copy/move command completes your tool throws a timeout.
  7. Nothing happened.

Also if you share your CloudDrive folder with Samba it is annoying to transfer several GB worth of data. The wait kills at least me. I don’t like to wait.

Let’s fix all that. Now.


Managing KernelCare with Puppet


If you haven’t felt it before: When Dirty Cow hit you did. The Linux Kernel is rock solid, proven but also has security issues. In this case: Root rights for everyone! And on top of that this bug is so trivially easy to exploit (several proof-of-concepts are out there that can easily converted into a life, working gun) that you had to update your kernels. On every server. And reboot.

The last part is especially evil because a reboot will be noticed by your customers if you are not employing some high-availability setup. And in the world of web hosting this is mostly not the case. So every reboot is a downtime, costs time and money. Plus, you have to update your servers in due time and plan said downtime accordingly. But for all this to happen your distribution must build and provide you with updates first. You can’t install non-existent patches.

Enter KernelCare.

KernelCare is a product from the folks that bring you CloudLinux, which solves all of the above problems. It consists of a kernel module that loads additional kernel patches for your kernel version and applies them in real time. The daemon checks for available updates every 4 hours (via cron) and patches are made available blazingly fast. To pick up the above Dirty Cow example, here is their incident reaction chart. To sum it up: You are days ahead. In a situation where remote root exploits is a thing, days can kill you.

Let’s rather kill the bugs.


Amazon Clouddrive, Encfs and Rsync

Cloudy, with a chance of Unlimitness.

Amazon.$tld just released an unlimited Storage Tier for non-commercial applications for ~70 Bucks a year (depending on your location). And they are not kidding. Not only does the Web-Interface say ‘x used of unlimited’ but once mounted you’ll notice 100TB Quota. And reports on github actually state people reached that limit and even increased that by contacting support. So, plenty of space for your ‘documents’.

In its most basic form you can manage, down- and upload your files via the web-interface. That’s nice for on-the-go, but we are talking serious data here. So we need a client. Amazon offers the usual stuff: Windows Client, OSX Client. No Linux? Don’t be sad. Those clients can’t even do a real ‘sync’. Plus so far all clients upload your files unencrypted. Of course amazon goes ad-fishing with buzzwords from the ‘secure’-department… But they do have a list of compatible file types available. This is just for the web-client that it can index and play media; but this also means they are scanning your files (even if only automatically).

Err, no thanks.

What we want is the ability to mount the CloudDrive in Linux and encrypt everything before uploading.
Let’s get cracking.


Installing Vaultier under Fedora


So there I was; trying to install Vaultier on a dedicated CentOS machine. Turns out there is only a docker installation, an installation script for Ubuntu and manual install. And for the latter it’s only for Ubuntu (or Debian). Tough luck.

But how hard can it be to install this in CentOS? Next to impossible. The software shipped in the default repositories (and epel) are too old to actually get it to work (without compiling a lot on my own, but that would break the nice updates). And updates are a must on a server that handles sensitive data.

So I took the next best thing: Fedora 24 Server. Even that turned out to be ugly; but in the end it worked. Here is how I did it.


XenServer, Patch 22 and Crypto

The What

I am using XenServer as my private solution for my network. It’s fast, reliable, open-source and free (as in free beer). I am sort of a fanboy. That said we are using XenServer at work, too.

Somewhat recently Citrix, maker of XenServer released hotfix XS65ESP1022 aka Patch 22, release notes:

This hotfix supports the upgrade of OpenSSL package to version 1.0.1.

Files Updated


The bold one however, introduces some issues. If you (like everyone) installed extra packages in the Dom0 (the hypervisor) and maybe even used packages from epel then stuff will break apart. For example:


Implementing DNSSEC


DNS is on of the most basic and needed database structures on the planet. Its’ hierarchical, and goes from top to bottom.
If you have no clue on how DNS works, you should check out this fine tutorial over at Once you did that, come back here and we talk security. But seriously, read that. Now.

The issue

Like I said (and you read, if you haven’t: shame on you!) DNS is hierarchical, top-to-bottom approach. And spoofing or tampering can happen in any level, even at the bottom (dns server hijacking).

Enter DNS-SEC.


Fedora 23 and fwbuilder

Fedora & Fwbuilder

For both work and private entertainment I have been using fwbuilder, a graphical clicky-clicky firewall configuration tool that totally rocks when you have a shitload of servers to manage. Added a new trusted ip? net? *click*, done. And by deploying the rules with puppet it’s a breeze and almost fun.

As only of recently I began using Fedora (both vanilla and kde spin) as my workstation OS, and so far I like it. Before that I have been using Ubuntu 12.04 LTS since it came out. But it was aged and the upgrade kind of failed (ati, *cough*).

So back on my new and shiny Fedora Station I typed the magic words:

~ $ sudo dnf install fwbuilder
Last metadata expiration check: 2:05:43 ago on Tue May 3 13:21:44 2016.
No package fwbuilder available.
Error: Unable to find a match.

Yikes! Fwbuilder is not in the main repo. I googled and it turns out fwbuilder was removed back in Fedora 21, running 23 any hope of using a 21’ish rpm is gone. Tarball of binaries? No. Nothing.

rpmfind et all does not yield any sort of result. So we need to get our handy dirty.

http public key pinning (hpkp)

Quick trusted overview.

We now live in a work of free ssl certificate for everyone [startssl, let’s encrypt]. This is awesome not only for security, as in you encrypt your in-transit data, but also for authenticity, that is, you prove (by proxy) to everyone that the data stream originated from your server. This is done by trusting the certificate, or more precise: the certificate path.

A trusted certificate path.

A certificate path starts from the certificate authority, the company or entity you gives you your certificate. Most of the time your certificate is signed not directly by your certificate authority (CA) but by/through an intermediate certificate authority. Your browser (Firefox, Opera, Iexplore, Safari et all) comes with a list of trusted ‘root certificates’. All certificates signed by any of those trusted root-CAs are automatically trusted as long as the chain of trust is complete. So your browser trusts, for example, startssl root-CA. StartSSL created and signed an intermediate-CA so that is trusted, too. This one, in turn, issued your server CA. As your browser trusts the entire chain you should not see any security warnings when you visit your site.

The problem.

So what is the issue here? Your browser already trusts all the certificates issued by any trusted root-CA. There you have it. The problem. You trust any issued certificate. So when an obscure Chinese root-CA issues valid certificates for your domain, that certificate is automatically trusted by anyone. A man in the middle or imposer attack was never this easy. Think this is a theoretical problem? It’s not. There are many cases out there.


PFsense and XenServer

The players

I like XenServer. It’s a rock solid (albeit basic) virtualisation platform that’s not only open-source but can handle any OS you throw at it. Management is a bliss and in my many years of using it both professionally and privately I yet have to encounter a (non-hardware related) crash or other issue with it.

Installation is a breeze. All you need is the ISO, burn it an install it on some hardware you have lying around (it works even inside VirtualBox for a Test-drive; and yes: It also works inside XenServer. Xenception.). The hardware specifications are based on what you are going to do with it. From basic testing to high end computing with several hundreds of cores– no problem there.

The other piece of software I totally like is pfsense, a software based firewall distribution. With some minor tweaking you can get a real neat setup working.

Those two are just screaming to get together and have  party. Bring the party hats!


Lenovo S21e, Linux and the Touchpad

The ‘Ahh’.

I recently bought a Lenovo S21e notebook. I wanted something light, thin and before all: cheap. The usage of a notebook is restricted on doing stuff on the balcony or garden; “stuff” being puppet code, general server management and light web applications. For that the tiny S21e for a mere 180€ at amazon (note: the price actually increased since I bought it) seemed good enough. Sharp display, full size keyboard and no fans or other moving parts. It has no SSD either; the mass  storage is an embedded 64Gb flash card which speed is in between a native spinning hard disk and a SSD. The soldered 2gb ram seemed enough for it’s task and the quad core Celeron; well, it’s a Celeron.

It came with Windows 8 & Bing pre-installed. I always boot into the pre-installed system at least once to test the hardware for defuncts. Later on you can’t tell if it’s a hardware or software problem. A practice that sure helped me…