Handling the Load

So yesterday two Micronetia hosted sites were "Instalanched" thanks to two different instapundit contributors posting links to them.

This site was one and it handled the load just fine. The summary stats are below. The peak unique visitors was just under 1000/hour or one every 3-4 seconds.

The site is running on a Pi3B so far as I can tell had absolutely no problems handling the load. It is worth noting that this site is heavily textual and the text had mostly been cached by Cloudflare over the preceeding hours and days

As you can see from the graph, traffic went from near zero to ~600MB/hour (the free cloudflare plan doesn't give greater granularity, my guess is that a significant chunk of that 600MB was in the first few minutes after the link was posted). But the vast majority (90%) of that load was cached and the cache is now serving almost 95% of all requests which is precisely what the cloudflare part of the micronetia concept is supposed to be about.

In brief it all worked exactly as it should have. Yippee!

Uncached Graphics Are a Problem With a Pi3

Another site "Glorious Meme Commissary of Proletariat" got hit harder. It is also much, much more graphics heavy - pretty much every page is a meme and there is very little text. It was also initially on a Pi3B but that server struggled to deliver the initial requests so that Cloudflare could cache them. Putting it on a 4GB Pi4 and it handled the load with no problem at all. It too is now at a shade under 95% cached and so could probably be moved back to the Pi3

Looking at the bandwidth is instructive

The first hour the Pi3 had a problem fulfilling requests from about 600 users of about 130MBytes. In fact at one point when I went to take a look, the Cloudflare proxy reported that the site was down although, as I understand it, the pi itself was doing OK if you were logged in via SSH. However it did report very high memory usage.

Once the Pi3 was shut down and the SD card was plugged into the Pi4 the problems went away. The Pi4 was able to handle over 8000 requests in the hour (15-20% of the total) and deliver over 600MB of data (= 10MB/minute) without any problem at all. However if you look at the graph at 7pm-9pm that was about 10% of the total load so the Cloudflare cacheing really made a difference.

Lessons

Lesson one is that cacheing in advance of heavy traffic is critical. Once cloudflare can handle most of the load a Pi3 is perfectly fine as a host. However in the initial hit, if things aren't cached a Pi4 with its extra memory and fast processor is needed to be able to serve uncached requests. There's no doubt in my mind that it should be possible to serve more data from the Pi3, my guess is that Ghost isn't very efficient and that if there were a way to have a static blog platform the Pi3 would be fine handling the hundreds of requests and megabytes of data per second that would be required to handle a similar traffic spike.

Lesson two is that if you have more than one pi, you can swap out your hardware really quickly. It takes maybe 30 seconds to shut one down and no more than a couple of minutes after that to eject the SD card, insert it into a new pi, power that one on and have it up and running. At least that's the case for ghost. It can take a couple of minutes longer for rocket.chat if you have installed that too - as I discovered when trying to do some minimal performance testing.

The nice bit about swapping out the SD card is that the new hardware keeps the exact same configuration in network etc. terms as the old one so the IP address is the same, SSH still works, cloudlared etc. still connects and so on.