A quick look at how (and why) our platform is built…

25 September 2020
Ben Waters
Ben Waters Technical Director

I’ve written before about how our platform is built, but haven’t really gone into too much technical detail. Having had quite a few questions lately on how we assign CPU etc, I thought a bit more information would be useful for those that are interested in how we’re actually structured.

In a nutshell, our platform works using Debian and Unprivileged LXC Containers. Virtually all the actual management of these containers is done via our own bespoke platform, which we’ve been writing since mid-2019. We’ve got a mixture of Java, PHP and Node to give us the full functionality we need.

Each website is hosted inside its own LXC Container, and each container is by default, assigned 5GB of SSD Storage, 512MB of RAM, and 4 x (Shared) CPU Cores. We assign the multiple cores to enable multithreading applications to take advantage but in reality, those cores are shared across the platform. Each server has 32 cores available, and we limit the number of containers per machine to limit the number of shared cores. So for 10 sites, you’d have 50GB of storage available, 5GB of RAM, and 40 x Shared CPU Cores.

One of the reasons I haven’t really spoken publicly about this, is right now this is still in a flexible state for us. We’re gauging how well this performs as we scale up, and those individual resources per machine may increase, or decrease. Sticking to the ethos of providing permanently high quality, fast dedicated hosting – we will never allow a server (or website) to get to the point of reduced performance. We’re reserving a relatively substantial portion of resources on each server to ensure that the server always has extra capacity, to deal with unexpected spikes or additional loads.

As far as the actual hardware goes, we use high performance Dell servers, and we have built everything on the principle that there should be no single point of failure. Each server as dual power supplies with A/B power feeds, and each individual server houses multiple solid state drives, which are then pooled together over our network using a ‘vSAN’ storage technology called Ceph. What that means, is that multiple disks can fail, without impacting on our performance or stability whatsoever. In fact, we can lose multiple entire servers out of the cluster, without this impacting on our stability or long term performance.

Each server has 4 x 1Gb cables running to two separate switches, giving a throughput of 4Gbps – although we’re in the middle of building our new production platform which will have 2 x 10Gb fibre links giving us 20Gbps throughput per server. Our switches have independent feeds to our supplier, who also have diverse connectivity to ensure no single point of failure.

As each machine runs as its own environment, we looked for server management software that would fit the bill… but nothing seemed to really tick all the boxes and give us what we wanted. So, we took the opportunity to write our own management software. This gives us the ability to allow you to customise your server through a control panel, simply by clicking a few buttons. From changing the PHP version, to switching between Apache or Nginx, and full control over the configuration of each. We also give single-click WordPress install, and will be expanding further on control panel functionality over time.

Each instance runs a backup agent which is configured to run a backup every 15 minutes off-site. We also use have monitoring (currently running every minute) which provides us with realtime monitoring for each website. It checks for uptime/availability, response time, and the validity of SSL Certificates. Currently, that monitoring is limited to just DataLords engineers, but the information is embedded inside the portal, and we hope to expand options on this in the future, allowing you SMS / Telegram / WhatsApp notifications for any website issues.

Additionally in development, is the ability to be able to dynamically scale the resources assigned to you. We know, ultimately, managing a range of 500 sites, at least 80% of those will be very low usage, but 20% will require significantly higher resources. Our control panel will allow you the ability to choose which websites get the priority power/resources, still ensuring that every website you host have enough resources to keep it fast and responsive.

If you’re interested in giving the platform a test, and would like to apply to join our beta program, click here to find out more.