Deadwood and 64-bit systems; a highly scalable load balancer

People looking at the source code of Deadwood will observe some of the code is 32-bit specific; There are a lot of int32_t and uint32_t variables and functions in the code. The reason for this is to make porting to 64-bit easier; while not as efficient as using 64-bit ints everywhere, it's pretty much guaranteed that the code will run the same on a 32-bit and 64-bit CPU/operating system (my development machine has a 64-bit CPU, but is currently using a 32-bit operating system). However, one thing I have been thinking about is how to scale the code up to use 64-bit ints.

Most of the code can be pretty easily scaled up; use a 64-bit int instead of a 32-bit int for various constants. The random number generator, Radio Gatun, has both a 64-bit and 32-bit version.

The one bit of code that can't easily scale up quickly is the code that generates the 31-bit primes. I use a simple trial division that doesn't scale up. This code will have to be rewritten to use something fancier; Chris Caldwell has a lot of really good pages about how to quickly find a big prime. Using this method, along with this should be able to find a number that is almost certainly a 63-bit (18-digit) prime without being too much slower than finding a 31-bit prime.

---

It is possible to make a DNS load balancer that can handle far more than 3,500 queries a second. The bottleneck with Deadwood-1 is how I use TCP/IP (or, should I say, UDP/IP). The trick is to have both the load balancer and the upstream DNS servers have public and private IPs. The load balancer would get its query on a public IP, and send it over a private IP to the upstream DNS server. There will be a firewall in front of the load balancer that rejects all packets with private IPs coming in from outside.

For each upstream load balancer, there can be about 64000 connections at the same time. Given 10 upstream load balancers, thats 640000 connections a second. That's, with a seven second timeout, over 90000 DNS queries a second.

However, I'm not going to develop this code. Instead, I'm going to work on the caching DNS server. My plate is already full.

- Sam