Deadwood snapshot: RR rotation now works

Today's Deadwood snapshot has working Resource record rotation.

What's RR rotation?

RR rotation allows DNS to work as a primitive load balancer. If you make a DNS query with Deadwood, and the answer has multiple IPs, Deadwood now rotates the IPs so that the first IP in the list always changes.

This makes it so simple DNS clients that connect to the first IP they see when given a list of IPs over DNS will now get a different first IP each time they do a query.

OK, polish it up (make sure there's no memory leaks or other nasty bugs, allow people to disable it because I do a lot of massaging of strings to rotate the records which eats CPU on a loaded server, etc.) then work on TTL aging.

It can be downloaded here

New Deadwood snapshot

I have a new Deadwood snapshot; I'm continuing work on RR rotation. In addition, I added a document about the RRs Deadwood uses internally (0xff70-0xff71; "private use" in the IANA DNS parameters list) so I can keep track of how I used these numbers; 0xff71 now means "an error happened when we tried to get the DNS RR type".

It can be downloaded here

New Deadwood snapshot

Since the functions that make and manipulate "DNS string" objects will be used for more than DNS compression, I have created new source files, DwDnsStr.c and DwDnsStr.h, which will handle DNS strings and things like RR rotation and TTL aging. This lets me keep the number of public functions in DwCompress.c down to two: One for compressing DNS packets, and another for decompressing packets.

To do:

• Function to convert a "DNS string" back in to a uncompressed DNS RR object as stored in the cache.
• RR rotation
• TTL aging
• Release Deadwood-2.4.05

It can be downloaded at the usual place.

Deadwood snapshot update: New cache file format

I have released a new Deadwood snapshot today.

Warning: This new snapshot of Deadwood uses a new format for the cache file, where DNS packets in the cache are stored in an uncompressed format, instead of the compressed format earlier Deadwood releases have used for the cache file stored to disk.

Please delete any cache file from an older version of Deadwood before using this Deadwood release.

It can be seen at the usual place

ObHack EOL announcment

I'm just letting people know I currently have no plans to continue development of my ObHack fork of Oblige. This includes bug fixes and feature improvements. If you want to become the maintainer of this project, let me know.

I've decided the "you've found a secret area" message popping up outside the secret door once in a blue moon is not a serious enough bug to merit making a new ObHack release.

I recently posted, on my personal blog, an entry about no longer playing deathmatch games that were in vogue in the 1990s but have gone out of style. Most people play games on dedicated consoles, or play MMORPGs (multiplayer role playing games, like World of Warcraft) these days.

While I still enjoy the occasional Doom single player game, it's no longer something important enough to me to merit devoting a programming project to it.

Firefox 3.5 now works; Deadwood minor update

OK, a week or so ago I reinstalled my operating system, zapping all of my Firefox configuration data. Once I did this, I tried installing Firefox 3.5 again. This time, everything worked without significant problem; I am able to use Firefox 3.5 without significant freeze-ups while loading background tabs.

For other people who are seeing these freeze-ups, try resetting your Firefox profile to see if the problem goes away.

---

I have released a minor update to Deadwood today.

New Deadwood snapshot

There was a single SQA regression that didn't pass in Deadwood 2.4.04: The sqa_tcp test. The test does not pass because BIND (at least the example.com server's install of BIND) doesn't use compression for DNS-over-TCP. I have revised the test to not care if the DNS-over-TCP packet uses DNS compression. Now that I have done this, the Deadwood snapshot passes all SQA regressions again.

It can be downloaded at maradns.org/deadwood/snap.

I've removed all pre-2.4.04 snapshots of Deadwood.

- Sam

Deadwood 2.4.04 released

OK, after over a day of testing, it looks like I have definitely fixed the crash bug. Cityhopper showed me how to look at Dr. Watson logs in Windows XP, which I did in order to verify all of the crashes we have seen are caused by the code I have since fixed. When they say C is portable assembler, they mean it; the compiled assembly looks a lot like the C code.

That in mind, I have released Deadwood 2.4.04. The only change from the 20080718 snapshot is that we only log times when we get a valid DNS packet, but the packet is not correctly decompressed-then-compressed.

I did a lot of testing this weekend and I have yet to see a DNS packet that doesn't decompress-then-compress correctly. If people see this (it's logged in dwlog.txt or on standard output in CentOS 5), let me know with the full DNS packets.

Now that Deadwood can compress and decompress DNS packets (it takes some 5k of code to do this), the next step is to add support for TTL aging and RR rotation.

It can be downloaded at maradns.org/deadwood/testing

- Sam

Deadwood update

Last night, after coming home from a party, I turned on my computer and Deadwood crashed (alas, without a stack trace or any useful info) during Windows start up. I tried to reproduce the bug last night and this morning without success; I set up some stress tests and ran Deadwood both with debugging symbols turned on, turned off, with Valgrind, etc. Deadwood looks good in Valgrind (no errors with compiled with -DVALGRIND_NOERRORS and no leaks), and I have not been able to reproduce the crash.

I'm thinking at this point that maybe I used an older version of Deadwood without the "don't reference a null pointer" fix from yesterday, even though the crashed program had a recent timestamp. I've done some hardening of all of the new compression code, have been completely unable to reproduce this crash, so I made sure to recompile Deadwood to use the latest version with the crash fix (and I can verify the crash fix works when stress-testing it), and will run this version of Deadwood for about a day and a half. If I don't get any crashes again by tomorrow around noon, I will make the HEAD snapshot Deadwood 2.4.04 in the early afternoon and see if the one crash I saw last night doesn't pop up again.

- Sam

MaraDNS and Deadwood update

I just posted an update to Deadwood; this hopefully solves the once in a blue moon crashes I'm seeing. I didn't make sure the values of certain variables were not null before referencing the pointers; I have fixed this issue and will run Deadwood for 24 hours to make sure nothing else comes up. It can be downloaded at the usual location.

---

I'm getting good results from my "Don't bug me in private email if you don't want to pay up. That includes non-security bug reports" policy; someone has just offered to pay me a little to give them a little MaraDNS support. My prices are very reasonable and reflect the current economic situation we are in.

Another person got upset when I told them to pay up to talk to me about MaraDNS in private email. I've set up three more different form replies to handle people like this. The form replies can be summarized as "You're rude, go away", "Report the bug to the MaraDNS mailing list", and "Really, I mean show me the money or take it to the list". These four form replies (the original one being "Show me the money or take it to the list") should handle pretty much any user wanting private email support for MaraDNS.

Remember: MaraDNS is free, support on the email list is free, but support by private email is not free.

---

I've been getting a lot of comments about DNScurve, which I just approved the majority of (the anonymous ones were flamebait or downright wrong, so weren't published). No, I'm not going to support it. Not only have I had personal issues with DJB, but Mr. Kaminsky (the person who discovered how to poison DNS caches) has told me DNScurve isn't as secure as DNSSEC. I don't have time to look at the two proposals; I'll just take Kaminsky's word for it.

I'm back!

I'm back!

I've uploaded a new Deadwood snapshot today. I've done some hardening to the DNS compression code; it now has no warnings nor leaks reported by Valgrind. I'm running this code right now as my DNS server to see if the problem with the server crashing comes up again; if it doesn't, I'll do the other tests and make a testing release of Deadwood early next week.

It can be downloaded at the usual place

Revised Deadwood roadmap

OK, in light of the hard-to-reproduce crashes of Deadwood, I need to bunker down and harden the code for Deadwood:

• Make sure the compression code makes no warnings (or leaks) when using Valgrind and compiled with -DVALGRIND_NOERRORS
• Harden all routines in DwCompress.c; always check return codes and always make sure something is not null before doing a->b type code. Ignore the "52 lines per function" rule when doing this hardening but follow all other coding style standards
• Once the code is hardened, go back and split up functions as needed to keep everything below 52 lines in length again

The compression code works but there are subtle bugs which result in a crash once every two or three hours which are a pain to track down.

When will I do that? Starting on Friday, July 17th. I am now officially on a hiatus from Deadwood and MaraDNS development. Unless it's a bugtraq-worthy security report, I will not make any changes to Deadwood nor MaraDNS for the next two weeks.

In addition, I will not post any more blogs here for a while.

Have a happy 4th of July, Americans, and a happy 2-week summer break everyone!

Deadwood 2.4.04 release delayed

There are still bugs in Deadwood that make Deadwood segfault. I've modified the code a little to give me verbose messages and will see if I can recreate the bug. Until I can find and fix this issue, I'm not going to release Deadwood 2.4.04.

- Sam

Deadwood snapshot update: Some more compression bugs fixed

I have fixed a couple of bugs that made some DNS packets not decompress-then-recompress properly, and have released a new testing release of Deadwood. As before, hit on this release and lets see if we can find any other DNS packets which make Deadwood complain the packet doesn't decompress or recompress properly.

I will run this for a few hours; if I don't see any more problem packets, I will make the 2.4.04 and release it early this afternoon.

It can be looked at by going to maradns.org/deadwood/snap.

It's nice to be wrapping things up with the decompression/compression code; DNS compression is a royal headache and one of the parts of DNS I loathe the most.

Testers wanted for Deadwood snapshot

I just made a Deadwood snapshot update which is intended for people to more widely test Deadwood. I would like more people to test the compression code with Deadwood.

Testing release: The compression code does not affect packets coming to and from the resolver. However, every time Deadwood gets a DNS packet, it decompresses then recompresses the packet. If the recompressed packet differs from the packet received from the upstream DNS server, Deadwood logs a "WARNING: Compression problems with packet" error, followed by the DNS packet causing problems. This log message is then followed by the length of the compressed packet, the packet as actually compressed by Deadwood, and finally the string value and length of the question.

This allows me (and anyone who wishes to help) to test Deadwood's compression core with real-world DNS packets. In addition, I will make some acid tests for the compression core (tests like making sure we never compress SRV RRs but can understand packets with compressed SRV records, etc.) which I will use to further test Deadwood's compression core with.

Since it would be nice to see some wider testing in this snapshot, I have made both the source code and a 7zip compressed Windows binary of the snapshot available here.

I will release Deadwood 2.4.04 when I'm satisfied that testing has ironed out the bugs in the compression core.

- Sam

Deadwood update: DNS compression code finished

I have finally finished up the DNS compression code for Deadwood.

The reason for the slow progress is because my personal life has been a lot more busy and I have had less time to devote to Deadwood. This will not change for the foreseeable future and Deadwood progress will continue to be slow.

In today's snapshot, the underlying compression and decompression libraries work. Now I have to go through and make sure they are bug-free; it's a matter of a bunch of SQA testing at this point.

People can look at the code at maradns.org/deadwood/snap

Memories of Usenet

Usenet is dead.

Usenet has been dead for a few years now.

Yeah, sure, there is this thing out there that works like The Pirate Bay using NNTP instead of BitTorrent packets, but that isn't the Usenet I knew and loved in the 1990s.

Usenet is dead.

Sure, you get a few crotchety oldtimers who get their panties in a bunch when you mention that Usenet is dead, just as you got people who denied the Amiga was dead for years and Japanese soldiers fighting World War II as recently as the 1970s. People often react to a loss or defeat with denial.

The last time I significantly used Usenet was back in the early 2000s. Before the college I was going to at the time set up Wi-Fi everywhere, allowing wireless internet access from my laptop, I would use Leafnode to download a number of Usenet newsgroups I would read during the day. I dialed up to my ISP (yes, I had dialup as recently as 2003), downloaded all the newsgroups I wanted to read, uploaded any posts I made while offline, and updated my local Usenet spool. I could then, over the day, read my daily set of new Usenet postings and post replies as appropriate.

I still kept this setup to read Usenet after they added Wi-Fi at school. I finally canceled my dial-up ISP, then did things in reverse: I would upload and download Usenet postings at school and read Usenet offline at home.

By then, alt.hackers, a fun little group about clever solutions to problems in the mid-1990s, was comatose and in its death throes. rec.audio.pro, a newsgroup discussing methods to record audio, was still alive and well, but would soon get replaced by Gearslutz. comp.os.linux.* was already overtaken by spam and flame wars; Linux users had taken their discussions to /., Kuro5hin, among other places.

By 2004, I realized that little worthwhile discussion was still taking place in Usenet, and moved on to web-based discussion boards. The way to be a part of a community on the internet changed in the 2000s; I had my first blog in 2003 (back then, it was called a "journal"), got a MySpace account (my first social networking account) in 2005 and started this blog in early 2007.

Usenet is dead.

I remember the day I discovered usenet. It was in the fall of 1993, and I was at the computer lab of a university I just transferred to. I was on a Macintosh and saw this icon marked "newsreader", opened it up, and discovered Usenet. It was incredible. I could use a computer to connect to this worldwide network of computers and talk with anyone in the world about any topic I wanted to talk about. It blew my mind away.

I soon learned UNIX and the TRN newsreader because I didn't have a computer at the time (this was when a basic computer would set you back $2000; these days a basic notebook or desktop is about $400) and wanted to be able to read and post to Usenet all night--the only computer labs open all night were running UNIX.

The skills I learned spending so much time reading Usenet translated in to me getting a job at Netcom, then one of the most prestigious internet providers, a couple of years later. I saw the internet experience an explosive growth in the late 1990s, with Mosaic and later Netscape giving the internet a user-friendly GUI, but still read and posted to Usenet using the TRN client on my computer running Linux.

In was around late 1997 that I first discovered a forum I liked that wasn't a Usenet forum, a place where men and women both talked about relationships, sex, and personal details about their life the way nobody in Usenet talked about their lives. I found the place wonderful, particularly since it had a lot of girls; something Usenet never really had.

I also was told by a co-worker about Slashdot, got a 4-digit account in 1998 and starting reading and posting there.

I continued to read Usenet for many things throughout the 1990s, but with the discovery of web-based forums in the late 1990s, my interest in Usenet started waning and the number of interesting discussions Usenet used to have (or never had) moved to the web. I had a friend who gave me access to Usenet II in 1998, a late-1990s attempt to revive the Usenet of the early 1990s, but that never went anywhere and was little more than a hierarchy of empty newsgroups; the only active group there talked about Usenet II and making sure Usenet II postings did not make it in to ordinary Usenet.

Usenet is dead.

The last really interesting thing to happen with Usenet was when Google was able to, in 2002, recover the majority of pre-1995 Usenet postings from archives individuals had. It was interesting to finally read entire legendary ancient Usenet threads, such as the legendary 1992 "Linux is obsolete" debate and see what people had to say about BIND and other DNS servers in the 1980s and early 1990s.

Usenet is dead.

It was fun while it lasted, but the internet has moved on and, in truth, Usenet really wasn't that great. There was no moderation, so no way to keep flame wars or spam under control. It was a place with a lot of arrogance and elitism; a place where experienced users took a sadistic delight in flaming newbies (this was even worse in IRC, the 1990s version of MSN and instant messaging); a place where finding an answer to a technical question was a hit-and-miss affair. A place without graphics or multimedia; the interface was nothing more than ugly fixed-width text on an 80-column screen.

But, it was the best we had at the time and I thank you for letting this aging geek talk about the way things used to be.

Usenet is dead.

It's time to move on. I have a lot of things today I didn't have when I was a Usenet junkie in the mid-1990s: A girlfriend who loves me and who I love; a community of real friends in the United States who I keep in touch with via Skype and social networking sites. I also have an open-source project I am eager to finish up.

I just looked at the code for Deadwood, and I haven't quite finished up the compression code. Things look good, however, and I just need to do a little more last-minute touchup before the compression code is done.