New Deadwood snapshot: Parser bugs fixed

I have been uploading new Deadwood snapshots to fix some bugs in the dwood3rc parser; the parser now has the += (append) operator work correctly with dictionary variables, as well as having more stringent controls for dictionary indexes (dictionary variables must be initialized; a dictionary index can not be appended to without first being set; etc.) so that dwood3rc files must be working Python 2 scripts to correctly parse.

In addition, I have refined the documentation a little.

They can be looked at here:

http://maradns.org/deadwood/snap/

New Deadwood snapshot: Two questions and answers added to FAQ

I have uploaded a Deadwood snapshot today that adds two questions and answers to the FAQ. One is about blacklisting domains with Deadwood, and the other is a brief note about how, as much as I would like Deadwood to have DNSSEC support, I am not in a place in my life where I have the time to implement DNSSEC “for fun and for free”.

The updated FAQ can be looked at here:

http://maradns.org/deadwood/doc/FAQ.txt

New Deadwood snapshot: Improved documentation

I just uploaded a Deadwood snapshot; this snapshot has updated and improved the Deadwood manpage (typos corrected; Windows Reference.txt version of manpage updated). In addition, I have added install.bat and uninstall.bat batch files to the Windows port of Deadwood, and revised README.txt for Windows accordingly, so Windows XP users can now install (or remove) Deadwood with just one click.

It can be looked at here:

http://maradns.org/deadwood/snap/

Update: I just tested this in Windows 7, and Deadwood is not a one-click nor two-click install, since right-clicking on a .bat file and choosing “run as administrator” changes the current working directory to \WINDOWS\system32. To install on Windows 7:

• Go to the start menu and search for “cmd”
• When found, rick click on it and run “cmd” as an administrator
• At the cmd prompt, use the “cd” command to find and enter the directory where Deadwood is unpacked
• Invoke the “install.bat” script from this directory

I will have to update Vista.txt and README.txt to point out how to install Deadwood in Windows 7 (and, presumably, Windows Vista) in the next couple of days.

New Deadwood snapshot: Resolve speeds comparable to Unbound

It looks like I have solved the issues people have reported with Deadwood being sluggish. The two bugs I fixed were one where Deadwood would delay timeout_seconds (normally 2 seconds) if there was a problem getting an upstream IP from its cache, and another where Deadwood would delay timeout_seconds if there were upper case letters in the name of a DNS reply.

I just ran Juergen’s test with Deadwood and Unbound. Deadwood is able to resolve the list of domain names on my network in 2:42 (two minutes forty-two seconds) and 2:54 (the 2:42 figure was done starting Deadwood with no cache); Unbound needed 2:57 to resolve the same list. In addition, namebench shows Deadwood doing a lot better than Unbound (far fewer lost queries and better average response time to the queries Deadwood does resolve) on my machine.

Steve Gibson’s DNS benchmark tool is buggy and incorrectly reports that Deadwood drops most DNS packets; I have set up Deadwood to report every reply it sends as well as every incoming query, and have verified that Deadwood replies to most, if not all, of its incoming queries when Gibson’s DNS benchmark is run against it.

In addition, I have revised Deadwood’s internal hash algorithm to be more secure: The algorithm, in addition to using a multiplication constant that randomly changes every time Deadwood is compiled in CentOS Linux or a new Deadwood release is made (which Deadwood has always done), now uses an additive constant that is different every time Deadwood is started. Deadwood is hard-coded to use /dev/urandom (secret.txt in Windows systems) for entropy to generate this additive constant from (we also get a bit of entropy from the timestamp); if this file is not found, Deadwood will fallback to using a hard-coded additive constant for its core hash.

I would like to thank Juergen Daubert, Wayne, and Marlon for their bug reports, feedback, and suggestions getting programs to help debug Deadwood’s performance.

The snapshot can be downloaded here:

http://maradns.org/deadwood/snap/deadwood-H-20100908-3.tar.bz2

Windows users can use this snapshot that has the improved performance (but not the improvements to Deadwood’s internal hashing function):

http://maradns.org/deadwood/snap/Deadwood-H-20100908-win32.zip

Please report any bugs to the mailing list [1].

- Sam

[1] Crash reports are useless unless they are accompanied by a stack trace or a recipe for reproducing the crash; Valgrind errors are invalid unless Deadwood is compiled with -DVALGRIND_NOERRORS

Note: Yeah, I’m still looking for work. Send me an email if you have a job for me.

Update: Anyone who downloaded deadwood-H-20100908-3 should download deadwood-H-20100908-4; this update is a hotfix that fixes dictionary variables which I broke in deadwood-H-20100908-3. Note also that dictionary variables work with the most recent Windows binary; it was the improved hash compression security which temporarily broke dictionary variables.

Yes, I do perform regressions to catch things like this, but only for numbered release (e.g. I’ll do the regressions before releasing Deadwood 2.9.07) and not for snapshot releases.

New Deadwood snapshot: Queries nearly three times faster

There has been some discussion today about the fact Deadwood, while very good at resolving names in general, is a good deal slower than other DNS servers.

Looking at the code, I already found the #1 offender that was responsible for about two thirds of the slowdown, and have patched this bug in today’s Deadwood snapshot. Anyone who feels Deadwood is a bit sluggish is encouraged to download and use today’s snapshot.

In addition, one issue that I have been thinking about for a while is that the hash compression function is not quite random enough. So, I’ve revised the hash compression core to add a random 16-bit number for each iteration of the hash compression. The number is randomly generated with Deadwood’s secure PRNG (Read: RadioGatún[32]) and is different every time Deadwood is started.

Deadwood snapshots can be looked at here:

http://maradns.org/deadwood/snap/

New Deadwood snapshot: MX record lookups disabled by default

Over the years, people have reported issues using MaraDNS as part of a mail hub, processing MX queries. This is a manner of using MaraDNS which I plain simply do not perform, and therefore have not adequately tested. The fact that releases of Deadwood with broken MX lookup support have been out for over a year, with no one letting me know about this bug on the MaraDNS mailing list (or me becoming aware of it myself with my own “dogfood” testing; I only became aware of it when I decided to make a MX lookup with Deadwood on a lark) indicate that this is not something widely used, much less tested.

My general feeling is that, if an average client machine is making a MX query, most likely it has been taken over by a spambot sending out spam. I have been using Microsoft Outlook and other MUAs (mail user agents) with Deadwood for over a year while its MX processing was broken and never saw a problem; a MUA does not perform MX lookups to send mail, but instead connects to the SMTP server, which does the actual MX lookups.

That in mind, in today’s snapshot of Deadwood, I have explicitly disabled MX lookups; if an MX lookup is made, the query is rejected and the IP is logged as being a possible spam zombie. There is a parameter which I have documented that will enable MX lookups again.

The fact of the matter is this: Using Deadwood with a mail hub making a large number of MX requests is completely untested. If one wants to use Deadwood in this manner, please subscribe to the MaraDNS mailing list and express the intention to help beta-test Deadwood for this use case which I do not have.

In the meantime, by disabling and logging MX lookups in Deadwood, we can make networks a little more secure and resistant to spam zombies commandeering machines on our networks.

New Deadwood snapshot: Non-A records now resolve

I guess not that many people are testing Deadwood.

Let me be honest about the type of testing I’m doing with Deadwood: I am testing Deadwood’s as a DNS server used for casual web surfing on today’s internet. That means that most of the queries done are A queries; I don’t do much testing with AAAA (IPv6) queries because, quite bluntly, my ISP doesn’t offer IPv6 service and I can think of only one site on the Internet with an AAAA record: ipv6.google.com.

Likewise, I haven’t done any testing with MX records. I haven’t touched MX records at all because I’m not using Deadwood with a mail hub. So, it came as an unpleasant shock to me when I discovered last night that Deadwood hasn’t been properly resolving MX records for over a year.

In Deadwood 2.4.05 (released August 9, 2009), I added the ability to rotate resource records. Unknown to me, this feature broke Deadwood with any variable-length record, such as a MX record pair where the hostnames are of different lengths.

The advantage of using Deadwood’s robust string library with almost all of Deadwood’s DNS processing is that this bug did not result in any memory corruption or cause any other problems; the only issue has been that Deadwood would not resolve MX or other variable length records.

The idea that some freetards advocate that open source software is magically tested and that all bugs become shallow is nothing more than so much—how should I say this—mental masturbation. The author of Mailman, John Viega, points out how wrong this is in his excellent essay “The Myth of Open Source Security”.

Excuse me for the blunt wording, but I am a little frustrated right now: I have devoted a decade of my life to MaraDNS, and I haven’t even been able to get a job because of my hard work; while I have had a couple of interviews I would not have had because of MaraDNS, so far I have gotten no offers. Yes, those are interviews I would not had have if it weren’t for MaraDNS, and, yes, MaraDNS kept my skills from getting completely rusty during the ’00s when I was concentrating on learning Spanish and living in Mexico. But, looking back, I really wish I had spent more time learning C++ and object oriented programming and less time editing the Wikipedia or posting to /..

It looks like Deadwood isn’t getting much external testing; not one person saw Deadwood’s issue with MX records in the year Deadwood has had this bug. So, yeah, like everything else with MaraDNS, I’m responsible for just about all of the testing. This isn’t a complete loss; people have asked me in interviews about my testing methodology for MaraDNS.

I have just uploaded a snapshot which fixes the issue with rotating records like MX records:

http://maradns.org/deadwood/snap/

As a side benefit, since I have fixed this issue, TTL aging and RR rotation now work with ANY records.

Next: Set up a test for this issue, as well as DNS compression tests for all of the resource record types MaraDNS supports. If I get time, I will also set up tests to make sure Deadwood correctly handles things like SRV records with compression pointers, and that Deadwood doesn’t compress SRV records.

New Deadwood snapshot: Better heuristics for empty DNS packets

When a remote server sends us an empty DNS packet, what does that really mean? Well, it usually means that the hostname does not always exist. But, not always.

Let’s take the hostname ibnlive.in.com. It has three DNS servers: 123.108.40.31, 123.108.40.32, and 123.108.40.81. Two of the servers (123.108.40.32 and 123.108.40.81) do the right thing: They redirect the name to Akamai’s horrendous mess of DNS resolution. However, 123.108.40.31 is broken and gives us a blank DNS packet with a SERVER FAIL message.

The DNS administrators for in.com aren’t trying to tell us that ibnlive.in.com doesn’t resolve. No, they’re trying to tell us that their DNS administrator is asleep and misconfigured one of their three DNS servers.

So, I’ve tweaked the heuristics of Deadwood to treat a DNS packet with a “SERVER FAIL” RCODE as being one where the DNS server is asleep at the wheel, so instead of treating the packet like “this name does not exist” (like what we normally do with a blank DNS reply), we ignore the packet and bug the next DNS server on the list. The code isn’t perfect: We wait timeout_seconds before contacting the next server, but it’s good enough to have domains mismanaged like this correctly resolve.

I should note that the bugs I’m finding in Deadwood right now are not bugs caused by Deadwood not being RFC-compliant—the last bug I fixed like that was on August 9. No, at this point, I’m fine-tuning Deadwood to more or less do the right thing with mismanaged domains and broken DNS servers.

As always, the snapshot can be looked at here:

http://maradns.org/deadwood/snap/

New Deadwood snapshot: Better handling of empty packets

I have uploaded a new snapshot of Deadwood which improves Deadwood’s handling of empty packets. A number of broken DNS configurations and servers out on the internet, such as the DNS servers answering the IPv6 (AAAA) query for wwwchase.gslb.bankone.com, send out an empty DNS packet instead of a correctly made SOA-in-NS-section DNS reply when a host name can not be found.

Another situation where a DNS server gives out a reply like this is when someone stops paying for their domain, so the ISP suspends their account by removing the zone for their domain. However, this information has not been changed upstream. As a result, when we contact the DNS servers for a given zone, the DNS servers give us a blank QUERY REFUSED reply.

Since these do happen, Deadwood now pretends that a blank DNS reply is a low-TTL “SOA-in-NS-section” reply. This should speed up some queries, especially on IPv6-capable computers that perform AAAA queries before doing A queries.

It can be looked at here:

http://maradns.org/deadwood/snap/

I should point out that I wrote the code to do this in a way to minimize the amount of code added to Deadwood and to maximize code reuse by having the relevant code in DwRecurse.c call the already created make_synth_not_there_answer() function made a little-more general purpose.

MaraDNS progress report: Documentation updates

I am continuing to update MaraDNS 2.0’s up and-coming documentation to correctly reflect the fact that all recursion is now done by the separate daemon Deadwood. I have also updated Deadwood’s manpage to state that Deadwood is a fully recursive DNS server.

http://maradns.org/download/1.9
http://maradns.org/deadwood/snap

New MaraDNS 1.9 snapshot

I made some progress with updating the documentation for MaraDNS 2.0; I have updated the mararc man page to remove parameters that are used for MaraDNS 1’s recursion, as well as adding a section in the update document on updating from MaraDNS 1.4 to MaraDNS 2.0.

I plan on adding a couple of details from the Deadwood FAQ to MaraDNS’s upgrade document, as well as updating MaraDNS’ tutorials to discuss using Deadwood for all recursion.

I am considering MaraDNS 1.9.x releases “snapshot” releases since I am making them fairly quickly right now.

They can be looked at here:

http://maradns.org/download/1.9

MaraDNS 1.9: Moving towards MaraDNS 2.0

I have made the changes to MaraDNS’ codebase to prepare her for the MaraDNS 2.0 release. The changes are (drum roll) pretty simple, actually: I have gone through all of the #ifdef AUTHONLY statements to either keep them as AUTHONLY statements, or to change them to #ifdef IPV6 for statements which enable IPv6 support (MaraDNS 1 has no recursive IPv6 support, so AUTHONLY is also used to enable IPv6).

The only other change I have done is to revise the build scripts to:

• Always define AUTHONLY when compiling MaraDNS
• Update the configure script and Makefiles to allow one to enable or disable IPv6
• Not link MaraDNS to the pthreads library (MaraDNS 2.0 will be thread-free)
• Compile Deadwood as MaraDNS 2.0’s recursive resolver
• I removed makefiles for all operating systems besides Linux, Windows, and a generic *NIX “noflock” makefile

It can be looked at here:

http://maradns.org/download/1.9/

At this point, the code for both MaraDNS 2.0 and Deadwood 3.0 (the recursive resolver MaraDNS will have) are done: The only changes will be bug fixes. The thing I should do at this point is revise MaraDNS’ documentation to reflect the MaraDNS 2.0 changes, as well as removing any MaraDNS 1 SQA tests that test MaraDNS’ recursor.

One issue MaraDNS 2.0 will have is that the recursive resolver will require a separate IP, and that there isn’t an automated way to convert a MaraDNS 1 mararc file in to a MaraDNS 2 mararc file and Deadwood 3 dwood3rc file. This in mind, I have no plans to stop supporting MaraDNS 1.4 for critical security fixes and other important bugfixes and updates (such as updating the default list of root nameservers) for the foreseeable future, as well as supporting MaraDNS 2.0 for “this host does not resolve” bugs, as well as other bug fixes.

New Deadwood snapshot; allowing anonymous comments

I have just uploaded a new Deadwood snapshot; this release does not change the program, but it has updated the INSTALL.txt file to be current and all compile-time flags Deadwood has are now documented.

It can be looked at here:

http://maradns.org/deadwood/snap/

This may be my last snapshot announcement for Deadwood snapshot updates when the actual Deadwood program has not been changed.

---

I have enabled anonymous comments in my blog again, since it looks like Google has enabled comment spam protection. As always, comments posted here are moderated and Deadwood/MaraDNS support requests will not be published, nor will flames, trolling, and spam.

New Deadwood snapshot: mkSecretTxt added; doc/internals updated

In today’s snapshot of Deadwoood, I have added the mkSecretTxt program I made for the 1.4.04 release of MaraDNS. When adding it to Deadwood, I have added a test to make sure secret.txt does not exist before overwriting it. The thinking is that Windows shies away from potentially dangerous commands more than *NIX does; since overwriting secret.txt without prompting is an example of the old dangerous (and, IMO, obsolete) *NIX way, mkSecretTxt no longer does this.

In addition, I have gone through the doc/internals folder and updated the files there. In the case of a couple of files, I just added a note that the file in question is out of date.

It can be downloaded here:

http://maradns.org/deadwood/snap/

OK, back to learning C++.

Deadwood snapshot update: Working around aplus.net’s broken DNS server

Aplus.net’s DNS server is broken.

Let’s contact their DNS server for the A record for bookride.com:

$ dig @64.29.144.70 www.bookride.com

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_4.2 <<>> @64.29.144.70 www.bookride.com
; (1 server found)
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 10397
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;www.bookride.com. IN A

;; ANSWER SECTION:
www.bookride.com. 3600 IN CNAME ghs.google.com.

;; AUTHORITY SECTION:
google.com. 86400 IN SOA ns1.aplus.net. hostmaster.aplus.net. 1007 86403 3600 3600000 86400

;; Query time: 304 msec
;; SERVER: 64.29.144.70#53(64.29.144.70)
;; WHEN: Sat Aug 14 01:18:15 2010
;; MSG SIZE rcvd: 119

This is an invalid packet: It is marked as a NXDOMAIN (complete with a SOA record in the NS/Authority section), but it is actually a CNAME.

I have updated Deadwood to treat these broken packets like ordinary CNAME packets. The snapshot can be downloaded here:

http://maradns.org/deadwood/snap/

I am also in the process of trying to file a bug report with aplus.net.

New Deadwood snapshot: neustar.biz fixed

While doing my own “dogfood” testing for Deadwood, I discovered that the host name “neustar.biz” did not resolve. The issue was that I never bothered to make DNS queries case-insensitive.

The solution is to convert all DNS queries in to their lower-case form before looking them up in the cache or sending another query out to resolve a glueless NS/incomplete CNAME (in the case of “neustar.biz”, the issue was that the CNAMEs were in upper case). In addition, since there may very well be some stub resolvers out there that won’t accept an answer unless it’s in the same case as the question, I now have Deadwood preserve the case in the original form of the query to give back to the stub resolver, using a variable called “orig_query”.

It can be downloaded here:

http://maradns.org/deadwood/snap/

The patch can also be looked at.

I can see the program is getting some attention by the number of clueless emails I am getting asking for private Deadwood support. It would be nice if these people would pay me once I give them the usual form reply asking for money. That has not happened; people who do not have enough courtesy to heed multiple disclosures that email is only for paying customers are not courteous enough to decide to pay for support. It’s a form of the internet dork rule: The anonymity of the internet coupled with the ease of getting on a soapbox attracts all kinds of unpleasant people; the corollary to this rule is that any unmoderated forum is soon overrun by unpleasant people, since anyone who doesn’t want to waste time getting in to a flame war is soon chased away by the trolls and flamers.

Hopefully this attention will soon translate in to a job with a living wage. Things are tough in this economy, and looking for work is sometimes very frustrating. The only serious interest I have gotten is from people who know about MaraDNS or from connections. Simply putting my resume out there on Monster or Dice so far hasn’t resulted in anything.

Deadwood snapshot update

I have updated Deadwood to fix two bugs:

• Large packets that needed DNS-over-TCP were not working when Deadwood was using full recursion. Fixed.
• It is now possible to have a recursive_acl without the recursive_acl having a netmask (there is a mailing list posting with a description of the issue)

The updated snapshot can be downloaded here:

http://maradns.org/deadwood/snap/deadwood-H-20100802-2.tar.bz2

Deadwood snapshot update

Since Deadwood is in beta-testing, the changes at this point will by and large be bug fixes. That in mind, I am going to start announcing Deadwood snapshot updates on my blog again, since they are more interesting for end users than “I’ve started work on converting the code to handle NS referrals”, “I’m making more progress on converting the code to handle NS referrals”, “I’m making more progress on converting the code to handle NS referrals. No, it doesn’t work yet, but, really, I’m making progress”, “Yet more progress on converting the code to handle NS referrals”, etc.

Today’s snapshop fixes the bug reported in a mailing list posting. [1]

I have fixed the bug, but I’m not 100% happy with the way I’m fixing it, as described in this mailing list posting.

Anyway, people can look at both the patch (in the “update” directory) and the patched Deadwood here:

http://maradns.org/deadwood/snap/deadwood-H-20100728-1.tar.bz2

Update: I have updated this with a better patch that, instead of deciding all SOA records are always “in bailiwick”, has it so bailiwick tests are only done if the record is determined to be a NS referral. See here:

http://maradns.org/deadwood/snap/deadwood-H-20100728-2.tar.bz2

- Sam

[1] The guys at Marc.info need to fix things to recognize a name on the From: line is UTF-8 encoded when it says it’s UTF-8 encoded. “From: =?UTF-8?B?U2ViYXN0aWFuIE3DvGxsZXI=?=” is “Sebastian Müller”, not “Sebastian Müller”.

Firefox does pretty agressive DNS pre-fetching

As I am performing some real-world testing with the (hopefully) soon-to-be-released Deadwood 2.9.01, I discovered something interesting that Firefox does.

The copy of Firefox 3.6.4 I have [1] in CentOS 5 looks at all of the links on a given page I am reading, and uses DNS to look up the hostnames of all of the links. This way, if I click on a given link, I don’t have to wait for the DNS lookup to be performed. I can see why Firefox implemented this.

Anyway, I usually don‘t announce snapshots here, but I am very pleased that Deadwood is finally, after nearly three years, a fully recursive DNS server. Snapshots are here:

http://www.maradns.org/deadwood/

Look in the snap directory for the latest snapshot.

Todo before releasing Deadwood 2.9.01:

• Make a quick Deadwood FAQ
• The usual Deadwood SQA regressions I do before any release

(Footnote follows)

[1] Yes, I know, the current version is Firefox 3.6.7, but I’m using CentOS 5, whose most current version of Firefox is 3.6.4. Out-of-date versions of programs is a very common problem in Linux, since the distributor of a given program has to approve and then send a program downstream before the users of the distribution get the update. It’s worse with CentOS, where a given update has to get approved by RedHat, then, in turn, processed by CentOS before it gets here.

One issue I have had a lot with MaraDNS is that people sometimes email me or post to the mailing list bug reports for issues I have already fixed, but the user in question is using the version of MaraDNS that comes with Debian or whatever, which hasn’t been updated.

Deadwood milestone reached: Recursion is starting to work

Finally, well over two years after starting the Deadwood project, recursion is starting to work. The operative word is “starting”; Deadwood is not ready for general testing yet. However, I was able to resolve “google.com” starting from the root name servers with the latest Deadwood snapshot today, since no glueless NS referrals need to be followed to resolve that name.

That in mind, here is a TODO list for releasing Deadwood 2.6.02:

• Make sure we don't follow any referrals if we're following an upstream server. Make sure RD is set to 0 if talking to a root server, and 1 if talking to an upstream server.
  
• Show an appropriate error message if we try to follow a glueless NS referral or CNAME referral.
  
• Run all of the SQA regressions in the 32-bit and 64-bit compiles of Deadwood, starting with the 32-bit compile. Fix things.
  
• Make a good default hash value for the tarball, and which will be in our source release.
  
• Make sure it compiles in Windows, both under MinGW and with TCC.
  
• Once all of this is done, make the 2.6.02 release: Source tarball, Win32 binary (.zip, .7z), and TCC .7z source release.
  

As always, snapshots can be looked at here:

http://maradns.org/deadwood/