Friday, July 3, 2009

Revised Deadwood roadmap

OK, in light of the hard-to-reproduce crashes of Deadwood, I need to bunker down and harden the code for Deadwood:
  • Make sure the compression code makes no warnings (or leaks) when using Valgrind and compiled with -DVALGRIND_NOERRORS
  • Harden all routines in DwCompress.c; always check return codes and always make sure something is not null before doing a->b type code. Ignore the "52 lines per function" rule when doing this hardening but follow all other coding style standards
  • Once the code is hardened, go back and split up functions as needed to keep everything below 52 lines in length again
The compression code works but there are subtle bugs which result in a crash once every two or three hours which are a pain to track down.

When will I do that? Starting on Friday, July 17th. I am now officially on a hiatus from Deadwood and MaraDNS development. Unless it's a bugtraq-worthy security report, I will not make any changes to Deadwood nor MaraDNS for the next two weeks.

In addition, I will not post any more blogs here for a while.

Have a happy 4th of July, Americans, and a happy 2-week summer break everyone!

Deadwood 2.4.04 release delayed

There are still bugs in Deadwood that make Deadwood segfault. I've modified the code a little to give me verbose messages and will see if I can recreate the bug. Until I can find and fix this issue, I'm not going to release Deadwood 2.4.04.

- Sam

Deadwood snapshot update: Some more compression bugs fixed

I have fixed a couple of bugs that made some DNS packets not decompress-then-recompress properly, and have released a new testing release of Deadwood. As before, hit on this release and lets see if we can find any other DNS packets which make Deadwood complain the packet doesn't decompress or recompress properly.

I will run this for a few hours; if I don't see any more problem packets, I will make the 2.4.04 and release it early this afternoon.

It can be looked at by going to maradns.org/deadwood/snap.

It's nice to be wrapping things up with the decompression/compression code; DNS compression is a royal headache and one of the parts of DNS I loathe the most.

Thursday, July 2, 2009

Testers wanted for Deadwood snapshot

I just made a Deadwood snapshot update which is intended for people to more widely test Deadwood. I would like more people to test the compression code with Deadwood.

Testing release: The compression code does not affect packets coming to and from the resolver. However, every time Deadwood gets a DNS packet, it decompresses then recompresses the packet. If the recompressed packet differs from the packet received from the upstream DNS server, Deadwood logs a "WARNING: Compression problems with packet" error, followed by the DNS packet causing problems. This log message is then followed by the length of the compressed packet, the packet as actually compressed by Deadwood, and finally the string value and length of the question.

This allows me (and anyone who wishes to help) to test Deadwood's compression core with real-world DNS packets. In addition, I will make some acid tests for the compression core (tests like making sure we never compress SRV RRs but can understand packets with compressed SRV records, etc.) which I will use to further test Deadwood's compression core with.

Since it would be nice to see some wider testing in this snapshot, I have made both the source code and a 7zip compressed Windows binary of the snapshot available here.

I will release Deadwood 2.4.04 when I'm satisfied that testing has ironed out the bugs in the compression core.

- Sam

Deadwood update: DNS compression code finished

I have finally finished up the DNS compression code for Deadwood.

The reason for the slow progress is because my personal life has been a lot more busy and I have had less time to devote to Deadwood. This will not change for the foreseeable future and Deadwood progress will continue to be slow.

In today's snapshot, the underlying compression and decompression libraries work. Now I have to go through and make sure they are bug-free; it's a matter of a bunch of SQA testing at this point.

People can look at the code at maradns.org/deadwood/snap

Wednesday, July 1, 2009

Memories of Usenet

Usenet is dead.

Usenet has been dead for a few years now.

Yeah, sure, there is this thing out there that works like The Pirate Bay using NNTP instead of BitTorrent packets, but that isn't the Usenet I knew and loved in the 1990s.

Usenet is dead.

Sure, you get a few crotchety oldtimers who get their panties in a bunch when you mention that Usenet is dead, just as you got people who denied the Amiga was dead for years and Japanese soldiers fighting World War II as recently as the 1970s. People often react to a loss or defeat with denial.

The last time I significantly used Usenet was back in the early 2000s. Before the college I was going to at the time set up Wi-Fi everywhere, allowing wireless internet access from my laptop, I would use Leafnode to download a number of Usenet newsgroups I would read during the day. I dialed up to my ISP (yes, I had dialup as recently as 2003), downloaded all the newsgroups I wanted to read, uploaded any posts I made while offline, and updated my local Usenet spool. I could then, over the day, read my daily set of new Usenet postings and post replies as appropriate.

I still kept this setup to read Usenet after they added Wi-Fi at school. I finally canceled my dial-up ISP, then did things in reverse: I would upload and download Usenet postings at school and read Usenet offline at home.

By then, alt.hackers, a fun little group about clever solutions to problems in the mid-1990s, was comatose and in its death throes. rec.audio.pro, a newsgroup discussing methods to record audio, was still alive and well, but would soon get replaced by Gearslutz. comp.os.linux.* was already overtaken by spam and flame wars; Linux users had taken their discussions to /., Kuro5hin, among other places.

By 2004, I realized that little worthwhile discussion was still taking place in Usenet, and moved on to web-based discussion boards. The way to be a part of a community on the internet changed in the 2000s; I had my first blog in 2003 (back then, it was called a "journal"), got a MySpace account (my first social networking account) in 2005 and started this blog in early 2007.

Usenet is dead.

I remember the day I discovered usenet. It was in the fall of 1993, and I was at the computer lab of a university I just transferred to. I was on a Macintosh and saw this icon marked "newsreader", opened it up, and discovered Usenet. It was incredible. I could use a computer to connect to this worldwide network of computers and talk with anyone in the world about any topic I wanted to talk about. It blew my mind away.

I soon learned UNIX and the TRN newsreader because I didn't have a computer at the time (this was when a basic computer would set you back $2000; these days a basic notebook or desktop is about $400) and wanted to be able to read and post to Usenet all night--the only computer labs open all night were running UNIX.

The skills I learned spending so much time reading Usenet translated in to me getting a job at Netcom, then one of the most prestigious internet providers, a couple of years later. I saw the internet experience an explosive growth in the late 1990s, with Mosaic and later Netscape giving the internet a user-friendly GUI, but still read and posted to Usenet using the TRN client on my computer running Linux.

In was around late 1997 that I first discovered a forum I liked that wasn't a Usenet forum, a place where men and women both talked about relationships, sex, and personal details about their life the way nobody in Usenet talked about their lives. I found the place wonderful, particularly since it had a lot of girls; something Usenet never really had.

I also was told by a co-worker about Slashdot, got a 4-digit account in 1998 and starting reading and posting there.

I continued to read Usenet for many things throughout the 1990s, but with the discovery of web-based forums in the late 1990s, my interest in Usenet started waning and the number of interesting discussions Usenet used to have (or never had) moved to the web. I had a friend who gave me access to Usenet II in 1998, a late-1990s attempt to revive the Usenet of the early 1990s, but that never went anywhere and was little more than a hierarchy of empty newsgroups; the only active group there talked about Usenet II and making sure Usenet II postings did not make it in to ordinary Usenet.

Usenet is dead.

The last really interesting thing to happen with Usenet was when Google was able to, in 2002, recover the majority of pre-1995 Usenet postings from archives individuals had. It was interesting to finally read entire legendary ancient Usenet threads, such as the legendary 1992 "Linux is obsolete" debate and see what people had to say about BIND and other DNS servers in the 1980s and early 1990s.

Usenet is dead.

It was fun while it lasted, but the internet has moved on and, in truth, Usenet really wasn't that great. There was no moderation, so no way to keep flame wars or spam under control. It was a place with a lot of arrogance and elitism; a place where experienced users took a sadistic delight in flaming newbies (this was even worse in IRC, the 1990s version of MSN and instant messaging); a place where finding an answer to a technical question was a hit-and-miss affair. A place without graphics or multimedia; the interface was nothing more than ugly fixed-width text on an 80-column screen.

But, it was the best we had at the time and I thank you for letting this aging geek talk about the way things used to be.

Usenet is dead.

It's time to move on. I have a lot of things today I didn't have when I was a Usenet junkie in the mid-1990s: A girlfriend who loves me and who I love; a community of real friends in the United States who I keep in touch with via Skype and social networking sites. I also have an open-source project I am eager to finish up.

I just looked at the code for Deadwood, and I haven't quite finished up the compression code. Things look good, however, and I just need to do a little more last-minute touchup before the compression code is done.

Tuesday, June 30, 2009

I've been updating Deadwood

I've been updating Deadwood; right now all of the DNS compression code is written and now I just have to go through and debug the code until it works. This should take a couple of days. Once Deadwood is able to compress DNS packets, I will change the code to store packets internally as uncompressed packets, then release Deadwood 2.4.04 with a different, incompatible format for the DNS cache written to a file (since DNS packets will be uncompressed internally).

After that, Deadwood 2.4.05 with RR rotation and TTL aging.

People can look at Deadwood here.