::::: : the wood : davidrobins.com

My name is David Robins: Christian, lead developer (resume), writer, photographer, runner, libertarian (voluntaryist), and student.

This is also my son David Geoffrey Robins' site.

DTI Data: case concludes on a high note

News, Technical ·Friday November 16, 2012 @ 13:24 EST (link)

I mentioned earlier some dissatisfaction with DTI Data, the company that I used to recover my lost disk. I went in today to return the last loaner USB drive they had given me, with the raw disk image on it rather than their rather poor attempt at recovering files. I called first, since they said they were trying another slower recovery method to recover more files, but that had not yielded any fruit (again, they had not called me, and I don't think they ever did, no matter what they promised). I talked to the technician and explained how I had used the second superblock to mount the drive (and was surprised their software had done so badly) and had been able to recover just about everything using that and various disk scanning to find orphaned files and directories.

Since, he said, I had to do most of the software recovery myself, he gave me the 1T Toshiba USB disk that I was returning. That was pretty good of him, and likely accounted for the cost of software recovery, with hardware recovery—that I couldn't do myself—being the bulk of the total cost and the value-add they provided me. That did make me feel better about the company; it was a nice gesture. I would recommend their services; just stay on top of them regarding communication, and realize their Linux (ext2/3) filesystem recovery at least is somewhat lacking (it'll strew files all over the place and miss a lot). (However, I know both filesystems even better than I did before, and in the past I have implemented drivers for them; if you have a Linux disk that needs recovery, if you throw enough money at me I might be able to help you out….)

Books finished: A Dance With Dragons.

Downtime due to disk failure

News, Technical ·Saturday November 10, 2012 @ 03:12 EST (link)

The server's primary disk, a 500GB Seagate SATA drive, crashed hard around October 24. The kernel image loaded but the main partition would not mount. Attempts to copy the disk to an image to work on it were continuing to time out. I searched around and found a local data recovery company, DTI Data Recovery that seemed to know what they were doing (I shopped around a little and another one I did talk to for a while gave me no confidence at all).

DTI Data occupies offices squeezed into a side tunnel in a strip mall in South Pasadena, about 20 minutes drive from home/work. It's convenient that they're that close, though; turnaround time by mail, which seems to be usual for many of their customers across the country, would be much worse. We brought the troubled drive in on October 26, and were quoted two prices: $495 if data could be retrieved without opening up the disk, and $895 if it had to be opened. The second one turned out to be necessary.

The first data I got back—on a 1TB Seagate drive I provided—was a real mess, and it only had /home on it. Directories and files were scattered all over the place; for example, photos were in my mail directory at random locations; at first I thought the drive was just that scrambled, and figured on reorganizing them back based on file timestamps and EXIF data. Much more data—they said all of it—was available, and apparently there had been an internal miscommunication. The paperwork required filling out vital directories, so I listed several, with /home first (and /var, /etc, and others, and then "everything you can find"), and someone apparently translated the list as "he only wants /home". DTI Data was also not very good about communicating with me; I generally had to call them to get any updates; once I was specifically promised a call later that day, but then the recovery ended up taking until Monday, and another time a call back in a half hour, but nothing for three hours until I called them.

They did recover some more data the next time, but there was still a lot scrambled and missing. Was the disk really that bad? I eventually asked for an image—a block-by-block copy of the disk, rather than recovered files, which they gave me, as with the recovered files, on a Toshiba 1TB USB drive. They warned me it wouldn't mount, but I had figured on that.

It turned out it did mount with the most basic data recovery due diligence: use the second superblock (short but key drive information, which is fortunately duplicated in every group, of which there are about 4000). First I had played around with it a little using libext2fs (from e2fsprogs) (old but relevant PDF documentation), skipping to the second superblock in the ext2fs_open call, reading the root inode, etc., and figuring the disk wasn't in completely terrible shape. I ran dumpe2fs to dump the superblock and group descriptors for reference, and played around a little with debugfs. I couldn't mount the image as ext3, which it was, because the journal was corrupt (even with noload), but mounting it as ext2 and skipping to the second superblock (read-only, of course) did the trick and I was able to access most of the files. (At this point I was working on a copy of the image on another 1TB Seagate SATA drive I had bought: copied using rsync, it took 4h21m to copy the 500GB image file from the USB drive.)

In particular, files that DTI had missed completely or scrambled into the strangest locations were plainly accessible; I worry about the recovery software they're using. I ran a find . -type d from / and got a basic idea of what inodes were bad (at this point after considerable investigation I believe damage is concentrated in a couple blocks worth of inodes (32 per 4K block). This is painful for the files that are lost, but then, recovering orphans is just mark-and-sweep (figuring out which goes where is tricker).

By this time I had set up a new Gentoo install on yet another 1TB drive and had it running well enough to provide NAT (local networking) services (and access it from my laptop wirelessly). But there was a long road to go to restoring web and mail even if most of the files were there.

For directories—most of which were either caches, which don't matter, or mail files, which do, since the . entry at the start points back to the directory's own inode, tracking down most was easy (although an entire scan of the disk, even just reading the first 24 bytes of each block, takes 43 minutes); I also logged children of the bad inodes (matching the .. inode). I ran another scan using a heuristic and then boost::regex (g++'s regex support is terrible) to find blocks that looked like Maildir directories (only a few of the missing directories were more than one block, and not all of their block entries were bad). From there it remains to determine the known good blocks, subtract them from those found, and figure out which directory the found blocks belong to (order doesn't really matter; filenames do not cross block boundaries—good design there, Rémy/Theodore).

Services are gradually coming back up: the Apache servers (dynamic and static) for davidrobins.net were first, with i4031.net, my resume, internal address list (needed some recovery), SSL (likewise), my Voluntaryist Wiki, and Code Visions. The next big one to tackle is mail, which involves several programs and their config files (which all need to be copied over or merged appropriately from the old mounted image): Qmail (and Fastforward), Courier-IMAP, Maildrop, Mutt, and Horde/IMP (for Honey). The work continues.

Books finished: Smart and Gets Things Done Joel Spolsky's Concise Guide To Finding the Best Technical Talent, Peopleware.

Kayaking

News ·Sunday September 30, 2012 @ 20:04 EDT (link)

I went back to Lettuce Lake regional park again—just me this time—to kayak for the first time. $2 entrance, $25 kayak rental. I had to let them hold on to my driver's license, and picked up a paddle, life jacket, and key at the entrance station. I put the paddle in the trunk but it stuck out so I used a tie-town. Walk down to the launch, unlock the kayak, put it in the water; about the same as a canoe, as was getting in: it's all about staying low and maintaining balance (after I cleared out or killed the large spiders in the back of the kayak). (This was a sit-on-top kayak.)

There really wasn't much to it: grasp the paddle with the concave edge of the blade toward you and alternate left/right strokes. Sometimes I'd double-paddle on one side to help turn, or back-paddle; but it later became easier to take a stronger stroke on one side to turn while maintaining rhythm. I was out for a couple hours; I went past most of the boardwalk, including the lookout. Saw several wading birds and ducks close-up; no gators; no other kayaks out, but a few canoes. Altogether a good experience.

Open source = ability to fork

News, Technical ·Saturday September 29, 2012 @ 15:56 EDT (link)

My response to a TED talk How the Internet will (one day) transform government by Clay Shirky:

The fundamental misunderstanding between open source projects and law is that open source projects are private property (and by that I mean the main repository, not the content) whereas law is imposed on people by force. This so extremely dwarfs how either are produced that it amounts to a red herring, a mere distraction. And it's shatteringly depressing that he almost got it, too; because if he had talked about forking (where someone takes a copy of an open source project and makes a new "main" repository going a different way, because they disagree with the existing owners), then he would have almost made it.

If you talk about open source and don't talk about forking, you don't understand open source. Even if an open source project never forks, the ability to do so—the threat—keeps the keepers honest. And even if a fork doesn't attract many people, or is entirely private, it represents ultimate control over the configuration of a project that you run on your system without having to be dependent on anyone else's vision.

So all he's really got is: law and its composition (inputs) should be more open, such that it's clear where inputs come from, and, in the other direction, what its effects are. But that doesn't change the fundamentals of democracy, which is majorities (or plutocracies) forcing their will on other individuals.

If he actually took the lesson of open source projects to heart, though, he'd be able to make a much more powerful point. First, he needs to understand that open source projects are not democratic: they are focused on a privately-owned main repository. Sometimes democracy (but not of everyone, but of people admitted to the inner circle, usually from the value of their changes, i.e., on merit) is used, but it is far from central. Linus (and other open source project leaders) are referred to as a BDFL—Benevolent Dictator For Life. But the ever-present threat is that someone can take the code and start a new "main" repository and that he'll be the dictator of a kingdom of 1, or at least that there will be kingdoms outside his control.

Now, what do you get if you apply "forking" to law? Recall what it means in the software context: if a person doesn't like the "main repository" of a program, or that their ideas are not being incorporated, they make a copy, publish it as a new "main repository" (usually under a different name to avoid confusion), possibly attract others that prefer their version, and, most importantly, use that version of the software. For example, if people thought that Apache (a web server) should instead be a mail server (silly example), then they could branch it and call it "Comanche" or something, and start working on it, and they could run their version on their computer and nothing compels them to keep running Apache. (And since Apache's changes are still being published, if they see changes they like they can incorporate them back into "Comanche", and vice versa.)

So apply that to law: if a person doesn't like the main repository of law (the state's version), or their ideas aren't being incorporated, they could make a copy, publish it as a new set of laws ("Dave's Law", which, say, strips out all victimless "crime" laws, leaving the other 12 pages alone for now), possibly attract other people that like Dave's Law, and, most importantly, only _follow_ "Dave's Law". (Note also that a person doesn't have to be a contributor to a project to use it, and similarly, a person would not have to contribute to "Dave's Law" to elect to follow it instead of the state's law.)

To answer our original question ("What do you get?"), the answer is, of course, anarchy. Possibly, just done exactly like this, anarchy in the sense of "chaos" as well as "no (imposed) masters". For example, (as written) people could opt out of murder laws and, if law is the only reason a murderer can be brought to justice (it isn't; see the Voluntaryist wiki page on Justice), there would be no penalty. So clearly that wouldn't quite work. In fact, anyone about to commit a premeditated crime would naturally opt out of the law against it! Yet this works both ways (somewhat like Molyneux describing someone dropping their DRO coverage, which you can read about in his book Practical Anarchy): it is a signal that some mischief may be planned. Others might refuse to do business with someone who has chosen to be bound to a system of law that they disagree with. And which law would the police enforce? It's easy for victimless crimes: that of the actor (if you choose to follow a law that says you can be jailed for 10 years for using drugs, then so be it, but if I don't, then leave me alone); what about at the boundaries (my law says I can sleep on your lawn, your law says you can shoot me if I try)? (Butler Shaffer's Boundaries of Order comes to mind: rights are all about boundaries.) (Especially consider if my law says I don't need to pay taxes to fund your police….)

So two things would come from this: distributed law (see, e.g., Hasnas' "The Myth of the Rule of Law"), and an appreciation that justice (right to restitution and retribution) does not come from arbitrary words on a page. We'd be getting into an anarchist private law (DRO, private protection agency, rights enforcement agency, etc.) system.

That is the parallel to distributed version control, not merely that people can see where the bits of text come from and who contributed. The guy misses the forest for the trees.

Upgrading the Myth box

News, Technical ·Friday September 28, 2012 @ 17:32 EDT (link)

The "Myth box" is what we call the computer connected to the TV that we use to play most shows and movies, because it runs MythTV (and XBMC for movies, with a planned cut over to XBMC for both shows and movies when I get around to it); its "official" machine name, is (following the theme of mythical locations) cirith-ungol. Anyway, a little over a year ago, while we were still in Washington state, the old case fan gave up the ghost, causing some overheating issues with the motherboard (including some visibly burst capacitors) before I replaced the fan (records indicate in July 2011).

However, the machine still ran well for the most part, so there was no need to replace anything else. But lately it had been acting up again: freezing up at boot, or not detecting disk drives, stopping at other points in the boot, or, when it did manage to boot, losing contact with the disk drives at a later point. For a while, connecting the disks to different SATA ports made the problem go away; but eventually the machine was so unstable I took it to Genie Computer Systems, a local computer place, for evaluation. I paid them $50 for diagnosis, and they told me it found the disks just fine and that my "Linux install was corrupt". OK; we developers have all seen "not repro" bugs before—it fails for the tester but works fine on our machine, either because the steps aren't quite right or the environment is different. So I can't fault them for that. And they did some work, so I don't resent paying them. But the "Linux install is corrupt" diagnosis was weak and unwarranted. If you don't know, say so! (It also took them a day or so longer than expected to come up with this useless conclusion.)

So, it was time for a new motherboard! I decided to stick with MSI, and did a little research and decided to go with the 990XA-GD55. I found out that neither the memory nor CPU were compatible, so to that I added an AMD Athlon II X2 (Dual-Core) 270 and 8 Gb of memory; I was going to go with some DDR3 1333, but Newegg (online retailer) had a sale on Team Xtreem Dark Series 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800) for a little less than the other. All told, just over $200 for am upgrade that was rather long overdue anyway. We anxiously awaited the package from UPS….

The Newegg box arrived, packed with their usual care; so I set out removing the old motherboard (an MSI K9N Platinum): I took some pictures in case I had trouble remembering what went where, but I didn't end up needing them. Power, USB, power/reset switches/LEDs, SATA cables; all removed, then unscrew the board and ease it out. Replace the rear connector cutout. Install the new memory and CPU in the new motherboard; easy enough; don't get any thermal paste on you! Now, ideally I should have been powering on the board at each step to check for failures; but I didn't; so I had everything attached—SATA cables were a bit awkward since they were at the far side, but I could get to them through the other side of the box. The 12V power connector I had was 4-pin and the board had an 8-pin socket, but apparently it was fine to connect it up where it fit. (Tom's Hardware Forum was one of the most frequent hits with good information, plus a few overclocker forums (even though I wasn't overclocking the CPU).)

I connected the power cord and a video cable and flipped the power switch.

Nothing: no fan, no buzz from the optical disks, no LEDs.

Great, what did I break? Did I kill it? I didn't see any magic smoke escape!

I even pulled out the multimeter and checked the voltages from the PSU (Power Supply Unit): all within tolerances. I was all set to RMA (return) the motherboard as a dud—after all, the old motherboard came right on when powered on—when I had a thought about checking if it required a power switch… and so it did, so I shorted pins 6 and 8 of "JFP1" and the thing whirred to life. My fault for not knowing that first, the board could be powered on without anything attached (even memory or CPU) (although it wouldn't be happy about it), and not incrementally testing it with each addition, and, second, knowing that the power switch connection was required on in this board. (Later I was able to set the system to automatically start when powered on, through the BIOS.)

The system booted past GRUB, loaded the kernel, but was unable to find any disks. The new motherboard didn't have an IDE connector, so I couldn't boot from a LiveCD/DVD; but fortunately, it could boot from a USB stick; so I made a Gentoo 12.1 LiveUSB from a LiveDVD ISO (just dd it!), and built and installed a new kernel with AHCI support (fortunate educated guess) via a chroot; and then it got a good deal further, but failed to bring up the network interface.

This was also obvious: the new motherboard has different on-board network hardware—old was nVidia nForce "forcedeth", now it was a Realtek chip needing the "r8169" driver (as you may guess, I build a custom kernel and don't build random stacks of modules). Even after setting that up, it wanted firmware (according to dmesg), so I emerged the linux-firmware package from the chroot; and (modulo a tweak so that udev didn't reassign the interface as eth2), finally it boots correctly!

And it's been stable for the last few days. It's nice to be able to watch shows again (I hear that too much reading rots the brain).

Lettuce Lake

News ·Sunday September 23, 2012 @ 10:01 EDT (link)

We just got back from walking at Lettuce Lake Regional Park; small, but has some scenic boardwalk trails through the swamps, with great old trees, hanging moss, and a few gators. There's a lookout, too, and canoe or kayak rentals, which we may take advantage of next visit.

Books finished: A Feast For Crows, Counter-Clock World.

AAA x2, new battery for the Solara

News ·Thursday September 13, 2012 @ 20:57 EDT (link)

Had to call AAA yesterday morning (dead battery); they charged it with a portable battery pack, but since there was no reason for it to be low (no lights left on or anything like that) I had my doubts about its long-term survival. I managed to start it to drive home from work, but barely—it was clearly Not Happy—the drive in isn't really long enough to charge it; and this morning it again wouldn't start. (I should say it wasn't actually dead; there was enough juice to turn on a few indicator lights and make a clicking sound.) So this time I bought a battery from the AAA guy—about $120 with a few forms of state extortion added in (obviously if it would hold a charge I didn't want to buy one, but as I said, I had little hope).

Both of the AAA guys said that Florida heat killed batteries—they had a couple pallets of dead ones each of the past few days. And maybe it was just time; it's been a couple years. Waiting on them made me a little late to work (around 1030) but I logged in remotely a little to make up for it.

Code Visions technical blog goes live

News, Technical ·Saturday August 25, 2012 @ 21:51 EDT (link)

I decided to split off a technical blog on software development, with an eye to improving software quality (for customers) through better software methods (the code we write, how we write it, how features are designed, test methods, etc.). It's called Code Visions and I hope to be contributing to it frequently. (And now I can keep my technical and political/personal posts separated.)

Books finished: The Honest Truth About Dishonesty.

Hurricane Isaac: batten down the hatches…

News ·Saturday August 25, 2012 @ 05:59 EDT (link)

Hurricane Isaac is approaching; we just went to Wal-Mart to get some "emergency" bottled water—it wasn't busy and had plenty of supplies.

At work, they put up some metal storm shutters over the high windows and had us all drape plastic over our electronics (computers, mainly, and any of our peripherals intended for blind users), and ensure electronics were off the floor.

It's expected to hit further north—Alabama or Louisiana—but time will tell. First hurricane, it's a little exciting; don't really know enough to be worried (ignorance is bliss).

The state extortion machine, hard at work (II)

News ·Saturday August 18, 2012 @ 17:09 EDT (link)

Honey writes: (continued from part 1)

On August 18, 2012 I had to go to court for the supposed seat belt violation that took place on May 31, 2012 on the way to my doctor's office. There is a separate write-up for the events that took place on May 31 and a write-up of my complaint of Officer McKenzie to his supervisor Sgt. Leschick.

David and I went to the court house at my specified time (around 4:00 p.m.) on August 18, 2012. I went in to the courtroom armed with my photos and notes about the incident that took place on May 31, 2012. We sat there and waited until we were called into the court room and the names of people that were going to "trial" were called alphabetically by last name. Several people got off and according to my memory most of them were in fact men. The women tended to be treated unfairly by the judge in my honest opinion.

Finally they got to the R's and the man who went before me was there for a seat-belt violation as well. He told how he was wearing his seat belt and taking his kid to school. He mentioned how he had never gotten a ticket before and the judge let him off. Then my name was called and I went up and told my side of the story. I didn't notice until it was too late that the police officers changed their stories from what Officer McKenzie had told me at the stop to what they told the judge during court. Office McKenzie had told me on scene at my stop that he saw me putting on my seat belt while being pulled over. However, in court he said that my seat belt was not seen and then he and Officer Jacobson (the actual "witness" to the violation) both said that my seat belt looked as though it was on, but tucked under my arm while I was driving. I missed the change in story (mostly because I didn't think they would lie), and told the judge that I had been wearing my seat belt the entire time and that I am just too short to see the seat belt from outside of my vehicle unless you are right next to me. I proceeded to show the officers and the judge my pictures showing how difficult it is to see me with my seat belt on (unless you are looking closely). The judge didn't even bother to look at the pictures, he just flipped through them as though he didn't care. I also mentioned that the officers didn't bring their "evidence" to court with them (the video tape and the audio recording) and the judge didn't care. The officers mentioned that my seat belt was not visible from behind the vehicle, but that it didn't matter because they had both seen me from the side and I didn't have my seat belt on (or it was tucked under my arm).

The judge told me that he was going to fine me for not wearing my seat belt but that as consolation, that this wouldn't go against my driver's license and would not increase my insurance (as if that should console me) and that he would not make me pay court costs either.

The real reason that Officer McKenzie did not want to bring the video/audio tapes in is because it would show him to be the liar that he is. It would show how rude he was to me and how judgmental and how he just dismissed me without bothering to care how I felt about things. It would also shed light on his statement that I was lying about wearing my seat belt because he saw me put it on while I was driving. He didn't want his lies to be shown for what they were so he conveniently did not bring the so called "evidence" with him because he didn't want to be seen for the villain that he is.

Officer McKenzie had a vendetta against me because I filed a complaint against him and he was getting his revenge by having his buddy Officer Eric Jacobson (I think that's his name) lie for him and back him up under oath.

This entire situation has made me realize how totally unnecessary traffic cops are under most circumstances and how voluntaryism is the answer! I have been extremely angry for almost a year now and nothing can ever give me my innocence back about this. I truly believed that if the truth was on my side that I would not be treated this way, but in the end it was a money grab and they (the officer's and the judge) did not care at all that I was wearing my seat belt, they just wanted to get their money.

What is more, I had to sign a paper in court admitting to the violation and then I had to go pay the fine. I had to lie in court on a piece of paper handed to me by the state and I was told that if I didn't sign it that I wouldn't be allowed to leave. I was forced to lie and pay the fine when I should never have been in that situation in the first place.