Unifying Contradiction
[Recent Entries][Archive][Friends][User Info]
Below are the 20 most recent journal entries recorded in the "Unifying Contradiction" journal:[<< Previous 20 entries]
|
04:18 am
[Link] |
Ouch
I woke up after yesterday's party feeling fine. No hangover or anything. However, as the day progressed I noticed my right shoulder getting stiff, followed by the right side of my neck and back. Even after a few hours application of a heating pad to the area in question, moving my head is both difficult and painful. I wonder how I'm supposed to sleep like this.
|
|
11:02 pm
[Link] |
Torchwood.
Just saw the last of this season's Torchwood episodes. It was dark, dark, dark. Really not sure where they think they're going with the series. Right now it looks like the existence of a 4th season is still up in the air. They sure seem to have a strange way of running series in the UK. (Not that N.A. doesn't have its share of WTF decisions in TV-land.)
|
|
06:45 pm
[Link] |
Tree?
The Linux 'tree' command conveniently outputs a display tree of files and associated data. That's just what I want, only I want to be able to provide my own (potentially very large) list of files. Anyone know how to do that? Tree only seems to accept file names on the command line, and I'll quickly blow through that.
|
|
01:24 am
[Link] |
Argh.
Helped Terry move in today. I left there at 9:30-ish and figured I'd get home by 11:00 pm, even on a Sunday Schedule. Well, I just dragged myself in the door. What with traffic jams due to the St. Jean Baptiste celebrations and a drunken fight on our bus which caused the police to be summoned, and the fact that they didn't get us a replacement bus, and the next bus never came, it took so long to get home that my transfer expired and the Opus system charged me another token.
I think they're gonna get a letter from me at the STM. THEM fucking up shouldn't mean that *I* have to pay extra. In fact, it should be just the opposite.
|
|
04:27 am
[Link] |
Spurious Sparseness.
I'm STILL working on recovering the dead server. Part of the problem has been a lack of adequate tools. I must say I am surprised at how hard it is to find tools to do the (I would think) relatively simple tasks required.
My latest hurdle was that reiserfsck can produce spurious sparse files when recovering a file system. How spurious? I have several hundred files (in a tree of hundreds of thousands) each claiming to have sizes in the Hundred Terrabyte to Hundred Petabyte range. Normally I would just shrug and tell the various copy programs that can handle sparse files to copy them sparsely. After all, there might be a few blocks of useful info in these files.
Well, that's not possible. Try as I might, I could find no 'sparse-aware' program that could process the allocated blocks in a sparse file in less than a time proportional to the reported size of the sparse file. For multi-Petabyte files, we're talking weeks to months. Each.
So, not being able to even find a tool to copy them, or to even selectively delete them for me (which baffles me -- Find will report the 'sparseness' of a file, but you can't test the value and perform an operation contingent on its size. In fact find seems to have no inequalities whatsoever, and you can't just pipe the info into bash because bash doesn't do floats).
In the end I took three days to find out what I could about linux filesystems and sparse files and how to write a recursive descent walker over a filesystem that can handle randomly-generated symlinks (because reiserfsck generates them too) and I'm now just deleting all these impossible-to-process files with a hand-written perl script. Which kind of scares me as I haven't done a lot of mucking around in linux filesystem internals, and I'm doing recovery work here.
What gives? I expected there to be a VFS API specifically for returning a map of the allocated blocks in a sparse file, but I could find no such thing. Without it, there's not much I can do. So, I am deleting them.
It shouldn't be the end of the world if there IS some important data there, as I'm going to regenerate the entire file system three times, once per pair in the three partitions, and merge the results (which is also going to be fun -- whee!).
So far, I'm almost done the first copy but I expect the 2nd and 3rd will go much faster as I now know what tools I will use to do what.
|
|
04:05 am
[Link] |
Dreamwidth Invite
I've got a spare invite to Dreamwidth. Does anyone want it?
|
|
04:29 pm
[Link] |
Drunken Debauchery.
It appears that ![[info]](http://l-stat.livejournal.com/img/userinfo.gif) taxlady and I shall be joining the regular gang at Hurley's tonight.
|
|
03:07 am
[Link] |
Progress
Sometimes progress consists of working out what it is you've been doing wrong for the last week. So, roughly a week lost in the reconstruction of webigail's data while I bashed my head against the wrong wall... Hopefully now that I've worked out what I should have been doing, things will go faster.
|
|
08:13 pm
[Link] |
Not My Day.
So, I'm in the middle of rebuilding the server and I start getting strange errors. Since there are known bugs in the VT8237r chip that's in the server, I immediately suspect that its having trouble with talking to two 1TB drives at once, or one of the data channels is bad, or something.
Nope, in the end it turns out that one of the three brand new drives was failing and has now completely failed. Even the diagnostic software from Western Digital says (in so many words) "Its toast -- ship it back".
Luckily, it wasn't the drive that I did 90% of the imaging to. I still have the majority of the image data. Alas it DID contain the image of the largest raid array in assembled form. I have all the individual partition images (plus whole drive images -- I get paranoid when doing this stuff, and I hope you can see why) but it would be a royal pain to rebuild the array image by hand, and I dislike remounting the failed drives except as a last resort. Luckily, I've been told that Trinity Rescue Kit is a LiveCD that contains software intended to automate building assembled raid images from partition images, so I'm going to try it out. If worst comes to worst, I've just lost a few days work.
But still: GRUMPH!
|
04:08 pm
[Link] |
-1 for Documentation, but +2 for Stable Methodology.
So, I am in the middle of reconstructing a broken raid array, copying the images onto new media and essentially resurrecting my mail/web server. Actually, I was in the middle of all of that last weekend when I ran out of time and had to concentrate on other things that were momentarily more pressing.
So, today when I finally manage to get back to the job, I find I have 3 new drives, all 1 TB, all visually identical, some partitioned, some not, and I realize I haven't documented a single thing I was doing last week. Now, typically, if I am working on a server with critical data, like this one, I keep a log of what I'm doing as I go along. Both to make sure I didn't forget anything crucial, and to ensure that if I ever have to go back and solve a similar problem on that hardware, that I have a record of what worked and what didn't.
Well, I've been rather under a lot of pressure lately, and quite frazzled, and so I completely forgot to maintain a log of my work. At first I was completely dismayed. Why had I set up the partitions the way I had? What was on them? Were they raid images? Filesystems? Unformatted?
Then, luckily I was distracted and had to help taxlady put away groceries. While I was at it, I mentally started at square 1 and tried to decide how I would typically solve the problem of rebuilding the array. By the time I had gotten stuff put away, I had worked out a reasonable procedure in my head. Now, returning to the job at hand, its obvious exactly how far I had gone, and what I had been up to when I had to stop. Even the names I gave to certain mysterious image files suddenly made sense. Lets hear it for having a stable methodology that comes up with the exact same solution to a problem, when given the same inputs twice.
I shall now proceed to continue from where I left off last week.
|
04:35 am
[Link] |
I know what I want for Christmas.
I've recently been rereading the Godstalk series of books by P. C. Hodgell. Truly, they are amongst my most favorite fantasy novels ever. This time through I'm amazed to notice how many things I thought I had come up with for my own RPG writing was inspired by, or directly borrowed from these books.
I actually own a signed, limited-edition leather-bound set of her writings as this was the only way I could get my hands on "Seekers Mask", the third book in the series.
On a whim I decided to check up on the books on Wikipedia, under the assumption that the series had come to an untimely end since it had been so long since the last book came out (1994). Well, it seems that the series hasn't been abandoned, nor has P. C. Hodgell passed away as I was beginning to fear. She released a fourth novel in 2006, that I had never heard of, and has a fifth one due out any time now.
I am SO happy about this, you cannot believe. I started reading these books when they first came out, when I was just out of high school, and am still in love with them now some 25-or so years later. I'm going to be picking these books up as soon as I possibly can -- in hardcover, money permitting.
|
|
11:37 pm
[Link] |
Waking Up in Light.
I just slept for 12 hours. I could, perhaps, have rolled over and gotten some more sleep, but that would have been inviting a migraine, so I did the prudent thing and got up. I had been awake for the previous 24 hours and had only had 6 hours of sleep the night before, but I'm still surprised I slept that long and solidly considering how much coffee I had in me when I went to bed. Usually that leads to short and broken sleep, but I guess I was tireder than the caffeine could interfere with.
In any case, I awoke to a better day (evening?) as the dishwasher, which has been broken for weeks was repaired under warranty while I slept. In addition, the work I stayed up all night to finish is 99.44% done. It would have been completely finished but the client called at the last minute to delay picking it up by a day, so we collapsed rather than do the last little part. I'll probably finish my part of it during the night, but that is such a tiny amount compared to what I've already done on it that I don't anticipate it taking very long (famous last words, I know). EDIT: And now I've finished that last bit too. Yay me!
So, for the last month or so Linda and I have been simultaneously dealing with health issues in my family, health issues in her family, Linda having huge backlogs of work that needed to be done before we could get paid for it (which I've helped out with, wherever a non-bookkeeper can), a broken mail/web server, a corrupted hard drive containing my most precious backups, a non-functioning TV system in the bedroom, an internet connection that resets every few minutes and a monitor on my desktop machine that is giving signs of imminent failure and making it very hard to work on at times.
All of these are small enough problems that I could have easily (for some value of 'easy') dealt with them one by one. All at once it has been a huge burden. Adding all of these to the fact that this happened while Anticipation was looking for me to do some work for them and while I was writing up the GM notes for the latest game that I've running (which is supposed to lower my stress levels) and my stress has been through the roof. Its been so bad that I nearly had a breakdown when I had to deal with our friend Terry's moving crisis in addition to everything else.
Anyway, at this point I am thinking that the worst is over and that I will now start to see my problems get resolved faster than they occur. That would be good. I had planned to start spamming the world with resumes at the beginning of May, before all of this hit. Its been a full-time job (plus lots of overtime) to deal with all of this so I haven't even begun to properly look for work or track down the leads which I've been given.
Maybe soon I'll be able to start that. I could use a job that actually pays.
|
|
01:10 pm
[Link] |
Oh My Dog!
In the latest cartoon episode of "Batman: The Brave and the Bold", they brought back Ace the Bat-Hound. Urp. I was expecting Bat Mite (because of the episode title) but not Bat-Dog as well.
|
|
07:56 am
[Link] |
Too Early.
I am awake far too early, in case I am needed in helping sort out the cluster-fuck that has become of my friend Terry's move. If things go according to plan, his stuff will be arriving at the apartment (without him) in about 2 hours. According to this plan I won't be needed, but since nothing has gone according to plan yet, I'm half expecting to hear from him any minute that I will be needed after all. So, I've dragged myself out of bed, having had far too little sleep and am sincerely hoping that it was all for nothing.
Edit: And... it looks like I will be needed after all. *sigh*. So I get dressed and head out there. Again, IF things go according to plan I won't be needed for long. Can you spot the pattern?
|
|
02:23 pm
[Link] |
Plan 'A' Seems to Have Worked.
How often does that happen, anyway?
Over the last three days I've managed to build an image of the failed raid partition in its active state. (Imaging the failed individual drives happened earlier as a precaution, but I wasn't looking forward to raiding the images to get something that looked like a filesystem...).
In any case, the image appears to be a real filesystem. I can mount it and everything, although I can't do a fsck because reiser needs to be able to write to a partition to do a 'read-only' fsck. WTF? In any case, we'll be abandoning reiser in the very near future anyway. I have already partitioned a new drive to mirror the partitions in the old array and as soon as I can clear enough space to physically open up the webserver and swap drives, I'll be able to start dd-ing the images into their respective partitions and then seeing what fsck says about their consistency.
So, barring a catastrophe, it looks good to be up and running again by the weekend.
|
|
04:38 pm
[Link] |
Good News
Our dishwasher died a few weeks ago, at the height of tax season, no less, so neither of us had any time to deal with it. We've been doing the dishes by hand ever since. Since our warranty must have run out ages ago, we thought, we've been wondering at how we'd ever afford a replacement.
Well, Sears just called us to remind us that our warranty runs out at the end of this month. :-)
So, we've got a repairman coming next Tuesday to fix it, and we've agreed to extend the warranty yet again. So far, the extended warranty has paid for itself.
|
12:39 am
[Link] |
A favorite movie quote.
Its sad, but what's really sad is, it never got weird enough for me. I moved to the country when the boat got too crowded. Then I learned that President Nixon had been eaten by white cannibals on an island near Tijuana, for no good reason at all.
Golly, you hear a lot of savage and unnatural things about people these days.
Well, Lazlo and Nixon are gone now, but I don't think I'm going to believe that until I can gnaw on both of their skulls with my very own teeth. Fuck those people, huh? If they're out there, I'm going to find them, and I'm going to gnaw on their skulls, because it still hasn't gotten weird enough for me.
|
|
07:34 am
[Link] |
Not in the Mood
My mail and webserver just crashed. At first it looked like the usual problem that one of the 3 drives in the main RAID10 array had gotten 'stuck' which, so far as I can tell, is caused by a bug in the raid software itself.
A quick reboot and the system marked the 'stuck' drive as dirty and started a rebuild. So far, so good.
3 hours later I noticed the system was unresponsive and upon closer inspection was spitting out warning messages to the console so fast that I couldn't read anything but the work 'rebuilding'.
After trying everything I could to make the system calm down, I had to power it off. I restarted it again (in single user mode this time) and it seemed fine, and I told it to go back to rebuilding the array, which it did.
3 hours later it announced that a completely different drive had failed and as it now had 2 dirty drives in one 3-drive array, it was stopping right there. And that's the current state. I have NO IDEA how to tell it to assume that a 'spare' drive is actually not a spare. I also don't know if its complaint of a failing drive is correct or not. I can't even check the logs, since the logs are stored on the drive that was rebuilding...
So, right now I don't have my main mail server, and I have no real idea on how to fix the problem. So, for the moment I'm going to stop fiddling with it and see if some extended thought might solve things.
|
|
11:40 pm
[Link] |
Dreamwidth.
I've got a couple of Dreamwidth invite codes. Anyone want to be invited?
|
|
05:49 pm
[Link] |
Hurleys
Heading out to Hurleys for the usual Wednesday Debauchery. Hope to get there before 7:00, depending on traffic.
|
[<< Previous 20 entries] |
