« Scriptions | Main | The Pain of Filesystems »
October 20, 2004
Kerberos 'n Shit
The laptop came back yesterday. That was 5 days (over a weekend, no less) out of an anticipated 14-21 days. Go Apple! Next time I’m definitely getting the fucking AppleCare.
I’ve got Kerberos working at home, not quite sure why the fuck I care because it’s not exactly going over the network. I guess it was partly because I want to be using AFS or something similar to it for Project SOULTRAIN. There isn’t much to be done about the fact that files need to be shared, and though I’m happy to implement all of the meta crap in Lisp and Prolog, I just don’t feel like implementing a distributed filesystem in it.
Since my blog tends to have lots of lists, here’s me knocking down all the network filesystems out there:
- AFS or Andrew FileSystem if you feel like being ancient. Works great… assuming you only need a client (ARLA). If you need a server, you get to use OpenAFS, which currently works with OpenBSD, AIX, Windows XP and Linux. Doesn’t that seem like a strange list of supported platforms? Apparently FreeBSD isn’t supported. The reason seems to be, because OpenAFS keeps track of what FS nodes are its, as does FreeBSD, and unless these two mechanisms are identical, you’ll have catastrophic data loss. Everyone using OpenAFS should pay attention to those words: catastrophic data loss. Your NFS is looking at the disk directly going, “Hmm, can I steal that node from the OS without it noticing?”
- GFS, oh wait, that only works on Linux. My bad. (It was ported to Linux from IRIX! You’d think they’d have thought to include a little portability!)
- Coda, which was intended to be sort-of next generation AFS at one point. It had looked abandoned for some time but lately they’re making new releases—though they haven’t had a binary for FreeBSD in 1 major and 8-some-odd minor releases. I’m downloading it now, here’s hoping it at least compiles before shitting all over me, but I don’t see any talk about it or FreeBSD in the past several years, which isn’t a good sign.
- Lustre, oh wait, that only works on Linux. You sense a pattern happening here?
- InterMezzo looks good, but they don’t mention platform support so I’m guessing it’s also… you get the picture. I’ll give it a shot later on though.
Apparently, the concept of a portable, distributed/cluster/network filesystem is just a far-off dream. You can have portable, or you can have distributed/cluster/network filesystem. Oh wait, I just realized, I forgot NFS4, which is like NFS but without the locking issues, with Kerberos authorization, with failover and volume management, and without a single open source implementation. :-P Someone needs to get out there and implement this in a way that’s secure, that scales, that doesn’t suck and maybe works on all free and a few commercial UNIXes. This can’t be that hard.
Some fuck is going to reply saying “Why don’t you just run Linux?” Well, the point isn’t that I hate Linux or won’t use it again. In fact, today I found out at work that I’m going to be in charge of a new hosted server, probably from 1and1, and you just know I’m putting Gentoo on that mofo. The point is, when I have a computer in the back room running some-fuck-platform, and a Mac, and this ancient cube POS I just adore running OS-du-jour, I’m going to want to be getting at my files from any of ‘em. I’m not installing Linux on my Mac to get at my music. So, it’s got to be cross platform. Which means, in all probability, it’s got to be NFS, or Samba, or Netatalk, or some other ridiculous thing that is meant to work (poorly) anywhere.
Code Note: The ./configure didn’t work, but according to README.BSD, I’ve already got Coda in the kernel. Which would be a great alternative to all this sucking, now it’s just a matter of making it work with Kerberos and, I dunno, building the fucking server.
On the job, today I wrote my secondish threaded program. It was a complete disaster, and I don’t think it was my fault (though I’m not sure I can prove it). Basically, I was creating 50ish threads to do DNS resolutions and write them to the database. With a timeout of 5 seconds, we should have seen them all retire within 5 seconds. Instead, they were all retiring for the first time in ~4 minutes, which is consistent with multiplying the numbers together. In short, what I’m afraid may have happened is this: App creates a thread 1, which creates threads 2 through 51. Thread 2, upon waking, resolves some server, resulting in an OS-level block. If the threads were implemented as user-threads, in other words with RB cooperatively tasking all of my Threads in the one “real” OS thread, then I would see this behavior, and the linear behavior we see. If the threads were implemented as OS threads (the right way, we’ll call it) then we should have seen no lag, non-deterministic responses, and it would have been fast.
Well, what we did see was random in a deterministic way. What I mean is, it would do 3, then 10, then 13, then 14, then 15, then 21 etc. The second time around, it would do 3, 10, 13, 14, 15, 21. Not linear, definitely deterministic, but not… correct. Furthermore, it would always go down the list, wrapping once it hit 50 or so.
This smells like shitty threads to me, not really my fault. Tomorrow I’m rewriting it to work with a Ruby script executing 50 of these in parallel. I think I can do this with something gross like this (in sh-like pseudocode, for those not fluent in Ruby):
function next_id() { ... }
function more_ids() { ... }
# start the initial 50
for i in $((seq 50)); do
looker-upper next_id &
done
# start another process everytime one of the 50 dies, until we run out
while more_ids && wait; do
looker-upper next_id
done
# cleanup; wait for our kids to die, or else we'd just slay 'em
# and nobody wants that in software
# (49 because one of 'em already died to get us here, or else I'm thinking
# too hard)
for i in $((seq 49)); do
wait
done
So, yeah, I’m pretty much basically just offended. Fuck.
In better news, I’m looking forward to 49’ers. I’ll probably get drafted to carry or something, which I’ll do a little bit of in a half-assed way, then realize I’m not even a student and fuck it. Should be fun, I haven’t seen any of my friends in the flesh in months. It will be good to get away from the computer for a little while, and go fix Harry’s. :)
Posted by FusionGyro at October 20, 2004 10:52 PM
Trackback Pings
TrackBack URL for this entry:
http://www.clanspum.net/~fusion/blog/admin/mt-tb.cgi/37