I have seen the future.

In this case they were "the same process, running with the same command line, serving up the same files from the same shared folder on the NAS that both servers could see" - but SOMEHOW, without a clustering setup that anyone is aware of, suddenly they were running on Server3 instead of Server2.

And, of course, people connecting to Server2/file were getting errors, because it wasn't bloody running on Server2.

Bonus points: This is repeatable. It happens every time that Server1 is rebooted while Server2 and Server3 are both online. We've added it to the documentation as something we wish we understood but that we can't troubleshoot properly during business hours because Server1 is the primary domain controller and also serves all the file shares to all the Windows machines.

Extra bonus points: Server3 is a DNS server. *Just* a DNS server. DNS is all it does. It's not supposed to be failover for ANYTHING.

From:

pappy-legba.livejournal.com

ALIENS

From:

From:

skiriki.livejournal.com

Well apparently the process had not done enough to reincarnate as a higher process! Still a case of process reincarnation!

From:

zenmonkeykstop.livejournal.com

Check your init scripts on server 1 for ssh/rsh commands with hostnames generated on the fly?

From:

I'm too busy wibbling about the awesome that is the virtual replacement.

Seriously, I had a creaking piece of mission-critical shit running an obsolete OS on antiquated, failing hardware, where the average disk temperature is reported as 90 degrees regularly, and where we were terrified that it would crash unrecoverably at any time, and that we couldn't make software changes to for fear that the change would break something and we wouldn't be able to back it out.

Now, I have terrible software on an obsolete OS on a rock-solid running-cool VIRTUAL machine. Where I can snapshot it. And make changes. And then revert them if they go wrong.

It's a massive load off my mind, seriously.

Figuring out the WHYS of the insane organic network are a project for "not Friday afternoon".

From:

le-trombone.livejournal.com

I like this answer. Even if it turns out to be incorrect, I like it.

From:

What type of OSes, Unix-y or Redmond-y? AD involved?

Also ARP/-caches, where those are concerned there might be timeouts beyond "server 1 was rebooted".

From:

Ancient Debian, Old Centos, Geriatric Debian, in fact.

And no, it happens repeatably when server1 reboots. It's nuts. It's also easy to fix, once you know about it, has apparently been happening forever and just nobody had mentioned it to me before, and I refuse to care on Friday. Caring is for next week. Maybe the week after.

From:

Caring about weird shit like this needs to be tested by alcohol.

If you get to the bottom of a bottle of spirits AND you still care, then by all means look at it.

From:

But you should probably wait until every last particle of whatever you were imbibing, plus any and all after-effects, has left your body at that point.

Which ties nicely into "don't give a shit on Friday".

From:

I don't really start any new work after 3pm any day. And I sure as shit don't implement anything new on a Friday.

Thats Thursday's business, so if we have to, we pull it the fuck out on Friday.

From:

Ancient and geriatric Debian. Woody and Potato? Or would Potato be from the Dark Ages in your book, and not just geriatric?

From:

Woody and Etch, respectively. And CentOS 4.

From:

Way cool, somehow.

Even I pulled the last Woody off our 'net last year (and, I think, only some dom0s are still on Etch).

From:

As I said upthread, it is "a creaking piece of mission-critical shit running an obsolete OS on antiquated, failing hardware, where the average disk temperature is reported as 90 degrees regularly, and where we were terrified that it would crash unrecoverably at any time, and that we couldn't make software changes to for fear that the change would break something and we wouldn't be able to back it out.

Now, I have terrible software on an obsolete OS on a rock-solid running-cool VIRTUAL machine. Where I can snapshot it. And make changes. And then revert them if they go wrong."

I would be ecstatic if I could get it all the way up to Squeeze. If nothing else, a Samba upgrade might let us add the Win7 machines to the domain.

(Linux domain controllers: KILL ME NOW. The guy who set this entire network up is one of those crazy Russian linux guys who has a favourite way of doing everything and makes it all Just Work with strings and bailing wire, homebrewing up solutions without documenting them and tossing services onto whatever machine has the hardware available right that second. And he left the company in late 2008. "Redo it all, from scratch, RIGHT" is high on my list of things to do, but it's hard to justify when it's technically working now and downtime is down.)

From:

Can you add AD controllers to Samba, seize the roles and never ever talk about the old Samba ones again?

I've not looked at Samba in a while (from a serious, infrastructure point of view), but wasn't that where 4.x was going?

Edited Date: 2012-03-16 09:58 pm (UTC)

From:

I don't actually know. My most sincerest hope is to install a Windows Server 2008 R2 Domain, create new accounts from scratch, tell all the LDAP shit to authenticate against it, join all the machines to it fresh, and forget that the previous domain and the previous LDAP servers ever existed. Just to get a clean start.

But I don't think I'll get to do that, which means I may have to worry about that kind of thing.

From:

I prescribe a round of drinks, on me, if you're ever around. Ah, whatever, make that "enough rounds", and still on me.

And THEN, when you're helpless, I'll start with telling you about the mission-critical swerver that ran then-ancient Debilian, and would only boot from floppy. For over half a decade (which might still be around). And we can compare war-stories from that point on. though you should stop me when I get to Portmaster2's.

From:

You're in Austria?

That makes it a little harder just hook up for drinks, but, hey, if I'm ever on the continent again I'll try to stop by.

From:

You're in Austria?

There is that, yes. Though I'm usually in North America at least once a year (security-conferences'r'us or so).

From:

Because I don't know how to tell theweaselking otherwise...

Borderlands 2 PC Promises (http://www.borderlands2.com/loveletter?x)

* FOV Slider
* Mouse usable in menu's
* PC Specific UI for crazy resolutions
* Native Multiplayer Matchmaking
* PTT VOIP
* Logitech keyboard support (I spose this means G1x keyboard LCD's and Macro's)
* Control Pad
* VSync options
* High Resolutions (multi-monitor?)
* Mouse Smoothing options
* Cloud Save Support
* Achievement support
* Friends List support
* No Port forwarding required.

As someone who can't really play borderlands without about 40 minutes of pissing around with ini files and fov hacks, I hope this is true. I remember the downright nightmare that was trying to get Borderlands multi over the internet at launch.

From: