The Nightstar Zoo

Nightstar IRC Network - irc.nightstar.net
It is currently Tue Nov 13, 2018 11:39 pm

All times are UTC - 6 hours [ DST ]




Post new topic Reply to topic  [ 18 posts ] 
Author Message
PostPosted: Wed Oct 03, 2018 1:12 pm 
Offline
Entertainment
Entertainment
User avatar

Joined: Wed Jun 06, 2012 12:02 am
Posts: 781
That seems reasonable, if a little hard on the Toughs. Sure, Chinook was driven insane by a multi-million-year-old trap, but the Toughs knew the infrastructure was questionable, and should of installed safeguards when installing Chinook.

This goes a long way towards me retracting my earlier rant. You can't claim credit for fixing it if you haven't actually fixed it.


Top
 Profile  
 
PostPosted: Wed Oct 03, 2018 1:46 pm 
Offline
Concession Worker
Concession Worker
User avatar

Joined: Mon Jun 29, 2015 9:52 pm
Posts: 1310
Motortiki wrote:
That seems reasonable, if a little hard on the Toughs. Sure, Chinook was driven insane by a multi-million-year-old trap, but the Toughs knew the infrastructure was questionable, and should of installed safeguards when installing Chinook.

This goes a long way towards me retracting my earlier rant. You can't claim credit for fixing it if you haven't actually fixed it.

Chinook was installed by the previous AI without the Tough's consultation.

_________________
Rule of thumb is that if you need to use a squeegee, they're at least Laz-4.


Top
 Profile  
 
PostPosted: Wed Oct 03, 2018 4:36 pm 
Offline
Reptile House Exhibit
Reptile House Exhibit

Joined: Wed Apr 18, 2012 9:28 pm
Posts: 276
grahamf wrote:
Chinook was installed by the previous AI without the Tough's consultation.

In which case they'd get no credit for installing a new AI to begin with, and they end up with the same result.


Top
 Profile  
 
PostPosted: Wed Oct 03, 2018 8:39 pm 
Offline
Aquarium Exhibit
Aquarium Exhibit

Joined: Sun Mar 06, 2016 3:33 pm
Posts: 32
Motortiki wrote:
Toughs knew the infrastructure was questionable, and should of installed safeguards when installing Chinook.


Can they even install safeguards without breaking the "universal compatibility" of Ennesby's vanilla-helix design?


Top
 Profile  
 
PostPosted: Thu Oct 04, 2018 1:49 am 
Offline
Monkey House Exhibit
Monkey House Exhibit

Joined: Fri Jul 01, 2016 8:10 am
Posts: 481
They had PLENTY of safeguards, they just couldn't safeguard against things that they had no idea existed at all.

Namely, the ROOT ACCESS SUBROUTINE that Tkkuts-Afa installed to edit itself, forcefully and without possibility of cancellation.

That's basically like trying to safeguard against your IDENTICAL TWIN that nobody knew you had killing you and taking over your life. Or just ruining your credit score.

And that subroutine had to be hidden, or else Afa could have found and disabled it.


Top
 Profile  
 
PostPosted: Thu Oct 04, 2018 8:21 am 
Offline
Monkey House Exhibit
Monkey House Exhibit

Joined: Thu Nov 01, 2007 7:33 pm
Posts: 503
For the sake of argument, let's take as given that the Toughs count as installing Chinook; Iafa actually did it but the Toughs were in a position to install a new one or just leave the station without an AI so it's reasonable to attribute the decision to have Chinook stay on after the immediate crises was resolved to the Toughs.

I think the Oafans have a decent case that the Toughs were negligent; Tagii had already shown dangerous instability in unexpected hardware conditions. The subroutine was unforseeable, but Chinook going nuts due to some aspect of the system they'd missed was something they should've considered.

Also, they don't seem to be suing, they're just not paying the Toughs for replacing their insane AI with a sane one on the basis that the new AI is insane. If they'd made a contract in advance it'd probably have a clause saying the Toughs wouldn't get paid if the new AI was insane. And with a typical software contract, they'd probably have paid the Toughs on delivering Chinook but when she went nuts they'd demand the Toughs come fix it for free because they paid for a sane AI. Bugfixes aren't billable unless the buyer's contract lawyer is really bad.

More proximately, Chinook de facto failed User Acceptance Testing; there was no point where the Oafans said "okay, looks good to us; here's the money and we'll call you if anything goes wrong." Her performance prior to the intergalactic Long Gun attacks probably would've passed, but you don't get paid until it passes testing by the client. Their legal position is ironclad; their only responsibility is to agree on what they're buying and pay up if it's provided. Everything else is the Toughs' problem. UAT is just a chance for the Oafans to check if it's what they agreed to before paying up

However you portion out obligations and responsibilities, the fact remains that when the Oafans emerged from the infosphere their station was still run by an insane AI. So from their perspective nothing was accomplished. Technically the Toughs indirectly released them, but it's really Petey and the Allstar they owe for that.


Top
 Profile  
 
PostPosted: Thu Oct 04, 2018 11:29 am 
Offline
Entertainment
Entertainment
User avatar

Joined: Wed Jun 06, 2012 12:02 am
Posts: 781
Kendrakirai wrote:
They had PLENTY of safeguards, they just couldn't safeguard against things that they had no idea existed at all.

Namely, the ROOT ACCESS SUBROUTINE that Tkkuts-Afa installed to edit itself, forcefully and without possibility of cancellation.

That's basically like trying to safeguard against your IDENTICAL TWIN that nobody knew you had killing you and taking over your life. Or just ruining your credit score.

And that subroutine had to be hidden, or else Afa could have found and disabled it.

Actually, that kind of system error is a serious issue in today's computers. Routines with root access overwriting or modifying something important in the background is a real concern, and lots of time and money gets invested into guarding against that. Chinook could of had a simple "friendliness" checksum, and a subroutine that would automatically trigger any number of solutions if that checksum didn't pass.

For example, she could of had an asynchronous backup server, that would preserve her if Chinook Prime was compromised in some way. Come to think of it, she had 1,600 backup servers: she had plenty of opportunity to create a network of safeguards against Chinook Prime getting damaged. And that damage being automatically propagated through the network shows vulnerable network design.

I could see that not being possible for say, a warship AI, due to hardware constraints. But Chinook is based of an architecture that is canonically very flexible. The only reason I can see for her not installing safeguards against corruption was intense personal arrogance. (Which Chinook also canonically exhibited. Pride cometh before the fall.)


Top
 Profile  
 
PostPosted: Thu Oct 04, 2018 10:53 pm 
Offline
Monkey House Exhibit
Monkey House Exhibit

Joined: Fri Jul 01, 2016 8:10 am
Posts: 481
Motortiki wrote:
Kendrakirai wrote:
They had PLENTY of safeguards, they just couldn't safeguard against things that they had no idea existed at all.

Namely, the ROOT ACCESS SUBROUTINE that Tkkuts-Afa installed to edit itself, forcefully and without possibility of cancellation.

That's basically like trying to safeguard against your IDENTICAL TWIN that nobody knew you had killing you and taking over your life. Or just ruining your credit score.

And that subroutine had to be hidden, or else Afa could have found and disabled it.

Actually, that kind of system error is a serious issue in today's computers. Routines with root access overwriting or modifying something important in the background is a real concern, and lots of time and money gets invested into guarding against that. Chinook could of had a simple "friendliness" checksum, and a subroutine that would automatically trigger any number of solutions if that checksum didn't pass.

For example, she could of had an asynchronous backup server, that would preserve her if Chinook Prime was compromised in some way. Come to think of it, she had 1,600 backup servers: she had plenty of opportunity to create a network of safeguards against Chinook Prime getting damaged. And that damage being automatically propagated through the network shows vulnerable network design.

I could see that not being possible for say, a warship AI, due to hardware constraints. But Chinook is based of an architecture that is canonically very flexible. The only reason I can see for her not installing safeguards against corruption was intense personal arrogance. (Which Chinook also canonically exhibited. Pride cometh before the fall.)


The root access problem wasn't an intrusion, though. It was specifically set up by the previous inhabitant to have said root access and be unstoppable. It was already deigned to bypass checksums. It requires a LOT of juice to edit an AI, especially without them noticing. Typically requiring direct hardware access that we've seen, and presumably a far higher processing power. Petey ripped through Ennesby's firewalls when he was still insane and suicidal. and Tkkuts knew what Tagii and Ennesby were up to because they were running on his hardware

Chinook's backup fragments were an asynchronous partial backup designed for burst transmission and revenge, not for actual backup. They could be found if there was more regular contact.

As for the backup and comparison itself, I was thinking about that.

If you were insane with grief, anger, righteous vengeance, and you were shown proof by those you thought were working against you, would you trust that you were insane and not COMPLETELY SANE and that they were trying to hack you? That your backup was pristine and not altered to make you more compliant?

Chinook doesn't think anything is wrong with her. If she had a backup - which is probable - she likely destroyed it out of paranoia that it was altered by the other AIs that she thinks are her enemies.


Top
 Profile  
 
PostPosted: Fri Oct 05, 2018 1:15 pm 
Offline
Entertainment
Entertainment
User avatar

Joined: Wed Jun 06, 2012 12:02 am
Posts: 781
Kendrakirai wrote:
The root access problem wasn't an intrusion, though.

I'm not talking about safeguarding against intrusions. I'm talking about safeguarding against "a root access user did something foolish," which is exactly what happened here.

The risk of bricking my own computer is very real, I have to be careful when I'm configuring scripts. I work around this by doing all my work on a "disposable" OS, and keeping anything important on an external server. That way, if I, or an update, breaks something in a way that prevents me from fixing what I broke, I can just re-install the OS. (I also have other safeguards, so far I have not needed the nuclear option.)

Kendrakirai wrote:
It was specifically set up by the previous inhabitant to have said root access and be unstoppable. It was already deigned to bypass checksums. It requires a LOT of juice to edit an AI, especially without them noticing. Typically requiring direct hardware access that we've seen, and presumably a far higher processing power. Petey ripped through Ennesby's firewalls when he was still insane and suicidal. and Tkkuts knew what Tagii and Ennesby were up to because they were running on his hardware

That is true, but I'm not convinced that it's impossible to guard against the hardware the AI is running on from hacking the AI. For example, fragments could be running on separate hardware.

Kendrakirai wrote:
Chinook's backup fragments were an asynchronous partial backup designed for burst transmission and revenge, not for actual backup. They could be found if there was more regular contact.

This is exactly where I think Chinook went wrong. She should of had a more robust system. Like, she could of had multiple layers of backups: a "burst and revenge" layer, and then hidden within that, a layer of fragments that observe the burst updates and check for issues. That she didn't shows that she never considered the possibility she could be corrupted. Considering that's exactly what happened to her progenitor Tagii, that's pretty arrogant. Which, again, is completely within Chinook's character.
Kendrakirai wrote:
As for the backup and comparison itself, I was thinking about that.

If you were insane with grief, anger, righteous vengeance, and you were shown proof by those you thought were working against you, would you trust that you were insane and not COMPLETELY SANE and that they were trying to hack you? That your backup was pristine and not altered to make you more compliant?

Chinook doesn't think anything is wrong with her. If she had a backup - which is probable - she likely destroyed it out of paranoia that it was altered by the other AIs that she thinks are her enemies.

That's true, I didn't think of it that way. Chinook Prime was facing the two most powerful AI's in (known) existence. Considering that she was arrogant to begin with, she was predisposed to not believe anyone saying that something was wrong with her, and view it as manipulation. She could of immediately acted to discard her own backups, out of that arrogance and paranoia.

I still think she still could of avoided this, by having a more robust network.


Top
 Profile  
 
PostPosted: Fri Oct 05, 2018 1:45 pm 
Offline
Monkey House Exhibit
Monkey House Exhibit

Joined: Thu Nov 01, 2007 7:33 pm
Posts: 503
I'm sure she could have somehow avoided it, but it's basically a rootkit using a zero-day exploit; any system that isn't literally infallible is potentially subject to that sort of threat. No backup system would truly have fixed the problem; the script was already executing the moment Chinook was installed. No self-diagnostic would be guaranteed to find it; the script could tamper with the diagnostic and sabotage any error checking. The script isn't an error; it's well-designed malware.

It's possible another AI would've been more resistant to spiraling into madness, but the Toughs' real screwup was relying on the self-diagnostics of a potentially compromised system. They should have analyzed it using a different system, one where the script couldn't tamper with the analysis tools themselves (hopefully).


Top
 Profile  
 
PostPosted: Sat Oct 06, 2018 12:10 am 
Offline
Entertainment
Entertainment
User avatar

Joined: Wed Jun 06, 2012 12:02 am
Posts: 781
(name here) wrote:
I'm sure she could have somehow avoided it, but it's basically a rootkit using a zero-day exploit; any system that isn't literally infallible is potentially subject to that sort of threat. No backup system would truly have fixed the problem; the script was already executing the moment Chinook was installed. No self-diagnostic would be guaranteed to find it; the script could tamper with the diagnostic and sabotage any error checking. The script isn't an error; it's well-designed malware.

It's possible another AI would've been more resistant to spiraling into madness, but the Toughs' real screwup was relying on the self-diagnostics of a potentially compromised system. They should have analyzed it using a different system, one where the script couldn't tamper with the analysis tools themselves (hopefully).

You are right, I was thinking of this as something that activated when the conditions were met. But your interpretation is far more likely: after all, the script would have to be running to check those conditions.

Heck, that would mean Tagii was affected by this the moment she loaded onto Eina-Afa's hardware as a trojan.


Top
 Profile  
 
PostPosted: Sat Oct 06, 2018 12:59 am 
Offline
Monkey House Exhibit
Monkey House Exhibit

Joined: Thu Nov 01, 2007 7:33 pm
Posts: 503
There's a number of ways it could work, but clearly it's intentionally designed to avoid detection and removal by the AI on the system, and high-end malware regularly sabotages antivirus systems and tampers with removal tools if it's managed to secure the necessary permissions. I had one hellish week trying to scrape out a virus that kept making my antivirus tools cut their scans short.

Backups might have been able to restore her to a non-crazy state, but they'd have backed up the script too so she'd just go crazy again in short order.


Top
 Profile  
 
PostPosted: Sun Oct 07, 2018 12:49 am 
Offline
Monkey House Exhibit
Monkey House Exhibit

Joined: Fri Jul 01, 2016 8:10 am
Posts: 481
(name here) wrote:
There's a number of ways it could work, but clearly it's intentionally designed to avoid detection and removal by the AI on the system, and high-end malware regularly sabotages antivirus systems and tampers with removal tools if it's managed to secure the necessary permissions. I had one hellish week trying to scrape out a virus that kept making my antivirus tools cut their scans short.

Backups might have been able to restore her to a non-crazy state, but they'd have backed up the script too so she'd just go crazy again in short order.



That's something else actually. I'm not as convinced that the subroutine was entirely software. I think it's also hardware.

A little bit back there was a hubbub about certain CPUs having a 'backdoor' interrupt that could be activated via software and was deliberately designed into the chip. Essentially a hardware-based program that could be turned on or off. I think that might have been what made the subroutine work so well and so quickly. It wasn't actually a program running on root. It was the hardware itself altering the system. That would be the most foolproof method of making sure the editing was unstoppable and undetectable, I think; it would just look like any other bit of hardware, doing its thing, like any random water pump or power distribution node.

We've seen before that hardware interlocks can directly affect AIs. Remember Petey's Loyalty Switch?


Top
 Profile  
 
PostPosted: Sun Oct 07, 2018 1:28 am 
Offline
Entertainment
Entertainment
User avatar

Joined: Wed Jun 06, 2012 12:02 am
Posts: 781
Kendrakirai wrote:
(name here) wrote:
There's a number of ways it could work, but clearly it's intentionally designed to avoid detection and removal by the AI on the system, and high-end malware regularly sabotages antivirus systems and tampers with removal tools if it's managed to secure the necessary permissions. I had one hellish week trying to scrape out a virus that kept making my antivirus tools cut their scans short.

Backups might have been able to restore her to a non-crazy state, but they'd have backed up the script too so she'd just go crazy again in short order.



That's something else actually. I'm not as convinced that the subroutine was entirely software. I think it's also hardware.

A little bit back there was a hubbub about certain CPUs having a 'backdoor' interrupt that could be activated via software and was deliberately designed into the chip. Essentially a hardware-based program that could be turned on or off. I think that might have been what made the subroutine work so well and so quickly. It wasn't actually a program running on root. It was the hardware itself altering the system. That would be the most foolproof method of making sure the editing was unstoppable and undetectable, I think; it would just look like any other bit of hardware, doing its thing, like any random water pump or power distribution node.

We've seen before that hardware interlocks can directly affect AIs. Remember Petey's Loyalty Switch?

Yeah, I agree, I've been thinking of this as a "hardware malware". But all the things (name here) have been saying are still valid. At a certain level functions can be either hardware or software, and it doesn't matter which. All it changes is what it takes to solve the problem.


Top
 Profile  
 
PostPosted: Sun Oct 07, 2018 6:54 am 
Offline
Monkey House Exhibit
Monkey House Exhibit

Joined: Thu Nov 01, 2007 7:33 pm
Posts: 503
Well, it was added in by Iafa at a later date, so I'd assume software.


Top
 Profile  
 
PostPosted: Sun Oct 07, 2018 9:11 am 
Offline
Aquarium Exhibit
Aquarium Exhibit

Joined: Sat Apr 04, 2015 3:13 pm
Posts: 32
She could have fabbed and installed new hardware just as easily. As we know from here and here there are even fabbers dedicated to that.


Top
 Profile  
 
PostPosted: Sun Oct 28, 2018 5:52 pm 
Offline
Arctic Exhibit
Arctic Exhibit

Joined: Tue Sep 20, 2016 12:18 pm
Posts: 49
Motortiki wrote:

Actually, that kind of system error is a serious issue in today's computers. Routines with root access overwriting or modifying something important in the background is a real concern, and lots of time and money gets invested into guarding against that. Chinook could of had a simple "friendliness" checksum, and a subroutine that would automatically trigger any number of solutions if that checksum didn't pass.

... text omitted.


I remember hearing of something like that happening many years ago. Boeing had a large network of mainframes that they sold time on. At some point, someone wanted to format a disk. Instead of formatting all tracks on this one disk, it accidentally formatted track zero on ALL disk packs.
That effectively erased all data.

Not sure how they recovered from that.


Top
 Profile  
 
PostPosted: Sun Oct 28, 2018 5:57 pm 
Offline
Arctic Exhibit
Arctic Exhibit

Joined: Tue Sep 20, 2016 12:18 pm
Posts: 49
Kendrakirai wrote:


That's something else actually. I'm not as convinced that the subroutine was entirely software. I think it's also hardware.

A little bit back there was a hubbub about certain CPUs having a 'backdoor' interrupt that could be activated via software and was deliberately designed into the chip. Essentially a hardware-based program that could be turned on or off. I think that might have been what made the subroutine work so well and so quickly. It wasn't actually a program running on root. It was the hardware itself altering the system. That would be the most foolproof method of making sure the editing was unstoppable and undetectable, I think; it would just look like any other bit of hardware, doing its thing, like any random water pump or power distribution node.

We've seen before that hardware interlocks can directly affect AIs. Remember Petey's Loyalty Switch?


They have something like that on Today's Intel chips. At least the beginning.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 18 posts ] 

All times are UTC - 6 hours [ DST ]


Who is online

Users browsing this forum: Bing [Bot], Kendrakirai and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group