Great Hack of 2008 - Revision history

Lithium at 15:59, 26 April 2010

2010-04-26T15:59:52Z

← Older revision		Revision as of 15:59, 26 April 2010
Line 241:		Line 241:
	We backed-up the rest of the stuff on Deathray, and re-installed that on Monday afternoon. Bitlbee was also setup late Monday night.		We backed-up the rest of the stuff on Deathray, and re-installed that on Monday afternoon. Bitlbee was also setup late Monday night.


			== The simplified version ==

			[[Image:GreatHack.jpg]]

	[[Category:Redbrick History]]		[[Category:Redbrick History]]
	__NOTOC__		__NOTOC__

Werdz: /* Wednesday */

2009-05-16T00:01:04Z

Wednesday

← Older revision		Revision as of 00:01, 16 May 2009
Line 176:		Line 176:
	Phaxx & atlas come to DCU to help make exim and mailman work. We also eat chinese.		Phaxx & atlas come to DCU to help make exim and mailman work. We also eat chinese.

	I (werdz) think it was around now (possibly Thursday or Friday) that I got ubuntu installed on murphy. Manually. After downgrading the firmware to version Poxy.ancient. The installer didn't seem to want to work at all (can't remember the exact list of problems), so it was installed using a rescue shell to create partitions, dpkg-bootstrap the system and install frivolous things like the kernel manually. Once this was done, Ubuntu booted, and things generally worked, but the serial console (well... what you got by typing console -f into the ALOM) randomly locked up. And some other things randomly broke. We decided this level of reliability wasn't what we were looking for in an OS. See [[~~Hardware:~~Murphy]] for the rest of this epic tale, and how it eventually ended up with Solaris 10, a full year later.		I (werdz) think it was around now (possibly Thursday or Friday) that I got ubuntu installed on murphy. Manually. After downgrading the firmware to version Poxy.ancient. The installer didn't seem to want to work at all (can't remember the exact list of problems), so it was installed using a rescue shell to create partitions, dpkg-bootstrap the system and install frivolous things like the kernel manually. Once this was done, Ubuntu booted, and things generally worked, but the serial console (well... what you got by typing console -f into the ALOM) randomly locked up. And some other things randomly broke. We decided this level of reliability wasn't what we were looking for in an OS. See [[Murphy]] for the rest of this epic tale, and how it eventually ended up with Solaris 10, a full year later.

	=== Thursday ===		=== Thursday ===

Werdz at 23:59, 15 May 2009

2009-05-15T23:59:52Z

← Older revision		Revision as of 23:59, 15 May 2009
Line 49:		Line 49:

	* 10:30 - final mass hey goes out before shutdown		* 10:30 - final mass hey goes out before shutdown
	* 10:32 - access to the network is cut off.		* 10:32 - access to the network is cut off. A few root holders take refuge in #blueblock on irc.linux.ie to discuss meeting later on.


Line 77:		Line 77:
	=== Friday Evening ===		=== Friday Evening ===

	The admins meet up in DCU about 5/6pm. Atlas also comes down. There is much drawing on a whiteboard about what our plans should be. Most of the night is spent taking backups.		The admins meet up in DCU about 5/6pm. Atlas also comes down. There is much drawing on a whiteboard about what our plans should be. Most of the night is spent taking backups. Getting some sort of mail processing in place is a priority, as we're not sure how long HEAnet/DCU will store our incoming mail for before giving up. Minerva's OS (and feck all else) is wiped and reinstalled by around midnight (after some extremely odd keyboard controller-related difficulties), and by the time we leave at around 1am, Deathray (not quite reinstalled, but isolated as much as possible from everything) is routing mail to inboxes.


Line 84:		Line 84:
	We meet up again in DCU early on Saturday and start the re-install process. One of the red dells is setup to allow admins external access to the network so stuff can be done remotely.		We meet up again in DCU early on Saturday and start the re-install process. One of the red dells is setup to allow admins external access to the network so stuff can be done remotely.

	~~Minerva is reinstalled.~~ Murphy continues to be shit.		Murphy continues to be shit.

	Cynic is re-installed with solaris, and DNS moves there.		Cynic is re-installed with solaris, and DNS moves there.
Line 139:		Line 139:
	===Monday ===		===Monday ===

	Halenger builds ircd on minerva. Ubuntu starts working on carbon on Monday evening, after we discover that the ubuntu-server kernel hates the disk controller, and ~~get~~ a different kernel.		Halenger builds ircd on minerva. Ubuntu starts working on carbon on Monday evening, after we discover that the ubuntu-server kernel hates the disk controller, and manually install a different kernel using a rescue shell.

	The new passwords went out on Monday. This required some epic hack scripting, since useradm was on one of the shite reddells, but that couldn't send email.		The new passwords went out on Monday. This required some epic hack scripting, since useradm was on one of the shite reddells, but that couldn't send email.


	=== Tuesday ===		=== Tuesday ===
Line 150:		Line 149:
	I fell asleep on the bus back from work, and woke up at Ballymun shopping center.		I fell asleep on the bus back from work, and woke up at Ballymun shopping center.

	Murphy still refuses to work		Murphy still refuses to work. The Ubuntu installer CD refuses to talk to the upgraded version of the T2000 firmware (which we were told to upgrade by the installation instructions).

	{0} ok WORK DAMN YOU		{0} ok WORK DAMN YOU
Line 177:		Line 176:
	Phaxx & atlas come to DCU to help make exim and mailman work. We also eat chinese.		Phaxx & atlas come to DCU to help make exim and mailman work. We also eat chinese.

			I (werdz) think it was around now (possibly Thursday or Friday) that I got ubuntu installed on murphy. Manually. After downgrading the firmware to version Poxy.ancient. The installer didn't seem to want to work at all (can't remember the exact list of problems), so it was installed using a rescue shell to create partitions, dpkg-bootstrap the system and install frivolous things like the kernel manually. Once this was done, Ubuntu booted, and things generally worked, but the serial console (well... what you got by typing console -f into the ALOM) randomly locked up. And some other things randomly broke. We decided this level of reliability wasn't what we were looking for in an OS. See [[Hardware:Murphy]] for the rest of this epic tale, and how it eventually ended up with Solaris 10, a full year later.

	=== Thursday ===		=== Thursday ===

Receive at 23:29, 15 May 2009

2009-05-15T23:29:05Z

← Older revision		Revision as of 23:29, 15 May 2009
Line 240:		Line 240:

	We backed-up the rest of the stuff on Deathray, and re-installed that on Monday afternoon. Bitlbee was also setup late Monday night.		We backed-up the rest of the stuff on Deathray, and re-installed that on Monday afternoon. Bitlbee was also setup late Monday night.


			[[Category:Redbrick History]]
			__NOTOC__

Receive at 23:25, 15 May 2009

2009-05-15T23:25:51Z

← Older revision		Revision as of 23:25, 15 May 2009
Line 15:		Line 15:
	== Timeline of Events ==		== Timeline of Events ==

	In an attempt to procrastinate from real work I've tried to document what happened on the week of no sleep, and insane amounts of coke. To give a bit of background, the hack happened just a few weeks after myself and johan were elected, lil_cain was the third admin, and werdz had recently moved from admin to webmaster. All of us were working full time on Intra, except for cian, who had a real job. Murphy was the main web server, running Ubuntu 6.06. Minerva was the main login server, also running Ubuntu.		In an attempt to procrastinate from real work I've tried to document what happened on the week of no sleep, and insane amounts of coke.

			To give a bit of background, the hack happened just a few weeks after myself and johan were elected, lil_cain was the third admin, and werdz had recently moved from admin to webmaster. All of us were working full time on Intra, except for cian, who had a real job. Murphy was the main web server, running Ubuntu 6.06. Minerva was the main login server, also running Ubuntu.

			Mostly, I've pieced this together from memory, email and irc logs. Some other people who were there may be able to fill in more of the details.

	--receive (May 2009)		--receive (May 2009)
Line 21:		Line 25:

	===Friday Morning ===		===Friday Morning ===


	* Around 8.30am (ish) - People start to notice something is up. Most people are on the way to work and stuff. Pretty sure I got a text on the bus, can't remember who from.		* Around 8.30am (ish) - People start to notice something is up. Most people are on the way to work and stuff. Pretty sure I got a text on the bus, can't remember who from.
Line 47:		Line 50:
	* 10:30 - final mass hey goes out before shutdown		* 10:30 - final mass hey goes out before shutdown
	* 10:32 - access to the network is cut off.		* 10:32 - access to the network is cut off.


	=== Friday Afternoon ===		=== Friday Afternoon ===
Line 69:		Line 73:
	Dano		Dano
	Secretary@redbrick.dcu.ie		Secretary@redbrick.dcu.ie


	=== Friday Evening ===		=== Friday Evening ===

	The admins meet up in DCU about 5/6pm. Atlas also comes down. There is much drawing on a whiteboard about what our plans should be. Most of the night is spent taking backups.		The admins meet up in DCU about 5/6pm. Atlas also comes down. There is much drawing on a whiteboard about what our plans should be. Most of the night is spent taking backups.


	=== Saturday ===		=== Saturday ===
Line 136:		Line 142:

	The new passwords went out on Monday. This required some epic hack scripting, since useradm was on one of the shite reddells, but that couldn't send email.		The new passwords went out on Monday. This required some epic hack scripting, since useradm was on one of the shite reddells, but that couldn't send email.


	=== Tuesday ===		=== Tuesday ===
Line 162:		Line 169:

	At this point, Deathray hadn't been touched yet, since mail needed to be migrated before that could happen, and Murphy was generally being a cunt, and wasn't working at all.		At this point, Deathray hadn't been touched yet, since mail needed to be migrated before that could happen, and Murphy was generally being a cunt, and wasn't working at all.


	=== Wednesday ===		=== Wednesday ===
Line 168:		Line 176:

	Phaxx & atlas come to DCU to help make exim and mailman work. We also eat chinese.		Phaxx & atlas come to DCU to help make exim and mailman work. We also eat chinese.


	=== Thursday ===		=== Thursday ===

	Svan gets fap working, and and syslog-ng doing remote logging.		Svan gets fap working, and and syslog-ng doing remote logging.


	=== Friday ===		=== Friday ===

	Get mailman working about 1am. Continue testing mail and stuff until about 2/3am		Get mailman working about 1am. Continue testing mail and stuff until about 2/3am


	==== Saturday ====		==== Saturday ====
Line 216:		Line 227:
	All the best,		All the best,
	andrew.		andrew.



Line 221:		Line 233:

	Welmar is backed up, and re-installed, followed by Obelisk. Anyterm starts working again late Sunday night.		Welmar is backed up, and re-installed, followed by Obelisk. Anyterm starts working again late Sunday night.


	=== Monday ===		=== Monday ===

Receive at 23:23, 15 May 2009

2009-05-15T23:23:42Z

← Older revision		Revision as of 23:23, 15 May 2009
Line 13:		Line 13:


	=== Timeline of Events ===		== Timeline of Events ==

	In an attempt to procrastinate from real work I've tried to document what happened on the week of no sleep, and insane amounts of coke. To give a bit of background, the hack happened just a few weeks after myself and johan were elected, lil_cain was the third admin, and werdz had recently moved from admin to webmaster. All of us were working full time on Intra, except for cian, who had a real job. Murphy was the main web server, running Ubuntu 6.06. Minerva was the main login server, also running Ubuntu.		In an attempt to procrastinate from real work I've tried to document what happened on the week of no sleep, and insane amounts of coke. To give a bit of background, the hack happened just a few weeks after myself and johan were elected, lil_cain was the third admin, and werdz had recently moved from admin to webmaster. All of us were working full time on Intra, except for cian, who had a real job. Murphy was the main web server, running Ubuntu 6.06. Minerva was the main login server, also running Ubuntu.
Line 20:		Line 20:


	==== Friday Morning ====		===Friday Morning ===


Line 48:		Line 48:
	* 10:32 - access to the network is cut off.		* 10:32 - access to the network is cut off.

	==== Friday Afternoon ====		=== Friday Afternoon ===

	D_fens, svan & drag0n go down to the server room and start tracing through the rest of the logs. The rest of us can't do much with no access.		D_fens, svan & drag0n go down to the server room and start tracing through the rest of the logs. The rest of us can't do much with no access.
Line 70:		Line 70:
	Secretary@redbrick.dcu.ie		Secretary@redbrick.dcu.ie

	==== Friday Evening ====		=== Friday Evening ===

	The admins meet up in DCU about 5/6pm. Atlas also comes down. There is much drawing on a whiteboard about what our plans should be. Most of the night is spent taking backups.		The admins meet up in DCU about 5/6pm. Atlas also comes down. There is much drawing on a whiteboard about what our plans should be. Most of the night is spent taking backups.

	==== Saturday ====		=== Saturday ===

	We meet up again in DCU early on Saturday and start the re-install process. One of the red dells is setup to allow admins external access to the network so stuff can be done remotely.		We meet up again in DCU early on Saturday and start the re-install process. One of the red dells is setup to allow admins external access to the network so stuff can be done remotely.
Line 83:		Line 83:


	==== Sunday ====		=== Sunday ===

	Minerva is finished off on Sunday. We also started scripting people's web directories back to normal. Temporary ldap is setup on one of the red Dells.		Minerva is finished off on Sunday. We also started scripting people's web directories back to normal. Temporary ldap is setup on one of the red Dells.
Line 131:		Line 131:


	==== Monday ====		===Monday ===

	Halenger builds ircd on minerva. Ubuntu starts working on carbon on Monday evening, after we discover that the ubuntu-server kernel hates the disk controller, and get a different kernel.		Halenger builds ircd on minerva. Ubuntu starts working on carbon on Monday evening, after we discover that the ubuntu-server kernel hates the disk controller, and get a different kernel.
Line 137:		Line 137:
	The new passwords went out on Monday. This required some epic hack scripting, since useradm was on one of the shite reddells, but that couldn't send email.		The new passwords went out on Monday. This required some epic hack scripting, since useradm was on one of the shite reddells, but that couldn't send email.

	==== Tuesday ====		=== Tuesday ===

	Phaxx gets IMAP working on minerva on Tuesday afternoon.		Phaxx gets IMAP working on minerva on Tuesday afternoon.

			I fell asleep on the bus back from work, and woke up at Ballymun shopping center.

	Murphy still refuses to work		Murphy still refuses to work
Line 161:		Line 163:
	At this point, Deathray hadn't been touched yet, since mail needed to be migrated before that could happen, and Murphy was generally being a cunt, and wasn't working at all.		At this point, Deathray hadn't been touched yet, since mail needed to be migrated before that could happen, and Murphy was generally being a cunt, and wasn't working at all.

			=== Wednesday ===

			I make some progress beating boards into working over the afternoon, it finally starts working properly about 7pm

			Phaxx & atlas come to DCU to help make exim and mailman work. We also eat chinese.

			=== Thursday ===

			Svan gets fap working, and and syslog-ng doing remote logging.

			=== Friday ===

			Get mailman working about 1am. Continue testing mail and stuff until about 2/3am

	==== Saturday ====		==== Saturday ====

	~~The system comes~~ back at ~~lunchtime on Saturday~~.		Go down to dcu about 10am to put all the cables back to normal, and let users login again. Security finally let us in at about 12.30

			SSH access to minerva is allowed again about 8pm, and the mail goes out shortly after.

	From: Andrew Harford		From: Andrew Harford
	Sent: Saturday, May 3, 2008 3:02:~~11 PM~~		Sent: Saturday, May 3, 2008 23:10:06 +0100
	Subject: RedBrick Online		Subject: RedBrick Online

Line 200:		Line 217:
	andrew.		andrew.

	..
			=== Sunday ===

			Welmar is backed up, and re-installed, followed by Obelisk. Anyterm starts working again late Sunday night.

			=== Monday ===

			Monday was a bank-holiday, so we got into DCU about midday.

			We backed-up the rest of the stuff on Deathray, and re-installed that on Monday afternoon. Bitlbee was also setup late Monday night.

Receive at 22:30, 15 May 2009

2009-05-15T22:30:28Z

Show changes

Receive at 21:28, 15 May 2009

2009-05-15T21:28:01Z

← Older revision		Revision as of 21:28, 15 May 2009
Line 11:		Line 11:

	<i><lil_cain>: Thankfully in my time there have been no root exploits</i>		<i><lil_cain>: Thankfully in my time there have been no root exploits</i>


			=== Timeline of Events ===

			In an attempt to procrastinate from real work I've tried to document what happened on the week of no sleep, and insane amounts of coke. To give a bit of background, the hack happened just a few weeks after myself and johan were elected, lil_cain was the third admin, and werdz had recently moved from admin to webmaster. All of us were working full time on Intra, except for cian, who had a real job. Murphy was the main web server, running Ubuntu 6.06. Minerva was the main login server, also running Ubuntu.

			--receive (May 2009)


			==== Friday ====


			* Around 8.30am (ish) - People start to notice something is up. Most people are on the way to work and stuff. Pretty sure I got a text on the bus, can't remember who from.
			* 09:19 - people have posted to redbrick.computing.www about the problem.
			* 09:27 - people in #lobby start to notice.

			09:27 cambo anyone know what teh deal with the website is ?
			09:27 marvin singer is afk
			09:27 marvin tbh
			09:28 fatwa cambo: haxed. lil_cain quit his admin position, that's his 'goodbye' revenge
			09:28 tbolger entire site hax0rd ?
			09:28 cambo fatwa: srsly...
			09:28 fatwa tbolger: check www.redbrick.dcu.ie/~tbolger
			09:28 fatwa cambo: ya rly
			09:28 fatwa tbolger: it only seems to affect home directorys

			* 09:31 - ssh access to murphy is shut down.
			* 09:40 - apache is shutdown on murphy.
			* 09:42 - access is traced to the hurling club account, which is disusered.
			* 09:46 - checks on other servers reveal that auth.log is missing from minerva.
			* 10:00 - the decision is made to shutdown all access

			10:01 receive changed the topic of #lobby to: RedBrick access is about to be shut off, and will remain off until at least late tonight.

			* 10:30 - final mass hey goes out before shutdown
			* 10:32 - access to the network is cut off.

Lithium at 14:33, 26 April 2009

2009-04-26T14:33:00Z

← Older revision		Revision as of 14:33, 26 April 2009
Line 7:		Line 7:

	</center>		</center>
			<p><br>
			Redbrick died a death 25 April 2008, following the immortal words of lil_cain only a day before, on 24 April:<br>

			<i><lil_cain>: Thankfully in my time there have been no root exploits</i>

Fatwa: New page: Image:hax.png ;_;

2009-04-02T17:57:21Z

New page: <center> Image:hax.png ;_; </center>

New page

<center>
[[Image:hax.png]]

;_;

</center>