OLPC:Volunteer Infrastructure Group/2008-09-16

Action items

Sep 16 17:34:31 <hhardy>	ACTION ITEM: NDA for gangs
Sep 16 17:40:29 <hhardy>	ACTION ITEM: compartimentalize 
services to independent vhosts
Sep 16 17:46:23 <hhardy>	ACTION ITEM: owner for weka
Sep 16 17:50:56 <hhardy>	ACTION ITEM: gregdek to look 
for a buildmaster-general for weka
Sep 16 17:59:03 <hhardy>	ACTION ITEM: isforinsects to 
inventory databases and coordinate backup scripts
Sep 16 18:02:27 <hhardy>	ACTION ITEM: review solar with 
an eye to separating public-facing and confidential servers and 
services
Sep 16 18:16:09 <hhardy>	ACTION ITEM: future meeting to 
review "build process" and look for oppportunity to leverage help 
from say Fedora to keep devels focused on doing devel
Sep 16 18:21:10 <hhardy>	ACTION ITEM: check MX records for 
RT
Sep 16 18:26:52 <adric>	ACTION ITEM; Go on call for OLPC sysad. 
Roger
Sep 16 18:31:14 <mchua>	ACTION ITEM: mchua send notes, questions 
to i-gang list

Agenda

Sep 16 17:12:45 <hhardy>	Here is the draft agenda: 
Sep 16 17:12:56 <hhardy>	status of weka way forward 
for weka
Sep 16 17:12:56 <hhardy>	status of owl and swan 
need detailed transition plan for owl and swan deployment 
and pedal, crank upgrades
Sep 16 17:12:59 <hhardy>	what other resources should 
go to W-91 need detailed transition plan for moving things 
to W-91
Sep 16 17:13:02 <hhardy>	tsm backups client install 
and configure
Sep 16 17:13:05 <hhardy>	mock downtime; status
Sep 16 17:13:07 <hhardy>	database backups need 
step-by-step plan for database backups
Sep 16 17:13:10 <hhardy>	moving solar to W-91, move 
confidential stuff elsewhere (?)
Sep 16 17:13:12 <hhardy>	new rt instance
Sep 16 17:13:15 <hhardy>	UPS for critical machines
Sep 16 17:13:17 <hhardy>	rt status rt team "minions"
Sep 16 17:13:20 <hhardy>	big sister 24x7 oncall list
Sep 16 17:13:22 <hhardy>	new business
Sep 16 17:13:25 <hhardy>	any 
additions/corrections/deletions?
Sep 16 17:15:08 <cjl>	hhardy, maybe there should be a 
general follow-up line item for keeping track of things 
tha will move slowly (e..g spam, etc.)
Sep 16 17:15:28 <adric_>	Spam moves pretty quick 
:D They have good cable modems.
Sep 16 17:15:40 <cjl>	low on hte list, just before new 
business call it "old business" a:-)\
Sep 16 17:15:42 *	adric_ is now known as adric
Sep 16 17:15:46 <isforinsects>	Yeah, ones that belong to 
other people :S
Sep 16 17:15:51 <hhardy>	let's add spam to the topic 
isforinsects and kimquirk and I discussed issue this morning 
in a ftf
Sep 16 17:16:04 <hhardy>	to the agenda rather
Sep 16 17:16:09 <adric>	agree
Sep 16 17:16:17 *	gregdek hullos.  :)
Sep 16 17:16:26 <cjl>	It's also a corodination thing, good to track 
Sep 16 17:16:38 <isforinsects>	Alright, let's get started

Meeting notes

Sep 16 17:16:39 <hhardy>	gregdek: salutations
Sep 16 17:16:44 *	cjl not expecting any miracles
Sep 16 17:17:10 *	mchua pops in, waves
Sep 16 17:17:15 <mchua>	here to watch and learn about 
what's going on
Sep 16 17:17:37 <adric>	mel !!! yay :)
Sep 16 17:17:37 <mchua>	(and doc mtg notes to teamwiki, 
if hhardy so desires) ;)
Sep 16 17:17:54 <mchua>	or publicwiki, better yet, for 
stuff that can go there.
Sep 16 17:18:16 <isforinsects>	 /topic Status of Weka
Sep 16 17:18:24 *	isforinsects isn't OP ;(
Sep 16 17:18:34 *	adric consoles isforinsects 
Sep 16 17:18:36 <hhardy>	mchua: yes great notes 
are good to public wiki page 
http://wiki.laptop.org/go/OLPC:Volunteer_Infrastructure_Group
Sep 16 17:19:03 <isforinsects>	mchua, I've been logging
 the notes.  I have a couple scripts from mstone that aid 
in this
Sep 16 17:19:18 <mchua>	isforinsects: cool, can please pass 
these on to me post-meeting?
Sep 16 17:19:43 <cjl>	What is role of weka?
Sep 16 17:19:46 <mchua>	hhardy: (also for post-meeting) 
if there isn't yet an "intro to our existing infrastructure" 
page I'd be happy to ask the stupid questions required to 
make one ;)
Sep 16 17:20:08 <mchua>	...which should include things like 
"what machines we have and what they do" (as per cjl's question)
Sep 16 17:20:09 <cjl>	mchua like that one ^
Sep 16 17:20:14 <mchua>	aye-aye.
Sep 16 17:20:23 <hhardy>	mchua: we have a number of 
pages on iternal wiki which need to be traged for sensitive 
information and majority moved to the public pages
Sep 16 17:20:31 <adric>	I was so happy when I found out machine 
RT was on :)
Sep 16 17:20:33 <hhardy>	*triaged
Sep 16 17:20:40 <mchua>	hhardy: happy to do that - will ping 
you after this meeting on it.
Sep 16 17:21:05 *	ChanServ gives channel operator 
status to isforinsects
Sep 16 17:21:21 <mchua>	woo isforinsects has the power!
Sep 16 17:21:25 <mchua>	anyway, weka

Weka

Sep 16 17:21:32 *	isforinsects has changed the topic to: 
Status of Weka
Sep 16 17:21:42 <hhardy>	Weka is a HP DL 185
Sep 16 17:21:55 <hhardy>	it has 8 GB ram and 4.5 tb 
diskarray
Sep 16 17:22:12 <hhardy>	it is intended to be a build 
environment
Sep 16 17:22:25 <hhardy>	replacement for xs-dev and 
possible mock, pilgrim
Sep 16 17:22:57 <hhardy>	a community member has taken 
responsibility for setting it up but I haven't had any updates 
from Dennis as to status
Sep 16 17:23:00 <mchua>	xs-dev, mock, and pilgrim are all 
old/out-of-commision/deprecated machines, or soon-to-be?
Sep 16 17:23:13 <hhardy>	they are used atm
Sep 16 17:23:20 <hhardy>	xs-dev is way overloaded
Sep 16 17:24:13 <hhardy>	mchua: I should have had 
introductions in agenda would you say a few lines about who 
you are/what role you forsee in the infra-gang pls?
Sep 16 17:24:40 <hhardy>	{quick takes a bite of burrito}
Sep 16 17:24:58 <mchua>	Mel Chua, new QA/Support engineer as 
of yesterday, longtime volunteer, [[User:Mchua]] for more details
Sep 16 17:25:10 <cjl>	hhardy, is there someone that can 
take on the job of bugging that vol (nicely)?
Sep 16 17:25:33 <mchua>	i'm hoping i-gang will become part 
of what I do in my "off-hours" for fun/volunteeringness... 
interested in the infrastructure for QA/Support volunteers 
in particular, for obvious reasons
Sep 16 17:25:38 <hhardy>	who wants to shepherd weka?
Sep 16 17:26:07 <mchua>	(and when I say infrastructure, I 
mean non-technical as well, but for i-gang purposes, the 
technical ones mostly.)
Sep 16 17:26:09 <mchua>	</intro>
Sep 16 17:26:20 <hhardy>	dgilmore: ping?
Sep 16 17:26:38 <cjl>	Who has a good relationship 
with that vol and can bug (nicely)?
Sep 16 17:27:14 <cjl>	hhardy dgilmore is on #olpc on 
freenode
Sep 16 17:27:21 *	m_stone (~mstone@dhcp-47-72.media.mit.edu) 
has joined #olpc-admin
Sep 16 17:27:40 <hhardy>	could someone jump over and 
ping him see if he can update us ok weka?
Sep 16 17:27:44 *	cscott (~cscott@wireless-19-94.media.mit.edu) 
has joined #olpc-admin
Sep 16 17:27:47 <isforinsects>	m_stone, and cscott are going to 
show up and say something about this
Sep 16 17:27:48 <hhardy>	*on weka
Sep 16 17:28:06 <hhardy>	regarding the role of mock 
and pilgrim
Sep 16 17:28:20 <cscott>	oh, is that what we're 
talking about?
Sep 16 17:28:21 <hhardy>	we had downtime on mock 
on Sat which is unexplained as yet
Sep 16 17:28:29 <hhardy>	topic is weka
Sep 16 17:28:31 <cscott>	i was just going to 
mention how mock/pilgrim were set up
Sep 16 17:28:45 <hhardy>	thats fine we might as 
well capture that info now
Sep 16 17:28:54 <hhardy>	topic mock/pilgrim setup
Sep 16 17:28:59 *	hhardy has changed the topic to: 
mock/pilgrim setup
Sep 16 17:29:09 <cscott>	Briefly: 
https://laptop.org/internalwiki/index.php/Machine:mock#Basic_setup_and_package_installation
Sep 16 17:29:19 <cscott>	(likewise the section on pilgrim)
Sep 16 17:29:24 <hhardy>	most people here dont have 
internal wiki access
Sep 16 17:29:33 <mchua>	hhardy: can that setup info move 
to public, btw?
Sep 16 17:29:37 <hhardy>	we hopefully will migrate 
most pages to public wiki
Sep 16 17:29:39 <cscott>	user accounts on mock/pilgrim 
are set up with locked passwords, to prevent password-guessing 
attacks
Sep 16 17:30:04 <cscott>	recent accounts added to 
the machine have had passwords set, though
Sep 16 17:30:16 <hhardy>	such as mine I just made
Sep 16 17:30:29 <hhardy>	which I will fix
Sep 16 17:30:34 <cscott>	so i just recently changed 
PasswordAuthentication no in /etc/ssh/sshd_config
Sep 16 17:30:38 <hhardy>	nod
Sep 16 17:30:54 <cscott>	and locked the passwords 
on all the existing accounts (including hhardy)

Information access

Sep 16 17:30:55 <lfaraone>	hhardy: is internalwiki 
different than teamwiki?
Sep 16 17:31:01 <cjl>	Can't see internal wiki, maybe 
(at some point) you might migrate some of that to team 
wiki as an interim step?
Sep 16 17:31:11 <hhardy>	lfaraone: yes
Sep 16 17:31:28 <cjl>	lfaraone: I think it is the 
mysterious and never spoken of "third wiki".
Sep 16 17:31:30 <hhardy>	cjl: those things which 
can't go to public wiki can go to teamwiki
Sep 16 17:31:42 <cscott>	public wiki is public
Sep 16 17:31:56 <cscott>	internal wiki == olpc-internal 
stuff, people's addresses and cell phone #s, etc
Sep 16 17:32:08 <cscott>	HR stuff, plus network 
config, passwords for accounts, etc.
Sep 16 17:32:22 <lfaraone>	cscott: other words, stuff 
I should be NDA'd for :)
Sep 16 17:32:34 <cscott>	teamwiki was set up to 
provide a place for NDA stuff that could be shared with 
others who had signed an NDA
Sep 16 17:32:41 <cscott>	like schematics and etc.
Sep 16 17:32:54 <cscott>	teamwiki was explicitly 
set up to be *not* employee only
Sep 16 17:33:05 <cscott>	we wouldn't necessarily 
give NN's phone number to our AMD partners
Sep 16 17:33:20 <cscott>	but we would give them 
access to the XO schematic (which Quanta requires an NDA for)
Sep 16 17:33:25 <hhardy>	kimquirk today suggested 
moving the internal sysadmin docs to teamwiki, those which 
can't/shouldn't be public
Sep 16 17:33:45 <lfaraone>	hhardy: or you could NDA the 
VIG members.
Sep 16 17:34:04 <hhardy>	that is a possibility
Sep 16 17:34:05 <kimquirk>	and we should all 'gang' 
members to get to team... they have to understand an abide 
by the nda of our partners.
Sep 16 17:34:31 <hhardy>	ACTION ITEM: NDA for gangs
Sep 16 17:34:51 <kimquirk>	we had also discussed 
setting up parts of the teamwiki that were just for the 
community members... such as support-gang, 
infrastructure-gang, and not worry about getting to 
partner data.
Sep 16 17:35:01 <cscott>	we've been around this 
topic a number of times before, and seem to get hung up 
on how/where we store the root/database/etc passwords for
 our services
Sep 16 17:35:11 <cscott>	not on the public wiki, 
presumably
Sep 16 17:35:14 <mchua>	kimquirk: that's actually 
something I was wondering, if we should make signing an 
NDA a part of the teamwiki account application process, 
for everyone.
Sep 16 17:35:16 <m_stone>	:)
Sep 16 17:35:42 <lfaraone>	cscott: well, if you're 
gonna have the VIG assisting in admining your servers...
Sep 16 17:35:52 <cjl>	at moment, SG'ers get "Team" 
space on teamwiki, but no other "spaces".
Sep 16 17:35:53 <m_stone>	I believe we posted 
some discussion of this subject...
Sep 16 17:35:54 <kimquirk>	mchua: I think it is 
important to protect our partners...
Sep 16 17:36:13 <cscott>	lfaraone: yes, there 
wants to be some space for VIG
Sep 16 17:36:18 <isforinsects>	Ideally we can create 
a 'clear bright line' between public and internal.
Sep 16 17:36:20 <cscott>	lfaraone: the question 
is, where that space should be
Sep 16 17:36:35 <cscott>	lfaraone: and how to 
manage matters when VIG members join/leave
Sep 16 17:36:36 <isforinsects>	That way we can give 
people a lot more free rein (root, etc) in the public 
servers.
Sep 16 17:36:56 <lfaraone>	cscott: that's the 
idea of sudo.
Sep 16 17:37:05 <hhardy>	yes sudo, not root
Sep 16 17:37:07 <m_stone>	cscott: 
http://wiki.laptop.org/go/OLPC_talk:Volunteer_Infrastructure_Group, 
http://wiki.laptop.org/go/User:Mstone/Commentaries/Infrastructure_1, 
and http://wiki.laptop.org/go/User_talk:Mstone/Commentaries/Infrastructure_1 
seem apropos
Sep 16 17:37:14 <lfaraone>	cscott: you have auditability.
Sep 16 17:37:25 <cscott>	lfaraone: how does sudo manage 
database passwords?
Sep 16 17:37:37 <lfaraone>	cscott: hrm.. that's a 
different matter.
Sep 16 17:37:45 <lfaraone>	cscott: NDAs manage database 
passwords.
Sep 16 17:37:46 <cscott>	is the exact network configuration 
of our hosts sensitive?
Sep 16 17:37:51 <cscott>	if so, where is that stored?
Sep 16 17:38:14 <hhardy>	let's carry on this topic to 
the discussion page for the mstone commentaries
Sep 16 17:38:24 <hhardy>	anything further on mock/pilgrim?
Sep 16 17:39:01 <hhardy>	someone said dgilmore had been seen? 
ping him?
Sep 16 17:39:06 <cscott>	well, one last thing for VIG: my 
long term proposal has been to split up our services into separate 
vhosts as much as possible
Sep 16 17:39:23 <cscott>	so that we can give people root 
on (say) wiki.laptop.org without worrying that we're going to 
crash (say) dev.laptop.org.
Sep 16 17:39:39 <dogi>	hi all
Sep 16 17:39:52 <hhardy>	dogi: hail
Sep 16 17:39:56 <cscott>	having once brought all of dev, 
trac, pootle, etc down at once accidentally, i am partial to 
splitting services
Sep 16 17:40:06 <dogi>	sorry i m late
Sep 16 17:40:15 <cscott>	the counter argument made is the 
"put all of your eggs in one basket and *watch that basket*" one
Sep 16 17:40:25 >kimquirk<	is http://pastebin.ca/1204800 
pretty much what the various wikis are for / how access to them 
is handled? (trying to figure out where to put which meeting notes)
Sep 16 17:40:29 <hhardy>	ACTION ITEM: compartimentalize 
services to independent vhosts

Back to weka

Sep 16 17:41:22 <hhardy>	what IRC was dgilmore in?
Sep 16 17:41:30 <cscott>	remote power on/off would have 
made fixing mock over the weekend easier, too.
Sep 16 17:41:42 <cscott>	i think that's part of 
henry's action item to move servers to W20
Sep 16 17:41:50 <cjl>	I jsus pinged dgilmore in #olpc 
on freenode, no response.
Sep 16 17:41:57 <hhardy>	W19, the MIT colo facility
Sep 16 17:42:13 <hhardy>	ok I feel like we need a 
new owner for weka to move forward
Sep 16 17:42:17 <lfaraone>	weka?
Sep 16 17:42:26 <dogi>	weka?
Sep 16 17:42:29 <hhardy>	weka.l.o, new build environment 
server
Sep 16 17:42:41 <lfaraone>	ah
Sep 16 17:42:44 <dogi>	thx
Sep 16 17:42:55 <lfaraone>	damn, that'll ruin our w.l.o 
abbr for the wiki.
Sep 16 17:43:00 <lfaraone>	:)\
Sep 16 17:43:17 <adric>	is weka a bird or a power source, or both?
Sep 16 17:43:20 <hhardy>	it really needs someone with 
deep fedora build experience 
Sep 16 17:43:37 <hhardy>	bird, possibly powersource if
 we adapt the cow-powered generator
Sep 16 17:43:38 <lfaraone>	m_stone: you?
Sep 16 17:44:17 <hhardy>	m_stone is v busy managing 
release
Sep 16 17:44:19 <cjl>	lfaraone: I think m_stone might be 
busy right now with 8.2
Sep 16 17:44:54 <cjl>	Who has Fedora contacts?
Sep 16 17:44:54 <hhardy>	erikg or martin possibly
Sep 16 17:45:25 <hhardy>	gregdek: we need someone to
 replace/augment dgilmore in getting the new build environment 
server, weka.l.o, setup
Sep 16 17:46:23 <hhardy>	ACTION ITEM: owner for weka
Sep 16 17:46:24 <lfaraone>	cjl: bernie, but he left :(

Swan and owl

Sep 16 17:46:40 *	hhardy has changed the topic to: swan 
and owl status
Sep 16 17:46:41 <cjl>	maybe gregdek can help.
Sep 16 17:46:56 <cjl>	We don't have t osolve this here and 
now, we've got an action item.
Sep 16 17:46:59 <hhardy>	owl and swan are intended to 
be backups/mirrors of crank and pedal
Sep 16 17:47:08 <hhardy>	they are set up and work
Sep 16 17:47:19 <hhardy>	I have tested chrooting to 
the mirror of dev
Sep 16 17:47:25 <hhardy>	I have not tested all the 
critical apps
Sep 16 17:47:48 <hhardy>	I have not made a nice 
script, or better, init state 4 setup to do the changeover 
automagically
Sep 16 17:48:27 <hhardy>	the intent is to get these 
online at W-91, then schedule maintenence intervals to update 
crank and pedal
Sep 16 17:48:46 <hhardy>	per kimquirk, we need a 
detailled plan for the deployment and upgrade
Sep 16 17:48:59 <lfaraone>	W91?
Sep 16 17:49:10 <hhardy>	and documentation of how to 
rebuild crank and pedal if they die
Sep 16 17:49:11 <gregdek>	Sorry, got pulled away.
Sep 16 17:49:14 *	gregdek reads up.
Sep 16 17:49:21 <hhardy>	W-91, a building at MIT where 
the main colo facility is
Sep 16 17:49:35 <gregdek>	OK.
Sep 16 17:49:52 <hhardy>	gregdek: need to replace/augment 
dgilmore as owner of weka, the new xs-dev and build environment server
Sep 16 17:49:54 <gregdek>	How about a cattle call to 
the Fedora infrastructure team?  See if someone wants to step 
up to a larger role in OLPC-land?
Sep 16 17:50:02 <gregdek>	I will ask around.
Sep 16 17:50:03 <hhardy>	gregdek: yes
Sep 16 17:50:11 <isforinsects>	Are, it's time to ask the 
Fedora team
Sep 16 17:50:56 <hhardy>	ACTION ITEM: gregdek to look 
for a buildmaster-general for weka
Sep 16 17:51:41 <hhardy>	paulproteus had some helpful 
suggestions about owl and swan but havent seen him online recently

TSM backups

Sep 16 17:51:59 *	hhardy has changed the topic to: 
TSM backups
Sep 16 17:52:21 <hhardy>	TSM is Tivoli Storage 
Manager from IBM
Sep 16 17:52:31 <hhardy>	it is a big robot at W-91
Sep 16 17:52:39 <hhardy>	very cool thing
Sep 16 17:52:54 <dogi>	:)
Sep 16 17:53:01 <hhardy>	we need TSM ported to
 ubuntu and configured to work w/ the MIT system
Sep 16 17:53:40 <hhardy>	this is on my plate but
 I could use some help here
Sep 16 17:53:46 <hhardy>	any volunteers?
Sep 16 17:53:56 <dogi>	me?
Sep 16 17:54:19 <isforinsects>	paulproteus is online 
right now in #cc over at freenode
Sep 16 17:54:27 <hhardy>	great dogi do you want 
to set up a FTF to talk about this sometime this week?
Sep 16 17:54:31 <isforinsects>	shall I go invite him 
if we have specific questions?
Sep 16 17:54:38 <hhardy>	isfor you can ping him
 and invite him over here pls?
Sep 16 17:54:43 <dogi>	FTF?
Sep 16 17:55:04 <adric>	FTF == "face to face"
Sep 16 17:55:06 <lfaraone>	dogi: face to face
Sep 16 17:55:09 <dogi>	ok :)
Sep 16 17:55:48 <hhardy>	you know, now through a
 glass darkly, but then, face to face
Sep 16 17:55:53 *	paulproteus (~paulprote@wide-rose.makesad.us) 
has joined #olpc-admin
Sep 16 17:56:11 <hhardy>	greetings paulproteus
Sep 16 17:56:16 <paulproteus>	Howdy.
Sep 16 17:56:27 <hhardy>	we talked about mock
Sep 16 17:56:28 <paulproteus>	Personal load average: 9.45
Sep 16 17:56:39 <adric>	yikes

DB backups

Sep 16 17:56:43 *	hhardy has changed the topic to: 
database backups
Sep 16 17:57:04 <paulproteus>	MySQL?
Sep 16 17:57:10 <hhardy>	We need to ID all the mission 
critical databases and set up hot backup scripts
Sep 16 17:57:16 <hhardy>	mysql and lostgres
Sep 16 17:57:20 <paulproteus>	IMHO just daily save them 
to files and let the filesystem backup scripts save them.
Sep 16 17:57:32 <hhardy>	paulproteus yep
Sep 16 17:57:37 <paulproteus>	Great.
Sep 16 17:57:50 <adric>	mysqldmp usually locks the dbs, 
but you are on the right track
Sep 16 17:57:57 <hhardy>	does anyone have useful 
hot backup scripts for those?
Sep 16 17:58:13 <hhardy>	hot as in, non-locking
Sep 16 17:58:29 <hhardy>	*postgres
Sep 16 17:59:02 <paulproteus>	http://mysqlbackup.zmanda.com/ FWIW
Sep 16 17:59:03 <hhardy>	ACTION ITEM: isforinsects to 
inventory databases and coordinate backup scripts
Sep 16 17:59:03 <adric>	for mysql we often just rsync the data 
files, but it's slightly risky
Sep 16 17:59:15 <paulproteus>	I have heard good things but 
have no specific knowledge of that.
Sep 16 17:59:36 *	hhardy has changed the topic to: moving s
olar.l.o to W-91
Sep 16 17:59:55 <hhardy>	today isfor and kimquirk and I 
were talking about possibly moving solar to W-91
Sep 16 18:00:38 <hhardy>	that would mean moving any 
"sensitive" stuff off it and making it a "public facing" 
machine which would be a vserver environment as well
Sep 16 18:00:48 <hhardy>	there is still the open 
quesiton of xen v vserver
Sep 16 18:00:52 <hhardy>	we have both atm
Sep 16 18:01:22 <cjl>	hhardy: dgilmore is responding to 
ping in #olpc
Sep 16 18:01:29 <cjl>	may be joining shortly
Sep 16 18:01:33 <hhardy>	ok cool
Sep 16 18:01:33 <m_stone>	lfaraone, cjl, cscott: 
I have updated my statement of "communal maintenance requirements".
Sep 16 18:01:36 <m_stone>	please review.
Sep 16 18:01:45 <m_stone>	(based on your prior feedback)
Sep 16 18:02:06 <cjl>	m_stone: will do
Sep 16 18:02:27 <hhardy>	ACTION ITEM: review solar
 with an eye to separating public-facing and confidential 
servers and services

RT

Sep 16 18:02:42 *	hhardy has changed the topic to: rt
Sep 16 18:02:51 <hhardy>	adric: whats new with rt?
Sep 16 18:03:11 <adric>	Wel,, after the good meeting on Th,
 everyone wants to move forward with 3.8.x as soon as.
Sep 16 18:03:28 <isforinsects>	?
Sep 16 18:03:30 <adric>	There's plenty of work to prep test 
and execute that
Sep 16 18:03:36 <isforinsects>	our meeting last tuesday?
Sep 16 18:03:40 <hhardy>	which meeting was that?
Sep 16 18:03:41 <adric>	yes,
Sep 16 18:03:51 *	dgilmore (~dgilmore@anubis.ausil.us) 
has joined #olpc-admin
Sep 16 18:03:53 <adric>	I dunno what to call that meeting..
Sep 16 18:04:04 <hhardy>	VIG meeting
Sep 16 18:04:06 <adric>	it was S-G and  kim, and some other folk
Sep 16 18:04:12 <hhardy>	oh ok
Sep 16 18:04:18 <isforinsects>	It sounds like there is a 
requirement of a second RT to be set up for private OLPC things.
Sep 16 18:04:19 <adric>	Mr Kane was there for part of it, even
Sep 16 18:04:29 <adric>	isforinsects: Yes, there is.
Sep 16 18:04:34 <hhardy>	yes we need a "private" rt 
instance
Sep 16 18:04:36 <isforinsects>	Oh right.  That wasn't really
 a public meeting adric.
Sep 16 18:04:42 <isforinsects>	For instance, I wasn't
 invited to it :P
Sep 16 18:04:48 <hhardy>	that will have to be 
set up by employees and possibly contractors
Sep 16 18:05:03 <dgilmore>	isforinsects: i gave
 one to cscott and m_stone a couple of weeks back.  
ive not had a chace to do anything since then. and 
they chose to wait foir 9.1 to implement the changes i
 was working on 
Sep 16 18:05:07 *	adric consoles isforinsects 
Sep 16 18:05:10 <hhardy>	so that the main rt can 
be more user-administered
Sep 16 18:05:14 <hhardy>	dgilmore: welcome
Sep 16 18:05:26 <lfaraone>	m_stone: will do
Sep 16 18:05:30 <dgilmore>	hhardy: :)
Sep 16 18:05:32 <hhardy>	dgilmore: status of weka?
Sep 16 18:05:39 <dgilmore>	hhardy: ^^^

yet more weka

Sep 16 18:05:57 *	hhardy has changed the topic to:
 weka status
Sep 16 18:06:13 <cjl>	cscott m_stone  is weka on 
hold until 9.1 is out?
Sep 16 18:06:21 <adric>	Some other priorties were 
fleshed out at the unamed meeting Th, namely Spam, 
and the other need improvements to RT
Sep 16 18:06:21 <kimquirk>	whoooo... i hope not
Sep 16 18:06:53 <lfaraone>	m_stone: makes sense.
Sep 16 18:07:06 <dgilmore>	cjl: not until it is
 out
Sep 16 18:07:12 <cscott>	cjl, kimquirk: 
i don't think it's a good time to change our build
process for 8.2.
Sep 16 18:07:17 <dgilmore>	cjl: but once 8.2 is out 
Sep 16 18:07:27 <dgilmore>	in the 9.1 development
 cycle it will be implemented
Sep 16 18:07:41 <kimquirk>	right... so we're not
 going to wait to get weka up... what about hte 8.2.1 release?
Sep 16 18:07:58 <cscott>	kimquirk: will likely 
be done with the 8.2 process
Sep 16 18:08:04 <kimquirk>	or, I guess I would prefer 
to have a target date much sooner than March 2009
Sep 16 18:08:05 <hhardy>	dgilmore: is there any 
need for additional help to set up weka?
Sep 16 18:08:16 <cscott>	weka is set up
Sep 16 18:08:27 <cscott>	i'm not sure what we're
 talking about here
Sep 16 18:08:38 <kimquirk>	I thought we needed a 
new build server...
Sep 16 18:08:51 <cscott>	we will move joyride
 builds off xs-dev first
Sep 16 18:09:13 <kimquirk>	i thought that was why
 we bought weka... to start movign things off xs-dev
Sep 16 18:09:14 <cscott>	the stable builds are,
 well, stable -- they lag behind whatever else we're doing
Sep 16 18:09:33 <cscott>	kimquirk: yes, but m_stone 
and i are rather busy making your 8.2 build at the moment
Sep 16 18:09:43 <kimquirk>	(my 8.2 build... thanks!)
Sep 16 18:09:52 <dgilmore>	hhardy: help is always welcome. 
Sep 16 18:10:05 <cscott>	so it's not a good time 
for us to start moving services -- even messing with 
joyride at the moment
Sep 16 18:10:10 <hhardy>	ok gregdek is going to 
see if there are any fedora folks who can help with weka
Sep 16 18:10:10 <kimquirk>	so the question is 
whether builds can be owned by the community and we 
don't need to wait for m_stone and cscott to be available
Sep 16 18:10:17 <cscott>	at soon as 8.2 is 
"done" we can tolerate some joyride downtime to move 
stuff around
Sep 16 18:10:26 <dgilmore>	cscott: :)  yep  once
 8.2 is done we can turn the world upside down
Sep 16 18:10:35 <cjl>	big difference between getting
 weka built and putting it into "production use", it is
 built (that is the status question) and it will be
 phased into production as build server when people 
aren't to busy making builds to mess with infrastructure?
Sep 16 18:10:42 *	reubencaron_ has quit (Quit:
 reubencaron_)
Sep 16 18:10:50 <gregdek>	I've made the first
 call for volunteers to assist with weka.  We'll see 
how it goes.
Sep 16 18:10:51 <cscott>	cjl: that's my 
understanding, yes.
Sep 16 18:11:01 <kimquirk>	ok

RT

Sep 16 18:11:11 *	hhardy has changed the topic to: RT
Sep 16 18:11:16 <hhardy>	back to RT, adric?
Sep 16 18:11:22 <adric>	hhardy: Sure
Sep 16 18:11:24 <cscott>	if volunteers want to 
get started, you can start working from dennis' mash 
stuff w/o us
Sep 16 18:11:38 <adric>	So we have 1) Upgrade , 2) 
features, and 3) Spam
Sep 16 18:11:42 <kimquirk>	cscott: is there 
documentation that we can point to for that to happen?
Sep 16 18:11:42 <cscott>	i'm just not going 
to switch any of the 'production' builds over to any
 alternate infrastructure until 8.2 is in the can.
Sep 16 18:11:52 <kimquirk>	sounds good.
Sep 16 18:12:42 <hhardy>	dgilmore: thanks for
 your efforts on weka thus far
Sep 16 18:13:02 *	hhardy has changed the topic 
to: RT upgrade
Sep 16 18:13:33 <cscott>	kimquirk: michael 
and i wrote some notes; mine are at 
https://laptop.org/internalwiki/index.php/Machine:xo-build
Sep 16 18:13:43 <cscott>	m_stone should be
 poked to post his somewhere
Sep 16 18:13:49 *	cscott pokes m_stone
Sep 16 18:13:57 <cscott>	but the best 
documentation right now is the code
Sep 16 18:14:23 <lfaraone>	hhardy: 
it seems that spam has worsended.
Sep 16 18:14:27 <kimquirk>	cscott: thanks
Sep 16 18:14:41 <cscott>	m_stone: did we 
get the git repos on xo-build pushed to dev?
Sep 16 18:14:50 <lfaraone>	cscott: kinda 
hard for us to work on collab with you on that (xo-build)
Sep 16 18:14:58 <dgilmore>	cscott: i put 
them in my ~
Sep 16 18:15:27 <dgilmore>	will in my ~/public_git
Sep 16 18:15:46 <cjl>	hhardy, make note for future
 meeting to review "build process" and look for oppportunity to leverage help from say Fedora to keep devels focused on doing devel 
Sep 16 18:16:09 <hhardy>	ACTION ITEM: future 
meeting to review "build process" and look for 
oppportunity to leverage help from say Fedora to
 keep devels focused on doing devel
Sep 16 18:16:28 <hhardy>	ok dont want to 
run too much longer
Sep 16 18:16:37 <cjl>	All back to RT now?
Sep 16 18:16:41 <hhardy>	Adric, we need to get
 rt backed up and then make that upgrade
Sep 16 18:16:53 <adric>	hhardy: Agree. You have ticket?
Sep 16 18:16:57 <hhardy>	let's make a plan and
 associate a scedule with it
Sep 16 18:17:18 <hhardy>	yea getting number
Sep 16 18:17:37 <adric>	http://rt.laptop.org/Ticket/Display.html?id=20497
Sep 16 18:17:45 <lfaraone>	hhardy: shall we talk about openid?
Sep 16 18:18:09 <hhardy>	lfaraone: in new business if
 you like at end
Sep 16 18:18:34 <hhardy>	adric expressed a desire for 
minions
Sep 16 18:18:42 <hhardy>	anyone want to be an rt minion?
Sep 16 18:18:50 <lfaraone>	hhardy: what would such a task
 entail?
Sep 16 18:19:05 <hhardy>	working with adric to maintain rt
Sep 16 18:19:08 <cjl>	hhardy: given that RT may have a 
little downtime during upgrade, is there any other piecs that 
need to be lined up so tha it doesn't (for instance) drop mail?
Sep 16 18:19:37 <hhardy>	cjl: the upstream MX (spam.l.o)
 should queue and redeliver
Sep 16 18:20:06 <cjl>	Are we sure all MX records are in place now?
Sep 16 18:20:21 <cscott>	lfaraone: yes, it will be hard 
for m_stone and I to collaborate on this until 8.2 ships, sorry
Sep 16 18:20:26 <hhardy>	lfaraone is a pretty experienced
 rt user already (=ffm)
Sep 16 18:20:43 <cscott>	dgilmore, m_stone: releng repo
 pushed to http://dev.laptop.org/git/projects/releng
Sep 16 18:21:00 <hhardy>	I forgot to ask MB about the
 MX records after I got back from vacation
Sep 16 18:21:10 <hhardy>	ACTION ITEM: check MX records 
for RT
Sep 16 18:21:23 <lfaraone>	hhardy: sure.
Sep 16 18:21:43 <mchua>	speaking of RT and making it more 
community-maintained - are there any plans to offer community
 groups / deployments / repair centers / etc. RT hosting?
Sep 16 18:21:46 <adric>	Indeed, ffm is. I guess lfaraone 
is okay :)
Sep 16 18:21:51 <hhardy>	mchua: yes
Sep 16 18:22:05 <mchua>	hhardy: when/how/what's up with that? 
how do I help out / find out more?
Sep 16 18:22:08 <adric>	there's a queue for one repair 
center already
Sep 16 18:22:15 <lfaraone>	adric: huh?
Sep 16 18:22:21 <lfaraone>	ah
Sep 16 18:22:26 <cjl>	What else can go wrong when taking
 RT down briefly, or what might need to be repointed or 
renamed for mail to flow correctly after an upgrade?  
Sep 16 18:22:27 <hhardy>	send a ticket to sysadmin
 and ask for queue to be set up
Sep 16 18:22:30 <dgilmore>	cscott: thanks
Sep 16 18:22:31 <mchua>	adric: i saw the olpc-cinci queue
 - was thinking of separate RT instances, even, that they 
could admin/setup themselves
Sep 16 18:22:34 <adric>	The threading is meeting is lossy, sorry.
Sep 16 18:22:35 <mchua>	hhardy: ^^
Sep 16 18:22:40 <dogi>	cscott: i cannot enter the internal wiki ...
Sep 16 18:22:46 <lfaraone>	cscott: it'd have to be very 
short downtime, how long are we talking about?
Sep 16 18:23:07 <adric>	mchua: It's concevable after we get 
the main nstance upgraded and featureful that we can clone it
 out, yeah
Sep 16 18:23:08 <cscott>	dogi: yes, there's an action
 item to move machine documentation off the internal wiki
Sep 16 18:23:09 <lfaraone>	mchua: hrm... IMHO what we have
 now will work, but it wont't scale.
Sep 16 18:23:20 <hhardy>	having 20-some virtual machines 
seems like overkill
Sep 16 18:23:22 <cscott>	lfaraone: which downtime are
 you talking about?
Sep 16 18:23:26 <lfaraone>	cscott: rt
Sep 16 18:23:36 <dogi>	cscott: ok, thx
Sep 16 18:23:53 <cscott>	lfaraone: sorry, i don't 
know anything about rt
Sep 16 18:23:58 <cjl>	cscott: lfaraone I was the one that
mentioned downtime on RT.  Trying to make sure all dependencies
 are considered.
Sep 16 18:24:08 <hhardy>	mchua: coordinate with adric 
and ffm re: repair center queues?
Sep 16 18:24:59 <hhardy>	anything further on rt for now?

big sister

Sep 16 18:25:07 *	hhardy has changed the topic to:
 big sister
Sep 16 18:25:08 *	adric note the clock time.
 I have hungry SO here ..
Sep 16 18:25:21 <hhardy>	5 more minutes
Sep 16 18:25:53 <hhardy>	email me if you'd be
 willing to serve as part of a 24/7 group to watch big
 sis and be paged if things go down
Sep 16 18:25:55 <cjl>	hhardy: has tehre been any 
folow up with jgay and fsf tweaks to their RT that we 
might want to use?
Sep 16 18:26:07 <hhardy>	cjl: don't know about that
Sep 16 18:26:11 <mchua>	lfaraone: yeah, that's my worry 
- not scaling. Talk with you and adric about that post-meeting, 
if that's ok.
Sep 16 18:26:17 *	gregdek has quit (Ping timeout: 480 seconds)
Sep 16 18:26:26 <lfaraone>	mchua: sure.

new business

Sep 16 18:26:38 *	hhardy has changed the topic to: 
new business
Sep 16 18:26:43 <mchua>	hhardy: I'll be at 1cc absurdly
 late at least 2 nights a week; can take shifts for big 
sis if you show me what to do.
Sep 16 18:26:52 <adric>	ACTION ITEM; Go on call for OLPC 
sysad. Roger
Sep 16 18:27:22 <hhardy>	ffm, your idea on 
single-signon? quickly?
Sep 16 18:28:03 <lfaraone>	hhardy: actually, 
I have to be going, we'll do it next meeting.
Sep 16 18:28:04 <lfaraone>	(sorry)
Sep 16 18:28:06 *	lfaraone is afk
Sep 16 18:28:13 <hhardy>	ok done for now thanks folks