Game Jam Boston June 2007/Feedback
Judging Forms
These are the forms we will be using to collect data during the judging on the final day of the Jam. Please comment and edit, but make sure anything you add would be applicable to all games. Note that the parents are going to be filling these out for their kids.
- Game Jam Form/Child Survey - demographics, one per judge
- Game Jam Form/Child Feedback - feedback on specific games, one per judge per game
- Game Jam Form/Developer Observations - for developers to note interesting things during judging
- Game Jam Form/Adult Feedback - for non-judges to give feedback on specific games
Feedback on judging forms
- Are all 'child forms' to be filled by the parent? No direct comment from the child?
- Comments were mainly from the judges, who were children. Younger cihldren were helped by their parents to fill out forms quickly.
- Also, what about the type of games they normally play? (ie: browser based or cd/installed; action vs. puzzle; etc) Did they like the graphics? Sound? Idea? Were the input methods (keys, touchpad) good? Would they like to play it again? Would they add/remove something?
- We didn't ask enough normalizing questions about their background... did ask for general feedback about input methods, but it was generally hard for them to get used to the varying levels of lag.
- Another suggestion could be to record a video of the children playing the game, so that later you may see what interactions they had with other kids and the games... 2cts, --Xavi 02:34, 10 June 2007 (EDT)
Judges' comments
One parent said Kuku Anakula was "...a great game for 2nd to 3rd graders" - some judges asked for easier math questions, others for harder ones.
One judge liked the Labyrinth game so much they returned to play it afterwards. A pair of siblings had a small argument because one got to play it through twice.
One judge described 3dPong as "awesome... oh yes, I scored another point!"
Pythagoras was described as having the "...best intro of the session"
Reversi was so "awesome" and "very absorbing" that it was "only easy to leave because it was a two player game and the game ended."
Sprayplay was "really fun" and one young judge said it was the "best game yet."
Typeblocker was "addictive" and "hard to stop," and Sort spanned the widest age range, being enjoyed by a 12-year-old judge and a 3.5-year-old judge.
What kids said was good
- The games were good. (Big underlines and smiley faces on the board next to this one!)
What kids said we could change
- Put more than one game on each laptop so kids wouldn't have to run around the room so much. (Note: We deliberately didn't do this because we thought it would take them longer to switch games than to walk between laptops. Mel still thinks this is true.)
- Make it easier to play the games right away. This encompasses both "make it easier for us to figure out how to start up the games when we first see the laptop" and "make games where it is immediately obvious to us what to do in order to play it."
- Get a loudspeaker or something so you can make announcements in the crowded testing room and be heard.
What kids said was not good
- The room was too loud and you could not hear things.
- We needed more food.
- The backgrounds of the games were boring. They were not fun colored graphics, they were just blank.
- The games froze a lot. (Much agreement from the young judges here.)
- The games were not finished and did not always work. (Much agreement from the judges. We explained the games were made really quickly and would probably not have all the bugs worked out.)
- There were not enough computers for every judge to play at once (we expected about 15 judges and ended up with three times that).
- There was too much moving around. (Need longer time to play certain games?)
- It was hard to figure out how to play some games without an instruction manual. (Note: Mel thinks this is an indication that games should be made more intuitive to play and that non-written documentation should be produced - not necessarily that games should start having instruction manuals!)
Judging Data
See GameJam_BostonJune2007_JudgingData for numbers.
There were 18 data-judges (from 20 folders). I did not count the total number of judges. Folder #9 did not get filled out, folder #20 did not get returned.
Identifying information in comments (gender, age) have been stripped out and been replaced with generic [child] or [judge] nouns-in-brackets. You can correlate back to gender and age of judge by number, though.
Gender distribution: 4 female, 14 male. Check to see if your game appealed more strongly to one gender or another.
Age distribution: (6-8 years) - 8, (9-11 years) - 7, (12-14 years) - 3. Note that 2 of the female judges were 12-14 years old.
Gaming experience: All but 3 judges played video games at least once a week. All but 5 played video games at least 3 times a week. This may account for the "it needs instructions" mentality we occasionally encountered.
Computer experience: All but 3 judges (who started between 5-8 years old) started using computers between 0-4 years old. All judges used computers both at school and home. 7 judges had their own computers at home.
Some general comments: The judging was definitely biased towards easily demonstratable, fast-to-play games. Any suggestions on how we can change this would be very welcome. During the event, some children wanted to play with the "real computers" (non-XOs), some wanted to play with the XOs and not the "normal computers."
Participant feedback
User:MitchellNCharity#Game Jam feedback
From an adult judge's assistant: I believe that the judges who RSVP'ed should have been given priority to play with the games (at least initially). Perhaps a list of those pre-registered could have been read and those students instructed to go to the computer room first. (perhaps even have specific assignments for each judge to start at a particular game). It was very chaotic in the room and it was very difficult for my two judges to get to play games (in fact, they were "done" even before they had to leave because they were disappointed). A bigger room was needed, or perhaps the use of two rooms and then judges switch rooms after 30 minutes or so. I know you did not anticipate that so many people would show up, and you did the best you could with what you had. I'm sure it was very overwhelming to have so many judges show up! I just think those that RSVP'ed should have been given priority. Ultimately, it's a good lesson for kids that planning ahead sometimes has rewards! :)
From ?: An exceptional venue for the Jam. Easily accessible and convienent. Would really like to see the Gnash player revved up more. Whole team had a great time.
notes by SJ from feedback session
Mike G -
Coming from the industry, getting to go through a whole development cycle, trying something out, design, working through the realization that it won't work for your audience, changing your design goals and language and gameplay, going through the crunch, and seeing people use the result and enjoy it in three days, is what keeps me in the industry. If you want feedback having more of a flow for organization, giving people more lag time, some people were unable to come.
Drag: Then I saw the XOs for the first time and thought, this is them? And an organizer was telling me no kid has ever spent more than 30 sec on opening one and after two minutes I had to find another person on my team.
Drag- Then I saw the xos for the first time, and htought, this is them? and an organizer was telling me mno kid has ever spent more than 30 sec on opening one .l going from being upanable to poen to getting a gme to run in a lang i've never read before; wow.
kids these days don't really et to see how to hack their own mes 0 on an ibm gtame you can't really do tha. My understanding is that kids ill be able to get in there and change all the code and open it up; it's fascinating to see it happening and think about how long it's actuall taken. I'm 40, so that was 30 eyyrs ago that I started. Amazin g that it will take that long; then we need to see what they do with tech. had been fun to see the computers and what's going to be done; people come out here and donate their time and make something.
X - I was reminded this weekend that I'm not a game programmer; I mostly do web dev. Sort of gettin back into the middle of a project with all these complicated problems and remembering how to work a lang as rick h as pythog, as opposed to the stou[id thing you get when trying to work java or c [nb - pyheads rejoice!] was rewarding and really really frustrating. I found you learn a lot about someone by working with them... and having to sit next to tem and trade code back and forth and look over eachother's shoulders and say, why did you call it that?
M (fl) smoking a peace pipe tonight - reminds me of a retreat wihere everyone hwo talked had a talking stick. the first thing I wanted to say was thanks for hosting to the organizers. based on the need to do this, perhaps it should be a biannual and quarterly event!
- next time -- [we should] decide before we get there what the game is before arriving; it would be good to have one visionary and og with that, or kill it. kevin wrote a backup game for us in case [easel didn't work].
How many people realize that the concept of the game jam isn't limted to olpc. it was made by a bunch of pro developers 4-5 yrs ago... there will be other game jams -- same concept in a few days -- with different themes.
---
the coolest part of the game jam was to do something really cool and pove I could do it. I" ma mechE going to ollin comllege. have a few yrs of programming, did a lot in high school, but haven't -- most in the lats 2 yrs was making things to show beam deflection. I knew it wa sth I could do and get into; more of a learning curve I wanted there to be... I thought it was realy cool that I could work with all this new stuf I doidn't know how to deal with and make sth that was viable.
Angela -thanks for letting me be here. I'm not a programmer... have nevre done this in my life. intersted in creating one. william called me abot doing sth or not. I said ok... learned the sound, music, all this; in the future i will become a programmer.
william - v important thing that was successful -- there was gallons of coffee good tasting and strong. so people would keep awake when they changed their design. on a broader horizon - if you look at thea aspects here with the game jam, it's like the objectives of the olpc, but its us - we're looearnig by ding; we're poutting together this dynamic environment, freating a lot of thingscoming through and learn from them.
I also want to thank the support em, brian, roberto, matt, for the support - more than that, part of each team - in a short time period we ewre making a tame the way thte real kids did in the environmen. t his wmade th uality of games we could show in a short period of time -- if we were going to design by steeling images off the we to do sth we woul dhave lost fous on the game. the other side i find from a learning standpoint, personally: I learned more about pygame and game de in general in 3 days than If I'd been reading books and playing with mysefl in 6 monts. 1 it's forced focus, 2 it's just the synergy you get. on the othe tefchnical side, the feedback of talking to noah, and bernardo - don't want to miss names - I was able to refactor code without ever turning it into production code. could come by and say 'I know there's a better way - this is what I would do ' 0- it wasn't a matte rof "hey dummy, here's how you'd do it." Thanks noah on animation and bits. [I still have aquestion for you :-)]
bernie - I'm an alien here, behind the scenes doing low level stuff; it's been good to see so many people hacking on our laptops and the freal end users hacking on them; I hope we're groinw los of hakers here and overseas in the coutnreis geting the laptops. maybe weo could improve sth on graphics accelleration; Iveseen games crawling a little, probably by our graphics system not being tuned.
owen - also an oibserver heloing eople here and there. Iwas impressed with how patient everyone was. people who are not only gtk devs, they don't run linux, don't run pythong; we re vgiving them a laptop, and are like, these things work, but this other stuff doesn't work.. and everyone is like, ok, we'll run with it . I was really impressed with how good th games came out given those limitations. you realize you can't have so man ypdates per second so you lower them or oiptimize renderning and get really smoth motion. I never thought that would be possible on the b2 hardware.
and the kiids were really patient too.. you'd be sitting there trying ot restart sugar... and they'd roll with it . I found I was on an airplane with a 9 yr old, and et him play with th laopt for 2 hr, he was fascinated the wholet ime. you'd think they would be unhappy bc it's not a playstation but they're really accepting.
manu - from the technical side, I relize wthe importance o this event. on projects, it takes lots of months; getgting 11 games running in 3 days, that's amazing. you could ask 3-4 devs to work on this for weeks and they couldn't do it so soon. doing this in 3 days was amazing. I cn't forget the way things have been done and people, everyone, contriuted towards making them and hleping the others. it was a community sharing and learning experience.
molly - it wa sa good weekend; I came in not even think ing I'd make a game. I appreciate everyon'e patience in the snags along the way; we threw a lot of this together in the last 5 or 6 days.. thanks.
ol2 - I also found this wknd wa amazing. it was fulfuiling as well; you have to fcocus. a lot of you said soimilar things already. I learened a lot. jessie - I was running an getting food, was just sort of here; I kind of saw what was going on; it really impressed me what the environment wa like; how open and helpful everyone was.
omar - I'm not a programmer, just read about it on th internet and wnated to come check it out; Iveb been impressed with the collab and energy and genius in the afternoon; I'm a poilot in the af, i nteach people to fly. whatever way I can contribute to help people oganize;I have no back ground except cs 101 a long time ago. it wa sinteresting, I thin ktI might have to dive into pythong and be curious and cotnibute to os creativity; I think the awesome concept here - that some stuff wil get hacked and rehaacekd by comms and vilalges all ove reh orlld; change an icon into a goat b/c that's what they have in their town or a llama, it's genius, this whioe contecept; that kids can enjoy, play, interact on th emesh. tansk for letting me be a prt this weekend, look forward to maybe trying it atain some time.
bonnie 0 not developer, come at it from an educational point of ie; I'm interestdi nhow these games can help eduate people; woul love to have desvs find ways to addres multiple disciplesi with theyir games. osmeone mentioned it ofver ther,e so that you don't just get the kids exited about math, but math and somethign youdo with humanities or socil sciencei; that would be grea.t than yo uf or letting me e a part; I enjoyed it very much; and many other things that have already een said I feel strongly about and support.
h thei enthusiastm and gera tideas and art and musi, jazzed me up mroe aso aobut the proejct sand contribute in other ways. I think a lot of us feel that way and would come back.
jlucks - oen thing I really liekd was end user vfeedback; tabout 1 sec after the first kid triesd to play the game I knew we needed to add suport we ahadn't thought about. that was interesteing to see. other real enjoynment was tapping noah's vcast brain to teach me have lots ot learn.
noah - ive been wokring on on games stuff for the xo for a long time evern befoer I started full time for olpc; it's noice to see game stuff start to come togehte, bu not muh hasb een done wrt games; prioeri to kthat, ame stuff was the crappy tetris game that ships by deful;t it's nitce to see better games than flow framerate tetris. and I'm happy tos ee people find bus in my ocde all weekend and put up with me say just a second I'll fix that; thakn you all for not killing me for my code not woring.
juluis - will some of thsee games ship when we go out?
matt - it'w as fun tgerat o meet everone; great to be alued for what
realizing that I could actually take a nap now -- that's really nice! I';m going to put gtogethe rnotes on the wiki oer the nexst fe w days about how to and nhjow not to put togrether a gjam. not just games, also art, also other content. feedback ood and bad, there are all ot of things I'll write down that don't work for people; thell me what that was and I'll put giv read circles arond it.... like make sure you get food donations ahead of time. you all have the olpc-gameajam@hackruonym email in your inbos. x
Things to remember:
need to write "how to make an olpc jam" tutorial and other tutorials from the weekend collecting & dealing w/ feedback crunch through data thanks to helpers thanks to judges $accounting$ refunds to everyone "content release bundles" - work this out regular test groups of kids? matt + 3 jordans = 4 gdc tickets developer's kit -> pong3d
manu -- we have other projects for people to contribute to.
q: so the hardware is done, and you're working on the software; whre do you think you'll be at the end of september.
[2k laptops end of june: b4; after that there's 1 internal test, then mass production] the hardware is basically fixed. on emulators: getting an emulator that is speed-matched is hard.
noah: I think they're working on screen emulation; being able to emulate b/w is key. we release new images of the os areound once every other day; we were giving people 432, newer than tstable but not too.
levels.py - there were some kids who wanted to download ("can we buy it?") labyrinth and older ones who wanted to do level design. can we get this to the june 25 release? talk to developers, they want to do it!