Short Term Server Questions
From OLPC
This page discusses a number of issues surrounding the immediate deployment of School server, to support existing (Feb. 2007) XO laptops. These are not the services and interfaces that will be used in any real deployment --- see Server services for those.
Contents |
[edit] Immediate questions
Erik and Bakhtiar have been pushing to get some video creation/manipulation/sharing over the mesh to test within a week or two, but were mainly working towards a later deadline [and have hit mesh-related roadblocks]. It would be nice to see a simple version of their activity that stores video to a server and demonstrates video capture and sharing/replay running now.
Some simple unanswered questions that need to be resolved to make this happen, about what a solution will look like before the full journal and filestore are in place:
- where will files be stored, on the laptops and on the server?
- The current thinking is one or more dedicated directories in /home/olpc. The files and directories in this directory will be sync'ed to the server ASAP for backup and sharing.
On the server, a user's /home/olpc directory will be mirrored, and available at a URL with the general form of: http://school/nickname/
- how will one discover or save files? [insert interface query here]
- The file open/save dialogs which still exist in the applications running on the XO. These do need to automatically start in the directory selected above.
- how will one play (stream) audio or video files from the server and from the internet, and what size/format restrictions or recommendations are there?
- Through the browser? Possibly a dedicated player ?
- The directories on the server sync'ed from the laptop should be in the document tree of the web server.
- How does one find the school server ?
- A dedicated network name, inserted into resolv.conf on each laptop as part of the unlock process? Suboptimal, but doing this through DNS on the server will run into problems if CoDNS is deployed on the XOs.
[edit] Questions about the School server
Again as I understand it, the initial school server will be an XO with an external drive, and its own Fedora-based disk image... perhaps exactly the same base image as the other XOs, but with extra default content.--SJ
I would suggest avoiding any performance degradation introduced by an underpowered server at this point, before we really know what will be running on the server and have had time to tune the software. At this time I'm thinking something much more stock.--wad
- What should its image look like?
- Is there any benefit to starting from a base XO distribution? Given the differences between services deployed on the server and XO, and the lack of a user interface, perhaps a more standard RedHat/Fedora distribution for now ?
In terms of installed software packages, this is the current list (feel free to discuss/add):
- A router
- iptables in the kernel
- dhcpd (DHCP server, not used for XOs)
- Avahi (multicast DNS service)
- Bind 9 (DNS server)
- will we be acting as an authoritative nameserver for something? or just using BIND as a DNS cache?
- Apache 2 (HTTP server for library)
- are we going to need mod_php/python/perl/etc?
- Squid (HTTP caching)
- Not quite IPv6 compliant
- might be able to get away with just using apache2.2 + mod_proxy + mod_cache
- Rsyncd (file transfer service)
- ntpd (Network Time service)
- An X server will not be supported on the School server. All server management will be performed via an HTTP browser running on a laptop.
- How large should its disk(s) be?
- Given the small number of servers we are talking about deploying in the short term (20 to 30) and the state of software/hardware, the target size should probably be in the 300GB to 500 GB size.
- There is little reason for multiple disks, it simply decreases the MTBF unless we are willing to dedicate the disk space to redundant RAIDs.
- We should be able to easily add disk drives internally or externally and relocate the backup directories to expand the storage available to a school server.
- How long will it be in use before an upgrade?
- We can expect that any server deployed now for testing will be in use for roughly eight months before any hardware upgrade.
- What material will be on it (especially larger media)?
- All short-term deployments have regular internet access. This means that we don't have to be very concerned about what is shipped on the server. Instead we need to concentrate on creating a set of country specific content (library) servers, which will be cached by the school servers.
- How/where children will be able to save audio and video they record?
- Addressed above under immediate questions
- What about common web applications like Moodle and MediaWiki? Should these be part of the standard installation? If not, should there be a single yum command to install and configure these? Should MySQL be standard to support this type of application?
[edit] Short Term Questions about Use cases
Here are two use cases we should be able to support in the short term.
General assumptions (all numbers below are estimates):
- 100 children per server, sharing 25Mbps while close to the server.
- The laptops arrive with under 150MB of free space; servers have at least 300GB of space.
[edit] Streaming Media from the Library
Assumptions:
- Servers come with 12 15-minute and 12 30-minute videos in each of three languages, roughly 25 hours of video @ 640x480, roughly 1.5Mbps / 128kbps audio; and a lower bandwidth option. (400Kbps/64kbps? less?) We should advise a specific size/framerate/compression solution)
- For now, all material should be produced at standard definition (assuming square pixels, i.e. 640x480), and only lightly compressed, allowing compression to the final formats when selected.
- Servers come with 100 audio-books, averaging 30MB each @ 128kbps.
- We will need to provide recommendations for the max. number of simultaneous viewers.
Total size : 25GB of media
[edit] Storing and Sharing User Generated Content
Erik says everything but the save dialogue is working for storing videos.
Assumptions:
- Children will take a couple of 1-minute videos and a dozen photos each day... 20MB of material.
- Children will want to share videos with one another, yet XO applications are not yet mesh enabled.
- There will not be any automatic "move off of laptop onto server" or "delete unused material" processes in place immediately.
- While the final processes will use the Journal mechanism, a shorter term fix which works with the non-Journal enabled applications currently running on the XO needs to be put in place immediately.
- It will be useful to have a directory on the server where videos are stored by default, or perhaps one directory per XO.
Total new storage per day : 2GB per server (100 children)
[edit] Use Case Questions
- What happens to photo/video capture when the server is unavailable?
- It is cached on the laptop, just as when the server IS available. The proposed model for transfer of application data to the server is one of local storage on the laptop with periodic synchronization to the school server.
- How easy is it for one person to hog all available bandwidth?
- This is unlikely to be a problem in the short term, unless the mesh routing algorithm is broken, as TCP is fairly good at fairly balancing bandwidth over the short term. In the long term, this is certain to be a problem as P2P applications downloading large files to individual laptops degrade performance for quick individual accesses.
- How quickly will the school servers become memory/cpu limited? Are we completely committed to making these demo servers XOs?
- Absolutely not! Alternatives are already being considered.
- What happens when the laptop storage fills up?
- The least recently modified files need to be deleted as the storage fills up (i.e. the storage will always have some minimum X MB available, the rest will be a cache of accessed data.) The bigger problem is maintaining a consistent interface for the kids once their files are only accessible through the school server. Perhaps the only interface is through it ?
- We said earlier that existing file open capabilities would be used to access files. This leads to a stop-gap two tier approach, where kids can open and edit recently modified files easily, but have to download files that they created (weeks, months ?) earlier before modification.
- What happens when the server storage fills up?
- Good question! For now we just deploy another external drive...
[edit] Caching use case
Caching materials after viewing
- One laptop caches locally, both in memory and on disk (in case of failure?)
- A set of laptops shareing a cache that's too large for any individually
- A server caching and aggregating caches from individual machines -- can it uptake cached materials if a connecting laptop has something in its cache that's not on the server?
Queuing requests
- A laptop requests something from a local cache;
- A cluster of laptops listens to requests from individual laptops in the cluster;
- A server listens to requests from connected laptops (directly, or passed on from others not currently on the mesh)
Caching requests without viewing
- A laptop passes on a request, a server picks it up while the laptop is off-mesh and caches the result
- Some sort of notification or syndication of the incoming material
