View source for Pootle/Administration

{{Translations}}
{{olpcboxtop|toptext=}}
  {{OBX team|Sayamindu Dasgupta, [[User:RafaelOrtiz|Rafael Ortiz]], [[User:Alfonsodg|Alfonsodg]]|[[User:Xavi|Xavier Alvarez]]}}
  {{OBX irc|#olpc-pootle, #olpc-content}}
{{olpcboxbottom}}
{{TOCright}}

== Administrators ==

There are two kinds of administrators: '''[[#Site admin|site]]''' & '''[[#Project + Language admin|project+language]]'''. There's no concept of ''project'' administrator per se, nor a ''language'' administrator either; tasks & responsibilities that basically go to the ''site'' administrator(s).

== Project + Language admin ==

Users with this right have several responsibilities and obligations.  Foremost of all is to ensure that the POTs get translated to the target language. In order to do that, they can grant [[#User permissions|some rights]] to other users (within said ''project+language'').

[[Pootle]] includes several built-in tests that try to ensure some quality levels (or typical errors) in the translations. They can be found by following the ''Show editing functions'' in each ''project+language'' page.  Normally the ''project+language'' page displays the progress level of each file (translated/fuzzy/pending) bar. This link changes the way each file is displayed; from the progress bar view into a line of links with something similar to:
:<tt>Translate My Strings | Quick Translate My Strings | Review Suggestions | Quick Translate | Translate All | PO file | ''XLIFF file'' | ''Qt .ts file'' | ''CSV file'' | Update | Commit</tt>
; Note: Each one a link (that may or not be enabled).  Links above in italic can be ignored.
: Unless you know what you are doing, '''update''' & '''commit''' should '''NOT''' be used.
Besides this change on the per-file display, new links appear in the ''project+language'' 'gray zone':
:<tt>Show Statistics | Show Checks | Show Goals | Show Assigns | Translate My Strings | Quick Translate My Strings | Review Suggestions | Quick Translate | Translate All | ZIP of folder</tt>
The checks can be found in '''<tt>Show Checks</tt>''' above, producing a detailed list of the errors found in each file, with links that allow editing the offending messages.

=== Defining goals ===
: ''see [http://translate.sourceforge.net/wiki/pootle/goals Pootle::Goals]''

Goals are required in order to be able to assign translations, thus enabling the organizing and prioritizing of work ''within a given language-project'' (iow, no global goals). Also worth noting, goals seem to be nothing more than a 'bundle name', with no other data like deadlines, comments or anything else.

In order to define a goal you must enable the <tt>Show Goals</tt> in the project page, and on the right, there's an entry field to add new.

# Define a goal in the project page (ie: [http://solar.laptop.org:5080/es/olpc/ spanish-olpc]) toggle the <tt>Show Goals</tt> 
#* by default a null-goal (ie: [http://solar.laptop.org:5080/es/olpc/index.html?showgoals=1&goal= Not in a goal]) exists as a catch all.
#* on the right, a box with an entry field allows you to give a name and '''Add Goal'''.
# Assign files to a goal
#* as we are trying to include files into a goal, click on the [http://solar.laptop.org:5080/es/olpc/index.html?showgoals=1&goal= Not in a goal]
#* make sure that the <tt>[http://solar.laptop.org:5080/es/olpc/index.html?showgoals=1&editing=1 Show Editing Funcions]</tt> is enabled by clicking on it if necessary
#* once in the language-project-goals view (showing the 'Not in a goal' files), each file has a pull-down list to pick and set the goal.

After setting the goals you can view the files bundled in a goal, and later assign work.

=== Assigning translations ===
: ''see [http://translate.sourceforge.net/wiki/pootle/assigning Pootle::Assigning]''

This is a nice administrative functionality that will actually (or probably) make the translator's life simpler. As an administrator you can assign certain users to goals and also to specific files (either to translate everything, unassigned, and a couple other variants).

By assigning users and goals/files, those user will then be able to use special links (ie: <tt>Translate My Strings</tt> or <tt>Quick Translate My Strings</tt>) that will feed them with their workload, and thus allow focus where is needed for the project. Still, the interface and handling is not quite describable at this moment as many aspects come together and {{pending|some things need to be better understood}} in order to get the most out of it (while trying not to complicate things too much).

{{pending|how do you '''remove''' an user from a given goal?}}

=== Terminology project ===
: ''see [http://translate.sourceforge.net/wiki/pootle/adding_a_terminology_project Pootle::Terminology], [http://translate.sourceforge.net/wiki/pootle/terminology_matching Pootle::Matching], [http://translate.sourceforge.net/wiki/creating_glossaries Sourceforge.net::Creating glossaries], [http://translate.sourceforge.net/wiki/toolkit/poglossary Toolkit::POGlossary]''

This is an extremely helpful and relatively inexpensive feature, particularly if we want to open the translation process to non-professional and [[#Opportunistic translator]]s because it aids them in preserving the terminology that has been decided upon by the more involved segments of the community (ie: will the olpc.es translate "Home" as "Hogar" or "Casa"?)

The basic structure of the terminology file is a standard .PO file (ie: msgid / msgstr). There are some rules about the name (and location) of these files. Documentation is a bit blurry on which alternatives apply, but there are basically two levels (the first as a local mechanism to override the global glossary):

; Specific to a <tt>project+language</tt> : override the global glossary by adding a <tt>pootle-terminology.po</tt>
: ''note: in projects with the GNU layout where all .po files are kept in one directory&mdash;iow, d.l.o&mdash;you can not use this approach''
; Global to the site (as a project) : several POT ''glossary'' files can be added to the [http://solar.laptop.org:5080/projects/terminology/ Terminology project] (ie: <tt>colors.pot, hig.pot</tt>, etc. each dealing with a specific domain for example)
: each language would have its corresponding PO file &mdash; that will be the one used to provide [[#suggestions]] (and edited if given the [[#User permissions]])

''' Note: ''' It would seem that a <tt><project>-terminology.pot</tt> in the [http://solar.laptop.org:5080/projects/terminology/ Terminology project] will automatically match said project (as a source), but {{pending|it may or may not}} affect the lookup of other terms in other files in the [http://solar.laptop.org:5080/projects/terminology/ Terminology project]. The precedence rules (if any) are unknown. The lookup is dynamic so there is no need to restart the server.

----

;[[#User permissions|Administrator]]:
# Enable the languages for which you want to have global terminology:
#* in the [http://solar.laptop.org:5080/admin/projects.html projects] page head into the [http://solar.laptop.org:5080/projects/terminology/admin.html terminology] project
#* verify that the language is listed (if not [[#Languages|add it]]) (ie: [http://solar.laptop.org:5080/es/terminology/admin.html spanish terminology])
# Create the glossary POT files
#* You can either create them manually from scratch, or
#* use [http://translate.sourceforge.net/wiki/toolkit/poglossary Toolkit::POGlossary]] to build the initial file from existing translations. Don't forget to clean it up.
# If you are going to allow editing the terminology through the web-GUI
#* verify that [[#user permissions]] are granted accordingly
#* as an user, in your [[#User options]] join the [http://solar.laptop.org:5080/projects/terminology/ Terminology project]
#* enter the project (ie: [http://solar.laptop.org:5080/es/terminology/index.html spanish terminology]
As any other project, if you have the [[#User permissions]] you can upload updates or totally new files to each language (but not to the templates) or edit the existing ones (ie: {{pending|1=[http://solar.laptop.org:5080/es/terminology/xavitest.po?translate=1&view=1 xavitest.po]}})

=== Taking advantage of translation memory ===
: ''see [http://translate.sourceforge.net/wiki/pootle/updatetm Pootle::TM]''

Very much like <tt>terminology</tt> but instead of working at a word level, the matching is performed at the whole string proposing full already used translations, saving some time but more importantly providing some level of consistency between translations. On the other hand, instead of being a dynamic feature, this is more of a batch or static process. Starting from a base <tt>translation memory</tt> file, one may generate suggestions for a specific file or a whole set of them.

; pre-test notes : This are some ideas, doubts and why-not stuff related to the reading of the documentation.
: File per file is probably not desirable (ie: use one PO to generate a memory for another PO), so some sort of 'olpc' translation memory file (per language) should be made.
: How do you generate the initial all-encompassing <tt>tm</tt> file?
: Every time a new language or PO is added, the associated memories for them (as targets) should be (automatically?) created...
: Updating a POT (iow, a new version) should trigger the update of all languages.
: The updating (which is not done in real-time) would still need to be done in a reasonable time period.

=== User permissions ===
: ''see [http://translate.sourceforge.net/wiki/pootle/permissions Pootle::Permissions]''

Permissions are granted or revoked within a specific language-project pair, and optionally, within that scope, to specific users. This means that the <tt>nobody & default</tt> users have local, instead of global, behaviors.

The following is the list of the available permissions handled by [[Pootle]]. The actual description is a result of observation and deduction (haven't found a specific documentation on them), so they are to be considered with <tt>#, fuzzy</tt> tag...

'''NOTE:''' The ''system'' administrator flag is not reachable through the GUI, as it resides as a flag (<tt>rights.siteadmin</tt>) in the <tt>user.prefs</tt> file. This permission allows the users possesing it to administrate all the projects, languages and functionality (you should always have one user with it).

{| border=1 cellspacing=0

|-
! Permission !! Description !! Default users !! Comments

|- valign="top"
! View 
| Allows the browsing of the PO files and their translation
| <tt>nobody, default</tt>

|- valign="top"
! Suggest 
| Allows to suggest a translation 
| <tt>default</tt>
| this should be enabled for <tt>nobody</tt> if we want to make things simpler for the [[#Opportunistic translator]].

|- valign="top"
! Translate 
| Allows to '''submit''' a translation.
| ''none''
| By default there are no users allowed to translate forcing the administrator to ''grant'' this right&mdash;bureaucratic and restrictive.

|- valign="top"
! Overwrite
| When uploading a PO file, it allows the user to overwrite any existing file (iow, no merge of changes)
| ''none''
| Handle with care.

|- valign="top"
! Review
| Allows to approve/reject ''suggestions'' made by users capable of ''suggesting'' translations.
| ''none''
| ''see [[#Reviewer]]''

|- valign="top"
! Archive
| Would allow the user to download the a set of PO files in zip format. Probably those of a language/project.
| ''none''

|- valign="top"
! Compile PO files
| Allows an user to generate / compile the PO into MO files.
| ''none''
| It is unknown where the MO files will reside, or how they will be transfered or made available. {{pending|need to explore}}

|- valign="top"
! Assign
| Allows an user to assign files {{pending|(or chunks of files)}} amongst users that have been granted the '''translate''' permission.
| ''none''
| {{pending|need to explore}}

|- valign="top"
! Administrate
| It's granularity is not well defined (or understood): administrate everything (seems like it), or just a project? or just a language? or just a language-in-a-project?
| ''none''
| {{pending|need to explore}}

|- valign="top"
! Commit
| The holy grail or point of all this: make a translation available.
| ''none''
| Again, the granularity is not well defined or understood. {{pending|need to explore}}

|}

== Site admin ==

As an administrator, in your home page you have access to the [http://solar.laptop.org:5080/admin/ '''Admin page'''] which offers: [http://solar.laptop.org:5080/admin/users.html Users], [http://solar.laptop.org:5080/admin/languages.html Languages] & [http://solar.laptop.org:5080/admin/projects.html Projects].

; [http://solar.laptop.org:5080/admin/users.html Users] : is a simple interface allowing the manual addition of users, edition of their names & (invisible) passwords (ie: resetting them) and where you can activate, de-activate and remove an user.

; [http://solar.laptop.org:5080/admin/languages.html Languages] : allows the maintenance of the list of available languages (based on the [[ISO 639]] codes, a descriptive text, [http://translate.sourceforge.net/wiki/pootle/characters special characters] (used in the translating UI), defining the number of plurals and its equation. Note: removing a language here would seem to affect only the ability to associate them to projects, {{pending|with no apparent impact on the previously defined associations.}}

; [http://solar.laptop.org:5080/admin/projects.html Projects] : this is the ''initial'' page where things start to come together. Besides being able to add a project by defining the values for the <tt>Project Code, Full Name, Project Description, Checker Style, File Type</tt> and <tt>Create MO Files</tt> fields, you can also <tt>Remove Project</tt>. The most {{pending|mysterious parameter is <tt>Checker Style</tt>}} which offers <tt>Standard, creativecommons, kde, openoffice, mozilla</tt> and <tt>gnome</tt> as options.

The above covers the broad, high-level configuration, which must be followed by the project and language configuration:

; [http://solar.laptop.org:5080/projects/olpc/admin.html Project languages] : A project needs to be informed of which languages it will have. This is accomplished by following the link of a specific project resulting in a page where you can add them. The only apparent way to remove a language is to delete its directory from outside of [[Pootle]].

; [http://solar.laptop.org:5080/es/olpc/admin.html Project language permissions] : Each language mentioned above is a link that allows you to configure the [[#user permissions]] in said project-language combination. You can grant/remove specific rights to a particular user, that will only apply in said project+language context.

; ''NOTES'' : Several languages were removed from the initial (default) list with the intention to limit or focus on the core green languages: {{Status green language codes}}.
: {{pending|Still pending}}: Obtain and verify the plural formulas for all.

== Primary administration ==

This administration level refers to the process of ensuring a functional and operative interface between the development environment and the translating environment. In other words, it must cover the largest probable set of situations and describe the appropiate steps to cope with it. Some things that need to be defined before starting are:

'''Number of Pootle-projects''' &mdash; {{pending|options are}}:
# single 'mega-olpc' project where all d.l.o. projects are kept
# one-to-one mapping between a d.l.o & pootle project
# ad-hoc granularity in the mapping (ie: core, bundled, extras, prototypes, etc.)
Some of this granularity may be handled by the concept of <tt>goal</tt> in [[Pootle]] (but they only work within a project+language context). On the other hand, although less likely, is that in a 'mega-olpc' we may be faced with name conflicts (ie: two or more projects using the same filename). This could be solved by adopting a standard naming convention within [[Pootle]](ie: prefixing all [[Pootle]] files with d.l.o's path to the project).

'''Number of Pootle-languages''' &mdash; {{pending|options are}}:
# strict minimal number for the 'green' countries (ie: {{Status green languages}})
# the above plus certain 'typical' languages (ie: german, japanese, etc.) plus some 'red' or 'orange' languages
# totally free as long as a ''language administrator'' can be identified and named for each language

'''Notification channels and/or mechanisms:'''
* developer's changes to the POTs (creation, changes and elimination) must reach the translating environment.
* translator's work and generation of POs (and possibly MOs) must reach the development environment.
How? This could be performed through the filing of tickets, sending mails, scripts monitoring both sides, etc. And in the future it may be modified to include or consider the back-end integration to git. {{pending|A clear primary channel must be determined}}. The periodicity and/or milestones for re-injecting the POs back into the development environment for their testing {{pending|must also be determined}}.

{| border=1 cellspacing=0
|-
! d.l.o<br>project !! POT !! PO !! Actions
|-
| rowspan=4 | New
| rowspan=2 | ''none'' 
| ''none'' || open a d.l.o ticket requesting missing POT file
|-
| > 0 || PAD & developer should decide if a PO may be used as the basis to generate the POT.<br>If appropriate, PAD {{pending|creates the POT}}.<br>: ''switch to 'new, POT > 0 and PO > 0' case below''.<br>If not, open a d.l.o ticket requesting missing POT file.<br>Developer eliminates offending PO(s).
|-
| rowspan=2 | > 0
| ''none'' || PAD saves POT in <tt>po/''project''/templates</tt>.<br>Performs a <tt>update from templates</tt> (in order to {{pending|generate the required PO for each language}}<br>Notifies the LADs who can then decide to include into their goals, assign it, etc.
|-
| > 0 || PAD processes the POT as above.<br>: ''switch to 'old, POT none and PO > 0' below.''
|-
| rowspan=4 | Old
| rowspan=2 | updated
| ''none'' || PAD saves POT in <tt>po/''project''/templates</tt>.<br>Performs a <tt>update from templates</tt> (in order to update existing POs).<br>PAD notifies LADs of changes as it may change the workload of goals and assignments.
|-
| > 0 || PAD saves POT in <tt>po/''project''/templates</tt>.<br>Performs a <tt>update from templates</tt> (in order to update existing POs).<br>PAD notifies LADs of changes as it may change the workload of goals and assignments.<br>: ''switch to 'old, POT none and PO > 0' case''
|-
| rowspan=2 | ''none''
| ''none'' || ''ignored &mdash; nothing to do''
|-
| > 0 || LAD verifies that PO matches existing POT.<br>If valid, <tt>uploads</tt> the PO through the web-GUI (either merging or overwriting the exising file)
|}
; Notes : The columns '''POT''' & '''PO''' refer to the external or incomming files generated outside of [[Pootle]].
: If a POT exists in [[Pootle]], the corresponding POs must exist for each language.

=== Terminology & roles used ===

; POT & PO : name given to the files depending on the purpose: '''P'''ortable '''O'''bject '''T'''emplate, for 'master copies'; '''P'''ortable '''O'''bject for their localized versiones (one per language). {{pending|naming convention}} would probably be "<tt>[[ISO 639|xx]].po</tt>"
; project administrator (PAD) : person in charge of overseeing that a project has the correct (latest) set of POT files and coordinates with the ''language administrator'' in order to ensure that the PO files reflect the POTs. Has also the responsibility to decide which languages to use.
: {{pending|to be determined}} if there's a single PAD for all d.l.o projects or more. Every d.l.o project should have one PAD.
; language administrator (LAD) : person in charge of ensuring that PO files are available to be translated while taking the appropriate measures to guarantee that the PO reflect the latest POT, merging, reviewing, etc. the translating community effort. Can also [[#user permissions|assign]] tasks to specific ''registered translator''
: There should be at least one LAD for each of the 'green' languages.
; registered translator : person actually performing the translation of a PO file within a specific project and language.
; developer : person that is (ultimately) responsible of providing POT files

== Setting up version control systems ==
: ''see [http://translate.sourceforge.net/wiki/pootle/version_control Pootle::Version control]''

'''These are some notes of the trials, not actual documentation.'''

Things have been messier than expected. First of all, GIT is not supported in the stable version, and the newer (unstable) version is reorganizing several aspects of [[Pootle]] making it too risky to jump onto the bleeding edge... Thus the decision to backport the git support.

On top of this, although volunteer gnrfan joined the fray (doing the backport), our experience with git is way below average and we are totally in the dark about how does [[Pootle]] actually obtain and store the POT files. We seem to have found a way (using symlinks in the [[Pootle]] subdirectories pointing to the repository files) but this solution although it may work, is a bit obscure and not even hinted by the documentation of [[Pootle]]:
<blockquote>
''To have any sort of integration with version control from within Pootle, it is necessary to check out the translation files into their correct places in their Pootle projects. The CVS or SVN meta files (CVS/ or .svn/) need to be there. This has to be done outside of Pootle.''
</blockquote>
Our efforts to contact or elucidate some sort of answers from the [[Pootle]] community have not been very effective for the moment... but hope is not lost! :)

=== Pootle workflow as per #pootle channel ===

Pootle has some very nice features, but integration to the repositories is not really one of them. Particularly the initial bootstrapping of a project. As extracted from a chat with <tt>friedel in [[IRC]]#poole</tt>, the usual / standard workflow with no repositories is as follows:
# The POT file is manually injected into <tt>/po/''project_name''/templates</tt>
# Doing the <tt>Updtate from templates</tt> for a specific language in a project basically re-syncs the existing PO (doing a merge, and in the case of conflicts demotes them as ''suggestions''{{pending|need to test}}) or makes available the particular PO for said language (in the <tt>/po/''project_name''/''lang_code''</tt>).
# In its origins, [[Pootle]] was finished, and the re-injection of the PO & MO files into the original project was left for the language or project coordinator to do.
With the inclusion of repositories, apparently the only part really integrated is the <tt>commit</tt> phase which would somehow trigger a push of the PO files. Unfortunately the 'pulling' from the repositories is not so automatic. Therefore, the probable (and suggested in [[IRC]]) workflow we'll implement requires manual intervention and/or the development of some scripts, and would probably look like this:
# Have a local git repository that syncs with d.l.o (as any other repository)
# Manually create the project
# Manually inject the POT file (most likely a symlink to the POT in the local repository)
# <tt>Update from templates</tt> would be performed for all languages
#: This step would actually create (or sync) the PO files for all languages in the project together with some associated internal files to [[Pootle]]. The actual PO file should then be ''symlinked'' and added to the appropriate place in the (local) repository.
#: Another possibility would be to create the project directory as a symlink into the repository (and probably use the <tt>.gitignore</tt> to filter out the [[Pootle]] files). This alternative (which could be simpler) seems to clash with the idea that all POT and PO files are stored in the <tt>/po</tt> directory in d.l.o because [[Pootle]] handles each language as a subdirectory (under the assumption that each project has several POT files, something that even [[Etoys]] has reverted and now has just one big POT).
# Translate at will & perform <tt>commits</tt> at will / as necessary, staying alert for:
#* If the POT changes along the way, this {{pending|should be somehow noted}} and an <tt>Update from templates</tt> should be carried out.
#* If a ''new'' POT file (not an update) is generated, it must be ''manually injected'' together with the corresponding PO files per language.
#* If a POT is deprecated (''eliminated'') the appropriate local removal should ensue.
The manual nature of [[Pootle]]'s repository integration is currently the weakest point as it adds administrative overhead and coordination issues. Some of it could be eased by either developing some scripts or modifying [[Pootle]] itself (ie: the creation of a new PO file could be tweaked to automatically generate the symlink instead of the local file and the issuing of the appropriate command to add it to the repository; or, just settle for a simple symlink of the project's directory and all files would reside in the local repository).
{{Pending|verify any possible conflict in naming conventions of the target PO&mdash;particularly with t.fp.o }}

=== Twilight Zone ===

; Testing setup : Is based on a two-layered clone of d.l.o projects. The first clone acts as the pseudo-d.l.o, and the second clone (a clone of the first) is where [[Pootle]] files will be symlinked. Currently the first clone (a.k.a. the d.l.o-clone) is in [[User:RafaelOrtiz|Rafael's]] home dir, and the second clone (a.k.a. pootle-clone) in [[User:Xavi|Xavi's]] home dir. Each d.l.o-project is cloned individually, as you can't actually clone d.l.o as a whole. The way each project is cloned into the pootle-clone git is by first cloning from d.l.o:
 #
 # For every project in d.l.o with [[Localization|L10n support]]
 #
 cd __d.l.o-clone__                                            # path to the d.l.o-clone (for testing purposes: /home/rafael)
 git-clone git://dev.laptop.org/git/''__project_name__''           # ''see [http://www.kernel.org/pub/software/scm/git/docs/git-clone.html git-clone doc]''
 #
 cd __Pootle-git__                                             # for testing purposes /home/xavi/clone2
 git-clone __d.l.o-clone__/''__project_name__'' ''__project_name__''   # could be manual or by scripts.(clone.sh) 

; Hooking [[Pootle]] into the pootle-clone : This is a manual step involving the symlinking of the appropriate POT file(s) and possible PO previously done.
 cd /var/log/pootle                  # home ''podirectory'' specified in /etc/pootle/pootle.prefs
 cd ''__pootle_project__''/templates     # a test directory for OLPC -- for testing purposes /olpctest
 # symlink the POT file and associate an unique name withing [[Pootle]] by prefixing it with ''__project-name__''
 ln -s __Pootle-git__/''__project_name__''/''__project.pot__'' ''__project-name__''.''__project__''.pot
 #
 # Enter the GUI and in the project languages perform an ''update from templates''
 # this will create the appropriate PO files for each language in 
 #  /var/log/pootle/''__project_name__''/xx
 # (where the ''xx'' is the [[ISO 639]] language code)
 #
 #
 # For each PO that didn't exist in a language in the pootle-project (for testing purposes: ''olpctest''):
 #
 cd /var/log/pootle/''__pootle_project__''/''xx''
 mv ''__unique__''.po __Pootle-git__/''__project_name__''/po/''xx''.po
 ln -s ''__unique__''.po __Pootle-git__/''__project_name__''/po/''xx''.po
 #
 # If a given language PO existed in the project:
 # (and you want to keep it)
 #
 cd /var/log/pootle/''__pootle_project__''/''xx''
 rm ''unique''.po
 ln -s ''unique''.po __Pootle-git__/''__project_name__''/po/''xx''.po

The reason that we are symlinking files instead of directories is that the structures differs:
{| style="font-size:90%; "
|-
! [[Pootle]] directories
! OLPC git directories
|- valign="top"
|
 /var/lib/pootle/''__pootle_project__''
   /templates
      /''__project_name_1__''.pot
      /''__project_name_2__''.pot
   /es
      /''__project_name_1__''.po
      /''__project_name_2__''.po
   /pt
      /''__project_name_1__''.po
      /''__project_name_2__''.po
|
 /__Pootle-git__
   /''__project_name_1__''
      /po
         /''__project_name_1__''.pot
         /es.po
         /pt.po
   /''__project_name_2__''
      /po
         /''__project_name_2__''.pot
         /es.po
         /pt.po

|-
! git integration
|-
| colspan=2 |
 /var/lib/pootle/''__pootle_project__''
   /templates
      /symlink ''__project_name_1__''.pot -> __Pootle-git__/''__project_name_1__''/po/''__project_name_1__''.pot
      /symlink ''__project_name_2__''.pot -> __Pootle-git__/''__project_name_2__''/po/''__project_name_2__''.pot
   /es
      /symlink ''__project_name_1__''.po -> __Pootle-git__/''__project_name_1__''/po/es.po
      /symlink ''__project_name_2__''.po -> __Pootle-git__/''__project_name_2__''/po/es.po
   /pt
      /symlink ''__project_name_1__''.po -> __Pootle-git__/''__project_name_1__''/po/pt.po
      /symlink ''__project_name_2__''.po -> __Pootle-git__/''__project_name_2__''/po/pt.po
|}

The initial failure of [[Pootle]] to identify a symlinked file as being within the domain of a version control system was fixed with a small patch.

Apparently, the layout of the OLPC's git follows what the [[Pootle]] documentation refers to as a ''GNU layout'' (in the <tt>/etc/pootle/pootle.prefs</tt> file):
  # pootle.podirectory
  #
  # All projects are stored in this directory in this layout:
  #   $podirectory/$project/$language
  # Projects can also be stored according to the GNU convention with one PO file
  # per language, and all files in one directory.
  podirectory = "/var/lib/pootle/"

[[Pootle]] handles GNU layout automatically (just symlink the <tt>/po</tt> directory in the d.l.o project and add the pootle-project). Although this reduces the hassle of setting up the version control, there are some practical disadvantages:
* No pootle-project specific glossaries.
*: This is not a great loss as the global glossary is still active.
* Multiplicity & size of POT files per project
*: Given the nature of our [[activities]], POT files are on the smallish end of the spectrum (even minimal, like [[Kuku]]) and each activity, has just ''one'' POT file. ** This means that the statistics are rather shallow and not useful to keep tabs on the advances of the localization effort. 
** Some of the intended tools (like goals & assignments) loose their appeal,
** Adds other administrative tasks (associating languages, permissions and users), and
** Users will now have to register in each of the projects+languages.
Overall, we could say that the usability (both in administration & user interaction) would suffer, while the only gain is obtained on the initial setup and some follow-up maintenance.

==== setup ====

These are the steps being taken in order to properly setup the server:
# Execute Sayamindu's <tt>populatepootle.py</tt> script that given the ''pootle_project, git-url & destination_directory'', does the following:
## creates a clone of the d.l.o-git (GNU layout <tt>/po/xx.po</tt>)
## creates a symlink in <tt>pootle_project/templates</tt> to the POT file, and
## for each <tt>xx.po</tt> file it adds a symlink from <tt>pootle_project/xx/</tt> to it.
# In order to trim, or ensure that the POs match their POTs, we recreate them by executing:
#: <tt>pot2po --input=_/templates_dir/_ --template=_/old_po_dir/_ --output=_/new_po_dir/_</tt>
#: The resulting new PO files should be copied over the old PO

== Advanced site administration ==

; Lowering latency through web servers : ''see [http://translate.sourceforge.net/wiki/pootle/apache Pootle::Apache]''
: ''see [http://translate.sourceforge.net/wiki/pootle/nginx Pootle::NginX]''
; Translator statistics : ''see [http://translate.sourceforge.net/wiki/pootle/logstats Pootle::LogStats]
; Configurable logos : ''see [http://translate.sourceforge.net/wiki/pootle/important_changes Pootle::Changes]
; Removing / Renaming files : According to user friedel in [[IRC]]#pootle, you can 'safely' delete a file (and associates&mdash;ie: <tt>xyzzy.es.po.pending</tt>) without much to worry about. This has to be done outside of [[Pootle]].

== File structure ==

{| border=1 cellspacing=0
|-
! F/D+Type !! width=40% | File/dir !! Comments
|-
| D-base
| <tt>/var/lib/pootle/</tt> ''(using <tt>__pootle-dir__</tt> below) || root directory of the Pootle data directories and files (POT & PO)
|- valign="top"
| D-base
| <tt>/var/lib/pootle/terminology/</tt> || directory representing the pootle-project dealing with the <tt>terminology</tt> that will be used throughout the site. Also known as the ''glossary''
|- valign="top"
| D-base
| <tt>/var/lib/pootle/terminology/templates/</tt> || directory where <tt>terminology</tt> POT files are stored. Also known as ''glossary'' files
|- valign="top"
| D-auto
| <tt>/var/lib/pootle/terminology/''xx''/</tt> || directory where each language keeps their <tt>terminology</tt> or ''glossary'' files
|- valign="top"
| D-gui
| <tt>/var/lib/pootle/''_project_''/</tt> || directory of a prticular ''pootle-project'' (note that there's no 1:1 correspondence with d.l.o) '''Note:''' the removal of a ''pootle-project'' has no effect on the directories and files (which must be eliminated manually)
|- valign="top"
| D-manual
| <tt>/var/lib/pootle/''_project_''/templates/</tt> || directory where the POTs associated with the pootle-project go (not accessible through the web-GUI)
|- valign="top"
| D-gui
| <tt>/var/lib/pootle/''_project_''/''xx''/</tt> || directory where the PO files in an associated language (using [[ISO 639]] coding) for a given ''pootle-project'' are kept. Created when a language is added to the project.
|- valign="top"
| F-auto
| <tt>/var/lib/pootle/''_project_''/''xx''/pootle-''_project_''-''xx''.prefs</tt> || file that overrides the global .prefs in the context of a specific [language+project], holding the [[#User permissions|<tt>rights</tt>]] on a per user basis (for nobody, default & specific users). Also keeps track of the <tt>goals</tt> (name and list of files), 
|- valign="top"
| F-auto
| <tt>/var/lib/pootle/''_project_''/''xx''/pootle-''_project_''-''xx''.stats</tt> || file holding for each PO file in the project six numbers (pressumably translated words, translated strings, non-translated words, non-translated string, total words, and total strings)
|- valign="top"
| F-gui/rcs
| <tt>''somefile''.po</tt> || the actual PO file.
|- valign="top"
| F-auto
| <tt>''somefile''.po.pending</tt> || if <tt>suggestions</tt> have been made, they are stored here in a pseudo-PO format that modifies the msgid (appends a <tt>"_: suggested by ''username''\n"</tt> line). See [[#Bug dealing with Suggestions]].
|- valign="top"
| F-auto
| <tt>''somefile''.po.stats</tt> || statistics about the internals of the file, apparently only about the <tt>checks</tt> and their results.
|- valign="top"
| F-manual
| <tt>{{pending|''somefile''.po.tm}}</tt> || the results of applying a translation memory to that file, pressumably the suggested string translations.
|}

== To Do == 
=== Usage ===

* Merge of uploaded PO
* Check if there any kind of verification when a PO file is uploaded to ensure the up-to-dateness of the original POT on which it is based? (ie: a PO is loaded but the POT on which it is based is a an older version)
* Update from templates (this needs the git interface on one side, but also if a POT is updated by whichever means, the PO should be merged/solved)
* {{Pending|Define a workflow!}} &mdash; the [[#Basic Scenarios]] above just cover the translator part. After the git interface is tested, the handling of the developer input, and the [[Pootle]] output must be defined.

=== Config ===

* Interface with GIT &mdash; both on importing POTs & POs from it & exporting POs back to it.
** [http://translate.sourceforge.net/wiki/pootle/version_control Version Control] doc ([http://translate.sourceforge.net/wiki/pootle/configure_version_control_authentication scant doc on configuration])
** [[User:Alfonsodg|Alfonso]] has patched the server with the git support and installed git in the server. So we should be almost in condition to start testing against dev.laptop.org.
* For completion's sake, language plurals & associated formulas must be defined (a bit pointless if developers don't actually do i18n right though, but...)

==== setting up ====

Please note that we are grouping the projects found in d.l.o into two base projects for [[localization]]: '''core''' & '''bundled'''. Other projects that are not found in the builds, will have a case-by-case decision on where to included them (either in an independent pootle-project, or grouped with other 'extras'). Given this grouping, we must ensure that no files will conflict in their naming, so the first approach is to prefix their POT filenames with their project name, ie: the <tt>Chat.pot</tt> of the <tt>[[Chat|chat-activity]]</tt> will be <tt>chat-activity.Chat.pot</tt> in [[Pootle]]. This in turn means that all the .PO files will be named <tt>chat-activity.Chat.po</tt> (but in different per-language subdirectories).

# First obtain a local clone in the [[Pootle]] server machine of the project(s) that needs to be included (see [[Pootle/Files]] for some notes).
#* Create the appropriate pootle-project for the git-project
#* Copy the .POT file from the git clone into the <tt>/templates</tt> directory of the pootle-project
#*: (may need to create this directory as a subdirectory of the pootle-project directory)
#* Associate the initial languages ({{Status green language codes}})
#* Select the languages and perform an <tt>Update from templates</tt> in order to generate the initial (empty) PO files for each language
# If the project includes pre-existing PO files, they need to be merged and included:
## '''merge''' previous work by executing <br><tt>msgcat --use-first -o /__git_path__/xx_YY.po /var/lib/pootle/__project__/xx_YY/__project__.__foo__.po /__git_path__/xx_YY.po</tt>
##: this takes the initial clean Pootle generated PO file and populates it with the <tt>msgstr</tt> from the git PO file; while it also ensures that Pootle's basic and standard header is preserved (more specifically the timestamp of the source POT file)
## '''copy''' the file back into [[Pootle]] with: <br><tt>cp /__git_path__/xx_YY.po /var/lib/pootle/__project__/xx_YY/__project__.__foo__.po</tt>
## '''delete''' (the now obsolete) [[Pootle]] statistics (that will be re-generated on-demand): <br><tt>rm /var/lib/pootle/__project__/xx_YY/__project__.__foo__.po.stats</tt>

=== Done ===

* user creation (both through the admin & registration interface that uses confirmation codes via mail)
* add / remove languages (basically reduced the set to green countries languages in order to focus attention)
* add project (only one for 'olpc' &mdash; deleting is performed through the GUI.
* associate languages & projects
* define permissions for nobody, default and specific user
* upload PO file (note: it doesn't verify against the templates, so you may end up uploading anything anywhere... handle with care)
* translate on line &mdash; may be a bit slow although it could be the browser, connection or server (see [[#Advanced site administration]])
* used the ''suggest'' funcionality, hit bug with ''review'' (see [[#Bug dealing with Suggestions]])
* Define goals
* Assign translations
* Delete files (this is done outside of [[Pootle]] directly on them... some things will protest, but is the 'accepted procedure')

== Glitches ==

* Something related to encodings was wrong during the setup
*: Alfonso commented several language specs that apparently were iso8859 instead of UTF ??

=== Bug dealing with Suggestions ===

There seems to be something awry with the '''suggestions'''... somehow there seems to be a mismatch between the '''msgid''' for which the suggestion was made, and the '''msgid''' displayed in the review process. IOW, you suggest ''foo'' as a translation of ''bar'', but in the review process it will show it as a suggestion for ''xyzzy''!! {{pending|review problem!}}

Talking in [[IRC]]#pootle, the following emerged: the mechanism used to associate a given suggestion is based on the <tt>#:</tt> comments (usually documenting the source code line where the string is located). This mechanism fails when there are more than one strings extracted from the same source line.

; Possible solutions & workarounds : the problem is relatively serious as it inhibits the [[#Opportunistic translator]] (or any other translator) from making suggestions (as they may get lost).
:* modify the source code to avoid having more than one <tt>gettext</tt> string per line.
:* modify the POT to avoid duplicate comments (iow, a manual process)
:* disable the suggestions
:** {{pending|explore}} the possibility of having the ''offending'' POT in another project where suggestions are not allowed (thus avoid reviewers get bitten by it).
:* patch [[Pootle]] (or file a bug)