This is the fifth and sixth post (combined into one) in the series of my weekly GSOC Sugar Labs, where I summarize my week of working with Sugar Labs under GSOC. This post was published little late due to the delays on my side, since I wanted to merge merge this and next weekly post.
In the following bi-weekly period I worked on the aslo-v3 along with Samuel. Samuel and I disucssed potential architecture and work flow. Intial plan was to complete database less but later we relaized, a small database layer is needed to index packages. We settled on MongoDB because of it’s No-SQL capabilities and relaxing Schemaless architecture seems perfect for storing the activity meta data and translations. We re-iterated over workflow over and over and intial plan was to start with something simple. We started with a simple and high level which involved
1. Parse activity.info (forget about gtk3, etc). 2. Store in DB: activity name and version (you might look activity table in current ASLO). 3. Build bundle. 4. Show entry in web page.
Bundle_id would be used to uniquely identify identify which is just a Fully Qualified Package Name, similar to what Java does. A quick solution to convert activity meta data into db documents was to parse activity.info into a json and push the raw json directly into Mongo. A typical activity.info looks something like this. This one is of Turtle Art activity
Which is a simple config file and can easily parsed by ConfigParser and conversion to JSON was simple. MetaData to Json conversion can be visualized as
To interact with MongoDB, we intially went with Flask-Mongo but then changed it to pymongo upon which Flask-Mongo is built for more fine control.
Next was to build the actual bundles which is what end users expect to download. Now building part was little tricky because the Sugar Toolkit has no documentation. So I looked at SamP’s code to find how toolkit was installed. I expressed my concerns regarding with the community and a new pip package might become a reality in the coming time. There are two versions of Sugar Toolkit - old version which supported GTK2 only and a new GTK3 version. New Toolkit (GTK3) is all we need to build packages.
Since there is no pip package for toolkit yet. To install we need to move the source to
dist-packages folder and create a symlink of
sugar3 to build old gtk2 packages.
SamToday’s DockerFile had the instructions so it was pretty easy to follow along. Sadly toolkit is built onyl for Python2, so it won’t work with Python3.
Once we have the toolkit installed, building bundles is easy as
python setup.py dist_xo --no-fail
Aslo-v3 was written in Python2 when I started but Samuel suggested (a right suggestion) to move to Python3, since it was early stage it was quite easy to move from Python2 to Python3, only change was to use new ConfigParser and a slighltly different instantion instructions. One advantage of
virtualenv is how easy it is to switch between interpreters along with dependencies without touching files outside /home.
Next move was to dockerize the build for security reasons. I looked at SamToday’s code and migrated from
fedora build image to
python-minimal image and since it’s based upon Debian, we can still install foriegn packages (so we get best of both worlds).
My approach was to use spawn a docker container build from a the above Dokcerfile, building docker image is easy as doing
docker build . -t provided Dockerfile is also in the same directory. We pass name of the activity along with docker spawn command and open two volume directories to interact with the host , to build the activity and move the activity to other folder for serving, script provided in the Entypoint does the actual building.
What gave me hard time was the permission managment between docker and host. Since Folders where activities and bundles are stored are owned by the aslo user which runs the backend server but Docker runs as root and the final image copied to folders have different (root) permissions and cannot be accessed/modified by the actual user, this was a big issue and I managed to solve it by using
setfacl and running docker as current user by passing current user id to docker which then emulates the user actions under the user id.
Also any file of a shared volume deleted under docker will still remain in the host system but you can change permissions and can move files
Docker part was the biggest time consuming part
In between I did a rough sketch of UI
Samuel also suggestes to clone only the latest tag branch instead of master. Also bundles can be as big as 150 MB we looked on the file size limits for Github releases and as per documentation single file can be as big as 2 GB . We also discussed using Github status api in the coming future to inform users of build status and a way to support screenshots , i18n/l10n.
Goals for Next Week
I plan on to fix and improve ASLO to make it ready for the first GSOC review. If you find any mistakes/typos, do let me know and I’ll fix them.