Mashoom 5 Release Notes

Version 5 is by far away the biggest step Mashoom has taken in its history, the first commit was on 3rd April 2017 @ 19:27; half a year of development. Essentially the aim has been simple, take what Mashoom has and make it public. So, here we go...

Front end website

If Mashoom is going to make it in the big wide world it needs a face, so here it is. Its based on Wordpress and isn't meant to be looked at as the new gold standard in website design, however I hope you agree its a nice place to be. Evaluating software is very difficult and often you want to hear about the background and the people making it over the actual features, this website aims to be able to do that.

Sub domain splitting

Mashoom 4 had a go at this before. *.mashoom simply directed all traffic to the mashoom server. Now we have 2 main sites:

• user.mashoom.co.uk - The login section of the site, where all the magic is
• front.mashoom.co.uk - The site you are currently on, the front end face of the site
• www.mashoom.co.uk and mashoom.co.uk - will direct you to either of the above sites based on if you have been logged in.

Merging team's data storage

Here we go, the technical stuff. This was the first job and afterwards it proved to myself Mashoom 5 was possible in the first place. OK, so, Mashoom database structure used to look a little like this:

• 1111_Items
• 1111_Deadlines
• 1111_Logins
• 1111_ etc.
• 1112_Items
• 1112_Deadlines
• 1112_Logins
• 1112_etc.
• Error log

So, all of the tables containing data specific to each account were separated, each one with an identical data structure. This made securing the accounts simple; the account number was added to the table names on the startup of the Mashoom server side program. The idea was actually nicked off Wordpress who use the same sort of system to split wordpress installs on a server.

However, a set of tables per account is unwise in the long run. What is much more efficient is to have one table of every team's data of that type (for instance "Items"), with an extra column of data specifying which account that row belongs to. Database tables are designed to do this; a properly indexed table can run into the millions of rows without too much of a problem.

This is would have been a nightmare, as it was, it got very interesting. This is down to the DBMinions, who perform every database connection in Mashoom. The important word is every; changing the DBMinion class can change every DB query Mashoom makes, without changing any of the individual pieces of code. I jumped 50 feet in the air when I worked this out and I'm pleased to say, the minions now add in these locking keys into every query as required. The file that tell the minions which table to secure is called Gru.

Users have their own personal space

Each team used to have their own list of users. This was part of the security process, the idea being that the team leader could reset passwords etc. Honestly, the biggest reason this was done is that email servers were scary, and I didn't want to write an email template etc etc.

Now users are discrete from teams, a user can request access to a team then switch between the different teams they are a part of. This is only possible thanks to a swanky new email service, password recovery is now done the safest way; through emails. The Mashoom framework has also been upgraded to have a half way house security clearance, where a user is logged in but not in a team.

New team signing up

This was the aim of this entire upgrade (well, as you can see I got distracted at various points, but that where we started). A completely new person can make a user account, then form their own team. Many people may wonder why this wasn't the case to begin with and all I can say it is proof that Mashoom wasn't conceived as a public product, I was adding teams manually and doing various bits and pieces of admin by hand. Honestly, I think this UI has a long way to go to improve the process, but I'm very happy there is a V1 at all.

User Interface V2

Mashoom has always been very jumpy... this is mostly down to the loading bar sliding down from the header, moving all the content down with it. I'm not sure if people were being polite when I coded this but I quickly found it very annoying, and like most framework features it was ignored in favour of making the content better. However, I couldn't let another major release go and not sort this out; the loading bar is now on the bottom of the screen.

Then there was the pages sliding in from the side of the screen, When I wrote this I was mega impressed with myself, some time down the line and I now class it as a nice idea with bad consequences. The consequences were how much effort it took and how much natural CSS has to be replaced for javascript. Then of course mobile browsers and slow PCs made an absolute pigs ear of making the animation smooth. Interestingly it was originally for the page to be preceded by a racing car going across the screen, that bit never happened. Anyway, now the page fades out and re-appears with the new content. Much easier to control, much more reliable and actually far less annoying in general.

The Mashoom logo now nicely collapses into a streamline bar when you are scrolling down the page, and the minimise button at the top right of the header (yup, go and have a look, least used feature by far) now has enough space to work properly.

I'm also very pleased to say that Mashoom now doesn't need to refresh, all the way from login and signup to logout. There is frankly mostly only a pride related reward as frankly many websites (including Facebook) employ a bit of refreshing as the user navigates. However, the advantage is that Mashoom feels very "fast", even when doing things like switching between different teams.

The sub menu

Truth be told we always had a submenu, it was only used for admin options and was a bit rubbish. Now its awesome:

This is a much clearer way of showing what is contained within each module, and which modules your team has available. Admin options has gone from being a dumping ground of miscellaneous functions to an actual set of features you would expect. At this point it would also be good to mention Font Awesome, who's icons, well... they look nice don't they?!

Big, fast and awesome searching

I'm pretty proud of the algorithm that is behind this search. The aim of the game is you can type what is in your head into the search bar and come out with a set of results which hopefully has what you are after. I decided to spice it up by saying this bar should be able to search every item from every module. And then to top it off, it will recognise part numbers using the same numbering engine as the rest of Mashoom.

This is the first time I had written something like this, and I had to tear myself away after a week to work on something else because it is both fascinating and something you can gently improve again and again. Safe to say I will be gathering input on if people like this feature and improving the search algorithms.

For those power users out there, Ctrl/Cmd + s will open the search bar, up/down arrows can be used to navigate the items and enter to go to their details.

Super powered downloads

This releases the full capacity of AWS's storage system straight to the user. Previously we were copying the file from our storage bin, saving it to the web server then sending it to the user. This process meant the performance bottleneck was in how fast the web server could write to disk, not something web servers are meant to be good at. This also provided memory issues, especially when bundling large assemblies.

Now we simply stream straight from AWS, zipping as we go. This is down to a few libraries and a lot of head banging but I'm pleased to say that Mashoom can now deliver files at some very very silly speeds, with no size limit.

Design Project Management

I wrote this ages ago, then promptly dug up the part numbering engine and subsequently broke this feature for any real use. OK, honestly I was thinking of killing this off in Mashoom 5 and making a proper project management module, however, this is a very useful tool for our Formula Student team, it is worth it being there even if it is somewhat basic. This will eventually get its own module and will pull together items from multiple modules.

It is worth saying that to get the most out of this feature, you need to use projects. They can be setup from team settings then parts added to them. One part can be in multiple projects.

Part Numbering Upgrade

Part numbering was one of the very first "clever" things Mashoom could do, Mashoom 5 sees its feature set completion, after 2 years of work. The biggest thing it was lacking was the ability for it to be changed by the user, up till now I manually set and edited numbering schemes for each team. Now this UI is available:

Part numbering is a fascinating thing to develop a program for. In one way it is very very simple, in another there are an infinite number of possibilities so no room for any "static" code. This UI will see many upgrade as it still isn't the easiest to use and requires some explanation, however, I hope this demonstrates for the first time what Mashoom has actually been running off for the last year. A part number can be split up into lots of sections, each section having a certain bit of information. These can be pieced together to form a number, via va la drag n' drop!

That, on its own, was a pain to code. Then you need to interpret this sea of html and update a reasonably complex data model... o, and you need to make sure that the user has made a valid numbering scheme otherwise Mashoom really would get into a pickle when it tries to auto increment revisions etc. Of course, it wouldn't be published unless I had managed it.

Indexes just made it into Mashoom 5 before it was released. The unique part of the number used to auto increment starting from 1, based on each part added to Mashoom. That was good for our purposes but not very good for an existing team with existing part numbers creating an account for the first time. For this I put in indexes, whereby you can set the next available unique number and specify a unique number when adding a part. These two features mean that a team can have new parts being added as per usual, whilst over time legacy designs can be uploaded as well.

Time Upgrade

I love being able to write this heading. Time is very very funny stuff when you get into computer world, you think it should be simple right? No. I shall write a blog post on this at some time but basically we are fully upgraded to using timestamps; defining a point in time as the number of seconds from the Unix Epoch (00:00 1st January 1970 GMT). This is the only way to store a definite point in time without worrying about timezones and daylight saving times.

I think the easiest way to describe what I've done is a comment from someone who looked at the log after I had done this upgrade; "O you fixed the times being wrong".

Notifications

This is down at the bottom of the list because it is a very quick pull together for a solution, and I'm sure it will need some proper infrastructure behind it for it to become a true notifications bar. For now, its very useful for telling people they are using a beta and getting them to verify their email address.

KNode tree

Mashoom is now really good at hierarchical data sets... however, the knowledge items never got updated and were still using a Mashoom 1 solution to having children, which at some point in time was broken anyway. They are now updated, whilst the functionality is still quite limited they now fully work. Watch this space though, next major upgrade is likely to be in this area.

We say goodbye to...

This release comes with a fair well to some pieces of code and functions, some of these were revolutionary for me at the time, now they look amateur compared to what they have been replaced by

• SQLFn - This was what the DBMinions have to thank for their creation. One function per select, update insert and delete, aaahhhh the simplicity...
• View Catalogue - There used to be a public facing list of all the parts on Mashoom. For obvious reasons this was made redundant a while ago. Mashoom 5 sees it actually deleted from the source code.
• PartNumFuncs - You can guess what this used to do. Infact, this has been the hardest set of functions to get rid of, as they were so damn good! They had one problem, they only worked for our FS part numbering scheme. To put into context the development this idea has been through, this file topped out at about 200 lines of code. ObjNum (A Marvin component that now performs these tasks) is 2498 lines long.

Lots of other pieces of code have been cleared out the source code as well to try and keep our line count down (a small obsession of mine), but most aren't as notable as the above.