Diaspora API Dev Progress Report 28

Today is a momentous day in the Diaspora API development saga.  Today we have completed primary development of the API, the unit tests, and the external test harness.  There are still two code reviews between that and the real code review for integration into the main development branch, but all of the major work is complete.  What does that mean exactly?

For those that aren’t in software development when they read the above it likely brings up visions of something being out soon.  That is of course a relative word however.  A maxim  I often use when it comes to software development is, "The first 90% of the work takes the first 90% of the time and the last 10% takes another 90% of the time."  When code is in the state that we are in now, especially on something as large as what we are seeing here, the real work begins once it starts hitting real world use.   To give you an idea of the scope of this change, look at this screenshot of the differences between the API branch and the main development branch:

Four people over the course of 127 commits (really more because a bunch have been squashed down into one already) added over 9000 lines of code and removed 153 lines of code across 108 files. Some of those changes are in pieces code that are central to the operation of the site.  Along with the usual meticulousness that the Diaspora team applies to code there is the matter of the shear volume of changes that has to be considered.

Even the best code reviews and unit tests only find so many bugs though.  That’s where real world testing comes in.  During a shake out period with real users doing real things with code edge cases and problems inevitably arise.  Playing whack-a-mole with those over time is just par for the course and just takes time.  After all that there is still the matter of getting it into a major release.  Yes, something that is this large and pervasive needs a major release milestone to be integrated.  While the Diaspora team does point releases regularly something on this order of magnitude isn’t appropriate for that.

So what does all of that mean practically speaking?  I bring up the celebratory moment because it is good to celebrate such a thing.  It’s good to look back on all the hard work everyone put in and admire the accomplishment.  It’s also good to take a positive deep breath before the hard part begins.  The hard part is about to begin now.  All of those code reviews, discussions around code and implementation minutia will be underway for quite some time.  It is not only inevitable but it is a good part of ensuring quality code.   Alongside that is the real world testing. This can’t show up on a server and break everything. I’ve been meaning to learn how to really set up a production server and to play around with some other smaller changes so this is a good opportunity to stand up a test server for those purposes.  Along the way of all of that will be countless death by a thousand cuts as bug reports roll in, pull request discussions unfold, et cetera.  That’s not a lament anymore than a marathoner pointing out the 20,000-30,000 steps they will have to take to complete a race they enjoy running.

Along with all of that work there is the question of making the API more accessible to developers.  These are raw HTTP REST endpoints.  They aren’t the hardest thing in the world to work with but accessibility is important.  That means having some good tutorials and use case documents.  That means potentially having some existing libraries for popular languages.  A good talk I saw at RubyConf covered auto-generation of these sorts of libraries.  All of the above is stuff I intend to work on along with writing a test application that can exercise this in a real world usage scenario.  Because of the time spans that will be involved I also intend to pick up some non-API Diaspora work along the way as well.

I haven’t decided if I’m going to keep documenting my open source development on such a regular basis.  When we were making some very concrete major steps each day it seemed natural.  I don’t want it to become just noise.  I’m not going to force the issue either way, but I bring that up as yet another reason I am celebrating this occasion.  The updates will get less frequent.  That doesn’t mean hard work isn’t happening it’s just a lot of minutia that aggregate to finally getting us to the API in production but in and of themselves aren’t noteworthy.

You can follow the status dashboard at this Google Sheet as well, but it’s all green :D