As I have spent most of March this year hassling with the tooling provided with Golang, I have not made half the progress on Halo I was supposed to. Moving into April with no end in sight, I have to stop for a moment and redo my assessment for Halo’s choice of language in backend server programming.
For those unfamiliar with Golang, it has this cornucopia called the “Go tool”,
invoked on the command line using the
go handles getting and
building programs, dependencies, test suites, and so forth. In exchange for
go expects developers to follow ‘best practises’ wherever
relevant, which is dictated by Golang community consensus. I think the benefits
of this should be somewhat obvious, but if they aren’t, the Golang community
seems to be more than happy to explain everything they’ve decided on.
This month of development has been dominated by my efforts to take manual control of Halo’s build process using POSIX shell scripts for automation. The reasons for doing this include:
The second problem is that Golang ‘best practise’ mandates putting all Golang
source files in the root of the Git repository. It also mandates that tests
be paired to source files 1:1 with a
_test filename suffix before the
extension. I found myself reminded of Java programming practises in trying to
grapple with these arcane rules, and the reminiscience set in once I realised
the third and final issue at hand: the tooling works to enforce these things
every step of the way.
Originally, I worded the third reason a bit generously. My problem is more
severe than Golang’s lack of support for doing things differently – had that
been the extent of my issue I would have sailed right on, writing the scripts I
need to automate the build process and get on with coding the application. No,
go tool actively obstructs me and undermines every effort I make to deal
with the natural implications of a manual build. It turned a detour that needed
a couple of days to complete into a month-long torture session with this
program as my inquisitor, and I’d like to enumerate a few of the insanities
this tool has brought to bat.
go will deliberately halt without doing what you told it to
if this variable contains two consecutive dots, or if it begins with a dot.
Ergo, it thoroughly rejects relative paths, so don’t even bother unless you’ve
got a fancy absolute path resolver.
go will also resign in protest if the
$GOPATH directory doesn’t follow the prescribed hierarchy, which makes it
impossible to reconfigure the variable to sanely point somewhere nearby for
dependencies not in your original
Within the last couple of versions, Golang recently received support for a new modules feature that sidesteps that circus. While this makes it practical to have projects with source code in a subdirectory of the repo (why wouldn’t you?), you’re still out-of-luck if you hoped to sidestep Golang’s dependency management involving an internet connection, Git clones, and everything else that comes with it. You cannot put shared code in a separate module folder, build it once as a static library, and pass that to the linker for each project that depends on it. You still need to follow the orthodoxy for tests.
If this tool were a human being, it would belong in a psych ward. There are a world’s worth of reasons substantiating best practises and its tooling support, but no reason with any technical substance justifies obstructing developers from disregarding those reasons and doing something different when necessary. Rules are made to be broken, and it seems the only recognised path for that process is through the Golang community itself, which is a hidden cost most people cannot bear. I am here to make Halo, not fix the problems of the programming language I happen to be writing it in.
While I held out as much optimism as I could, I have to admit I haven’t seen much better out of the community itself. Visiting Discord servers and IRC channels for Golang has shown me a kind of conceited hubris from the people active there that, in my opinion, more than explains the schizophrenia exibited by the tool, and this was to some extent corroborated by posts on help forums such as SO and GitHub. Lots of yapping about best practises, colouring inside the lines, the attitude of people acting like they know better no matter what the context or who they’re talking to. They’re of no help for dealing with programming problems when it requires thinking outside their predefined box of ‘best practises’. One Discord community of over 2,000 members even shared with me this jaw-dropping exchange: https://archive.fo/deMEm
This has cost a lot of time, and I have to make a call here on whether this is still worth pursuing through Golang instead of any other programming language. By far the most compelling benefits of Go lie with its language design, so it really couldn’t be more unfortunate to have that retarded by ideological convictions from its community.
As nice as it is to write Go, I can write in many other programming languages too. It’s time to make a feature assessment and choose another language to take Go’s place. Before I examine the contenders, I want to get out of the way languages not up for contest, and the (simpler) reasons for why they are ruled out.
Like Golang and D, Scala helps itself to performance through static typing, and like D its typing discipline is also strong for that. The JVM is also key in providing more performance boosts, and the language is incredibly expressive in both functional and object-oriented contexts. It also supports several concurrency strategies, backed up by its functional programming patterns, making it ideal for Halo’s bastion backends, especially with tools like Apache Spark.
Some downsides Scala presents relate to its functional expressivity, which is to a great extent unavoidable and those unable to cope should git gud. Arguably the largest drawback is its power curve: Scala absolutely requires competency and skill to effectively use, and those lacking that will struggle to use the language to its full effectiveness.
Python is a very popular and understandable programming language with very strong, diverse support for various problem domains. While Halo will probably forego the high-level Django framework, Python is still a compelling choice thanks to its library support, and its dependency management is as opt-in as anything.
Fine-grained optimisation is something that Python doesn’t support as cleanly. While this is not of immediate concern for Halo, it’s still important for the assessment as problems like this will crop up down the road. Compared to other languages, Cython and similar pathways for native performance are as clunky and ad-hoc as those of Node.js. This is important also because of Python’s typing discipline, as unlike Scala, D, and Go it doesn’t accomodate performance as well.
Python’s support for concurrency is present, but the expressivity of the language does not accomodate it as well as Haskell, Scala or other functional languages. Libraries are very prone to cause paradigm clashes, rendering them useless for their purpose and driving the developer to reinvent the wheel to solve their problems.
D’s syntax will be comfortably familiar for those versed in C♯, and veteran C++ developers will be pleasantly surprised by its extensive support for various paradigms, quality of life features like contracts and compile-time code execution, and of course its familiar syntax. Its main downside is its relative complexity and unpopularity, and dependencies aren’t always in stock, so C bindings may end up in order for special things like database drivers. D is as good as gold in regards to its community and ecosystem, so there should be no problems like those of Golang or Rust when it comes to dependencies and code accountability.
Although Scala well outdoes D in expressivity because of the latter’s derivation from C, it can still be very empowering for projects needing functional accomodations in the interest of concurrency. While Python presents problems with its open-endedness leading to paradigm clashes through third-party libraries, D suffers less from this as there isn’t much to count on outside the standard library, Phobos.
The strong suit of Haskell lies with its overtly functional design. It has Warp for an HTTP server, and there are implementations of OpenPGP and AES which can be integrated to start with. Its drawbacks mainly revolve around its unpopularity amongst the general public, and its build processes are more clunky than that of D. Thankfully, Haskell does not seem to suffer from performance problems, unlike most of its closest competitors Scala and Golang, and it does support C bindings to a similar extent as D. The main problem lies in what argument Haskell has in what it provides that cannot be found with Scala.
the Worldwide Web. Using Node.js, server-side code can be written with rapid
iteration, but care must be taken to work without
npm, opting for manual
dependency management. The async nature of JS can be difficult to apprehend
sometimes, and concurrency support is also minimal.
For Halo, the choice is Scala. I have given this a few days to simmer, bearing in mind everything explained above, and I have not found any other languages to be more appropriate.
It’s worth pointing out the largest criticisms of Scala—relating to thresholds of skill and competency—are not only non-issues for Halo as an open source project, but actually blessings in disguise: those who will be able to make contributions will be more skilled on the whole, and will be contributing better code in doing so. This benefit is also positively compounded by Scala’s functional, concise nature doing so much to minimise bugs in any case.
Another note: For those unaware, Halo’s development has moved off GitHub onto Bahariya, Arqadium’s primary web server. You can browse the source code and make contributions at https://trinityg.it/halo/halo.