Category: Software Development

Categories
Software Development Web Development

Why I Will Always Choose Firefox

Unlike most people who may be reading this, when I install a browser, it’s always Firefox. Being a software developer, and working around other tech-literate people, this simple fact seems to draw a fair bit of confusion, as everybody else seems to think of Google Chrome as their go-to option.

I’ve had a great many friends over the years try to convince me to switch to Chrome, usually, I presume, as an instinctive aversion to difference, and once because they wanted to prank me by installing the NCage chrome extension. On that last occasion, I installed it just to see the prank, and then promptly uninstalled it again.

In the past, my reasoning for doing so wasn’t so strong. When I first installed Firefox, it was when I was perhaps in junior high school, and at the time I wasn’t very tech literate. I had basically every browser installed on my old Windows XP system, and any given week I might favor a different one. At one point, I even recall being a staunch Safari for Windows supporter. Perhaps that’s slightly less of a sin than my occasional use of Internet Explorer, though.

It was a few years later, when I was in high school, that I finally made my final decision. I wanted to only use Firefox… because of the cute fox mascot (I am aware that a fire fox is a red panda, but their mascot isn’t, so your point is invalid). No joke, that was the decision that led to where I am today.

Things have changed an awful lot since I made up my mind about my choice of browser, but I have used every major browser there is, and I am still up to date on the technological details and features of most of them. To this day, I feel that I still made the correct choice, albeit for a silly reason at the time.

In the past, I felt happy to be different in my choice, because why not? Everybody else could use whatever they wanted, and so could I. The nature of the internet wasn’t at stake. But now, that has changed. With Microsoft choosing to switch edge to be based off of Chromium, this now means that there are only two remaining players in the browser engine arena: Google’s Chromium, and Mozilla’s Gecko.

Competition Makes Better Browsers

One of the benefits to what may seem like duplicated work is a parallel to one of the main benefits of capitalism in general: competition creates faster innovation. Because every browser team wanted theirs to be the best, they had no choice but to try to innovate faster, create richer features, and do whatever the people wanted as fast as possible, lest they fall out of favor and be replaced by their competitor. No doubt this is, and was, a stressful thing, but that stress undoubtedly yields a superior product from everyone in the game.

If Gecko were to fall, there would be only one browser engine left: Chromium. Do you think they’d need to innovate quickly? Absolutely not. They’d be in no danger of being replaced. Innovation would surely still continue, but not at the rate it has in the past.

Competition Enforces Standards

The web is entirely based on standardization right now. There are several large committees representing hundreds of companies and organizations that govern the standards of the web that we use today. These standards are designed to be written guidelines with very specific rules, which ensure that multiple different implementations produce the same result when given the same web content.

In the past, this has worked very well. Despite the occasional difficulty, browser engines were able to keep up with these standards, and that’s why somebody could create one version of their website for Chrome, Firefox, iOS, Opera, Edge, and so on, and it would work the same everywhere (except Internet Explorer, they could never truly get it right

The question it seems a lot of people might struggle to answer is why every browser cared so much about following standards. Why couldn’t chrome create its own method of drawing on a canvas, or aligning content, for example? Surely doing so would give them the ability to claim that they had a special, extra feature that nobody else did. In a way, it’d be much like Microsoft holding exclusivity on an API like DirectX. Anybody wanting to use that API needed Windows (until Wine happened, of course). The problem, which is on a smaller scale an issue faced by DirectX, is that developers will begin to prefer alternative APIs that support more browsers with the same code, and users of other browsers will begin to see that browser as something of a non-team-player. Every time they go to a website that doesn’t work correctly because of the lack of standardization, they’ll blame both the developer of the site, and the developer of the non-standard browser. Further, if the other browsers are all standardizing, and one isn’t, then they all become the browsers that “Just Work” and the odd one out is… well… the odd one out.

I do believe that the initial adoption of web standards was because browser developers are web developers too, and they likely did it out of a desire to make the web a better place, but years later, standards are no longer optional, and competition is why.

If Gecko dropped out of the competition and left Chromium all alone, would the Chromium team have to follow standards? Absolutely not. I believe they would try, but they’d have all the leverage. If they wrote an API, it’d become a de facto standard, because everybody has it. If they wrote an implementation wrong, it wouldn’t matter, because theirs would be the vast majority of the market share, so the actual standard implementation would actually become “wrong.” This might sound good to some out there, because it would surely streamline the process of the standards, but it puts all of the power in the hands of one project, one team, one company. It’d essentially be giving Google a monopoly on the internet. They wouldn’t necessarily use that for evil, the problem is that they easily could.

The Fight

Maybe I’m too idealistic, but now that Gecko is the last competitor to Chromium, this has become a real fight to me. I won’t switch to Chrome because I like Firefox more, but also because I believe in the open, competitive web, and that can only change over my cold, dead Gecko-based browser. And I hope some of you will join me in that fight.

Categories
Mobile Apps Software Architecture Software Development

HyperTrace 1.0 – My First Android Development Experience

This week, my company Danger Interactive LLC launched its first app, one for which all of the programming and art was done by myself, so in a sense, I launched my first app.

Learn more about that here, by the way: https://dangerzonegames.com/apps/hypertrace/

Experience

My preexisting experience with android development was very limited prior to developing this app. I had followed a tutorial one time over five years prior, before I even understood programming well. On another occasion, I planned to build an app for a friend, but once again, this was before my understanding of programming was sound enough to create even web apps. Thus, going into this project, I understood: basic Java programming concepts and how to launch Android Studio (although back when I did tutorials, there was no such program, it was all addons for Eclipse).

I spent approximately 2 months learning the Android API and building this app, which I think is a lot longer than it would take to duplicate it if I were to do it again, now that I know the ins and outs of the Android API and the Java programming language. Android is an excellent and easy to use API, and I am very happy I finally got around to learning it.

My Perspective on Android UI Reusability

In Android, the UI is composed of Activities, which is the main container for all other content. Usually only one activity is visible at a time, although there are ways to display partly transparent activities, but that’s not relevant for this discussion. Inside of these Activities, you have “Views” and “Fragments”. Everything that displays something to the screen is a View, and Fragments are a special kind.

Fragments, at first sight, seem to be an excellent way to create reusable chunks of Views. This is patently false. This is, in fact, what I suspect will be the primary mistake of new Android developers with regard to creating UI layouts.

There are a million reasons for this, but to make a long story short, Fragments are not meant to be a reusable chunk of views, in abstract. They are a special case. Fragments are not direct children of the Activity, and therefore they have an entirely different lifecycle. They are handled by the fragment manager. This separate lifecycle means that you can do really neat things like moving a fragment from Activity to Activity or changing the layout around.

However, if that’s not what you’re doing, then it is going to be a huge pain, because you cannot treat Fragments like regular views. All transactions must happen through the fragment manager. As you might imagine, this could also add additional overhead. So essentially, unless you need this special functionality, there’s a much better choice:

public class ReusableChunkOfLayoutView extends FrameLayout

This is what you should do. You will create a subclass of FrameLayout, which is your reusable chunk of layout, and then you’ll inflate a layout.xml file from that. It’s simple and straightforward, it offers the same ability to create XML layouts and build them easily, and it inherits all of the pre-built onLayout and onDraw code from FrameLayout, which is the rough equivalent of a “div” tag in HTML.

Reactive Code is Neat

I wrote the entire app using “Reactive” API design. What that essentially means is that each component that embodies some kind of state, network, or IO (especially network and IO actually) is capable of communicating with dependent objects. The means that I used to facilitate this communication channel was the Observer pattern.

public interface ObserverInterface {}

ObserverInterface was actually just an empty interface. It’s only there as a syntactic sugar to make the intent of a particular class clear to the programmer.

… public static class Observer implements ObserverInterface

Each class that needs to communicate with other classes creates a static inner class that implements this empty interface

… private SingleObservable<Observer> m_observable = new SingleObservable<>();

Then the class creates an instance of either SingleObservable or MultiObservable, through which all of the notifications will be passed. It is a generic class because ObserverInterface doesn’t offer any methods, so we need to have no erasure in order to be able to call the methods that are added to the observer. This is a good example of functionality via composition instead of inheritance.

public interface ObserverAction<T extends ObserverInterface>

The ObserverAction class represents a “notification” essentially. Once again, it’s generic to avoid the erasure issues. The interface contains a single public method called “run” that takes a single argument of type “T” (the observer). This observer argument is used to be able to call methods on the observer (the real notification channel). The SingleObservable and MultiObservable classes receive these objects and apply them to each relevant observer.

public class SingleObservable<T extends ObserverInterface>

The SingleObservable class was a relatively straightforward one. It just needs to have a single private member that holds one element of type T (the observer), and an ArrayList of ObserverAction objects. The ObserverAction list is there because of what I termed “final actions” which represent actions that should be run on every observer added after the action occurs. This is good for such things as observers with an “onComplete” method, where you can be certain that it will only be called once, and therefore the reasonable assumption is that later observers might want to be informed of this completion.

This class includes a “setObserver” method, and then a “run” and a “runFinal” method, which are provided the ObserverActions to be run once, and run forever, respectively.

public class MultiObservable<T extends ObserverInterface>

The MultiObservable class was a little more complex. There are thread-safety issues that need to be considered in this case. This observable needs two ArrayLists of objects of type T, one of which is the observers, and the other is the pending observers. Pending observers is the holding patter for observers that try to get added while a notification is being processed. Thus, there is a boolean flag that represents whether or not the observers are being notified, and if so, “addObserver” will place the observer in pending, and then the running “runAll” or “runAllFinal” methods will unset the flag and move all of the objects to the observers list and clear the pending observers.

This class includes different methods. There is an “addObserver” method, and then “runAll” and “runAllFinal” which do the equivalent of “run” and “runFinal” in SingleObserver, but they iterate through every object in the observers list and runs the ObserverAction on the current observer. As mentioned, on completion they move anything from pending to the regular observers list.

The Bad… The Ugly…

There are exactly two things that I hated to use when working with Android:

Scrolling

To do scrolling, there are three relevant views. There’s ScrollView, HorizontalScrollView, and NestedScrollView. What do you need to know about them? ScrollView is deprecated and useless, just use NestedScrollView instead, HorizontalScrollView is the only scroll view capable of horizontal scrolling, and there is no way to create a single view that scrolls on both axes. You must nest a HorizontalScrollView inside of a NestedScrollView (or vice versa).

Maximum Height/Width

There is no such thing. I have actually not found a good way to do this either. I guess I’ll update anybody on a solution if I ever find one. This is a tragic issue because max-height and max-width are extremely useful when creating responsive layouts. Obviously this is less of a concern on Android, since the screen and window size isn’t expected to change much.

Speaking of responsive layouts, your best bet for creating anything that can be termed “responsive” will likely involve a ConstraintLayout or two. Learn to use them. You won’t regret it.

Categories
Game Development Game Jams Software Development

Ludum Dare 41 Postmortem

Ludum Dare 41 came to an end just a little more than a week ago. I believe I’ve finally recovered from the mayhem and come through the better in the end.

Project Status

Unfortunately, my Ludum Dare 41 project, a “turn-based AI battle-royale game” called “Turn-by-Turn Deathmatch” was not finished when the 72-hour deadline hit. This was, unfortunately, my expectation for this one. I may continue working on the game in the future, but I am particularly averse to the theme and not satisfied with the idea that I was able to come up with.

The Theme

My struggle for this Ludum Dare originated at the outset of the theme announcement. The theme for 41 was “combine two incompatible genres.” I know a lot of people had differing opinions on this, but it is mine that it is the worst theme yet. For those of us that are relatively new to the game-design scene, the theme is asking us to throw out any convention of design that we may have relied upon to make a game that is not a complete flop. For those more experienced, I will grant that it does provide a unique challenge, and for that reason I was almost somewhat conflicted. That said, it is my opinion that Ludum Dare should be welcoming to new developers, as Ludum Dare is often some of the first exposures to the indie game dev scene that they will get. It should be a good impression. I believe this theme, by significantly raising the difficulty of creating a workable concept, likely created a bad first experience for them.

I created a very large, very time-costly spreadsheet of an incomplete list of game genres in an adjacency-martrix-like spreadsheet, which ranked the combination of any two genres as “good,” “neutral,” or “bad” based on how well it fit the theme, which is to say that a “good” combination was two genres that one would not expect to see combined. Even with this chart, I found myself struggling to come up with a theme, because as it turned out, many of the “good” combinations sounded like awful games, or they required some complexity that I knew would take longer than the deadline. For example, I really wanted to do a “first-person platformer,” which would be a 3D game where you basically see something like Super Mario from Mario’s eyes. It’s been done before, and I recall positive reception. Unfortunately, my 3D-focused game engine did not have a completed rendering engine, and with 3D being somewhat new territory for me, I scrapped that idea. Eventually I came up with combining “turn-based” and “shooter” and then eventually I realized that “shooter” could be replaced with “battle-royale” if the enemies are roughly equally equipped and trying to kill each other as well. I thought it was a neat concept, but I also knew that I had minimal experience with AI, and no experience with turn-based games. I was also looking at just over 24 hours left on the deadline when I had the theme nailed down, so I knew it would be a big challenge.

The Technical Struggle

I wanted to go into this jam using my custom 2D game engine, ArcticWolf. I saw this as an opportunity to, as I did last time, use the game jam as a sort of excuse and motivation to further my progress on the mundane core subsystems of my game engines. I had intended to work out a simple painterly blitting rendering engine, because I thought it would be the most straightforward design where the least issues could crop up. What I did not realize, however, was that rendering things in the correct order being mandatory in a painterly renderer, would actually prove to be a nightmare of a challenge.

In C++ there is no STL container that allows indexed access and automatically organizes them based on a comparison. There is a container called std::priority_queue, but that will only provide access to the front element in the queue. This could work if we rebuilt the entire rendering queue every frame, but I knew that would be a huge detriment to the performance of the game, since elements would not be frequently (probably never) changing their z-indices, and therefore what I really wanted was a std::priority_vector.

Well, I hate to break this to you all, but no such thing as std::priority_vector exists. So I made one. In the ArcticWolf code, there is a class called aw::PriorityVector, which does everything that I needed. It took a lot of work, but it is one of the pieces of my creation during this game jam that I am the most proud of. It’s very simple, and all it does is store elements into the vector in the correct order based on a comparison type, the same that you might pass to std::sort. If the attributes defining the comparable values of the contained objects change, there’s a method that allows reordering the entire container, which is expensive, but useful if z-indices change. Another couple of useful helper classes were created to be used with this: aw::PointerGreater and aw::PointerLess. They are equivalent to std::greater and std::less, but they dereference each element being compared. This was necessary because the aw::Renderable class was polymorphic and referenced via pointer only, which meant the usual comparisons would have actually compared their addresses, not their values.

The main technical struggle, however, was in the complexity of my ambitious rendering engine. I decided it to render a scene which contains polymorphic layer objects, each of which can contain a polymorphic aw::Renderer and a bunch of objects that contain aw::Renderables, getting passed as pointers back to the aw::Renderer when the layer is rendered. With this system, I could have a base that is a tile map, representing the game world (which I also intended to allow multiple layers with different parallax values to make it a 2.5D engine if I wanted), and then a sprite layer on top of that into which the player and other entities would be placed, and then a UI layer on top of that which uses a rendering pattern similar to HTML5 DOM.

I spent a lot of time on this rendering code, and I am glad to say that I made quite a bit of progress in the process, but it also means that I didn’t have enough time as the idea that I had would have required. I was able to get the preloading and menu completed, but not much beyond that. I actually ended up trying to temporarily bail on the new rendering engine and write raw rendering code within the last 12 hours of the jam as a last-ditch effort to get a workable game by the deadline, but by that point I had not slept in a long while and I didn’t have the focus to keep myself awake.

At some point, I decided that this jam and its very undesirable theme was not worth the suffering. I remember waking up with my face down on my laptop, completely disoriented, with the morning sun shining brightly on me, and at that point I decided to close the project on the ldjam website and go to sleep. I woke up at about 11PM that evening. So much for my sleep schedule.

Future Plans

I was quite disappointed in the theme for this Ludum Dare, and I can’t help but wonder if the reason we’ve started getting worse and worse themes over the last few years might be directly correlating to the improvements being made to the ldjam website and making voting easier. They received well in excess of three thousand votes for the theme, but based on past trends, I think we can expect approximately a third of those people to submit games. Thus, it seems likely that many of the people voting on the theme have little to no intention of making a game for it, and therefore they have no interest in making sure the theme is workable, reasonable, and doesn’t cut out new developers.

My suggestion of improvement would be to allow theme voting only for users with submitted, completed, and rated jam or compo submissions to vote. In my opinion this would drastically improve the themes. That said, I don’t think that is likely to happen. Given that, next Ludum Dare, I may or may not participate depending on the theme. If the trend of worsening themes continues, I will not be participating in Ludum Dare 42. Nonetheless, I do intend to do a game jam in August. That is not a question, what is a question is whether or not I will be following the chosen theme or the standard deadline. I’ve been toying with the idea of running my own week-long jam with a theme chosen by either myself or by taking one of the losing themes from a previous Ludum Dare. Obviously I’d lose out on some of the community features of Ludum Dare, but it would still provide more or less the same training effect, and I think in the course of a full week, I’d be able to create a more-than-trivial game design.

Categories
Game Development Software Architecture Software Development

The Scourge of Global Mutable State

I’m not a perfect programmer, but I like to think that I’m getting better every day. I’ll admit that one of the things that I’ve had the most difficult time adapting to was what I perceived as other programmers being somewhat pedantic about avoiding what they called “global state” or sometimes more specifically “global mutable state.”

In the past, I held onto some conviction that while this is something to be avoided where possible, there are actually cases where it is appropriate. Recently I’ve decided to not only better conform to this well-known best-practice, but also to understand why it is that this is an issue, and what can be done to achieve what I, in the past, mistakenly perceived as the reason that global mutable state made sense, without violating this fundamental rule.

To learn something, you must learn the rules, but to become an expert, you must learn when to break these same rules. Programming is no different. However, over several years of programming I’ve come to find that the “when” of “when to break the rules” is an exception, not the rule. If you’re breaking the rule more often than not, then you’re probably misunderstanding something or being lazy. It pays to heed the words of the experts superior to you in experience even when you don’t fully understand why, because eventually, with more experience, you will understand why, and if you failed to heed their advice, you’ll find yourself chastising your past self for it.

Let me start by explaining why it is that I’ve so often resisted this best practice and gone ahead with global mutable state anyway. When writing larger code bases, which most projects inevitably aspire to be, you will find that some of your code will be, as a matter of necessity, used by a large proportion of the code base. Using dependency injection as your only distribution technique, as an example, might drastically increase the verbosity of your code. Who wants to pass an instance a “Log” class to every single instance of a “GameState” object? I’ll admit that it’s not really THAT much extra work, but it has a distinct feeling of not being “DRY,” since I’m essentially repeating myself on every object instantiation. There are some subsystems which, by nature, the entire system will need to access, and would be better off being organized.

A pet peeve not only of myself, but of many people, is that some people seem to think that using the singleton pattern is a solution to this problem. Singletons are global state, and moreover, it appears to me that by now, the singleton is obsolete. Any problem solvable with a singleton can be solved with a static class with mutable private fields, and frankly these static classes don’t lie about their true nature. But both are global state, and both ought to be avoided anyway. But if you’re going to ignore the advice anyway, you might as well just use the static class instead, so users can recognize the warning signs of nondeterministic behavior before it’s too late.

I’ve learned, however, that all of my arguments came from a lack of understanding of the actual problem, and the actual solutions.

  • Global mutable state generally excludes communication channels that allow decoupled non-global objects to communicate (such as Observers or Service Locators). This is a significant solution to this problem.
  • Global mutable state implies that the global items in question contain actual values that change between subsequent calls. Global classes are still perfectly valid for resolving the problem of subsystems that need to be accessed from across the system as whole, so long as they incorporate the solutions from the previous bullet.

I’ll document one solution that I solved with my TimberWolf game engine.

In the past, I’ve had a static class named Log, which was declared as follows:

#ifndef H_CLASS_LOG
#define H_CLASS_LOG

#include <iostream>
#include <string>
#include <iomanip>
#include <sstream>
#include <fstream>
#include <ctime>
#include <mutex>
#include <exception>
#include <cstdlib>
#include <atomic>
#include "../../enum/LogLevel/LogLevel.hpp"

class Log {

public:

    Log () = delete; // static only
    ~Log () = delete;

    Log (Log&&) = delete;
    Log& operator = (Log&&) = delete;

    Log (const Log&) = delete;
    Log& operator = (const Log&) = delete;

    static bool cliOutputEnabled ();
    static void enableCliOutput ();
    static void disableCliOutput ();

    static LogLevel getCliFilterLevel ();
    static void setCliFilterLevel (LogLevel);

    static bool fileOpen ();
    static bool openFile (const std::string&);
    static void closeFile ();

    static bool fileOutputEnabled ();
    static void enableFileOutput ();
    static void disableFileOutput ();

    static LogLevel getFileFilterLevel ();
    static void setFileFilterLevel (LogLevel);

    template<typename ...T>
    static void log (LogLevel messageType, T&&... message) {

        std::string out;

        if (
            (m_cliOutputEnabled || m_fileOutputEnabled) &&
            (
                messageType == LogLevel::UNDEFINED ||
                messageType >= m_cliFilterLevel ||
                messageType >= m_fileFilterLevel
            )
        ) {
            out = Log::formatMessage(messageType, Log::concatMessage(std::forward<T>(message)...));
        } else {
            return;
        }

        if (m_cliOutputEnabled) {
            if (messageType == LogLevel::UNDEFINED || messageType >= m_cliFilterLevel) {

                if (messageType == LogLevel::ERROR) {
                    std::unique_lock<std::mutex> lock_stderr(Log::mutex_stderr);
                    std::cerr << out << std::endl;
                } else {
                    std::unique_lock<std::mutex> lock_stdout(Log::mutex_stdout);
                    std::cout << out << std::endl;
                }

            }
        }

        if (m_fileOutputEnabled && fileOpen()) {
            if (messageType == LogLevel::UNDEFINED || messageType >= m_fileFilterLevel) {

                std::unique_lock<std::mutex> lock_file(Log::mutex_file);
                m_file << out << std::endl;

            }
        }

    }

    template<typename ...T>
    static void verbose (T&&... message) {
        Log::log(LogLevel::VERBOSE, std::forward<T>(message)...);
    }

    template<typename ...T>
    static void notice (T&&... message) {
        Log::log(LogLevel::NOTICE, std::forward<T>(message)...);
    }

    template<typename ...T>
    static void warning (T&&... message) {
        Log::log(LogLevel::WARNING, std::forward<T>(message)...);
    }

    template<typename ...T>
    static void error (T&&... message) {
        Log::log(LogLevel::ERROR, std::forward<T>(message)...);
    }

    static void bindUnhandledException ();
    static void unhandledException ();

private:

    template<typename ...T>
    static std::string concatMessage (T&&... message) {

        std::ostringstream oss;
        (oss << ... << std::forward<T>(message));
        return oss.str();

    }

    static std::string formatMessage (LogLevel, const std::string&);

    static std::atomic<bool> m_cliOutputEnabled;
    static std::atomic<bool> m_fileOutputEnabled;

    static std::atomic<LogLevel> m_cliFilterLevel;
    static std::atomic<LogLevel> m_fileFilterLevel;

    static std::ofstream m_file;

    static std::mutex mutex_stdout;
    static std::mutex mutex_stderr;
    static std::mutex mutex_file;

};

#endif

As you can see, everything is static, and the class cannot be instantiated. This makes sense, because the Log class is something that needs to be accessed from all over the code base, and we generally want all log messages to funnel through a single location so none of them accidentally go astray before they are sent or stored. This was all fine and dandy when it was originally written, because it only had one purpose then, and that was to write error messages to std::cout or std::cerr. It interacted with a global I/O, but it didn’t store any kind of state.

After a while, though, I decided that a few features would be nice to have in the logging subsystem. First of all, I wanted to be able to filter log messages by their LogLevel enum. This was simple enough. Add a static field to store the minimum log level, and then compare the log level of each message before writing it to I/O. But now the log class contains the global state for the current log level. Anybody could change that, and it might make it hard to debug stuff because its behavior becomes nondeterministic. The behavior of the Log class depends on the order of modifications. If some code somewhere deep in the code base calls Log::setFilterLevel(LogLevel::ERROR) because the developer was getting inundated with verbose calls and wanted to temporarily silence them, and then forgot to remove it, all LogLevel::WARNING messages sent after that piece of code gets invoked will fail to display. The developer might mistakenly think that no errors are occurring. And there’s no way to receive those messages elsewhere unless the filter level is dropped again.

This hasn’t actually been a problem for me yet, but I was slightly aware of the possibility, but I hadn’t yet done anything to resolve it. I added additional features, such as allowing Log to write to a log file, and allowing a separate filter level for the log file. I noticed some additional potential problems after adding these features, however. First of all, I can only write to a single log file. What if I decide later on that I want to write only warnings and errors to one file, and all messages to another file. Or what if I want a console window in my game somewhere to display the error messages, without requiring the end-user to look at their terminal or open up a log file?

The Log subsystem is a problem that can be solved by the observer pattern. I can leave the API for sending log messages alone, because it implements everything the observables need (but I am going to add a context string that can be used for additional filtering, in case we want the log messages pertaining only to the sound system to go somewhere, for example). There will need to be a separate API for RECEIVING the log messages, however. One of these will be used to receive messages and route them to the command line. Another one will be used to put messages into a file. But more can be created and removed as needed.

If those changes are implemented, The Log class, which is the global portion of the subsystem, will no longer contain any global state at all (the vectors of observers are not technically “state” in this context, as they are not going to have any effect on one another, the log subsystem, or the game as a whole). Any code in the code base can send messages to it, but the output, which is the real tangible effect of the log subsystem, will only be controlled by non-global objects. Code far across the code base will not be able to create nondeterministic behavior in the logs of an unrelated system.

This is, for all intents and purposes, the best of both worlds. I can expose a global API without needing to define behavior globally. I can create a log subsystem that doesn’t require dependency injection, but also doesn’t expose any state to the global scope, while still supporting stateful behavior at one end of the pipeline.

With that explanation out of the way now, allow me to show you the finished solution. Note that between the old version and this current one, the namespace “tw” was added to everything.

Here’s the new Log class declaration:

#ifndef H_CLASS_LOG
#define H_CLASS_LOG

#include <iostream>
#include <string>
#include <vector>
#include <memory>
#include <sstream>
#include <mutex>
#include <functional>
#include "../LogObserver/LogObserver.hpp"
#include "../../enum/LogLevel/LogLevel.hpp"

namespace tw {
class Log {

public:

    Log () = delete; // static only
    ~Log () = delete;

    Log (Log&&) = delete;
    Log& operator = (Log&&) = delete;

    Log (const Log&) = delete;
    Log& operator = (const Log&) = delete;

    template <typename ...T>
    static void log (LogLevel messageType, const std::string& context, T&&... message) {

        std::unique_lock<std::mutex> lock(m_mutex);

        for (unsigned int i = 0; i < m_observers.size(); ++i) {

            m_observers[i]->notify(messageType, context, concatMessage(std::forward<T>(message)...));

        }

    }

    template <typename ...T>
    static void verbose (const std::string& context, T&&... message) {
        Log::log(LogLevel::VERBOSE, context, std::forward<T>(message)...);
    }

    template <typename ...T>
    static void notice (const std::string& context, T&&... message) {
        Log::log(LogLevel::NOTICE, context, std::forward<T>(message)...);
    }

    template <typename ...T>
    static void warning (const std::string& context, T&&... message) {
        Log::log(LogLevel::WARNING, context, std::forward<T>(message)...);
    }

    template <typename ...T>
    static void error (const std::string& context, T&&... message) {
        Log::log(LogLevel::ERROR, context, std::forward<T>(message)...);
    }

    static void bindUnhandledException ();

    template <typename T, typename ...Targ>
    static LogObserver* makeObserver (Targ&&... args) {

        auto observer = std::make_unique<T>(std::forward<Targ>(args)...);
        auto observerPtr = observer.get();

        m_observers.push_back(std::move(observer));

        return observerPtr;

    }
    static void registerObserver (std::unique_ptr<LogObserver>&&);
    static void registerObserver (LogObserver*);

private:

    template <typename ...T>
    static std::string concatMessage (T&&... message) {

        std::ostringstream oss;
        (oss << ... << std::forward<T>(message));
        return oss.str();

    }

    static std::mutex m_mutex;
    static std::vector<std::unique_ptr<LogObserver>> m_observers;

};
}

#endif

As you can see, the Log class is now nothing more than a shell, a notification system for a collection of observers (and also a system for creating and managing said observers). It presents a new class known as LogObserver, which is a virtual class.

#ifndef H_CLASS_LOGOBSERVER
#define H_CLASS_LOGOBSERVER

#include <string>
#include <set>
#include "../../enum/LogLevel/LogLevel.hpp"
namespace tw{ class Log; }

namespace tw {
class LogObserver {

// WARNING: Never call Log::* functions from inside subclasses of this.
// It will cause infinite recursion and you will be sad.

public:

    static inline constexpr unsigned int ALLOW_UNDEFINED = 0b00001;
    static inline constexpr unsigned int ALLOW_VERBOSE =   0b00010;
    static inline constexpr unsigned int ALLOW_NOTICE =    0b00100;
    static inline constexpr unsigned int ALLOW_WARNING =   0b01000;
    static inline constexpr unsigned int ALLOW_ERROR =     0b10000;

    LogObserver () = default;
    explicit LogObserver (unsigned int);
    explicit LogObserver (const std::set<std::string>&);
    explicit LogObserver (const std::string&...);
    LogObserver (unsigned int, const std::set<std::string>&);
    LogObserver (unsigned int, const std::string&...);
    virtual ~LogObserver () = default;

    LogObserver (LogObserver&&) = default;
    LogObserver& operator = (LogObserver&&) = default;

    LogObserver (const LogObserver&) = default;
    LogObserver& operator = (const LogObserver&) = default;

    void notify (LogLevel, const std::string&, const std::string&);

    bool undefinedAllowed () const;
    void allowUndefined ();
    void blockUndefined ();
    bool verboseAllowed () const;
    void allowVerbose ();
    void blockVerbose ();
    bool noticeAllowed () const;
    void allowNotice ();
    void blockNotice ();
    bool warningAllowed () const;
    void allowWarning ();
    void blockWarning ();
    bool errorAllowed () const;
    void allowError ();
    void blockError ();

    bool contextAllowed (const std::string&...) const;
    void allowContext (const std::string&...);
    void blockContext (const std::string&...);
    bool allContextAllowed () const;
    void allowAllContext ();
    void restrictContext ();

protected:

    virtual void notifyCallback (LogLevel, const std::string&, const std::string&) = 0;

    unsigned int m_allowedLevelFlags {
        ALLOW_UNDEFINED |
        ALLOW_VERBOSE |
        ALLOW_NOTICE |
        ALLOW_WARNING |
        ALLOW_ERROR
    };

    std::set<std::string> m_allowedContexts;
    bool m_allContextsAllowed {true};

};
}

#endif

LogObserver classes contain non-overrideable fields and methods for storing and manipulating data about the log levels and contexts that it’s interested in. The notify method is markedly non-virtual because it implements the code for determining if the LogObserver even cares about the message that’s being sent to notify. As such, notify is the method that is initially invoked whenever a log message is being processed. If all of the checks pass, then it is finally forwarded on to the protected virtual method notifyCallback, which is to be overridden to the tastes and needs of the implementor.

Let’s look at some examples of implementations, the few that I’ve already made, perhaps all that I’ll ever need:

#ifndef H_CLASS_LOGOBSERVER
#define H_CLASS_LOGOBSERVER

#include <string>
#include <set>
#include "../../enum/LogLevel/LogLevel.hpp"
namespace tw{ class Log; }

namespace tw {
class LogObserver {

// WARNING: Never call Log::* functions from inside subclasses of this.
// It will cause infinite recursion and you will be sad.

public:

    static inline constexpr unsigned int ALLOW_UNDEFINED = 0b00001;
    static inline constexpr unsigned int ALLOW_VERBOSE =   0b00010;
    static inline constexpr unsigned int ALLOW_NOTICE =    0b00100;
    static inline constexpr unsigned int ALLOW_WARNING =   0b01000;
    static inline constexpr unsigned int ALLOW_ERROR =     0b10000;

    LogObserver () = default;
    explicit LogObserver (unsigned int);
    explicit LogObserver (const std::set<std::string>&);
    explicit LogObserver (const std::string&...);
    LogObserver (unsigned int, const std::set<std::string>&);
    LogObserver (unsigned int, const std::string&...);
    virtual ~LogObserver () = default;

    LogObserver (LogObserver&&) = default;
    LogObserver& operator = (LogObserver&&) = default;

    LogObserver (const LogObserver&) = default;
    LogObserver& operator = (const LogObserver&) = default;

    void notify (LogLevel, const std::string&, const std::string&);

    bool undefinedAllowed () const;
    void allowUndefined ();
    void blockUndefined ();
    bool verboseAllowed () const;
    void allowVerbose ();
    void blockVerbose ();
    bool noticeAllowed () const;
    void allowNotice ();
    void blockNotice ();
    bool warningAllowed () const;
    void allowWarning ();
    void blockWarning ();
    bool errorAllowed () const;
    void allowError ();
    void blockError ();

    bool contextAllowed (const std::string&...) const;
    void allowContext (const std::string&...);
    void blockContext (const std::string&...);
    bool allContextAllowed () const;
    void allowAllContext ();
    void restrictContext ();

protected:

    virtual void notifyCallback (LogLevel, const std::string&, const std::string&) = 0;

    unsigned int m_allowedLevelFlags {
        ALLOW_UNDEFINED |
        ALLOW_VERBOSE |
        ALLOW_NOTICE |
        ALLOW_WARNING |
        ALLOW_ERROR
    };

    std::set<std::string> m_allowedContexts;
    bool m_allContextsAllowed {true};

};
}

#endif

ConsoleLogObserver is a specialization that sends the messages to the console.

#ifndef H_CLASS_FILELOGOBSERVER
#define H_CLASS_FILELOGOBSERVER

#include <string>
#include <set>
#include <ctime>
#include <iomanip>
#include "../LogObserver/LogObserver.hpp"
#include "../File/File.hpp"

namespace tw {
class FileLogObserver : public LogObserver {

public:

    FileLogObserver () = default;
    explicit FileLogObserver (const std::string&);
    FileLogObserver (const std::string&, unsigned int);
    FileLogObserver (const std::string&, const std::set<std::string>&);
    FileLogObserver (const std::string&, const std::string&...);
    FileLogObserver (const std::string&, unsigned int, const std::set<std::string>&);
    FileLogObserver (const std::string&, unsigned int, const std::string&...);
    ~FileLogObserver () override = default;

    FileLogObserver (FileLogObserver&&) = default;
    FileLogObserver& operator = (FileLogObserver&&) = default;

    FileLogObserver (const FileLogObserver&) = default;
    FileLogObserver& operator = (const FileLogObserver&) = default;

    std::string getFilePath () const;
    bool setFilePath (const std::string&);

protected:

    virtual void notifyCallback (LogLevel, const std::string&, const std::string&) override;

    virtual std::string formatMessage (LogLevel, const std::string&, const std::string&);

    File m_file {"./log.txt", File::ENABLE_WRITE | File::ENABLE_APPEND};

};
}

#endif

FileLogObserver is a specialization that stores a file handle and writes log messages to that file.

And finally, FunctionLogObserver is a specialization that stores and manages a collection of function objects that receive the message (so in a way, it can do anything).

#ifndef H_CLASS_FUNCTIONLOGOBSERVER
#define H_CLASS_FUNCTIONLOGOBSERVER

#include <string>
#include <set>
#include <vector>
#include <functional>
#include <mutex>
#include "../LogObserver/LogObserver.hpp"

namespace tw {
class FunctionLogObserver : public LogObserver {

public:

    typedef std::function<void(LogLevel, const std::string&, const std::string&)> Callback;

    FunctionLogObserver () = default;
    explicit FunctionLogObserver (const std::vector<Callback>&);
    explicit FunctionLogObserver (const Callback&...);
    FunctionLogObserver (const std::vector<Callback>&, unsigned int);
    FunctionLogObserver (const Callback&, unsigned int);
    FunctionLogObserver (const std::vector<Callback>&, const std::set<std::string>&);
    FunctionLogObserver (const Callback&, const std::set<std::string>&);
    FunctionLogObserver (const std::vector<Callback>&, const std::string&...);
    FunctionLogObserver (const Callback&, const std::string&...);
    FunctionLogObserver (const std::vector<Callback>&, unsigned int, const std::set<std::string>&);
    FunctionLogObserver (const Callback&, unsigned int, const std::set<std::string>&);
    FunctionLogObserver (const std::vector<Callback>&, unsigned int, const std::string&...);
    FunctionLogObserver (const Callback&, unsigned int, const std::string&...);
    ~FunctionLogObserver () = default;

    FunctionLogObserver (FunctionLogObserver&&) = default;
    FunctionLogObserver& operator = (FunctionLogObserver&&) = default;

    FunctionLogObserver (const FunctionLogObserver&) = default;
    FunctionLogObserver& operator = (const FunctionLogObserver&) = default;

    const std::vector<Callback>& getCallbacks () const;
    void setCallbacks (const std::vector<Callback>&);
    void addCallback (const Callback&...);
    void clearCallbacks ();

protected:

    virtual void notifyCallback (LogLevel, const std::string&, const std::string&);

    std::mutex m_mutex;
    std::vector<Callback> m_callbacks;

};
}

#endif

If you’d like to read the implementation .cpp files, those are available in the git repo. They didn’t seem to be necessary to illustrate the design pattern. Also, this example code isn’t perfect, and I’m sure in short order I’ll find reason to modify it. At the time of writing, TimberWolf doesn’t have a stable API, so please don’t use this as documentation for it.

Categories
Game Development Game Jams Software Development

Ludum Dare 38-40 Postmortem

It’s been nearly a year since I participated in Ludum Dare 38, which was my first ever game jam. I finished the game, and received decent ratings (I mean, not great over all, but given it was my first one), placing me between the 500th and 700th place in the rankings. I also participated in Ludum Dare 39, which unfortunately never made it to the playable stage by the 72 hour deadline. Ludum Dare 40 I likewise did not finish, however it was much more finished than 39, and was more technically advanced than either of the previous attempts.

Ludum Dare 41 is coming up in less than a month, and as I try to prepare my game engine implementations, now seems like as good a time as any to do a postmortem on each of my experiences.

Ludum Dare 38 – Asteroid Field

Let’s start with a simple explanation of my concept for Ludum Dare 38. The theme was “A Small World.” My game was called “Asteroid Field,” and was a game wherein you play as the planet earth, firing missiles into space to blow up large asteroids that are careening randomly toward your tiny planet. As time progresses, so does the average size, number, and speed of asteroids. The more points you score, the better your missiles get.

It was written in JavaScript using ES6 and transpiled and bundled with Webpack and Babel. The game engine was Phaser CE 2.

The majority of my time was spent learning the Phaser engine, which I had never used before. I had especially debilitating delays caused by issues with collision physics and particle emitters as implemented in Phaser then. I ended up implementing the physics collision math myself instead of going through the game engine, as the built-in arcade physics that I had initially intended to use gave strange behavior, as everything was expected to be a square, whereas my assets were generally better represented as circles. Unfortunately, my reliance on circles for hitboxes resulted in performance-killing trigonometry in the inner loop, which makes it difficult to play on older computers. I also made the mistake of starting the project without using the pushdown game state system, which resulted in having to significantly refactor my code when I decided I needed a game over screen.

My next biggest use of time was late-phase debugging, which I think is largely due to the fact that I went into this challenge knowing very little about the library I was using. I spent very little time on creating assets, game design, and play testing/balancing.

Overall, I would say I did a great job of limiting the scope of the project to make it realistic for my experience level with the tools, format, and process. Right from the start, my strategy was to create a game design with minimum complexity of mechanics, but with a lot of potential for cosmetic and very simple gameplay additions that could improve on the barebones concept. This strategy worked well for a first game.

Aside from inexperience, which can only be remedied over time, I think the primary lesson that I learned from this experience was that I needed to plan things more effectively. I think more complete mockups of the design, as well as figuring out the requirements for things like the physics engine going into the project would have helped me to avoid many roadblocks.

Ludum Dare 39 – Blackout (Incomplete, Unplayable)

A few months after Asteroid Field, I participated in Ludum Dare 39, for which the theme was “Running Out of Power.” My game was called “Blackout”, and the premise was that you would play as a dispatcher for a power company, and your screen would be a control panel showing a city and providing notices when the city’s power grid was damaged. Your job would be to dispatch your repair crews as effectively as possible to keep the power on for the greatest number of people. If you were not powering part of the city, that part would not be paying you, and if enough people are without power, the power company loses money. You lose when your company runs out of money.

This game was built on the same tech stack as Asteroid Field.

During this project, I spent the majority of my time, disappointingly, designing the digital representation of a city. This project was a lot more ambitious than my last one. From the design perspective, it didn’t seem that much more complex, certainly not enough to prevent me from completing the game on time. What I underestimated, however, was the complexity of the “street node” system, and the difficulty of implementing both the rendering and the physics of the vehicles travelling along said street nodes. I did get so far as determining that there needed to be object representations of “intersections” and “roads.” Intersections would contain references to each road that connected to them, and each road would contain references to each intersection acting as an endpoint. Additionally, roads would contain data on the amount of traffic, the speed of the road, and sometimes control points, through which a Catmull-Rom spline would travel to create curved or windy roads. The vehicles would be dispatched by passing their origin, destination, and the city grid to an A* pathfinding function, which would spit out the turn-by-turn directions. An update loop callback would then be responsible for moving the vehicle along this path based on the speed and traffic values for the street upon which the vehicle is currently traveling.

That design was straightforward enough. Unfortunately, that is more or less the entirety of the implementation that I had even figured out, let alone implemented. A number of problems appeared this time around. First of all, how do I draw this city? Well what I did was literally to draw it on a sheet of paper, and then draw a grid on top of it and manually write out a big long JavaScript object containing the intersections and streets and their coordinates as they appeared on the sheet of paper that I drew up. This took an ENORMOUS amount of time. I spent over a day and a half doing just this. With this task figured out, I also needed to implement the preloading system and game menu. Fortunately, I was able to complete those things, and I do have a menu complete with the ability to name your power company. After that, I needed to render the city. I had initially determined that a Catmull-Rom spline would be the best way to implement curves in the roads, since I could simply give it coordinates that fell on the path instead of off-path control points which Bezier curves need. Unfortunately, after completing the map, I found that the game engine I was using had already implemented Bezier curves, but not Catmull-Rom, which were more complex. At this point, I realized that I needed to implement the curve algorithms myself. This was about where I was when 72 hours came to an end. The extent of playability is that you can load the game, name your company, and start the disappointingly blank screen that is the “playing” state, even though somewhere in your browser’s memory is an invisible and very complex street map that I painstakingly wrote out by hand.

I think that the biggest lesson here, is, once again, to learn how to better plan ahead of time, better consider the technical limitations of the game engine at the time of the contest, and to better assess the difficulty and depth of the gameplay elements. I think this might have also gone better if I had been able to adapt the design as issues appeared, which might have been possible if I had more often benchmarked my progress according to a schedule.

Ludum Dare 40 – Speed Fishing (Incomplete, Kind Of Playable, Pretty Neat)

Okay, Ludum Dare 40 was the best of times, and the worst of times for me. I had hyped myself up for this challenge, for which the theme was “The More You Have, The Worse It Is.” I’m not going to lie, I hated the theme for this one. But that’s how Ludum Dare goes sometimes, and that’s what’s so cool about it. You show your adaptability by taking the theme and working with it, regardless of your feelings towards it. And so I did.

My game was called “Speed Fishing,” and its premise was that you are a fisherman in a fishing boat. You must collect as many fish as you can in a specific time limit. Before the timer is up, you must return to the starting place. You drive your motorized boat out into the water, avoiding obstacles such as land, sharks, and battleships, and collect the fish as you go. The more fish your boat contains, the more difficult it is to accelerate, brake, and turn, thus making travel more perilous.

This time, I broke from the web stack and decided to implement this game in C++ using SFML. I had already been working on the engine for a Minecraft-like game called Territory, and I ported much of the code from that, refactoring it to use SFML instead of GLFW. I chose SFML over GLFW because of its relative simplicity for 2D games, whereas GLFW was chosen for Territory because of its more do-it-yourself nature, and its support for Vulkan.

I spent a good part of my time working on real game features, believe it or not. That said, a few things took longer than expected because I had come from the web stack and expected functionality to be present that simply wasn’t, even in as high-level a framework as SFML. Namely, I had to figure out how to place elements on the screen correctly using direct coordinates (which I did in the previous games, but there seemed to be more helper functions available there to do things like center an asset), and I had to handle click events myself to determine if clickable elements had in fact been clicked. This click event handling was the biggest shock of all to me, although I was able to figure it out, and fortunately determining if a click lands inside a rectangle is fairly straightforward math. I also expected SFML to implement game states and associated functionality, but that was also left to me. Fortunately, I had already done most of that in the Territory project. As in the last Ludum Dare, however, the complexity of the game map implementation was one of the biggest consumers of time. In this case, I wanted the world to be mostly ocean with some land, and I wanted everything to be generated procedurally using perlin noise. I also needed to guarantee that the boat would spawn in water. Additionally, I needed to use the flyweight pattern for the rendering of the map itself, as I didn’t want to have millions of instances of multi-byte objects eating up large swaths of fragmented memory and slowing down the rendering because of CPU prefetch misses. I also needed to be able to move the player entity around through the world, all while keeping the camera centered on it. Because the world was larger than the screen, I also didn’t want to have to render the tiles that were outside of the visible screen area.

All of these features, I was able to implement, and just in time for the deadline no less. Unfortunately, things like the menu, score-keeping, fish entities, enemy entities, land collision, timer, and boat movement physics did not get completed within the time limit.

This time around, I will say that I had more distractions than before, and therefore I could have put more focus into this project than I did. Despite that, however, this project has more lines of hand-written code than both previous projects, and despite being incomplete, it is more technically complex than the completed Asteroid Field game, and I am truly proud of it. In all 3 Ludum Dare experiences, this one is the one of which I am the most proud. So that’s the good. The bad, however, is that the last day is one of which I did not sleep one minute. In fact, when the 72-hour mark was met, I had been awake for 30-something hours. I felt awful and tired. There were many moments between the 24-hour mark and the end that I was absolutely suffering. But I was determined to finish something, to be proud of something. I succeeded at that, and I am grateful.

I think one of the things that I learned with Speed Fishing was that I loved C++ more than I loved JavaScript. I also learned that code reuse is a beautiful and helpful thing, and I need to be doing more of that. This project, Speed Fishing, is the origin of a game engine I created called “ArcticWolf,” which comes after “TimberWolf,” which is the one that I used in Territory. Each of them is a specialization of a very deliberately similar API. I will be able to use these engines in not only future Ludum Dare projects, but also in free and commercial games that I will be releasing via my game company “Danger Zone Games.” So in a way, Ludum Dare 40 was the beginning of Danger Zone Games, it was the beginning of me making modular and reusable game engines, and created the ArcticWolf game engine (which was originally borrowed code from TimberWolf, which was borrowed code from Territory).

The Future!

I am very excited to participate in future Ludum Dare jams, and I am very excited to be creating games at long last. It’s been a dream of mine for a very long time, and it’s been a wild ride getting to where I am now. Even though I’m not making money on my games yet, I still love it. For me, game development isn’t a job or an industry. Game development is an amazing art. It incorporates technical skill through programming, graphic art and musical talent through game assets, and psychology through gameplay design. When you’re on your own, you have to be a skilled artist and technician in each of these fields. When you work with others, you may only need to specialize in one, but as a team your talents must stay in sync. I am proud to be a part of this community, and I am excited to see what we can all achieve in the coming years.

Categories
Programming Languages Software Development Web Development

JavaScript: Love, Hate, and Standardization

Recently, I’ve read a lot of hate toward JavaScript (which is easy to explain, see my previous post titled “Programming Language Wars“), and specifically there are a number of people who would request that browser vendors and the web standards organizations implement additional/alternative programming languages. This is a very naive request that I believe stems from lack of respect for standardization and the challenges facing browser development.

First of all, “JavaScript”, AKA ECMAScript AKA ECMA-262 AKA ISO/IEC 16262, exists primarily as a standardization, and that’s the important part here. Despite its origins, JavaScript has been around for decades, and likely will persist as such for decades, maybe even centuries to come. It was selected to be the core of the programming side of the standardized web. This standardization has been of particular benefit to making the web more or less completely compatible between a variety of browsers. JavaScript is defined in clearly written specifications, and as a result, each implementation, from Firefox’s Spidermonkey, to Chrome’s V8, to the dozens of other complete implementations, can be implemented separately and therefore be mutually independent from one another, while at the same time each supporting the same code so long as the developers making these products follow the standards correctly. This also affords freedom for competition to create distinct implementations instead of relying on a centralized one, which cultivates innovation.

There are many different languages that have a similar standardization and specification process, but this is not the only important characteristic of JavaScript. As a language designed for the web, it cannot make any presumptions of the architecture on which it will run. This means it would need to be provided as a scripting language (which JavaScript is), or a language of virtual machine bytecode (which WebAssembly is, which I’ll get into later). In addition to this, the language is supposed to be entirely forward compatible, so that code written a long time ago is capable of running on newer engines without modification. On some languages, this requirement can be, and is, safely ignored. Allowing things to be routinely deprecated and changed breaks old code. Often in these cases, the solution is simply to run the code on an outdated interpreter. This is not an option for browser vendors, however, whose releases already push several hundred megabytes in size, to say nothing of the nightmare of deciding which versions are still to be shipped, or the support nightmare of requiring end-users to manually choose their interpreter versions. These characteristics rule out a number of other possible languages.

Yet another issue with implementing other languages is the prevalence of JavaScript. Whether or not you like it, which is really not of concern here, JavaScript is arguably the most popular programming language in the world right now. Its forgiving nature cultivates ease of use for the beginner, largely due to being untyped and having automatic garbage collection. Additionally, the massive amount of development done due to the importance of the web has made JavaScript very performant. It also has a very C-like syntax, which is very popular and familiar to many developers.

Some people are hasty to criticize a language or library for catering to the needs of the less-experienced. These people usually complain about the loss of quality by making programming more accessible. But as is usually the case, ease of use does not preclude advanced functionality. JavaScript is a very mature language, and as such, it has a large user base that demands access to more advanced features and the ability to optimize performance more than an entry-level user requires.

And finally, and in my opinion, the biggest reason against implementing new languages for the browser is maintainability. As I write this, the Chromium JavaScript implementation, V8, has 1350 open bug reports. This is the number for a single programming language. If the web standards were to mandate the addition of another language, this would have to be implemented in yet another system of roughly equal complexity, and this would cripple the progress of development teams that even now are hard at work steadily advancing the single language implementation they have to maintain already.

All of these arguments aside, however, I fully understand the desire for developers to work in a language that suits their own style. I know the struggle of being forced to work with a programming language that I don’t like. So if I believe that, why do I still deny that other languages should be implemented in the web standards? First of all, I believe the benefit is greatly outweighed by the detrimental effect it would have on the speed of development for the open web standards. Second of all, if new languages are implemented, sooner or later, even those languages will come under fire for essentially the same reasons.

What we need isn’t to explicitly modify those standards to include languages, but to provide a common foundation on which new languages can be founded without requiring modification to the central standard itself. The simple answer to this is an intermediate language or virtual machine.

So far, JavaScript itself has acted as this intermediate language. Languages such as TypeScript, CoffeeScript, and Dart transpile into simplified JavaScript with boilerplate to implement missing features. Emscripten has even allowed the transpiling of languages such as C or C++ into JavaScript.

More recently, however, there is a new standard known as WebAssembly, which I believe solves this problem even better. One of the main purposes of WebAssembly was to provide an avenue for higher performance computing in a browser environment. One of the lowest levels of abstraction of an execution environment is assembly language. It encodes the primitive computational operations of the hardware itself, and has very few, if any, high level features. This lower level of abstraction also means that instructions written in it have little to no overhead. Due to the requirement for web languages to be architecture-neutral and to have some level of access control security, this had to be implemented in a virtual machine architecture that is conceptually close to a CPU architecture, but with a universal specification, and the ability to restrict the execution context to resources that it should have access to. This design is remarkably similar to the design of the Java virtual machine.

In the future, I suspect that many more programming languages will have either integrated or third-party backends that allow software to be compiled to run in this environment. This will require a fair amount of work to get this completed for each language, but I think it is the best way forward. With this method, any language could become a web-compatible language in the future, without needing to be integrated into the web standards, and without requiring extra work by the browser developers. By funneling all languages into the same intermediate language, it also ensures compatibility between them. And finally, because this method requires a compilation step, it means that if the programming language is updated and breaking changes implemented, previously compiled code will continue to function as normal until recompiled. The breaking changes may therefore still impact the developer, but not the end-user.

So to sum up: NO, new languages should not be integrated into web standards, but YES, developers of other languages should utilize transpiling and WebAssembly to make their languages available for web development. Other web-friendly programming languages should not be competing with JavaScript, they should rather see JavaScript and WebAssembly as their natural choice for an intermediate compilation target.

Categories
Software Architecture Software Development Web Development

Software Architecture and the YAGNI principle

I consider myself a developer with a good appreciation for a well-designed and carefully implemented architecture. I like to do things the “right” way whenever possible, instead of the quick way. I try my best to make everything I write modular, with minimal dependencies, and adaptable to any platform. Over the years I’ve been programming, however, I’ve learned that while that is a noble goal, there are certain occasions where the effort to conform to the aforementioned goal is not actually worth it. The key driving principle behind what I am referring to is the “YAGNI principle.”

YAGNI, meaning “You Aren’t Gonna Need It,” is the idea that you should never add a feature to your code base that you aren’t certain that you are going to need soon, or even eventually. I think many software developers who are relatively new to the craft, and who begin to learn about all of the architectural design patterns that can be used to keep your code base from turning into something that keeps you awake at night, sweating with fear, there is a tendency to begin developing a fear of cutting any corners ever. There is a general sense that making things more “smart” is always worth the effort, regardless of the cost. But YAGNI says that is not so. This was a principle that I unfortunately had to learn the hard way about 3 years ago.

I was working on an (unfortunately closed source) web-based asset management system software for my previous employer. Believe it or not, this was the first piece of professional software that I worked on (as in, the first that I was getting paid to make). I started work on it in mid 2014, and at that time I was making it with a fairly difficult-to-maintain procedural PHP code base, with no libraries or frameworks whatsoever. No functionality was being delegated to JavaScript, it was just going to be PHP doing all the work and sending a big chunk of HTML. In other words, it was an atrocious mountain of spaghetti at that point.

During that year, I worked on a lot of other projects, and began to learn quite a bit more about the PHP language, and above all, I decided that I was high time I learned how to use Object Oriented Programming to make my code a lot more maintainable, “DRY” if you will. So I dug into some tutorials and messed around with it, and got a general sense of how it worked. I also started learning about basic template systems, AJAX, and RESTful principles.

Some time in 2015, I decided to take all of this new-found knowledge back to the old asset management system code base, and try to apply it everywhere I could. And so I spent a large amount of time modernizing the code base and getting it up to par with what I thought was excellent design. And most of the ideas I had stuck around through the remaining years, more or less, so I know I wasn’t completely misguided.

There was one idea that I had, however, that was the biggest waste of time, and one of my absolute biggest regrets in the entire project. This is the purpose of this postmortem: to learn from this mistake that cost countless hours of valuable development time.

I decided that it would be cool to make this piece of software as portable as possible, so it was easy to install on any system my employer would ever use. I followed a lot of the patterns that I’d seen in other self-hosted web applications. Despite the fact that all of our apps only used MySQL as a database server, I wanted to make it possible to run it on MySQL, PostgreSQL, and SQLite. I also wanted to make this feature a library that I could use in other projects, and other people could use in theirs, so I did release that code under the MIT license on GitHub.

That idea, in itself, is not always bad. There are truly some occasions where doing such a thing would actually be valuable. They are all relational databases, so despite some syntactical and feature differences, they operate on the same core concepts. The way I decided to implement this cross-sql-platform support was by abstracting the entire concept of a relational database into PHP, and having that PHP act as a code generator writing SQL for prepared statements and submitting the relevant data. Essentially the idea was that instead of writing SQL queries, I would have database objects with methods allowing me to modify the structure and data in the database. The library would have to be pretty intelligent in figuring out how to safely and efficiently write a SQL query to perform the requested operation, so there were certainly security and performance risks associated with it.

I spent months of time adding each feature and testing them one by one as I started to integrate it with the existing code base. For all of the features that I actually integrated and tested, it actually worked quite well. The problem, though, was the incredible complexity of the system. SQL is implemented as a full programming language because of that complexity, and somehow I didn’t grasp that at the beginning of the project.

At some point, well into the project and not making the kind of traction that I wanted to make, I kept the library code on GitHub as before, but I scrapped integrating it into the asset management system, instead opting for another database abstraction design that I had learned and found to be much more reasonable for a project that is unlikely to ever be moved from the RDBMS that it started on (but is still flexible enough if it does). That approach was to create a single static class representing the database, with very basic abstraction methods as private, utilized by the public methods representing more high-level operations like inserting a new item, or editing an existing department. This means that all of the database-related code is in this single file, so it’s easy to change it whenever needed, but I can safely assume that any operation can be written specifically for MySQL. If the RDBMS needs to be changed one day, then the queries in this file would need to be rewritten, but the rest of the code base would not need to be touched, as the interface would remain entirely the same.

Most of the benefits of the abstraction library, at a fraction of the complexity. This design took far fewer lines of code to implement, and took a much shorter time. From the ground up, all features implemented in this static class, took less than half the time that I spent on a still-incomplete full database abstraction.

So the lesson to be learned here is what? Don’t implement a full database abstraction because it’s a waste of time? Absolutely not. Like I alluded to before, there are certainly some projects that would benefit from such a thing, and in those cases, you’ll just have to suck it up and trudge through thousands of lines and dozens of classes to make it work. But the question you should be asking before you go off on such a costly mission is an honest estimate of how often you expect the code base to switch its RDBMS. Maybe you’re marketing your software to a lot of varied users who may not want to install your database server of choice. In that case, you might want to consider it. But in my case, simply asking myself that question, the answer would have clearly been “probably never, maybe once in 5 to 10 years”. I should not have done it. Frankly, the cost of rewriting the queries in the static class MAYBE once would have been much less than the amount of time that I spent writing that database abstraction library.

So remember the YAGNI principle during the design phase of your project. Whenever you see something potentially complex getting added to the design, make sure to ask yourself, “Are you really gonna need it?” and if the answer isn’t a certain “yes,” then just remember that it’s okay to rewrite things later on when you have more information. Refactoring your code later just might be a lot cheaper and better than worrying about the minute details so early on. The finished product is rarely the same as the finished concept anyway, so be an adaptable programmer first, and write an adaptable code base later.