Decoding Kumu

Why we're moving away from users writing code

Technical tools trend towards code. The more extensible a tool needs to be, the more likely it is that someone outside of the development team has to write some code. Sometimes that’s people creating plugins, scripts, or extensions. Sometimes it’s people writing formulae or filters. The expressive power of code helps bridge a void that might otherwise involve incredibly complicated user interfaces.

You can’t go very far with Kumu without needing to roll up your sleeves, open up the docs, and get your hands dirty writing some code. It’s not a coincidence that many of the prolific and successful mappers in our community have a technical background, but Kumu isn’t a tool for programmers. It’s a tool for anyone that wants to make sense of connected complexity.

Kumu (as you know it today) uses a homemade language that is a dialect of CSS for defining views. It’s not a language that has a name, or any kind of formal specification. We started with CSS (the language browsers use for styling web pages) and translated some of the concepts to apply them to networks. Behind the scenes there’s a forked PostCSS parser, that we’ve extended to understand concepts like @settings, @controls, and traversal selectors.

The decision to define views with code has saved us a huge amount of incidental complexity over the years. Adding new style properties is trivially easy. We already have the machinery in place to support powerful ideas, such as partial views, without ever having to think about how to give them a user interface.

The problem is that the complexity is still there, we’re just offloading it to our users instead. To use Kumu, they’ll have to learn to write code.

To tell a compelling story using Kumu, you’ll need to spend far more time in documentation than you would with most tools. You’ll need to understand concepts like selectors. You’ll need to have an intuition for syntactic correctness, and fixing problems from our relatively vague parsing errors. Heck, for a long time, you even needed to know how to resolve Git-style merge conflicts when multiple people were editing concurrently.

If you have a technical background, this isn’t a huge ask. Compared to modern day text editors, you’ll probably notice the lack of automatic completions and intellisense though. There’s some jank as controls modify the current view and we serialise the internal syntax tree back to CSS (clobbering most comments and any personal formatting preferences). It can be frustrating, but armed with the documentation, you can be reasonably productive, reasonably quickly.

If you don’t have a technical background, Kumu can be a bit of a scary place. We have a “Basic Editor” which gives users access to an important subset of the power of views, but most projects need to go beyond that subset. Most people who use Kumu will need to learn to use the advanced editor, and learning to use the advanced editor is still a significant hurdle.

These are the kinds of hurdles we’re aiming to remove forever as we re-imagine the tool. It has become one of the principles that’s guiding our design. Users should never need to write code to make maps.

Advanced Editor

Let’s take a look at a fairly simple style rule in the advanced editor to get a sense of the level of complexity here (it is easy to forget, if you’ve been using Kumu for a while).

Legend Metadata

We’ll kick off with the odd way we encode legend information. The comment (the /* … */ syntax) above this rule is a directive which says:

• Include a preview of this style rule in the legend.

• Render the preview as an element rather than a connection.

• Use the label “Congress”.

This has some symmetry with the way programmers often use doc comments to annotate the details of functions, but in Kumu there’s nothing obvious that links that comment to the legend.

Selectors

The next thing you’re hit with is an attribute selector. It selects all elements with the type “Institutional Norms & Processes”. This is how you decide which elements or connections will be styled. Most selectors are easy enough to read, but for many users, writing them by hand will require a trip to the documentation.

This is one way to write this selector, but you also have access to shorthand syntax for writing type selectors.

["Element type"="Congress"] { ... }

/* is equivalent to */

congress { ... }

The shorthand here is quicker to type and feels more familiar to folks with a CSS background, but users have to guess what our rules for normalising types are.

What would the shorthand be for the selector from the example?

["Element type"="Institutional Norms & Processes"] { ... }

Well, it depends entirely on how our normalisation process handles the ampersand. It could be institutional-norms-processes or it could be institutional-norms-and-processes. I would bet on it being the former, but off the top of my head, I don’t actually know.

This also introduces an unfortunate bias towards people using the Latin alphabet. Want to take a guess at the shorthand selector for 使用高级编辑器? Software has a huge problem with Americentrism and here we’re part of the problem!

Some of the complexity in selectors is invisible! Trying to figure out why one of your style rules is overriding another? Time to learn about specificity!

Even the folks coming from CSS can have a hard time with selectors. Gone is the browser’s element tree, now you need to learn to think in graphs! There are no parent—child relationships, so you’ll need to learn about the syntactic extensions we’ve added, like traversals.

person --> organization

Does this select all people that have a directed connection to organizations? Or does it select all organizations that have a directed connection from people?

If you come from CSS, you’re likely to assume it’s the latter, because of the associativity of CSS combinators. If you didn’t come from CSS, then you’ll need the docs to resolve the ambiguity.

Style Properties

And finally we get into the style properties inside the rule. These follow the traditional CSS key-value model, and when sensible, we’ve kept keys that overlap with CSS, but we’ve tweaked others which created confusion.

["Element Type"="Institutional Norms & Processes"] {
  color: #63A3AE;
  font-color: #111;
  size: 70;
  border-width: 0;
  text-align: center;
}

For example, color will set the background color (not the text color like it would in CSS). Properties like margin and padding probably work like you’d expect, but they too come with caveats, such as requiring unitless values. No px or em here, please! Properties like text-align work on the vertical axis, because that’s more natural for element labels.

What’s this #63A3AE thing? To many people, it’s a meaningless jumble of letters. In fact (as you probably know if you’re reading this blog) it’s the hexadecimal representation of a color. The first pair of digits are the red channel, the second pair is the green channel, and the last pair is the blue channel. Even when you’ve gotten your head around hexadecimal numbers, RGB is a notoriously hard model for humans to work in. We’re terrible at figuring out how much green to subtract to make a colour less saturated, for example! Hex codes are a conveniently terse way to express color, but not to design or adjust it.

The example we’re looking at has five style properties and the syntax is approachable enough that it’s reasonably straightforward to read and to change. But consider that all style rules begin life as an empty block with no properties at all.

["Element Type"="Institutional Norms & Processes"] {
  
}

Your cursor is there, blinking emphatically, waiting for you to start typing. There isn’t any more guidance than that. Either you’ve memorised our property reference or your next trip will be to the docs. Every single time you have to leave your map to read documentation disrupts your cognitive flow and the visits to the docs are an inescapable part of using the advanced editor.

Syntax Errors

So far we’ve only talked about the happy path where everything works first try. What happens when you make a mistake?

Well, it depends on the kind of mistake. An innocuous misspelling isn’t considered a syntax error. Are you the kind person who writes colour instead of color? That will be on you to spot. It’s technically feasible for us to detect unrecognised properties, but it would be harder than you’d expect due to the feature-by-feature parsing architecture we use.

When you do hit a legitimate syntax error, such as a missing brace or an unclosed quote, your whole map will revert to its unstyled form and we’ll show you a “descriptive” error.

Maybe if we’d started building Kumu v2 in the days of Monaco and the Language Server Protocol we’d have an advanced editor that gave you red squigglies and autocompletions, but our CSS language predates them both and retrofitting them now would be hard, if not impossible.

New Editor

The advanced editor has served us and thousands of others well for over a decade now. It has enabled us to iterate on features far faster than we ever would have done if we’d been needing to build user interface for every single style property we support.

All good things come to an end, and in v3 we’re saying goodbye to the advanced editor and hello to a more intuitive way to edit views.

Let’s take a look at that same style rule inside an early version of the new editor in Kumu v3.

We’re bringing the editor much closer to what you’d expect to find in contemporary design tools, with Figma being a particularly strong influence on how we’ve structured the interface.

This is not the finished product. We expect to do multiple rounds of refinements between now and the release of Kumu v3, but there’s enough here to give you a flavour of how it will eventually feel to create views in Kumu.

Views & Style Rules

The first thing you may notice here is that we’re no longer looking at an entire view. Instead we’re focused in on a single style rule.

When you open a view, you’ll see a list of all the style rules it contains.

Each rule has a style preview (same as you’d see in the legend) and if you’ve added a legend label, you’ll see that there too. If not, then you’ll instead see the selector that the rule uses, which is often the most accurate descriptor (it doesn’t look like a selector but it is. Read on!).

We also removed the idea of selector specificity whilst we were at it. Now your style rules are applied in the order they are defined. A rule at the bottom of the list beats a rule at the top of the list and you can drag and drop to re-arrange them.

Selectors

You can’t have Kumu without the concept of selectors. They’re an essential part of style rules, but we also use them for controls, for configuring clustering, focus, and showcasing, and even for presentations and widgets inside markdown. But there’s no escaping it, selectors are code and users should never need to write code to make maps.

Part of redesigning the editor involved a significant rethink of our selector builder (a process which deserves a whole post of it’s own).

An early look at the selector builder from v3

Instead of leaving you to get the syntax right on your own, you get to type freely and we’ll suggest tokens that you can use to build a selector. Not only does this tend to be faster, it’s far less mistake prone, because it’s impossible to make syntactic errors, and the filtering will often guide you towards picking an existing value, rather than ever finding yourself typing out “Institutional Norms & Processes” by hand.

In those rare cases where you want to get at the underlying text, there’s still a way to hop back to editing manually!

This time, with red squigglies!

A lot of work went into figuring out a blend between a UI that would be approachable and UI that would be efficient compared to writing text. What you’re seeing above is actually “Selector Builder 5” but we finally feel like it’s into a really solid place.

I’m tactically sidestepping some of the unsolved challenges here. We’re still figuring out a natural way to build UI for traversal selectors and some kinds of parameterised psuedoselectors (think :to and :from) where you actually end up with selectors inside selectors. Maybe we’ll explore some of the options we’re considering in a future post about the selector builder.

Style Properties

Once you’ve tweaked the selector, all that’s left is to start editing the style properties!

Unlike the intimidating empty block in the advanced editor, everything you can possibly do to style an element is discoverable right there onscreen. Borders, shadows, bullseyes, flags, labels. They’re all right there.

We’ve gone to great lengths to make these controls as user-friendly as possible. Every number you see is draggable (no more guessing values). Every colour is actually a colour (and opens a colour picker when clicked). We’ve got buttons that toggle and buttons that cycle. We’ve got dropdowns for selecting fields.

We’re currently in the final stages of supporting dynamic style values (think scaling a property, or categorizing by color, based on a field value).

A great benefit of this approach will be knowing which properties can take can dynamic style values, just by looking at the editor.

Internal Representation

CSS is a great format for humans, but it’s kind of awkward for computers. To interpret CSS you need to parse it. What you get back from the parser is a syntax tree which is easy to program. If something changes the syntax tree, then you can serialise it to get new CSS. If something changes the CSS, you can parse it to get a new syntax tree.

In v3, we can skip the parser and serialiser completely and always work with the syntax tree instead. It’s not as readable as CSS but JSON is still readable enough.

Most Kumu users won’t ever need to know what language we’re using internally, but the decision to move away from CSS means we’ll no longer be maintaining a custom CSS parser. That’s less code we need to ship to browsers for Kumu to work and it’s less time spent parsing views before your map can render.

We’ll Miss Writing Code

Sure, the advanced editor has some rough edges. Sure it’s unusual for a tool with a primarily non-technical user base to expect people to write code. And sure, it has a steep learning curve. But, it has its own charm and some of our users have invested significant time developing competency there.

When you’re editing in the advanced editor, you’re just editing text. It means you’re susceptible to making all sorts of mistakes, but plain text also has super powers.

The browser’s native search just works. Undo and redo are well defined operations with predictable operational semantics and built-in keyboard shortcuts.

You can see the complete definition of the view in one text input. That means you can edit multiple style rules in parallel without having to navigate backwards and forwards.

You can select, copy, and paste arbitrary selections and it’s hard to overstate how important this is. You can copy style rules to duplicate them. You can copy your view into a text editor of your choice as a backup. You can copy the style properties from one rule to another. We’ll try to mitigate some of this by making it straightforward to duplicate style rules and even entire views. We’ll also think about making it possible to actually copy (or export) a view or a style rule directly to your clipboard as JSON that you can paste into other projects.

The advanced editor also allowed us to iterate rapidly for years and years. We could experiment with new types of decorations, settings, and controls, without changing the UI at all. No need to figure out where something will fit, or which icon to use, or how to make it accessible to keyboard users. We’d just document the new properties and people could start using them immediately. There is no doubt that we will iterate more slowly when it comes to the new editor, but maybe that’s fine. There’s not a lot of decorative space left for Kumu to explore.

What’s Next?

There’s stuff we can’t show yet, like controls and partial views and that’s because we haven’t implemented them! But rest assured, we’re not planning to remove any advanced editor functionality, you’ll just be controlling it with inputs and buttons and dropdowns, rather than with text.

There’s a bright future for the new editor, but the ideas presented here are not yet rigid. We have some strong opinions but we’re ready to engage with feedback that challenges them too. Hop into our Discord server for v3 or drop us a line at support@kumu.io and let us know!