Building a Discord clone in Elixir

Elixir is a functional programming language designed for high-concurrency. Lately I heard many cool startup using it (like tandem.chat from YC W20), so I want to give it a go. I am building clone of the messaging app Discord from scratch, it should be live at https://dixord.herokuapp.com.

shut up and show me the code

The MVC layer

I am following this phoenix-chat-tutorial, and I will be calling my Discord clone "Dixord" (from Elixir Discord). We are using the Phoenix web framework, that has a terrific documentation. The first concept I bump into is Phoenix channels. There is some awesomeness already, quoting from the docs

Channels support long-lived connections, each backed by a lightweight BEAM process, working in parallel and maintaining its own state. This architecture scales well; Phoenix Channels can support millions of subscribers with reasonable latency on a single box, passing hundreds of thousands of messages per second. And that capacity can be multiplied by adding more nodes to the cluster.

Channel is an abstraction over websockets, the Phoenix architecture lets you define and manipulate Channels inside the `lib` folder using Elixir controllers, and then the magic `socket` modules from phoenix let you subscribe to Channels using Javascript.

The Data layer

Elixir uses Ecto as a Database wrapper and query generator. It's not strictly a ORM since there are not objects in Elixir, but provides you a convenient way to speak with the database. A nice syntactic sugar I just discovered while writing the queries is the pipe operator, that work exactly like in Bash.

Writing database interactions highlights how Elixir is concise. Loading all the messages from a chatroom and push them to the front-end can be done just with a one-liner, using a pipe operator with a lambda function passed to Enum.each.

After 15 minutes of building that's what I have got. You can try it at https://dixord.herokuapp.com/

After setting up ExUnit (the unit test framework) and code coverage I am uploading my toy chat to Heroku. Unfortunately the process is not as smooth as Python, with some manual buildpacks step required, but the documentation is still great. The app is live at https://dixord.herokuapp.com/.

Server Side Rendering

Elixir allows you to have a scalable number of concurrent processes in your server, this fits well the Server Side Rendering pattern: instead of compiling the html on the front-end (using React), you compile the html on the back-end in the server. You can do that cause every client maintains an open socket connection with the server, and with that connections the client receives live updates in form of super small JSON. Among the other advantages, this makes the code complexity an order of magnitude easier, like building a Twitter clone in 15-minutes with real-time updates. Server Side Rendering is inside the PhoenixLiveView module in Phoenix, and if you still don't totally get you can watch PhoenixLiveView for non-Elixir developers.

To pick up this new pattern I found a code-guided tutorial on elixirschool.com LiveView with PubSub Presence. The tutorial hasn't that many stars on Github, but the author Sophie DeBenedetto has a bunch of talks at Elixir conference about LiveView so I will follow along.

Dixord with server side rendered messages, with 0 lines of javascript (and around 50 of elixir).

I went on and added the Server Side Rendering pattern to Dixord. I basically built a 'react component' in the backend, in what Elixir calls a LiveView. This component sends rendered html to the front end, and allowed me to add that discord styling you can see in the picture above.

LiveView allow us to have 0 javascript codebase thanks to server side rendering of html. We might still need some Javascript hooks though to implement some UX features, for instance we want the chat to scroll down all the time there is a new message. LiveView supports JS interop Hooks, so with just 5 line of Javascript I can complete a pretty functional chat that scrolls down at every new message received/sent.

More real-time features with Phoenix.Presence

The Phoenix.Presence module is an interesting abstraction to track presence (e.g. who is online) across processes or channels. Under the hood it works with the same broadcasting mechanics (Phoenix.PubSub) of LiveView that we already used before. In this section we are going to build a guests tracking system for Dixord, that assign a guests name to every connected user and shows you which guests are online at any given moment.

To track users inside the app we can use the concept of Plugs. Plugs is a Elixir specification to build composable modules in web application: you can build a authenticate_user function plug and then plug it into any Endpoint, Router or Controller on Phoenix.

With 50 more lines of plain Elixir I added real-time features like a list of guests that are online in the chat, and a real-time indicator of who is typing.

Once you pick up the Presence pattern (really similar to LiveView) is really easy to add a bunch of features. You just need to go in your .leex template, add some triggers like phx_change on a DOM element, and finally add a handler in the LiveView object that broadcast the update either through Presence or directly through the Endpoint. You can see in my ~50 elixir lines commit how I added all the features listed in the image above following this pattern.

Lines of code count for Dixord with the features in the screenshot above, 46 lineswith 80% test coverage ;)

Generating code with Mix.Phx.Gen

MIX is the Elixir package manager, that is a similar flavour to Ruby on Rails let you generate a bunch of scaffolding files with just one command. For example mix.phx.gen.html User generates for you a surprising rich amount features for the static resource User. Beside views and controllers it generates test (more than 40+) and documentations that are pretty useful to understand the best practices inside Phoenix.

I added User model to Dixord using the mix.phx.gen.html scaffolding utilities

I went on to build a User model, that came out pretty easy with the nice Ecto APIs (the Elixir database query language): after building the association between `message` and `user` I didn't need to user any JOIN statement but I just added the Repo.preload, that preload all the associations on the model you are querying.

I set up the Pow library for authentication together with my User model

I decided to add two other nice dependencies to Dixord. The first is Pow, a Elixir native authentication system for web app to complement my User model I created just before. The second is Surface, a server-side rendering component library that allows me to write React style front-end code instead of native Elixir EEx templates. With this two new entries it took me just another afternoon of coding to build the full authentication flow for my User model as in the screenshot above. Notably, Pow offers a sister library Pow_assent to enable different login providers like Facebook or Github, here is a nice video about buiiding e2e authentication in Elixir Easy Authentication in Elixir & Phoenix with the pow & pow_assent libraries.

A more complex system design with Chats and Servers

So far we built just a single view application, let's take a step further and add a system that supports a two-level hierarchy of Servers (set of chats) and Chats (set of messages). Phoenix had an extensive guide that articulate system design best practices around the Phoenix.Context module: a tool to modularise and contain the complexity of Elixir systems. I found a nice talk if you want to dig more in to this: Building Beautiful Systems With Phoenix Contexts and Domain-Driven Design from ElixirDaze 2018.

At end of the day, Phoenix.Context are just Elixir modules that serve the purpose of abstracting away specific design decision from the rest of the application. The powerful idea here is that decoupling contexts from one another will allows us later to use event-driven messaging, where different context communicate just by publishing and subscribing to specific events. This pattern is rooted in the singular OTP behaviour that Elixir is built in, more on that in Intro to OTP in Elixir.