Starting with rebar, live upgrades and packaging erlang code RPM

Rebar is something that I wish I was taught when I first started using erlang, it will help new programmers get started with erlang without having to do the boring "book-keeping" behind the setup/compile/test/release system.  It really is a good way to get started.

Some readers may note that there are many rebar tutorials out on the net, however nothing for Red Hat style Linux, and many of them more complicated than they need be.

Installing Rebar

I'm lazy and always use a relatively modern Linux distribution. If you're using Fedora 15/16 or Red Hat Enterprise Linux 5 or 6 (with EPEL) you can install rebar with the command

$ sudo yum install erlang-rebar

This will also install the necessary erlang runtime/vm and other related tools, correct for whatever architecture your system is running.

Creating your first erlang project

[wmealing@mutalisk wmealing] mkdir myfirstapp  
[wmealing@mutalisk wmealing] cd myfirstapp 

[wmealing@mutalisk myfirstapp] rebar create-app appid=something
==>; myfirstapp (create-app)
Writing src/something.app.src
Writing src/something_app.erl

Writing src/something_sup.erl

If you get the error "ERROR: Template simpleapp not found." when attempting to try the above commands, copy the templates into the user-specific template directory.

[wmealing@mutalisk myfirstapp] mkdir -p $HOME/.rebar/templates/

[wmealing@mutalisk myfirstapp] cp /usr/lib*/erlang/lib/rebar-2/priv/templates/* $HOME/.rebar/templates/

Backup cowboy, what just happened ? Rebar created an app based on the templates that are included.  These templates are located in the /usr/lib64/erlang/lib/rebar-2/priv/templates/ directory if you're on 64 bit  or if you're still using 32 bit erlang, /usr/lib/erlang/lib/rebar-2/priv/templates/ directory.  So it creates from the "simpleapp*" named files substituting whatever you supplied for the appid where it is mentioned in the templates.

Unsurprisingly this created the src directory and a simple "application" with the appid provided on the command line.  Can I suggest you be a little bit more creative than my example.

Even though you have not written any code yet, run compile and test that the skeleton that you have created compiles.

Compiling

[wmealing@mutalisk myfirstapp] rebar compile
==> myfirstapp (compile)
Compiled src/something_app.erl
Compiled src/something_sup.erl

And yeah! your first template-style erlang app compiled, and the "beam" code is in the ebin directory.  See the project layout so far in the example below.

[wmealing@mutalisk workspace] tree myfirstapp
myfirstapp
|-- ebin
|   |-- something.app
|   |-- something_app.beam
|   `-- something_sup.beam
`-- src
    |-- something_app.erl
    |-- something.app.src
    `-- something_sup.erl

2 directories, 6 files

The project so far is pretty much a basic supervisor for your application, this by itself isn't quite enough to be useful.  Usually you'd want your application to do something useful.  This typically involves creating  a "server" of some kind.  The most common is the gen_server.

Many of the other rebar tutorials seem to either enjoy making additional complex files however modern releases of rebar already include templates for a basic server.

Creating a server

[wmealing@mutalisk myfirstapp]$ rebar create template=simplesrv srvid=mine_srv
==> myfirstapp (create)
Writing src/mine_srv.erl

This creates a gen_server style server in the src/mine_srv.erl file that you can modify to how you see fit.

Lets make it greet the user in a stereotypical Australian greeting, for use later when we show hot-code-upgrading in the section "Running the code".

So, modify src/mine_srv.erl and its existing handle_cast function to print the Aussie greeting to the stdout in the gen_server cast.

-export([greet/0]).

greet() ->
    gen_server:cast(?MODULE, {matchallthethings}).

handle_cast(_Msg, State) ->
  io:format("Gday Mate, how are ya ?~n"), 

  {noreply, State}.

We added the exported "greet" call so we can abstract away the implementation from the caller.  We wont test it now, but we'll test it later.

To have mine_srv be supervised/started by the application, the something_sup (supervisor) will need to be instructed to start mine_srv as a worker.

In src/something_sup.erl lets change the init function to start the mine_srv worker.

init([]) ->
    MineSrvWorker = ?CHILD(mine_srv , worker),
    Children = [ MineSrvWorker ],
    RestartStrategy = { one_for_one , 4, 9600},
    {ok, { RestartStrategy, Children } }.

Cleaning up (make clean)

Just so we can test how it works, issue the command rebar clean and rebar compile to compile the supervisor and your new gen_server server.  Just as you can compile with erlang, you can also use rebar to  "make clean" and recompile as you would when any project is changed.

[wmealing@mutalisk myfirstapp]$ rebar clean
==> myfirstapp (clean)
[wmealing@mutalisk myfirstapp]$ rebar compile
==> myfirstapp (compile)
Compiled src/something_app.erl
Compiled src/mine_srv.erl
Compiled src/something_sup.erl

Running tests

You may or  of running unit tests or integration tests, rebar supports a few kind of tests, but i'll kick it off with eunit to get things started.

Rebar looks at eunits configuration options through a file named rebar.config  file. The erlang-rebar package puts one in /usr/share/doc/erlang-rebar-2/rebar.config.sample by default, but you should be able to make an empty one with just the following lines in it.

[wmealing@mutalisk myfirstapp]$ cat rebar.config

%% Erlang compiler options

{erl_opts, [{i, "test"}, {src_dirs, ["src"]},

            {platform_define,

            "(linux|solaris|freebsd|darwin)", 'HAVE_SENDFILE'},

            {platform_define, "(linux|freebsd)", 'BACKLOG', 128}]}.

{eunit_opts, [verbose, {report, {eunit_surefire, [{dir, "."}]}}]}.

{cover_enabled, true}.

This line ensures that eunit will include the test directory in its locations to search for "included" files.

Create a test directory

[wmealing@mutalisk myfirstapp]$ mkdir test

For each of the parts of your project you wish to test, add something similar to this at the end.  For my example I append the below to mine_srv.erl.

-ifdef(TEST).
-include("something_tests.hrl").
-endif.

The HRL file extention is uncommon and is intended to show that the contents of the file should be included in another erl file.

Next create the test/something_tests.hrl file.  This should contain standard eunit tests.  The name something_tests.hrl should accurately reflect what the "something" is that you are testing. It is a good practice to name it something related to the module or server  and type of testing you are doing. Here is a quick example of a very simple test in the the test/something_tests.hrl file.

-include_lib("eunit/include/eunit.hrl").

my_test() ->
    ?assert(0 + 0 =:=  0).

my_second_test() ->
    ?assert(0 + 1 =:= 1 ).

simple_test() ->
            ?assert(1 + 2 =:= 3).

To run these tests run the command:

[wmealing@mutalisk myfirstapp]$ rebar eunit
==> myfirstapp (eunit)
======================== EUnit ========================
module 'something_sup'
module 'mine_srv'
  mine_srv: my_test...ok
  mine_srv: my_second_test...ok
  mine_srv: simple_test...ok
  [done in 0.008 s]
module 'something_app'
=======================================================
  All 3 tests passed.

Cover analysis: /home/wmealing/Documents/workspace/myfirstapp/.eunit/index.html


A source code coverage analysis can be found in the the url above.
I'll leave it as an exercise to the reader to figure out what it shows when tests go wrong.

Creating Documentation

Rebar can build documentation on your API, or whatever additional information you wish to provide assuming you make it happen in the edoc format.

Edoc requires writing documentation in code. It uses the  @something style tagging and immediately documents the function right below it.  For example:

  %% @doc Greets the user on the standard output, returns nothing

  greet() ->

       gen_server:cast(?MODULE, {matchallthethings}). 

The tags must follow a comment (%%) followed by a space starting on the first character of the line.

There are a bunch of supported tags, the edoc welcome page has a bunch more on the topic.  Rebar makes it easy to generate html documentation from your code by running the command

[wmealing@mutalisk myfirstapp]$ rebar doc
==> myfirstapp (doc)


Stylized html format pages will  appear in the docs subdirectory, you can regenerate them at any time, but this will destroy any changes made to the files in the doc directory.

Running your code.

One thing that rebar does not do, is run your application for you.   You will need to run it yourself.  Fortunately its actually pretty easy.

[wmealing@mutalisk myfirstapp]$ erl -pa ebin/
Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false]
1> application:start(something).
ok

Now to test the greet function we wrote earlier:

2> mine_srv:greet().
Gday Mate, how are ya ?
ok


Building the first release

It is rarely the case that the software will only run on the system that it is developed for.  The smart cookies at Ericsson and rebar know this and have a release mechanism that can be used to deploy the "whole stack" as a release.  This includes all the files, applications and libraries that would be used to run the application.

Rebar simplifies the release requirements, but its still a little involved to "get right".   Lets walk through using the existing sample to build a first release to deploy on a system.

[wmealing@mutalisk myfirstapp]$ mkdir rel
[wmealing@mutalisk myfirstapp]$ cd rel/

[wmealing@mutalisk myfirstapp]$ rebar create-node nodeid=something_node

==> myfirstapp (create-node)
Writing reltool.config
Writing files/erl
Writing files/nodetool
Writing files/something_node
Writing files/app.config
Writing files/vm.args

The name here shouldn't be labelled after a release codename or version but the "service" or daemon name that you'd like to start the script with.


Hot upgrades

@fixme This is something I also plan to cover another day.


Wrapping it up.

So rebar is a very useful tool to add to your arsenal, and if it is used for all your projects it will make a good head start on creating standardized, simple maintainable code that other (seasoned) erlang programmers should be familiar with.


Footnotes

I know there could be a better project layout for this, but with the mindset of keeping it simple, I've negated mentioning multiple applications and some of the finer points of migrating state in the gen_server between updates, this should be considered when you update your erlang application.