Building Rust Devices with HyperBEAM M3 Beta: mini-Roam API (Vol. 1)

Inspired by ar.io's roam - a tool for stumbling upon permaweb content - we put together a guide for new HyperBEAM developers building custom Rust devices, using a mini version of roam as an example.

One of HyperBEAM's core strengths is its ability to wrap almost any code natively inside the node, giving it sensible REST standards and interoperability with the ao network.

In this tutorial we'll go from start to finish - by the end, you'll have a custom Rust device running inside your own HyperBEAM node. This should serve as a basis for bringing any arbitrary device inside HyperBEAM!

Prerequisites

To run a HyperBEAM node you need to have Rust, rebar3, erlang OTP27 and other dependencies available on your machine. For a full prerequisites setup installation, please refer to the official documentation of hyperbeam on hyperbeam.ar.io

Installation & initialization

First let’s clone the HyperBEAM repo and run it. At the time of writing this guide, we will use the edge branch as it’s the most up-to-date M3 beta branch. In the future, while referring to this guide, it’s advised to use the most recent stable branch (ideally, main).

git clone -b edge https://github.com/permaweb/HyperBEAM
cd HyperBEAM
rebar3 compile
rebar3 shell

After that, the node should be running at http://localhost:8734/

If rebar3 shell doesnt work (especially if you are on ubuntu), you can do the following:

erl -pa _build/default/lib/*/ebin

Then in the erlang shell:

application:ensure_all_started(hb).

Perfect. Now that you have installed and you are running hyperbeam node locally, let’s move to the core of this repo

my_device rust device: ~roam@1.0 API

Rust crate Initialization

cd native
cargo new my_device --lib 
cd my_device

touch .gitignore

And now let’s replace the content of Cargo.toml with this one:

[package]
name = "my_device"
version = "0.1.0"
edition = "2024"

[lib]
name = "my_device"
path = "src/lib.rs" 
crate-type = ["cdylib"]

[dependencies]
anyhow = "1.0.98"
rustler = "0.29.1"
ureq = {version = "3.0.11", features = ["json"]}
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"

And let’s add manually .gitignore content:

/target
.env

And let’s replace the content of lib.rs with this skeleton NIF rustler function (hello) - replace it and run cargo build to verify compilation. By doing that, we will have our first Rust NIF function:

use rustler::NifResult;

mod atoms {
    rustler::atoms! {
        ok,
    }
}

#[rustler::nif]
fn hello() -> NifResult<String> {
    Ok("Hello world!".to_string())
}

rustler::init!(
    "my_device_nif",
    [hello]
);

mini-Roam API functionality

As the core of this tutorial, let’s try to re-implement a mini-version of the famous permaweb app (Roam - roam.ar.io built by ar.io CEO, Phil (aka vilenarios)). For the sake of simplicity, we will limit the features and functionalities of the Roam app and solely focus on how to implement a very minimalistic version as a rust device. We will roam the most recent blocks searching for media content (images).

we will use ureq as it’s a safe, lightweight HTTP client with a great feature of blocking I/O instead of async I/O to query Arweave Goldsky GraphQL client.

Now let’s create a core directory and a file called arweave.rs to pack our Arweave utility functions:

mkdir src/core
cd src/core

touch arweave.rs mod.rs

In mod.rs add the following line pub mod arweave; and in the header of lib.rs add pub mod core;

And now, let’s add the core functionality of our device, the mini-Roam logic and communication with Arweave’s permaweb via the Goldsky gateway in arweave.rs add:

use anyhow::Error;
use serde_json::Value;
use ureq;

pub fn query_permaweb() -> Result<String, Error> {
    let url = "https://arweave-search.goldsky.com/graphql/graphql";

    let query = r#"
        query {
          transactions(
            first: 1
            sort: HEIGHT_DESC
            tags: [
              {
                name: "Content-Type"
                values: [
                  "image/png"
                  "image/jpeg"
                  "image/webp"
                  "image/gif"
                  "image/svg+xml"
                  "image/avif"
                ]
                op: EQ
                match: EXACT
              }
            ]
          ) {
            edges {
              node {
                id
                tags {
                  name
                  value
                }
                block {
                  id
                  height
                  timestamp
                  previous
                }
              }
            }
          }
        }
    "#;

    let body = serde_json::json!({ "query": query });

    let response = ureq::post(url)
        .header("Content-Type", "application/json")
        .header("Accept", "application/json")
        .header("DNT", "1")
        .header("Origin", "https://arweave-search.goldsky.com")
        .send_json(body)?;

    let json: Value = response.into_body().read_json()?;

    let tx_id = &json["data"]["transactions"]["edges"][0]["node"]["id"];
    let tx_id = tx_id.as_str().unwrap_or("No ID found").to_string();

    Ok(tx_id)
}

To test its functionality, let’s add a simple rust test for the query_permaweb() function, again, for the sake of simplicity, let’s add the tests module in lib.rs given our small library. At the end of the lib.rs file, add the following code then run cargo test -- --nocapture :

/// tests


#[cfg(test)]
mod tests {
   use super::*;


   #[test]
   fn test_query_permaweb() {
       let id = query_permaweb().unwrap();
       println!("TXID: {:?}", id);
       assert_eq!(id.len(), 43);
   }
}

You will see an output like this indicating success:

running 1 test
"TXID: H7o52GAEGrCoBBMaOQw0K6OV_K2Vvf2SsKk6GhaHmnc"
test tests::test_query_permaweb ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.54s

As a final step in rust, let’s add out the query() NIF function that uses DirtyCpu scheduler as the underlying query_permaweb() function requires more than 1ms for execution and makes network I/O calls – in lib.rs:

#[rustler::nif(schedule = "DirtyCpu")]
fn query() -> NifResult<String> {
    query_permaweb().map_err(|err| Error::Term(Box::new(err.to_string())))
}

// update this line adding `query()` function
rustler::init!("my_device_nif", [hello, query]);

Now that we finished building the core component of our rust device, let’s add this script file at the root of the HyperBEAM repo that will make it easier to bootstrap the node with the device building process later on. At root level run touch ./build.sh then add the following content

#!/bin/bash
set -e  # Exit on any error

# Function to build and copy NIF
build_nif() {
    local nif_name=$1
    echo "Building ${nif_name}..."
    cd native/${nif_name}
    cargo build --release
    mkdir -p ../../priv/crates/${nif_name}
    cp target/release/lib${nif_name}.so ../../priv/crates/${nif_name}/${nif_name}.so
    cd ../..
}

# Main script
echo "Starting deployment..."

# Build all NIFs
build_nif "my_device" // our current mini-Roam device

echo "All NIFs built and copied successfully"

Run chmod +x build.sh then execute it by invoking ./build.sh

Interfacing my_device with erlang

Now that we have our device successfully built, let’s interface it with the erlang part of the HyperBEAM node, along the rest of the devices stack, by creating the following erlang modules. From root directory:

cd src
touch my_device_nif.erl dev_roam.erl

In my_device_nif.erl module add the following code:

-module(my_device_nif).
-export([hello/0, query/0]).
-on_load(init/0).

-define(NOT_LOADED, not_loaded(?LINE)).

-spec hello() -> binary().
hello() ->
    ?NOT_LOADED.

-spec query() -> binary().
query() ->
    ?NOT_LOADED.

init() ->
    {ok, Cwd} = file:get_cwd(),
    io:format("Current directory: ~p~n", [Cwd]),
        
    PrivDir = case code:priv_dir(hb) of
        {error, _} -> "priv";
        Dir -> Dir
    end,
    
    % following the path created by our build.sh bash script to locate .so images
    NifPath = filename:join([PrivDir, "crates", "my_device", "my_device"]),
    io:format("NIF path: ~p~n", [NifPath]),
    
    NifSoPath = NifPath ++ ".so",
    io:format("NIF .so exists: ~p~n", [filelib:is_file(NifSoPath)]),
    
    Result = erlang:load_nif(NifPath, 0),
    io:format("Load result: ~p~n", [Result]),
    
    Result.

not_loaded(Line) ->
    erlang:nif_error({not_loaded, [{module, ?MODULE}, {line, Line}]}).

In dev_roam.erl add the following code:

%%% @doc A device that allows querying the Arweave permaweb, inspired by roam.ar.io
%%% built on top of ureq HTTP client, and goldsky GQL gateway the roam@1.0 device provides

-module(dev_roam).
-export([info/1, info/3, query_permaweb/3, hello/3]).

info(_) ->
    #{
        <<"default">> => dev_message,
        handlers => #{
            <<"info">> => fun info/3,
            <<"query_permaweb">> => fun query_permaweb/3,
            <<"hello">> => fun hello/3
        }
    }.

%% @doc return roam device info
info(_Msg1, _Msg2, _Opts) ->
    InfoBody = #{
        <<"description">> => <<"Roam device for interacting with my_device_nif">>,
        <<"version">> => <<"1.0">>,
        <<"paths">> => #{
            <<"info">> => <<"Get device info">>,
            <<"query_permaweb">> => <<"Query the Arweave permaweb">>,
            <<"hello">> => <<"Simple hello world test">>
        }
    },
    {ok, #{<<"status">> => 200, <<"body">> => InfoBody}}.

%% @doc simple hello world endpoint
hello(_Msg1, _Msg2, _Opts) ->
    try
        Result = my_device_nif:hello(),
        {ok, #{<<"status">> => 200, <<"body">> => Result}}
    catch
        error:Error:Stack ->
            io:format("~nError: ~p~n", [Error]),
            io:format("Stack: ~p~n", [Stack]),
            {error, #{
                <<"status">> => 500,
                <<"body">> => #{
                    <<"error">> => <<"Failed to call hello">>,
                    <<"details">> => Error
                }
            }}
    end.

%% @doc query the Arweave permaweb
query_permaweb(_Msg1, _Msg2, _Opts) ->
    try
        Result = my_device_nif:query(),
        {ok, #{<<"status">> => 200, <<"body">> => Result}}
    catch
        error:Error:Stack ->
            io:format("~nError: ~p~n", [Error]),
            io:format("Stack: ~p~n", [Stack]),
            {error, #{
                <<"status">> => 500,
                <<"body">> => #{
                    <<"error">> => <<"Failed to query permaweb">>,
                    <<"details">> => Error
                }
            }}
    end.

%  example calls:
% curl -X GET http://localhost:8734/~roam@1.0/hello
% curl -X GET http://localhost:8734/~roam@1.0/query_permaweb

Then in rebar3.config search for and replace it by this code block

{cargo_opts, [
    {src_dir, "native/dev_snp_nif"},
    {src_dir, "native/my_device"}
]}.

Finally, go to hb_opts.erl and in the preloaded_devices list add our device

#{<<"name">> => <<"roam@1.0">>, <<"module">> => dev_roam}

As a final step, let’s compile the node by running rebar3 compile and running it by rebar3 shell

Go to the browser and try http://localhost:8734/~roam@1.0/query_permaweb it should return an Arweave txid, congrats, now you have a minimalistic and simple mini-Roam API!

The source code of this demo is available in the load_hb repository under the edge-roam-demo branch; https://github.com/loadnetwork/load_hb/tree/edge-roam-demo

Conclusion

In this first volume of our HyperBEAM devices tutorial guide, we have successfully built a basic API with a functionality similar to and inspired by roam.ar.io – in the following series we will explore how to develop in-hyperbeam UI for the roam device, more GQL customization and in-browser data rending.

In volume 3, we will explore how to host the hyperbeam node in the cloud!