Bootstrap a CODE development environment

Introduction

The Olympe platform can be used and extended through CODE, which acts both as development framework and Olympe DRAW extensibility platform. This tutorial introduces how to start a CODE project.

Prerequisites

This tutorial assumes that the following subjects are known to the reader

  • Node.js development, in particular the Node Package Manager (npm)
    • The needed Node.js version for these tutorials (and for developping Olympe CODE projects in general) is the v14. We discourage (for the time being) the v15 as it brings some incompatibility on which our build tool depends.
  • A basic understanding of the Olympe platform

This tutorial also assumes that

  • a full Olympe backend (database + orchestrator) is running on the machine. As they are different ways of provisioning the backend environment (depending on whether you are a direct Olympe collaborator or not) the instructions are not listed here.
  • an access to the main olympe git repository - typically thru the Olympe VPN.

Outcome

At the end of this tutorial, you will have a running instance of DRAW and a setup ready for a CODE project:

outcome

Overview of the steps

Bootstraping an Olympe CODE project involves the following steps:

  • Describing the project as a Node.js package by redacting a package.json descriptor.
  • Describing the project as one or more Olympe modules, by redacting one or more module.json descriptors.
  • Indicating in a gulpfile.js file to the Olympe build tool, gulp, where to find other modules, to be able to use them as dependencies.
  • Embedding the Olympe IDE, DRAW, inside the projet

Once these steps are completed, the project is ready to received its own code contributions.

CODE project as Node.js package

Writing the package.json file

NPM is used as package manager to retrieve the dependencies, in particular the CODE framework, as well as the DRAW package. Similarly to other NPM projects, the first steps thus consists in creating a directory for the new project, in our case tutorials, and by redacting a package.json file into it, typically looking like the one displayed below:

{
  "name": "tutorials",
  "version": "0.0.1",
  "description": "Custom functional bricks tutorial",
  "dependencies": {
    "olympe-composer": "git+ssh://git@gitlab.caas.olympe.cloud:30001/olympe/composer.git#master"
  },
  "author": "Olympe S.A. <dev.olympe.ch>",
  "license": "SEE LICENCE IN LICENCE.txt"
}

Note the reference to the package olympe-composer (currently hosted on Olympe private git repository exclusively - which requires VPN connectivity if outside of Olympe network).

Performing the NPM install

Once the package.json descriptor has been written, typing npm install in a console installs the dependencies of our projects, i.e. olympe-composer, but also its dependencies. At the end of the install process, it is possible to consult the dependency tree by typing npm list -depth=2. This returns the following:

npm list -depth=2
tutorials@0.0.1 /Users/user/olympe/tutorials
└─┬ olympe-composer@1.8.2 (git+ssh://git@gitlab.caas.olympe.cloud:30001/olympe/composer.git#64d73d573550d571589269781ee1422551c1cb77)
 └─┬ olympe-framework@7.6.1 (git+ssh://git@gitlab.caas.olympe.cloud:30001/olympe/framework.git#504a8e02d80f07974f9e99d4649b45c1ef12b7d6)
   ├── eslint@7.12.1
   ├── ink-docstrap@1.3.2
   ├── jsdoc@3.6.6
   └── olympe-gulp-tasks@3.3.5 (git+ssh://git@gitlab.caas.olympe.cloud:30001/tools/olympe-gulp-tasks.git#843bcfdd2df627c05df8d1a905f3bd1c1f1fc66f)

Observe that olympe-composer has a dependency to olympe-framework which has itself several other dependencies:

  • olympe-gulp-tasks (and, for now, gulp) : for installing the Olympe development tools (which will be the subject of another tutorial)
  • jsdoc and ink-docstrap : for generating the documentation
  • eslint : for JavaScript linting

Describing project as one or more Olympe modules

The second step consists in describing one or more Olympe modules. They usually sit in a modules folder, as we will see soon.

Before writing any module, we should first add a "root" Olympe module by providing a module.json. We start with the following, nearly simplest description:

{
  "name": "tutorials",
  "version": "0.0.1",
  "dependencies": {
    "olympe-composer.targets": "*"
  },
  "targets": {
    "html": {
      "dependencies": {
        "olympe-composer": "*",
        "olympe.native.html.target.default": "*"
      }
    }
  }
}

With this new file, the directory structure should look as follow:

tutorial
├── package-lock.json
├── package.json
├── module.json
├── snapshot.sh
└── node_modules/

The snapshot.sh is a generated script that facilitate the exportation process of Olympe DRAW projects. We reserve the explanation of its usage for another tutorial.

Dependencies section

Exactly as in npm's package.json files, the dependencies section is used to designate which other Olympe modules (and their own Olympe dependencies) should be loaded alongside, and, actually, before, the current one. For this tutorial, we add the two dependencies required to import DRAW, olympe-composer and olympe-composer.targets.

Targets section

The root module needs to specifiy targets. A target describes a specific way to build a project. In our particular case, we simply want to build our project for a use in a browser. Thus, we add an html target.

Within each target, another set of dependencies can be listed, this time target-specific. Here, we add a third dependency through which the Olympe VM in its web-browser version will be loaded, olympe.native.html.target.default.

We will also fill the target dependency section by adding our created modules throughout this tutorial.

Note that would our project be targeting a deployment in Cordova instead of in HTML, we would describe another target cordova alongside or instead html.

Other sections

The module.json accepts many more configuration items; some of those will be described later on in this training, while others are described in the Reference Guide.

Providing the gulpfile.js

Olympe development and build tool chain is based on gulp, a popular Node.js package. We use Gulp as Make is typically used to build software in the Unix world.

Similarly to Make that requires a Makefile to be added in a project, Gulp requires a gulpfile.js file in the root directory of the project.

This gulpfile.js file is rather simple, as shown below:

require('olympe-gulp-tasks')({
    pathsToModules: [
        'node_modules/olympe-framework',
        'node_modules/olympe-composer',
        'modules/'
    ],
    defaultPassword: 'password1'
});

The file essentially consists in requiring (i.e importing) the "olympe-gulp-tasks" library, and then immediately calling it. The part of interest is the JSON structure given as parameter:

  • In the pathToModules array, paths where to spot Olympe modules (as described above) can be listed. These paths can be compared to a classpath in the Java world. We provide two paths toward the olympe-framework and the olympe-composer packages, which have been retrieved by npm.
  • The defaultPassword parameter is used to prescribe an initial admin password when resetting the olympe database.

With the addition of gulpfile.js, the directory structure should now look as follows:

tutorial
├── package-lock.json
├── package.json
├── gulpfile.js
├── module.json
├── snapshot.sh
└── node_modules/

Testing the new project with Olympe config application

Once the three configuration files package.json, module.json and gulpfile.js are ready, it is possible to test the setup by calling gulp. The following output is expected:

gulp_out

Notice the three last lines: gulp tells that it is serving target html, and that a server has started on port 8888 on the localhost. This means that everything is ready for the test app to be tested within the browser. It is now time to test it for real by visiting localhost:8888/?hc.app=olympe.config. Note that with hc.app=, we provide the Olympe VM with a parameter specifying which application should be loaded.

The following screen should be displayed:

olympe_config

which shows the Olympe VM configuration.

Adding code to the project

We will add some JavaScript code to the project and check that this code is included when the browser site is visited.

Creating the modules and helloworld directories

The convention is to write different functionalities into separate Olympe modules. These modules should sit inside a modules folder.

For this tutorial, we are interested in writing a "Hello World" module. To this end, we create a modules folder and create in it a folder named helloworld.

The directory structure should look as follows:

tutorial
├── modules
│   └── helloworld/
├── package-lock.json
├── package.json
├── gulpfile.js
├── module.json
├── snapshot.sh
└── node_modules/

Creating the source files

JavaScript code should be placed in a directory src inside the helloword directory.

As a second step, we need to create a JavaScript namespace into which the code will be stored. The convention there is to create an _ns.js file that solely creates this namespace:

/**
 *
 * @namespace
 */
const helloworld = {};

The third step consists in creating the source file containing the actual, real code. Within this tutorial, we only want to create a JavaScript ES6 class with a static method logging "Hello olympic world" on the console. We thus create an HelloWorld.js file in src and we place the following content in it

helloworld.HelloWorld = class HelloWorld {
    static sayHello() {
        console.log('Hello olympic world');
    }
};

helloworld.HelloWorld.sayHello();

Notice that we not only define the class, we also call the static method upfront.

At this point, our project directory should look like this:

tutorial
├── modules
│   └── helloworld
│       └── src
│           ├── HelloWorld.js
│           └── _ns.js
├── package-lock.json
├── package.json
├── gulpfile.js
├── module.json
├── snapshot.sh
└── node_modules/

Creating the module.json for the helloworld module

Since helloworld is an Olympe module, it needs to have its module.json:

{
  "name": "helloworld",
  "version": "0.0.1",
  "sources": [
    "src/_ns.js",
    "src/HelloWorld.js"
  ]
}

The project structure should look as follows:

tutorial
├── modules
│   └── helloworld
│       ├── module.json
│       └── src
│           ├── HelloWorld.js
│           └── _ns.js
├── package-lock.json
├── package.json
├── gulpfile.js
├── module.json
├── snapshot.sh
└── node_modules/

Registering the source files

We should first add the modules folder to the gulpfile.js dependencies, which should now look as follows:

require('olympe-gulp-tasks')({
    pathsToModules: [
        'node_modules/olympe-framework',
        'node_modules/olympe-composer',
        'modules/'
    ],
    defaultPassword: 'password1'
});

We should then inform Olympe to register the helloworld module by modifying the root module.json. The dependencies of the html target should include helloworld:

{
  "name": "tutorials",
  "version": "0.0.1",
  "dependencies": {
    "olympe-composer.targets": "*"
  },
  "targets": {
    "html": {
      "dependencies": {
        "olympe-composer": "*",
        "olympe.native.html.target.default": "*",
        "helloworld": "*"
      }
    }
  }
}

Once these changes have been saved, it is time to restart Gulp and visit localhost:8888. We expect to see our hello on the console.

Embedding DRAW in our project

Now that we know how to add code to our project, let us see how to embed DRAW within our project. For that, we need to initialize a datacloud on an Olympe Orchestrator since DRAW, in contrast to the "config" app, requires to be connected to a datacloud to function.

Initializing the datacloud

First of all, check that a fully functional Olympe orchestrator is running locally. The easiest way to do so consists of consulting this address localhost:8080/status/, which should display the following information:

OOStatus

If the page doesn't load or if there are any red or missing green labels, there is something wrong with your backend configuration. See at the end of this tutorial for a list of quick solutions to try.

If the status page displays correctly, you are ready to start the datacloud initialization. But before proceeding to it, think twice ! The current content stored in the database will be lost without recovery ways.

If you are sure of what you are about to do, type gulp rst in your console.

A slightly different process starts, and at some point you should start seeing blue lines corresponding to what the orchestrator responds:

resetDB

Once the re-initialization is over, it is time to access your local DRAW by visiting localhost:8888

The login screen of DRAW should be displayed:

draw_welcome

Caution! Contrarily to DRAW "deployed in the wild", here we do not prescribe a default domain as parameter. Thus, the login must be admin@bricks.olympe.ch. The password is what you have written in your gulpfile.js (password1 if unchanged).

draw_logged_in

Frequent errors to avoid

Src file order

Try switching the order of the source files in modules/helloworld/module.json :

"sources": [
  "src/HelloWorld.js",
  "src/_ns.js"
]

then restart gulp, and visit localhost:8888 after having cleansed the browser cache. You should now see an error: hence the class HelloWorld cannot be added to the namespace tutorial if the later has not been yet. This shows that ordering within a module is left to the responsibility of the programmer.

"Cannot GET /" message in Chrome

This means that the "bootstrap" index.html document that loads the JavaScript code has not been copied into the build. This is generally due:

  • to the fact that the html native implementation of the Olympe VM has not been put as dependency, namely the root module does not point (directly or indirectly) to `olympe.native.html.target.default
  • to the fact that the gulp watcher is dead.

Back-end not operational

If you have neo4j installed "natively" (i.e. not inside a docker container - the default configuration for developers), make sure neo4j is running by typing neo4j status and if it is not running, type neo4j start. Note that in most cases, neo4j should be restarted after a computer reboot or session closing.

The Olympe orchestrator generally runs as a docker container. Test if you have an olympeO helper installed on your machine by typing which olympeO. This helper, in its most recent version, takes two console arguments:

  • the version to run
  • the name of the container to run

To identify the version (if not provided by the trainer) consult the file node_modules/olympe-composer/package.json and look into the build section. The container name is generally orchestrator. All in all, this typically gives us the following command : olympeO 5.5.2 orchestrator.

Some additional notes

Gulp watcher

The gulp tool is able to some extent to watch files being edited and refresh them in the build, in order not to have to relaunch gulp each time. The watcher worker as long as existing files are modified, but generally fails when module.json files are edited. Improving the watcher is in our plans.

*.js wildcard

It is possible to add source files to a module using a *.js wildcard. Note, however, that in this case the ordering will be done alphabetically. This can be problematic, in particular when abstract class are used. Hence, abstract classes must always been defined first, followed by the classes extending them.

Directory exclusion in IDEs

Due to the amount of code available in the full node_modules directory, IDEs frequently ignore this directory when performing indexing for code completion. This is annoying in our case because we typically would like olympe-composer and olympe-framework to be indexed.

To circumvent this problem, a simple solution consists of creating two symbolic links, either manually, or using following bash script:

#!/bin/bash

##### PLACE IN THIS ARRAY THE NAME OF THE DEPENDENCY PROJECTS - a directory with the name provided there is expected
##### in the node_modules directory. If this directory exists, a symbolic link is created to it.
array=("olympe-framework" "olympe-composer")

for i in "${array[@]}"; do
   FILE="$i"
   echo "PROCESSING item: $i"
   DOWNFILE="node_modules/$i"
  if [ -d $DOWNFILE ]; then
    echo "-->$i detected in node_modules directory! Linking to it..."
    ln -s $DOWNFILE $FILE
  else
    echo "$i not detected, ignoring"
  fi
done

Next steps after this tutorial

This tutorial naturally leads to the following next tutorial: How to add a custom brick to DRAW

← CODE APIInitialize the versioning system →