code-executor

A library to execute code against test cases in various languages and obtain relevant results.
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

• About the Project
  • Built With
• Getting Started
  • Prerequisites
  • Installation
• Usage
• Roadmap
• Contributing
• License
• Contributors

About The Project

code-executor is a Node.js library built for the purpose of executing code in an isolated container against user defined test cases.

code-executor allows you to run arbitrary code in scalable, secure containers and returns metrics for each test case, such as the time taken, errors occured (if any), and the status (pass/fail).

This library uses a master-worker structure to run programs, which makes it scalable across servers as long as they use the same Redis instance. Visit the usage section to learn more.

Built With

• Typescript
• Node
• Bull
• Redis

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

• npm
• redis
• docker

Installation

You can install code-executor using npm.

npm install code-executor --save

Usage

code-executor exports a default CodeExecutor class and a Worker class. The following documentation describes these classes in brief along with examples of how to use them in your project. For the purpose of documentation, we consider a simple Competitive Coding website backend which uses code-executor to run programs submitted by the users.

TL;DR

1. In your backend, you can create a CodeExecutor object, which has a runCode method. This returns a promise which resolves when the program passed to it has finished executing. You can check out this example to find out how to use a CodeExecutor object.

2. Now, you can create a worker using the code-executor CLI. For that, you need to first install code-executor globally.

npm i -g code-executor

3. You can spawn a worker using the spawn-worker command (sw in short), and optionally pass the redis URL (default: redis://127.0.0.1:6379), the name of the queue (default: myExecutor), and languages to build (default: all).

code-executor sw

4. If you used some other redis URL or queue name in step 1, make sure you pass those to the CLI.

code-executor sw --redis <redis-url> --queue <queue-name>

Brief Description

• As mentioned before, code-executor is built using the master-worker strategy. Therefore, there is a master which is responsible for assigning work to a set of workers. These workers respond to the master on completing their tasks. In the case of the Competitive Coding website, the backend of the website is the master, whereas you can run separate Node.js scripts for spawning workers (a CLI is coming soon!).

• The backend assigns jobs to the workers through a queue, which is stored in the Redis instance and managed by the bull library. This abstracts process synchronization and ensures that no two workers are working on the same job. Once the workers finish executing the code, they respond to the master, and the backend can respond with success or failure, and other details returned by the worker.

• code-executor is built in a way that it allows multiple masters and workers to run parallely while internally handling synchronization. This allows you to have workers on different instances of a cluster as long as they use the same Redis instance. You can also scale your backend to have multiple masters.

CodeExecutor

The default export from the code-executor library is the CodeExecutor class. This is the master class, which can be run on the backend of your website. The purpose of this class is to assign jobs to the workers (through a queue as mentioned before, though this is abstracted so you need not worry about it). You can create an object of CodeExecutor and keep adding jobs to it as you keep getting submissions on your website.

You can create a CodeExecutor object in the following manner. You must pass the name of your queue and the redis instance you want the queue to be placed on. These details will later be used by the worker to identify jobs and run them.

Note: job refers to the task of execution of a single program which is passed using a queue to the workers.

import { CodeExecutor } from 'code-executor';
const codeExecutor = new CodeExecutor('myExecutor', 'redis://127.0.0.1:6379');

OR

const { CodeExecutor } = require('code-executor');
const codeExecutor = new CodeExecutor('myExecutor', 'redis://127.0.0.1:6379');

Now, say you received an a submission from a user, and you want to run their code against a set of test cases. You can use the following code inside your route handler.

async function routeHandler(code, language) {
  const input = {
    language: language,
    code: code,
    testCases: [
      {
        input: '',
        output: 'hello\n',
      },
    ],
    timeout: 2,
  };

  // We re-use the codeExecutor object that was created before.
  const results = await codeExecutor.runCode(input);

  console.log(results);
  return results;
}

That is all! codeExecutor.runCode() returns a promise which resolves when your code has been executed successfully by any of the workers. You can also stop a master from interacting with a queue using codeExecutor.stop().

codeExecutor.stop();

Worker

By now, we know how to use the CodeExecutor class to assign jobs to workers. Now, we see how to use the Worker class to create workers that will run your code.

The easiest way to spawn workers is by using the CLI provided by the library. For this, you need to globally install code-executor using:

npm install -g code-executor

Once you install it globally, you can run code-executor -h to see the options available to you.

$ code-executor -h               
Usage: code-executor [options] [command]

Options:
  -r, --redis <redis>     URL for the redis instance
  -q, --queue <queue>     name of the redis queue
  -l, --langs <langs...>  list of languages to build
  -h, --help              display help for command

Commands:
  spawn-worker|sw         spawn worker process
  help [command]          display help for command

To spawn a worker, run

code-executor sw

Here's another small example to spawn a worker supporting Python and Bash.

code-executor sw -l Python Bash

To spawn multiple workers, you can run the previous command several times.

Note: code-executor does not ensure that a worker will not take up a job if it doesn't support that language. Therefore, each worker you start should support all the languages you need to build. For example, if you need Python, Javascript and Bash, all the workers should be started with these languages. Since the workers build Docker containers, after 1 worker builds all its containers, the remaining workers will use the Docker cache and thus will be able to build much faster.

You can also use the API to build you own script to spawn workers. You can use the CLI in almost every use case, however, the API provides a lot more customization, as you can see below.

import { Worker } from 'code-executor';
const worker = new Worker('myExecutor', 'redis://127.0.0.1:6379');

OR

const { Worker } = require('code-executor');
const worker = new Worker('myExecutor', 'redis://127.0.0.1:6379');

You can create a new Worker object and listen with the same name and redis string you passed to the master class. There is another optional parameter called options, which is an object that may consist of the following parameters:

• folderPath, string: Will be discussed later.
• memory, number: The amount of memory assigned to every Docker container spawned by this worker, in MB. The default is 0 (no limit).
• CPUs, number: The number of CPUs assigned to every Docker container spawned by this worker. The default is 0.5.

For example, you could pass these values to the constructor.

const worker = new Worker('myExecutor', 'redis://127.0.0.1:6379', {
  folderPath: '/tmp/myFolder',
  memory: 100,
  CPUs: 1,
});

An object of the Worker class has the following important functions:

• build(langs)
• start()
• pause()
• resume()

worker.build(langs)

worker.build() is responsible for building docker images on the system. You can pass a list of languages to the function so that it builds images for just the specified languages.

• You can also call worker.build() without an argument to build all languages supported by code-executor.

async function build() {
  await worker.build(['Python', 'Bash']);
  console.log('Python and Bash containers built successfully!');
}

worker.start()

On running worker.start(), the current worker starts listening on the redis queue. After this function is executed, whenever there is a new job on the queue that has not been taken by another worker (if any), this worker will take up the job and run the code.

worker.start();

worker.pause() and worker.resume()

You can pause the execution of a worker with the help of the worker.pause() function. Executing the worker.resume() function resumes processing jobs from the queue.

worker.pause();

worker.resume();

The worker performs the following steps in order to execute a program:

• First, the worker builds all the images on the server. If the image is already present, it uses the cache.
• The worker listens on the queue for new jobs.
• Whenever it gets a new job, it calls a Runner object to run the code.
• The Runner object creates a folder with a random name in /tmp/code-exec. This can be changed with the help of the optional folderPath parameter passed to the constructor of the Worker class.
• This newly-created folder is mounted inside a docker container to execute the code.
• When execution is completed, or the time limit has exceeded, the worker responds to the master.

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'feat: Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

You are requested to follow the contribution guidelines specified in CONTRIBUTING.md while contributing to the project 😄.

License

Distributed under the MIT License. See LICENSE for more information.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Rohan Mukherjee 💻 📖

ashikka 💻 📖

Rahil Kabani 💻 📖

This project follows the all-contributors specification. Contributions of any kind welcome!