Skip to content

RISCV VM testing of popcorn

Jump to bottom
Ho-Ren (Jack) Chuang edited this page Mar 20, 2022 · 15 revisions

Overview

This tutorial details the steps to test x86 , riscv testing of popcorn kernel.

--------------   --------------
|   Node 0   |   |   Node 1   |
|            |   |            |
|    x86     |   |   riscv    |
| 10.4.4.100 |   | 10.4.4.101 |
--------------   --------------
|  qem-x86   |   |  qemu-riscv|
-------------------------------
|  linux-x86       linux-riscv|
|                             |   ------------
|         host (x86)          |   | Gateway  |
|          10.4.4.10          |---| 10.4.4.1 |--- Internet
-------------------------------   ------------

Prepare for the setup

  • The machine should have;

    • At least 20 GB of storage
    • 8 GB of RAM (qemu scripts below initiate VMs with 4 GB each. You can change -m option in the scripts to use different size of memory)

  • Let's start with updating apt repository and install some utilities:

$ sudo apt-get update
$ sudo apt-get install build-essential libssl-dev libncursesw5-dev git curl bc

Download the patch files and support files

  • Download following files and place them in the same directory on the host.
Item Filename URL
Popcorn Compiler Patch popcorn-compiler.diff https://drive.google.com/file/d/1QsdTbc9iwBlIBuCpPQRnNoE3sC_bNd2j
RISCV kenel config riscv64-config https://drive.google.com/file/d/1EKxnq8Jo4oIup9loN7Lf67uo157QmzfQ
Popcorn Kernel patch popcorn-kernel.diff https://drive.google.com/file/d/1eIBaIqbhBD1GnGcPR0uS6omkYl7Bk0zg
SBI loader fw_jump.elf https://drive.google.com/file/d/1S6mVxHlWdrDMtaqftwcllx2xUcj4NvzB
Riscv disk image riscv_diskimg.ext4 https://drive.google.com/file/d/1di06EiMiv-xqXiIQbaHGilSDlkx8gy1Y
RISCV QEMU startup script start-riscv64 https://drive.google.com/file/d/1akpnpCJSucmLQcK_ua-0chKen37cJ-jr
RISCV NPB benchmark tests.tbz https://drive.google.com/file/d/1i5UlRrkcWLet2-60i0C9QbEa8ovkzib5
x86 kernel config config-x86_64-qemu https://drive.google.com/file/d/1ebhRofZvVp7wg0MQUH4xluzZlnxWQuNS
x86 VM disk image x86.img https://drive.google.com/open?id=0B7RfKPGm2YZsaURxTVh5ZTMyTk0
  • The folder must look like this
$ls
popcorn-compiler.diff  riscv64-config       popcorn-kernel.diff  fw_jump.elf riscv_diskimg.ext4  start-riscv64
tests.tbz              config-x86_64-qemu   x86.img

Compile the RISCV QEMU

$ git clone --recursive https://github.com/riscv/riscv-qemu.git
$ cd riscv-qemu
$ ./configure --target-list=riscv64-softmmu
$ make -j$(nproc)
$ make install

Compiling the RISCV GNU toolchain

  • Follow the instructions to build and install riscv gnu toolchain.This build takes time.
$ git clone --recursive https://github.com/riscv/riscv-gnu-toolchain.git
$ ./configure --prefix=/opt/riscv --with-target-cflags="-ffunction-sections -fdata-sections"
$ sudo mkdir -p /opt/riscv
$ sudo make linux LINUX_TUPLE="riscv64-linux-gnu"

      $ export PATH=/opt/riscv/bin:$PATH
  • Test if /opt/riscv/bin is present in $PATH
      $echo $PATH 
      /opt/riscv/bin/:/home/userr/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin

Compiling the popcorn compiler with riscv support

  • Instructions to build popcorn-compiler. This build takes around one hour.
$ git clone https://github.com/ssrg-vt/popcorn-compiler.git
$ cd popcorn-compiler
$ git checkout stable-3.7.1
$ patch -p1 < popocorn-compiler.diff
$ ./install_compiler.py --install-path=/path/to/install --install-all

Compiling NPB applications for testing

  • The tests.tbz contains NPB benchmarking suite.
  • To compile the applications change the popcorn compiler path to the newly built compiler.You must change the compiler path in each individual application. (Currently 5 works. Soon will be updated to all 10 tests)
  • This will compile the NPBB applications for both the arhitectures. (x86 , RISCV).

Compiling the popcorn x86 kernel

  • We are using the mainlin 5.2.21 kernel. Apply the patch popcorn-kernel.diff.
  • Copy the riscv64-config file downloaded above into linux source as .config file.
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
$ git checkout v5.2.21
$ patch -p1 < popcorn-kernel.diff
$ cp config-x86_64-qemu .config
$ make -j24

Compiling the popcorn RISCV kernel

  • We are using the mainlin 5.2.21 kernel. Apply the patch popcorn-kernel.diff.
  • Copy the riscv64-config file downloaded above into linux source as .config file.
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
$ git checkout v5.2.21
$ patch -p1 < popcorn-kernel.diff
$ cp riscv64-config .config
$ ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- make -j24
$ cp arch/riscv/boot/Image ../riscv64-Image

Booting the RISCV kernel

  • The riscv_diskimg.ext4 is the disk image file.
  • SBI loader fw_jump.elf is necessary. Riscv QEMU needs this to boot the kernel.
  • Once the kernel is copied from the kernel directory , we must be ready to boot the kernel.
riscv_support_dir$./start-riscv
  • The ontents of the qemu boot. Fro explanation sake. Since the boot is a little different from traditional architectures.
sudo 	qemu-system-riscv64 \
	-machine virt -cpu rv64 -m 4G -nographic -s \
	-drive id=root,if=none,media=disk,file=riscv_diskimg.ext4 \    ---> The root file system
	-smp 4 \
	-device virtio-blk-device,drive=root \
	-device virtio-net-device,netdev=net0,mac=52:54:00:12:35:02 \
	-netdev tap,id=net0 \
	-kernel fw_jump.elf \                                           --->  "The SBI loader file "   
	-device loader,file=riscv64-Image,addr=0x80200000 \             --->  "Make sure you copied this file"
	-object rng-random,filename=/dev/urandom,id=rng -device virtio-rng-device,rng=rng \
	-append 'root=/dev/vda rw highres=off  console=ttyS0 earlycon=sbi '

Setting up messaging layer

Now execute the following command in both the machines:

In the riscv VM : root@riscv:~# modprobe msg_socket

In the x86 VM : user@x86 # modprobe msg_socket

  • If all works fine you are set with environment.

Testing the x86 <-> RISCV popcorn system

Warning : You must ensure that the applications are present in identical locations in both machines.

After booting the x86 and riscv QEMU VMs, copy the npb applications into the VMs.

  • Setup the messaging layer -- Create a folder /etc/popcorn in both VMs. -- Create a file inside /etc/popcorn/nodes in both VMs. -- The nodes file must contain the " same order " of IP addresses. In the above diagram both the VMs must contain the files in the below fashion.
$cat etc/popcorn/nodes
10.4.4.100
10.4.4.101

**IMPORTANT DISCLAIMER : In both the machines all three binaries need to be present. Without this you cannot get the binaries to work. ** In x86 machine : app_x86-64 app_riscv . Copy the app_x86-64 as app. **In riscv machine : app_riscv app_x86-64 . Copy the app_riscv as app. ** If you dont do this popcorn gods will haunt you. Booo !!!!!

  • Copy the executables into the VMs
$scp ep_riscv ep_x86-64 [email protected]
$scp ep_riscv ep_x86-64 [email protected]
  • In the x86 VM
x86 $ cp ep_x86-64 ep
  • In the riscv VM
riscv $ cp ep_riscv ep
  • Now you can start testing the ep by executing the application in the 0th Node. (in the x86 VM in this example)

Please continue to next page for trying popcorn on real Hardware.

Clone this wiki locally