Allow JIT compilation for system emulation #521

vacantron · 2024-12-17T08:16:54Z

This patch makes rv32emu support the just-in-time (JIT) compilation in system simulation. To achieve this goal, a "satp (supervisor address translation and protection)" field has been introduced to the block structure in JIT mode to ensure the block cache is replaced correctly.

The file src/system.c has been updated for the easier reuse of functions, and the MOP fusion and T2C are disabled temporarily. To boot the Linux Kernel, try the following commands:

$ make ENABLE_SYSTEM=1 ENABLE_MOP_FUSION=0 ENABLE_JIT=1 ENABLE_T2C=0
$ ./build/rv32emu -k <image> -i <rootfs> -b <dtb>

Summary by Bito

This PR implements JIT compilation support for RISC-V system emulation by adding SATP tracking and MMU-aware instruction handling. The changes include refactoring MMIO operations into separate helper functions, implementing comprehensive instruction handlers with proper memory access handling, and enhancing the block caching system with improved error handling for Linux kernel boot compatibility.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 5

jserv

Benchmarks

Benchmark suite	Current: `4ef61b8`	Previous: `7a032db`	Ratio
`Dhrystone`	`1284` Average DMIPS over 10 runs	`1340` Average DMIPS over 10 runs	`1.04`
`Coremark`	`972.508` Average iterations/sec over 10 runs	`947.06` Average iterations/sec over 10 runs	`0.97`

This comment was automatically generated by workflow using github-action-benchmark.

.gitignore

src/emulate.c

src/riscv_private.h

src/emulate.c

ChinYikMing · 2024-12-18T12:54:51Z

Testing Report:

x86-64 machine:
Issue: kernel panic.

Kernel message Logs:

[    0.000000] Linux version 6.1.119 (mirv32@node11) (riscv32-buildroot-linux-gnu-gcc.br_real (Buildroot 2024.08.3) 14.2.0, GNU ld (GNU Binutils) 2.42) #1 Sat Dec 14 22:54:23 CST 2024
[    0.00000] Machine model: rv32emu
[   .00000] earlycon: ns16550 at MMIO 0xf4000000 (options '')
[   .00000] printk: bootcons1655] enabled
[    0.000000] Zone ranges:
[    0.00000]   Normal   [mem 0x0000000000000000-0x000000001fffffff]
[   .00000] Movable zone start for each node
[   .00000] Early memory node ranges
[   .00000]   node   0: [mem 0x000000000000000-0x000000001ffffff]
[   .00000] Initmem setup node 0 [mem 0x000000000000000-0x000000001ffffff]
[   .00000] SBI specification v0.3 detected
[   .00000] SBI implementation 99 Version
[   .00000] SBI TIME extension detected
[   .00000] SBI SRST extension detected
[    0.000000] riscv: base ISA extensions aim
[    0.00000] riscv: ELF capabilities aim
[   .00000] Bui zonelists, mobility grouping on.  Total13004
[   .00000] Kernel command line: earlycon console=ttyS0
[   .000000] Dentry cache hash table entries: 65536 (order: 6, 26214
[    0.00000] Inode-cache hash table en3276 (or13107 bytes, linear)
[   .00000] mem auto-init: stack:off, heap alloc:off, heap free:off
[   .00000] 2428K avail323K kernel c33K rwd83K rod16K i131742K reserveK cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[   .000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.00000] riscv-in3 local interrupts mapped
[    0.000000] plic: interrupt-controller@0: mapped 31 interrupts with 1 handlers for 1 contexts.
[    0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.00000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff maxefdb196d, max_idle_ns: 440795204367 ns
[   .00000] sched_clo6 bits6MHz, resolut1n219902325555ns
[   .00068] Console: colour dummy device 80x25
[    0.00093] Calibrating delay loop (skipped), value calculated using timer frequency..
[   .00097] 00 BogoMI26000)
[   .00119] pid_max: de3276 mini30
[   .00203] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.00224] Mountpoint-cache hash table ent102 (orde409 bytes, linear)
[    0.00638] ASID allocator using 9 bits (512 entries)
[    0.008134] devtmpfs: initialized
[   .01275] clocksource: jiffies: mask: 0xffffffff max_fffffff, max_idle_ns: 7645041785100000 ns
[    0.013002] futex hash table entr25 (order: -1, 3072 bytes, l
[    0.015155] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.018778] platform soc@F0000000: Fixed dependency cycle(s) with /soc@F0000000/interrupt-controller@0
[    0.038375] clocksource: Switched to clocksource riscv_clocksource
[   .05459] sysfs: cannot create duplicate filename '/dev/char/4:'
[   .054745] CPU: 0 PID: 1 Comm: swapper Not tainted 6.1.119 #1
[    0.054941] Hardware name: rv32emu (DT)
[   .05504] Call Trace:
[   .05512] [<c0003ad>] dump_backtrace+03
[   .05531] [<c03228d>] show05
[   .05543] [<c032576>] dump_sta04
[   .05556] [<c032579>] dump02
[   .05568] [<c010e43>] sysfs_wa09
[   .05580] [<c010e76>] sysfs_do_create_l0c
[   .05594] [<c010e7a>] sysfs_creat04
[   .05606] [<c01fd13>] dev067
[   .05619] [<c01fd36>] device_re03
[   .05631] [<c01d117>] tty_register_devi021
[   .05645] [<c01d123>] tty_register_02
[   .05658] [<c01d13b>] tty_register01d
[   .05671] [<c033d8b>] v018
[   .05683] [<c033cd4>] t016
[   .05695] [<c033e6c>] chr_d010
[   .05708] [<c000110>] do_one_i026
[   .05720] [<c032c06>] kernel_init_f021
[   .05733] [<c0325a3>] kern011
[   .05746] [<c00023d>] ret_from_exc01
[    0.062338] Kernel panic - not syncing: Couldn't register console driver
[   .062494] CPU: 0 PID: 1 Comm: swapper Not tainted 6.1.119 #1
[    0.062688] Hardware name: rv32emu (DT)
[   .06279] Call Trace:
[   .06287] [<c0003ad>] dump_backtrace+03
[   .06306] [<c03228d>] show05
[   .06318] [<c032576>] dump_sta04
[   .06331] [<c032579>] dump02
[   .06343] [<c0322b8>] f
[   .06355] [<c033d7b>] v018
[   .06367] [<c033cd4>] t016
[   .06379] [<c033e6c>] chr_d010
[   .06391] [<c000110>] do_one_i026
[   .06404] [<c032c06>] kernel_init_f021
[   .06417] [<c0325a3>] kern011
[   .06429] [<c00023d>] ret_from_exc01
[   .06445] ---[ end Kernel panic - not syncing: Couldn't register console driver ]---

Aarch64 machine:
Issue: Coremark seems running forever, perhaps into a infinite loop. I have run about 30 mins but still no any output.

jserv

Rebase the latest master branch and resolve the conflicts.

bito-code-review · 2025-02-09T07:49:39Z

Code Review Agent Run #365781

Actionable Suggestions - 16

src/rv32_jit.c - 2
- Consider extracting duplicated MMIO handling logic · Line 282-301
- Missing error check for branch history parsing · Line 1036-1037
Makefile - 1
- Consider using more portable LLVM config option · Line 257-257
src/jit.c - 3
- Consider more descriptive assertion condition · Line 899-899
- Consider standard if condition syntax · Line 2260-2262
- Validate MAX_JUMPS constant before usage · Line 2288-2288
src/system.c - 2
- Consider retaining system functionality check · Line 10-10
- Consider visibility of vblk interrupt function · Line 22-22
src/rv32_template.c - 8
- Missing null check for cache_get return · Line 2426-2429
- Consider adding null check for next · Line 384-386
- Consider adding NULL check for cache_get · Line 421-423
- Potential null pointer dereference issue · Line 2426-2434
- Potential null pointer dereference issue · Line 2427-2428
- Potential null pointer dereference issue · Line 2428-2434
- Possible NULL pointer dereference issue · Line 2401-2402
- Consider adding null check for next · Line 2429-2434

Additional Suggestions - 10

src/utils.c - 1
- Consider using typedef for key type · Line 223-227
src/rv32_jit.c - 2
- Consider optimizing register reset calls · Line 430-431
- Consider extracting duplicated branch handling logic · Line 139-154
src/jit.c - 7
- Consider using assert(false) for unreachable code · Line 853-853
- Consider removing misleading UNUSED attribute · Line 401-402
- Consider more descriptive variable naming · Line 618-624
- Consider consolidating buffer overflow checks · Line 307-310
- Consider simplifying nested conditional structure · Line 2187-2192
- Consider more descriptive assertion message · Line 2284-2284
- Consider consolidating redundant satp checks · Line 2255-2264

Review Details

Files reviewed - 12 · Commit Range: 3d43867..d0ee6c4
- Makefile
- src/decode.h
- src/emulate.c
- src/jit.c
- src/jit.h
- src/riscv_private.h
- src/rv32_jit.c
- src/rv32_template.c
- src/system.c
- src/system.h
- src/utils.c
- src/utils.h
Files skipped - 1
- .gitignore - Reason: Filter setting
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- Fb Infer (Static Code Analysis) - ✖︎ Failed

AI Code Review powered by

bito-code-review · 2025-02-09T08:09:26Z

Changelist by Bito

This pull request implements the following key changes.

Key Change	Files Impacted
Feature Improvement - JIT Compilation System Support Enhancement	- `rv32_jit.c` - Added extensive instruction handlers for RISC-V base, M-extension and C-extension instructions with MMU support - `rv32_template.c` - Enhanced cache and branch history handling with SATP tracking for MMU support - `system.c` - Refactored system emulation code and made functions more reusable - `system.h` - Added new header file with MMIO and MMU support declarations - `utils.c` - Enhanced set operations with improved hash key handling - `utils.h` - Added support for 64-bit hash keys and SATP-aware block caching

src/rv32_jit.c

Makefile

src/jit.c

src/system.c

src/rv32_template.c

bito-code-review · 2025-02-11T16:52:22Z

Code Review Agent Run #708454

Actionable Suggestions - 1

src/utils.c - 1
- Consider adding NULL check for set · Line 190-190

Additional Suggestions - 1

src/utils.h - 1
- Consider more descriptive type name · Line 159-160

Review Details

Files reviewed - 3 · Commit Range: c2a1ec4..08bbf92
- Makefile
- src/utils.c
- src/utils.h
Files skipped - 0
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- Fb Infer (Static Code Analysis) - ✖︎ Failed

AI Code Review powered by

src/utils.c

bito-code-review · 2025-02-11T17:21:19Z

Code Review Agent Run #c0fb25

Actionable Suggestions - 10

src/system.h - 1
- Consider refactoring complex MMIO_OP macro · Line 31-65
src/jit.c - 3
- Consider using assert(false) for unreachable code · Line 2284-2284
- Consider handling set_add failure case · Line 2219-2220
- Consider using assert(0) instead of assert(NULL) · Line 2284-2284
src/utils.h - 1
- Consider adding size check for rv_hash_key_t · Line 162-162
src/rv32_template.c - 3
- Potential null pointer dereference issue · Line 2428-2429
- Potential NULL pointer dereference issue · Line 2427-2428
- Potential null pointer dereference issue · Line 2428-2429
src/rv32_jit.c - 2
- Consider consolidating branch handling logic · Line 35-54
- Consider improving floating point instruction handling · Line 1057-1060

Additional Suggestions - 10

src/system.h - 1
- Consider removing duplicate function declaration · Line 166-166
src/jit.c - 6
- Consider consolidating duplicate flush conditions · Line 307-310
- Consider simplifying IIF macro condition structure · Line 2229-2230
- Consider consolidating duplicate jump handling code · Line 1115-1121
- Consider removing misleading UNUSED attribute · Line 401-402
- Consider more descriptive jump location names · Line 1325-1345
- Consider removing redundant unreachable statement · Line 1389-1390
src/rv32_template.c - 1
- Consider simplifying nested conditional blocks · Line 171-182
src/utils.c - 1
- Consider moving bounds check before loop · Line 195-197
src/rv32_jit.c - 1
- Consider optimizing register reset timing · Line 430-431

Review Details

Files reviewed - 12 · Commit Range: 819000a..216be73
- Makefile
- src/decode.h
- src/emulate.c
- src/jit.c
- src/jit.h
- src/riscv_private.h
- src/rv32_jit.c
- src/rv32_template.c
- src/system.c
- src/system.h
- src/utils.c
- src/utils.h
Files skipped - 1
- .gitignore - Reason: Filter setting
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- Fb Infer (Static Code Analysis) - ✖︎ Failed

AI Code Review powered by

src/system.h

src/jit.c

src/utils.h

src/rv32_template.c

src/rv32_jit.c

jserv

Rebase the latest master branch and resolve the conflicts.

ChinYikMing · 2025-02-13T20:03:49Z

Run JIT enabled system emulation verification in CI?

The file is generated by "tools/gen-jit-template.py" To bring up the Linux Kernel to just-in-time (JIT) compilation, we need to update the memory-related operation with the memory management unit. However, the current "src/rv32_jit.c" was generated by the template. That template reduced the rework for the repeated statements, but also reduced the flexibility and the intuitiveness for bring up the new feature. In this commit, we deprecate that template and just use a regular file for the flexibility.

This commit introduces "satp" field to the block structure in JIT mode to ensure the block cache is replaced correctly. The MOP fusion and T2C are disabled temporarily. Use the following commands to boot the Linux Kernel: $ make ENABLE_SYSTEM=1 ENABLE_MOP_FUSION=0 ENABLE_JIT=1 ENABLE_T2C=0 $ ./build/rv32emu -k <image> -i <rootfs> -b <dtb>

vacantron · 2025-02-14T13:04:32Z

Run JIT enabled system emulation verification in CI?

It had been added in 8d46115 .

bito-code-review · 2025-02-14T13:20:58Z

Code Review Agent Run #77d5d2

Actionable Suggestions - 4

src/rv32_template.c - 3
- Consider adding NULL check for next · Line 2428-2429
- Add NULL check before dereferencing pointer · Line 2282-2283
- Possible null pointer dereference issue · Line 2428-2429
src/jit.c - 1
- Consider adding null check for block · Line 2272-2272

Additional Suggestions - 10

src/jit.c - 3
- Consider validating hash key before use · Line 2213-2213
- Consider using assert(false) for unreachable code · Line 853-853
- Consider consolidating buffer overflow checks · Line 307-310
src/rv32_jit.c - 3
- Consider consolidating duplicate PC store code · Line 139-154
- Consider optimizing register reset timing · Line 430-431
- Consider extracting common PC store pattern · Line 16-19
src/system.h - 1
- Consider removing duplicate function declaration · Line 166-166
src/utils.c - 1
- Consider consolidating duplicate assertions · Line 196-196
src/rv32_template.c - 2
- Consider consolidating cache_get and satp check · Line 2324-2325
- Consider simplifying nested conditional blocks · Line 171-182

Review Details

Files reviewed - 12 · Commit Range: 9cd48ea..8d46115
- Makefile
- src/decode.h
- src/emulate.c
- src/jit.c
- src/jit.h
- src/riscv_private.h
- src/rv32_jit.c
- src/rv32_template.c
- src/system.c
- src/system.h
- src/utils.c
- src/utils.h
Files skipped - 2
- .github/workflows/main.yml - Reason: Filter setting
- .gitignore - Reason: Filter setting
Tools
- Whispers (Secret Scanner) - ✔︎ Successful
- Detect-secrets (Secret Scanner) - ✔︎ Successful
- Fb Infer (Static Code Analysis) - ✖︎ Failed

AI Code Review powered by

src/rv32_template.c

src/jit.c

jserv · 2025-02-14T15:30:00Z

Thank @vacantron for contributing!

vacantron changed the title ~~Support just-in-time (JIT) compilation in system simulation~~ jit: Support just-in-time (JIT) compilation in system simulation Dec 17, 2024

jserv reviewed Dec 17, 2024

View reviewed changes

jserv changed the title ~~jit: Support just-in-time (JIT) compilation in system simulation~~ Allow JIT compilation for system emulation Dec 17, 2024

jserv reviewed Dec 17, 2024

View reviewed changes

.gitignore Show resolved Hide resolved

jserv added this to the release-2024.2 milestone Dec 17, 2024

vacantron force-pushed the wip/jit/mmu branch 2 times, most recently from 73b1ba5 to 09fc945 Compare December 18, 2024 11:57

jserv reviewed Dec 18, 2024

View reviewed changes

src/emulate.c Outdated Show resolved Hide resolved

jserv reviewed Dec 18, 2024

View reviewed changes

src/emulate.c Outdated Show resolved Hide resolved

jserv reviewed Dec 18, 2024

View reviewed changes

src/riscv_private.h Outdated Show resolved Hide resolved

jserv reviewed Dec 18, 2024

View reviewed changes

src/riscv_private.h Show resolved Hide resolved

ChinYikMing reviewed Dec 18, 2024

View reviewed changes

src/emulate.c Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

jserv requested changes Feb 2, 2025

View reviewed changes

vacantron force-pushed the wip/jit/mmu branch from 09fc945 to cf39d8f Compare February 9, 2025 07:47

vacantron marked this pull request as ready for review February 9, 2025 07:47