3GX Plugins are unable to call svcCreateMemoryBlock on their own heap memory

[LittleCube-hax]

I've been messing around with plugins quite a bit (the game I'm using is A Link Between Worlds USA) and something extremely disconcerting came up. Plugins cannot call svcCreateMemoryBlock on their heap memory in order to share it with a service. This is frustrating for using services like sockets, since obviously a call to socInit requires that this svc works.

I understand that all this is already known, interestingly enough #1777 references this problem but was closed as 3GX plugins were not an official feature of Luma3DS at the time. Now that they are, I'd like to open a new one in hopes of discovering an innovative and safe way to give plugins memory that works with services (easier said than done ofc). I am willing to help implement this change, but I didn't see an open issue referencing it so I thought I'd start the conversation. If this is unwelcome, I understand and will certainly not protest this issue being closed as not planned.

To outline the problem as clearly as I can: so far, I know that Rosalina is in charge of loading plugins. As far as I can tell, this is the code that instantiates the memory block that the plugin will use:

Luma3DS/sysmodules/rosalina/source/plugin/memoryblock.c

Lines 62 to 65 in a7bd320

	// Then allocate our plugin memory block
	if (R_SUCCEEDED(res))
	res = svcControlMemoryUnsafe((u32 *)&memblock->memblock, 0x07000000,
	g_memBlockSize, MEMOP_REGION_APP | MEMOP_ALLOC | MEMOP_LINEAR_FLAG, MEMPERM_RW);

Then, this is the code that maps executable and heap memory into the plugin's process (where header->heapVA is 0x06000000):

Luma3DS/sysmodules/rosalina/source/plugin/memoryblock.c

Lines 211 to 224 in a7bd320

	// Executable
	if (R_FAILED((res = svcMapProcessMemoryEx(target, 0x07000000, CUR_PROCESS_HANDLE, (u32)memblock->memblock, header->exeSize))))
	{
	error->message = "Couldn't map exe memory block";
	error->code = res;
	return res;
	}
	// Heap (to be used by the plugin)
	if (R_FAILED((res = svcMapProcessMemoryEx(target, header->heapVA, CUR_PROCESS_HANDLE, (u32)memblock->memblock + header->exeSize, header->heapSize))))
	{
	error->message = "Couldn't map heap memory block";
	error->code = res;
	}

Of course, the important part of this code is the fact that Rosalina calls svcMapProcessMemoryEx in order to map the memblock into the plugin's virtual memory space. This is helpful for giving the plugin access to a heap, but it prevents the memory from being shared "again" with svcCreateMemoryBlock. I could be wrong but I'm relatively sure this is the case, because even Rosalina is unable to use that call on the heap space after mapping it into the plugin (but is fully able to before it is mapped).

I have even tried calling svcUnmapProcessMemoryEx(CUR_PROCESS_HANDLE, ... from Rosalina on the heap memory once it has been mapped into the plugin's process, and even though this does not cause the plugin to crash (which is what I expected) it does not solve the issue, either.

Is there any way at all for Rosalina to put this memory into a state that will be accepted by svcCreateMemoryBlock? Perhaps it's worth trying to instantiate physical memory for the plugin the same way the os would for the game normally? As a less ideal option, is there perhaps a way to implement our own version of svcCreateMemoryBlock that bypasses the checks that error out? I understand that all this must sound terribly naïve, but if we got this to work then it would increase the quality and capability of plugins significantly.

[TuxSH]

Hi,

From a quick look at my kernel IDB SvcMapProcessMemory sets memory state KMEMSTATE_SHARED = 0x5806 (while accepting text/rodata/data/bss/heap), whereas SvcCreateMemoryBlock expects exactly KMEMSTATE_PRIVATE_DATA = 0xBB05, which corresponds to heap created the normal way via SvcControlMemory.

The "kernel extension" stuff Rosalina relies on is a huge pile of hacks I wrote in 2017 (back when I didn't fully understand kernel) and plugin stuff is even more hacks over it. Asking @PabloMK7 since he wrote the code you cited, however it is quite likely this requires reimplementing Kernel11 (which would fix the Rosalina cheat system in the process) -- though that won't be done before I reimplement NDS and GBA compatibility firmware first.

[LittleCube-hax]

Thank you for the quick response,

Yes! After posting I later noticed that svcQueryMemory returns SHARED for MemState, which as you pointed out seems to be exactly the problem here.

Writing an extra flag parameter to svcMapProcessMemoryEx to tell it to use KMEMSTATE_PRIVATE_DATA instead, and using it for the plugin's heap, results in svcCreateMemoryBlock succeeding! Not only that, but I tested a UDP socket from within the plugin and was able to send a string from my PC and output the string onto the 3ds screen just fine.

Unfortunately it seems to only currently work with plugins that use 2 MiB or 5 MiB of memory. On 10 MiB, the memory is still set properly, but the whole system crashes after a few seconds of the plugin running. 🥴 But with the other options, everything works as expected, other apps don't have problems afterwards, etc. It's also worth noting that 10 MiB heap plugins that don't use CTRPluginFramework do not crash, but work just as well as the others as far as I can tell.

I can make some changes that revert to default behavior for plugins that use 10 MiB, since using SHARED as normal causes those plugins to work just fine again. If all current plugins work and no other issues arise, is this something you'd be willing to consider merging if I made a PR? Obviously this would be a makeshift setup before the k11 rewrite, but for the time being it could encourage others to make some seriously interesting plugins.

[TuxSH]

Closed by #2086