Skip to content

Commit b11471b

Browse files
[NativeAOT] Improve managed typemap (#9910)
Context: #9846 Context: efbec22 Context: #2266 Context: 684ede6 Context: xamarin/monodroid@eb04c91 Context: dotnet/java-interop@397013e `Java.Interop.JniRuntime.JniTypeManager` supports bi-directional mapping between JNI type signatures and managed `Type`s: * `GetTypeSignature(Type)` returns the JNI type signature that corresponds to the `System.Type`. * `GetType(JniTypeSignature)` returns the `Type` that corresponds to a JNI type signature. One can imagine 1:1 *identity relation* between Java and managed types, in which `GetType(GetTypeSignature(t))==t`: static void Identity<T> () { JniRuntime.JniTypeManager tm = Java.Interop.JniEnvironment.Runtime.TypeManager; JniTypeSignature sigFromT = tm.GetTypeSignature (typeof (T)); Type? typefromSig = tm.GetType (sigFromT); if (typeof (T) != typefromSig) throw new Exception ($"Type({typeof (T)}) != .GetType({sigFromT}){{{(typefromSig == null ? "<null>" : typefromSig.ToString ())}}}"); JniTypeSignature sigIdentity = tm.GetTypeSignature (typefromSig!); if (sigFromT != sigIdentity) throw new Exception ($"{sigFromT}) != {sigIdentity}"); } This is *not* always true in .NET for Android. There *is* a required 1:1 relation between Java and managed types for "normal" user-written `Java.Lang.Object` and `Java.Lang.Throwable` subclasses, such as: partial class MainActivity : Activity { } There *may not* be a 1:1 relation between Java and managed types for: * Bindings/projections of Java types; that is, types which have `[Register(…, DoNotGenerateAcw=true)]` or `[JniTypeSignature(…, GenerateJavaPeer=false)]`. * *Arrays* of types. The same `JniTypeSignature` is generated for for `T[]`, `JavaArray<T>`, and other `Java*Array` types. For example, `Mono.Android.dll` contains *three* "binding aliases" for `java.util.ArrayList`: * `Android.Runtime.JavaList` * `Android.Runtime.JavaList<T>` * `Java.Util.ArrayList` Only `Identity<Android.Runtime.JavaList>()` passes on .NET 9. There are *two* binding aliases for `java.lang.Object`: `Java.Interop.JavaObject, Java.Interop` and `Java.Lang.Object, Mono.Android`. (Plus more in unit tests!) Only `Identity<Java.Lang.Object>()` passes on .NET 9, and that's because of special-casing (25d1f00, 7acf328). Which brings us to 684ede6 and #9846: there are two additional issues with the improved NativeAOT-compatible typemap: 1. It ignores assembly names when generating typemaps, and 2. It doesn't properly deal with binding aliases. ## Assembly Identity is important! It is not unusual for developers to copy and paste code "from elsewhere" into their project. It is likewise not unusual for developers to *not* rename such copied code, meaning it is quite possible for the "same" type to be present multiple times in an app: // Lib1.dll namespace Utilities { class GenericHolder<T> : Java.Lang.Object { public T Value {get; set;} } } // Lib2.dll namespace Utilities { class GenericHolder<T> : Java.Lang.Object { public T Value {get; set;} } } From a C# and .NET perspective, this is fine: `Utilities.GenericHolder<T>` has `internal` visibility; there is no conflict here. …except in pre-2014 Xamarin.Android, in which the package name of the Java Callable Wrapper (JCW) name was based on the namespace of the managed type. *Both* of the above types would have the *same* JCW name: `utilities.GenericHolder`. This would promptly error out: error : Duplicate managed type found! Mappings between managed types and Java types must be unique. First Type: 'Android.Support.V4.App.FragmentManager/IOnBackStackChangedListenerImplementor, Xamarin.Android.Support.v4-r18, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'; Second Type: 'Android.Support.V4.App.FragmentManager/IOnBackStackChangedListenerImplementor, Mono.Android.Support.v4' This scenario was improved in xamarin/monodroid@eb04c91c, which used an *md5sum* of the namespace and assembly name, ensuring that the two different `GenericHolder<T>` types would get different Java package names, as the assembly name was in play. (This was later changed to use a CRC64 instead md5sum in dotnet/java-interop@397013ed.) A problem in 684ede6 is that it would hash the managed type name for a lookup, but the hash ignored the assembly name. Consequently when it countered two different `Java.InteropTests.GenericHolder<T>` types in two separate assemblies, the assembly name was ignored, resulting in a duplicate hash. This would result in build failures: Fatal error in IL Linker Unhandled exception. System.InvalidOperationException: Duplicate hashes at Microsoft.Android.Sdk.ILLink.TypeMappingStep.<>c__DisplayClass9_0.<EndProcess>g__GenerateHashes|10(UInt64[] hashes, String methodName) at Microsoft.Android.Sdk.ILLink.TypeMappingStep.EndProcess() at Mono.Linker.Steps.BaseStep.Process(LinkContext context) at Mono.Linker.Pipeline.ProcessStep(LinkContext context, IStep step) at Mono.Linker.Pipeline.Process(LinkContext context) at Mono.Linker.Driver.Run(ILogger customLogger) at Mono.Linker.Driver.Main(String[] args) Fix this by hashing `Type.AssemblyQualifiedName` instead of `Type.FullName`. ## Binding Aliases #9846 had a unit test failure in `JavaObjectExtensionsTests.JavaCast_BaseToGenericWrapper()`: System.ArgumentException : Could not determine Java type corresponding to ``Android.Runtime.JavaList`1[[System.Int32, System.Private.CoreLib, Version=10.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]], Mono.Android``. Arg_ParamName_Name, targetType at Java.Interop.JniRuntime.JniValueManager.CreatePeer(JniObjectReference&, JniObjectReferenceOptions, Type) + 0x248 at Java.Interop.JniRuntime.JniValueManager.GetPeer(JniObjectReference, Type) + 0x77 at Java.Lang.Object.GetObject(IntPtr, JniHandleOwnership, Type) + 0x31 at Java.Interop.JavaObjectExtensions._JavaCast[TResult](IJavaObject) + 0x66 at Java.InteropTests.JavaObjectExtensionsTests.JavaCast_BaseToGenericWrapper() + 0x62 at libMono.Android.NET-Tests!<BaseAddress>+0x1489b83 at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0xf3 This is due to "binding aliases": there was no type mapping between `Android.Runtime.JavaList<T>` and `java/util/arrayList`, because the typemaps were assuming a 1:1 mapping, which isn't the case here. Update `TypeMappingStep` so that binding aliases are supported. Given (from `TypeMappingStep.cs`): partial class TypeMappingStep { IDictionary<string, List<TypeDefinition>> TypeMappings = new(StringComparer.Ordinal); } Then at the end of trimming, `TypeMappingStep.TypeMappings` will contain e.g. TypeMappingStep.TypeMappings = { ["java/util/ArrayList"] => new List<TypeDefinition> { typeof (Android.Runtime.JavaList), typeof (Android.Runtime.JavaList<>), typeof (Java.Util.ArrayList), }, ["java/lang/Object"] => new List<TypeDefinition> { typeof (Java.Lang.Object), typeof (Java.Interop.JavaObject), }, ["java/lang/Runnable"] => new List<TypeDefinition> { typeof (Java.Lang.IRunnable), typeof (Java.Lang.IRunnableInvoker), }, // … }; From `TypeMappingStep.TypeMappings` we produce two typemap "tables". Java/JNI class name to managed type resembles: partial class TypeMapping { static Type? GetTypeByJniNameHashIndex (int index) => index switch { 0 => Type.GetTypeFromHandle (typeof (Java.Lang.Byte).RuntimeTypeHandle), // `java/lang/Byte` hash=0x01cd624f1e38cc9f 32 => Type.GetTypeFromHandle (typeof (Android.Runtime.JavaList).RuntimeTypeHandle), // `java/util/ArrayList` hash=0x7b925bdca68a0101 56 => Type.GetTypeFromHandle (typeof (Java.Lang.Object).RuntimeTypeHandle), // `java/lang/Object` hash=0xbf6d427143271cb3 77 => Type.GetTypeFromHandle (typeof (Java.Lang.IRunnable).RuntimeTypeHandle), // `java/lang/Runnable` hash=0xfd2b1a3de667eb51 _ => null, }; static string? GetJniNameByJniNameHashIndex (int index) => index switch { 0 => "java/lang/Byte", // `Java.Lang.Byte, Mono.Android` hash=0x01cd624f1e38cc9f 32 => "java/util/ArrayList", // `Android.Runtime.JavaList, Mono.Android` hash=0x7b925bdca68a0101 56 => "java/lang/Object", // `Java.Lang.Object, Mono.Android` hash=0xbf6d427143271cb3 77 => "java/lang/Runnable", // `Java.Lang.IRunnable, Mono.Android` hash=0xfd2b1a3de667eb51 _ => null, }; private static ReadOnlySpan<ulong> JniNameHashes => new ReadOnlySpan<ulong>(ref s_get_JniNameHashes_data, 78); [StructLayout(LayoutKind.Explicit, Pack=1, Size=624)] private struct HashesArray_624 { } private static HashesArray_624 s_get_JniNameHashes_data = new ulong[78]{ // RVA data; insert hand-waving here 0x1cd624f1e38cc9f, // 0: java/lang/Byte // … 0x7b925bdca68a0101, // 32: java/util/ArrayList // … 0xbf6d427143271cb3, // 56: java/lang/Object // … 0xfd2b1a3de667eb51, // 77: java/lang/Runnable // … }; } Note that `TypeMapping.GetTypeByJniNameHashIndex()` and `TypeMapping.GetJniNameByJniNameHashIndex()` provides values for all indexes between 0 and the number of hashes within `s_get_JniNameHashes_data`. Not all entries are listed for exposition purposes. Managed type to JNI resembles: partial class TypeMapping { static string? GetJniNameByTypeNameHashIndex (int index) => index switch { 0 => "java/lang/IndexOutOfBoundsException", // `Java.Lang.IndexOutOfBoundsException, Mono.Android` hash=0x0242f4a673f183d1 2 => "java/util/ArrayList", // `Java.Util.ArrayList, Mono.Android` hash=0x07c2b62d20a668dd 4 => "java/util/ArrayList", // `Android.Runtime.JavaList`1, Mono.Android` hash=0x132055d1acecc87d 30 => "java/lang/Runnable", // `Java.Lang.IRunnableInvoker, Mono.Android` hash=0x386176afae6775ce 32 => "mono/java/lang/Runnable", // `Java.Lang.Runnable, Mono.Android` hash=0x408566f4617e204f 36 => "java/util/ArrayList", // `Android.Runtime.JavaList, Mono.Android` hash=0x48a63a89cfee545f 44 => "java/lang/Object", // `Java.Lang.Object, Mono.Android` hash=0x5b4a99c45538d0fb 51 => "java/lang/Object", // `Java.Interop.JavaObject, Java.Interop` hash=0x88a12107f88ba9a7 53 => "java/lang/Byte", // `Java.Lang.Byte, Mono.Android` hash=0x8ab15b52718902fb 96 => "java/lang/Runnable", // `Java.Lang.IRunnable, Mono.Android` hash=0xef2efa58a51bb883 _ => null, }; static string? GetTypeNameByTypeNameHashIndex (int index) => index switch { 0 => "Java.Lang.IndexOutOfBoundsException, Mono.Android", // `java/lang/IndexOutOfBoundsException` hash=0x0242f4a673f183d1 2 => "Java.Util.ArrayList, Mono.Android", // `java/util/ArrayList` hash=0x07c2b62d20a668dd 4 => "Android.Runtime.JavaList`1, Mono.Android", // `java/util/ArrayList` hash=0x132055d1acecc87d 30 => "Java.Lang.IRunnableInvoker, Mono.Android", // `java/lang/Runnable` hash=0x386176afae6775ce 32 => "Java.Lang.Runnable, Mono.Android", // `mono/java/lang/Runnable` hash=0x408566f4617e204f 36 => "Android.Runtime.JavaList, Mono.Android", // `java/util/ArrayList` hash=0x48a63a89cfee545f 44 => "Java.Lang.Object, Mono.Android", // `java/lang/Object` hash=0x5b4a99c45538d0fb 51 => "Java.Interop.JavaObject, Java.Interop", // `java/lang/Object` hash=0x88a12107f88ba9a7 53 => "Java.Lang.Byte, Mono.Android", // `java/lang/Byte` hash=0x8ab15b52718902fb 96 => "Java.Lang.IRunnable, Mono.Android", // `java/lang/Runnable` hash=0xef2efa58a51bb883 _ => null, }; private static ReadOnlySpan<ulong> TypeNameHashes => new ReadOnlySpan<ulong>(ref s_get_TypeNameHashes_data, 99); [StructLayout(LayoutKind.Explicit, Pack=1, Size=792)] private struct HashesArray_792 { } private static HashesArray_792 s_get_TypeNameHashes_data = new ulong[99] { // RVA data; insert hand-waving here 0x242f4a673f183d1, // 0: Java.Lang.IndexOutOfBoundsException, Mono.Android // … 0x7c2b62d20a668dd, // 2: Java.Util.ArrayList, Mono.Android // … 0x132055d1acecc87d, // 4: Android.Runtime.JavaList`1, Mono.Android // … 0x386176afae6775ce, // 30: Java.Lang.IRunnableInvoker, Mono.Android // … 0x408566f4617e204f, // 32: Java.Lang.Runnable, Mono.Android // … 0x48a63a89cfee545f, // 36: Android.Runtime.JavaList, Mono.Android // … 0x5b4a99c45538d0fb, // 44: Java.Lang.Object, Mono.Android // … 0x88a12107f88ba9a7, // 51: Java.Interop.JavaObject, Java.Interop // … 0x8ab15b52718902fb, // 53: Java.Lang.Byte, Mono.Android // … 0xef2efa58a51bb883, // 96: Java.Lang.IRunnable, Mono.Android // … }; } Note that `TypeMapping.GetJniNameByTypeNameHashIndex()` and `TypeMapping.GetTypeNameByTypeNameHashIndex()` provides values for all indexes between 0 and the number of hashes within `s_get_TypeNameHashes_data`. Not all entries are listed for exposition purposes.
1 parent ea399ed commit b11471b

File tree

4 files changed

+223
-126
lines changed

4 files changed

+223
-126
lines changed

src/Microsoft.Android.Runtime.NativeAOT/Android.Runtime.NativeAOT/NativeAotTypeManager.cs

+2-2
Original file line numberDiff line numberDiff line change
@@ -141,8 +141,8 @@ protected override IEnumerable<string> GetSimpleReferences (Type type)
141141
yield return r;
142142
}
143143

144-
if (TypeMapping.TryGetJavaClassName (type, out var javaClassName)) {
145-
yield return javaClassName;
144+
if (TypeMapping.TryGetJniName (type, out var jniName)) {
145+
yield return jniName;
146146
}
147147
}
148148

src/Microsoft.Android.Runtime.NativeAOT/Android.Runtime.NativeAOT/TypeMapping.cs

+49-36
Original file line numberDiff line numberDiff line change
@@ -10,67 +10,65 @@ namespace Microsoft.Android.Runtime;
1010

1111
internal static class TypeMapping
1212
{
13-
internal static bool TryGetType (string javaClassName, [NotNullWhen (true)] out Type? type)
13+
internal static bool TryGetType (string jniName, [NotNullWhen (true)] out Type? type)
1414
{
15-
ulong hash = Hash (javaClassName);
15+
type = null;
1616

1717
// the hashes array is sorted and all the hashes are unique
18-
int typeIndex = MemoryExtensions.BinarySearch (JavaClassNameHashes, hash);
19-
if (typeIndex < 0) {
20-
type = null;
18+
ulong jniNameHash = Hash (jniName);
19+
int jniNameHashIndex = MemoryExtensions.BinarySearch (JniNameHashes, jniNameHash);
20+
if (jniNameHashIndex < 0) {
2121
return false;
2222
}
2323

24-
type = GetTypeByIndex (typeIndex);
25-
if (type is null) {
26-
throw new InvalidOperationException ($"Type with hash {hash} not found.");
24+
// we need to make sure if this is the right match or if it is a hash collision
25+
if (jniName != GetJniNameByJniNameHashIndex (jniNameHashIndex)) {
26+
return false;
2727
}
2828

29-
// ensure this is not a hash collision
30-
var resolvedJavaClassName = GetJavaClassNameByIndex (TypeIndexToJavaClassNameIndex [typeIndex]);
31-
if (resolvedJavaClassName != javaClassName) {
32-
type = null;
33-
return false;
29+
type = GetTypeByJniNameHashIndex (jniNameHashIndex);
30+
if (type is null) {
31+
throw new InvalidOperationException ($"Type for {jniName} (hash: {jniNameHash}, index: {jniNameHashIndex}) not found.");
3432
}
3533

3634
return true;
3735
}
3836

39-
internal static bool TryGetJavaClassName (Type type, [NotNullWhen (true)] out string? className)
37+
internal static bool TryGetJniName (Type type, [NotNullWhen (true)] out string? jniName)
4038
{
41-
string? fullName = type.FullName;
42-
if (fullName is null) {
43-
className = null;
39+
jniName = null;
40+
41+
string? assemblyQualifiedName = type.AssemblyQualifiedName;
42+
if (assemblyQualifiedName is null) {
43+
jniName = null;
4444
return false;
4545
}
4646

47-
ulong hash = Hash (fullName);
47+
ReadOnlySpan<char> typeName = GetSimplifiedAssemblyQualifiedTypeName (assemblyQualifiedName);
4848

4949
// the hashes array is sorted and all the hashes are unique
50-
int javaClassNameIndex = MemoryExtensions.BinarySearch (TypeNameHashes, hash);
51-
if (javaClassNameIndex < 0) {
52-
className = null;
50+
ulong typeNameHash = Hash (typeName);
51+
int typeNameHashIndex = MemoryExtensions.BinarySearch (TypeNameHashes, typeNameHash);
52+
if (typeNameHashIndex < 0) {
5353
return false;
5454
}
5555

56-
className = GetJavaClassNameByIndex (javaClassNameIndex);
57-
if (className is null) {
58-
throw new InvalidOperationException ($"Java class name with hash {hash} not found.");
56+
// we need to make sure if this is the match or if it is a hash collision
57+
if (!typeName.SequenceEqual (GetTypeNameByTypeNameHashIndex (typeNameHashIndex))) {
58+
return false;
5959
}
6060

61-
// ensure this is not a hash collision
62-
var resolvedType = GetTypeByIndex (JavaClassNameIndexToTypeIndex [javaClassNameIndex]);
63-
if (resolvedType?.FullName != type.FullName) {
64-
className = null;
65-
return false;
61+
jniName = GetJniNameByTypeNameHashIndex (typeNameHashIndex);
62+
if (jniName is null) {
63+
throw new InvalidOperationException ($"JNI name for {typeName} (hash: {typeNameHash}, index: {typeNameHashIndex}) not found.");
6664
}
6765

6866
return true;
6967
}
7068

71-
private static ulong Hash (string javaClassName)
69+
private static ulong Hash (ReadOnlySpan<char> value)
7270
{
73-
ReadOnlySpan<byte> bytes = MemoryMarshal.AsBytes (javaClassName.AsSpan ());
71+
ReadOnlySpan<byte> bytes = MemoryMarshal.AsBytes (value);
7472
ulong hash = XxHash3.HashToUInt64 (bytes);
7573

7674
// The bytes in the hashes array are stored as little endian. If the target platform is big endian,
@@ -82,11 +80,26 @@ private static ulong Hash (string javaClassName)
8280
return hash;
8381
}
8482

83+
// This method keeps only the full type name and the simple assembly name.
84+
// It drops the version, culture, and public key information.
85+
//
86+
// For example: "System.Int32, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e"
87+
// becomes: "System.Int32, System.Private.CoreLib"
88+
private static ReadOnlySpan<char> GetSimplifiedAssemblyQualifiedTypeName(string assemblyQualifiedName)
89+
{
90+
var commaIndex = assemblyQualifiedName.IndexOf(',');
91+
var secondCommaIndex = assemblyQualifiedName.IndexOf(',', startIndex: commaIndex + 1);
92+
return secondCommaIndex < 0
93+
? assemblyQualifiedName
94+
: assemblyQualifiedName.AsSpan(0, secondCommaIndex);
95+
}
96+
8597
// Replaced by src/Microsoft.Android.Sdk.ILLink/TypeMappingStep.cs
86-
private static ReadOnlySpan<ulong> JavaClassNameHashes => throw new NotImplementedException ();
8798
private static ReadOnlySpan<ulong> TypeNameHashes => throw new NotImplementedException ();
88-
private static ReadOnlySpan<int> JavaClassNameIndexToTypeIndex => throw new NotImplementedException ();
89-
private static ReadOnlySpan<int> TypeIndexToJavaClassNameIndex => throw new NotImplementedException ();
90-
private static Type? GetTypeByIndex (int index) => throw new NotImplementedException ();
91-
private static string? GetJavaClassNameByIndex (int index) => throw new NotImplementedException ();
99+
private static Type? GetTypeByJniNameHashIndex (int jniNameHashIndex) => throw new NotImplementedException ();
100+
private static string? GetJniNameByJniNameHashIndex (int jniNameHashIndex) => throw new NotImplementedException ();
101+
102+
private static ReadOnlySpan<ulong> JniNameHashes => throw new NotImplementedException ();
103+
private static string? GetJniNameByTypeNameHashIndex (int typeNameHashIndex) => throw new NotImplementedException ();
104+
private static string? GetTypeNameByTypeNameHashIndex (int typeNameHashIndex) => throw new NotImplementedException ();
92105
}

0 commit comments

Comments
 (0)