Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dart_package option #298

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+294 −9
Open

Add dart_package option #298

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a4c3d98
first attempt
sigurdm Sep 27, 2019
e86f5f8
more tests
sigurdm Oct 2, 2019
3e3d5fb
Avoid using .dart for golden files
sigurdm Oct 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 23 additions & 9 deletions protoc_plugin/lib/file_generator.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,9 @@ class FileGenerator extends ProtobufContainer {
final FileDescriptorProto descriptor;
final GenerationOptions options;

// The relative path used to import the .proto file, as a URI.
// The path used to import the .proto file, as a URI.
// This is a package: URI if a `dart_package` option is given.
// Otherwise this is a relative path.
final Uri protoFileUri;

final enumGenerators = <EnumGenerator>[];
Expand All @@ -130,13 +132,22 @@ class FileGenerator extends ProtobufContainer {
/// True if cross-references have been resolved.
bool _linked = false;

FileGenerator(this.descriptor, this.options)
: protoFileUri = Uri.file(descriptor.name) {
if (protoFileUri.isAbsolute) {
// protoc should never generate an import with an absolute path.
throw "FAILURE: Import with absolute path is not supported";
static Uri calculateUri(FileDescriptorProto descriptor) {
// protoc should never generate an import with an absolute path.
assert(!Uri.file(descriptor.name).isAbsolute,
"Import with absolute path is not supported");

if (descriptor.options.hasExtension(Dart_options.dartPackage)) {
String dartPackage =
descriptor.options.getExtension(Dart_options.dartPackage);
return Uri(scheme: 'package', path: '${dartPackage}/${descriptor.name}');
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sigurdm I've been having a look at this, and I think this should be:

return Uri(scheme: 'package', path: '${dartPackage}/${path.basename(descriptor.name)}');

descriptor.name returns the full path of the proto file.

In my project the proto root is not the same as the package directory, so I get something like:

dartPackage: "foo/bar"
descriptor.name: "bar/baz.proto"

Combining them together gives "foo/bar/bar/baz.proto" when it should be "foo/bar/baz.proto".

I believe it would be better to ignore the path from the descriptor because the dart package name should always point directly to the directory that contains the dart source file. Right?

Copy link
Collaborator Author

@sigurdm sigurdm Nov 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the dart package should not have slashes. It should just be a single name (the name of the package where the proto belongs)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ooh interesting... That would be tricky in my project... I have one directory structure with several dart packages, and they all share a common proto root.

Copy link

@dave dave Nov 3, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it seems like an unnecessary restriction that your proto file structure should be the same as your dart package structure. Let me explain my project structure:

+ project root
  + server (go packages)
    + main 
    + repo1
    + repo2
    + ...
    + tests (go tests)
  + lib (main dart package)
  + repositories (other dart packages)
    + repo1
    + repo2
    + ...
  + proto (proto root)
    + main (proto files for main)
    + repo1 (proto files for repo1)
    + repo2 (proto files for repo2)
    + ...
  + test (dart tests)

It's convenient to have the proto files all in the same directory structure, but the file structure of the Dart packages doesn't match this. I'm trying to ensure my proto files stay platform independent, so I don't really want to rearrange my proto files to match my Dart directory structure. Nor do I really want to have a separate proto root for each package.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to have one proto-root per package, if the protos truly belongs in that package. (At least the generated files have to be moved into lib/ of that package.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I'll have a go at rearranging my project / protoc commands. I'll let you know how I get on!

} else {
return Uri.file(descriptor.name);
}
}

FileGenerator(this.descriptor, this.options)
: protoFileUri = calculateUri(descriptor) {
var declaredMixins = _getDeclaredMixins(descriptor);
var defaultMixinName =
descriptor.options?.getExtension(Dart_options.defaultMixin) ?? '';
Expand Down Expand Up @@ -194,15 +205,18 @@ class FileGenerator extends ProtobufContainer {
FileGenerator get fileGen => this;
List<int> get fieldPath => [];

Uri outputFile(OutputConfiguration config, String extension) {
Uri protoUrl = Uri.file(descriptor.name);
return config.outputPathFor(protoUrl, extension);
}

/// Generates all the Dart files for this .proto file.
List<CodeGeneratorResponse_File> generateFiles(OutputConfiguration config) {
if (!_linked) throw StateError("not linked");

makeFile(String extension, String content) {
Uri protoUrl = Uri.file(descriptor.name);
Uri dartUrl = config.outputPathFor(protoUrl, extension);
return CodeGeneratorResponse_File()
..name = dartUrl.path
..name = outputFile(config, extension).path
..content = content;
}

Expand Down
6 changes: 6 additions & 0 deletions protoc_plugin/lib/src/dart_options.pb.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,11 @@ class Imports extends $pb.GeneratedMessage {
}

class Dart_options {
static final $pb.Extension dartPackage = $pb.Extension<$core.String>(
'google.protobuf.FileOptions',
'dartPackage',
28205290,
$pb.PbFieldType.OS);
static final $pb.Extension imports = $pb.Extension<Imports>(
'google.protobuf.FileOptions', 'imports', 28125061, $pb.PbFieldType.OM,
defaultOrMaker: Imports.getDefault, subBuilder: Imports.create);
Expand Down Expand Up @@ -143,6 +148,7 @@ class Dart_options {
static final $pb.Extension dartName = $pb.Extension<$core.String>(
'google.protobuf.FieldOptions', 'dartName', 28700919, $pb.PbFieldType.OS);
static void registerAllExtensions($pb.ExtensionRegistry registry) {
registry.add(dartPackage);
registry.add(imports);
registry.add(defaultMixin);
registry.add(mixin);
Expand Down
3 changes: 3 additions & 0 deletions protoc_plugin/protos/dart_options.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ message Imports {
}

extend google.protobuf.FileOptions {
// The Dart package that should be used when importing code generated from
// this file.
optional string dart_package = 1073;

optional Imports imports = 28125061;

Expand Down
124 changes: 124 additions & 0 deletions protoc_plugin/test/file_generator_test.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
library file_generator_test;

import 'package:protoc_plugin/indenting_writer.dart';
import 'package:protoc_plugin/src/dart_options.pb.dart';
import 'package:protoc_plugin/src/descriptor.pb.dart';
import 'package:protoc_plugin/src/plugin.pb.dart';
import 'package:protoc_plugin/protoc.dart';
Expand Down Expand Up @@ -405,4 +406,127 @@ void main() {
expectMatchesGoldenFile(
fg.generateEnumFile().toString(), 'test/goldens/imports.pbjson');
});

test('FileGenerator generates correct dart package imports', () {
// This defines a.proto, same_package.proto, other_package.proto,
// no_package.proto
//
// a.proto:
// ---------------
// import "dart_options.proto";
// option (dart_options.dart_package) = 'abc';
// message A {
// }
//
// same_package.proto:
// ---------------
// import "a.proto";
// import "dart_options.proto";
// option (dart_options.dart_package) = 'abc';
//
// message B {
// optional A a = 1;
// }
//
// other_package.proto:
// ---------------
// import "a.proto";
// import "dart_options.proto";
// option (dart_options.dart_package) = 'def';
// message C {
// optional A a = 1;
// }
//
// no_package.proto:
// ---------------
// import "a.proto";
// message D {
// optional A a = 1;
// }
//
// import_no_package.proto
// ---------------
// import "no_package.proto";
// import "dart_options.proto";
// option (dart_options.dart_package) = 'abc';
//
// message E {
// optional D d = 1;
// }

FieldDescriptorProto makeField(String name, String typeName,
{int number = 1}) {
return FieldDescriptorProto()
..name = name
..jsonName = name
..number = number
..label = FieldDescriptorProto_Label.LABEL_OPTIONAL
..type = FieldDescriptorProto_Type.TYPE_MESSAGE
..typeName = typeName;
}

DescriptorProto mA = DescriptorProto()..name = 'A';
FileDescriptorProto fda = FileDescriptorProto()
..name = 'a.proto'
..messageType.add(mA)
..options =
(FileOptions()..setExtension(Dart_options.dartPackage, 'abc'));

DescriptorProto mB = DescriptorProto()
..name = 'B'
..field.addAll([makeField('a', '.A')]);
FileDescriptorProto fdSamePackage = FileDescriptorProto()
..name = 'same_package.proto'
..messageType.add(mB)
..dependency.add('a.proto')
..options =
(FileOptions()..setExtension(Dart_options.dartPackage, 'abc'));

DescriptorProto mC = DescriptorProto()
..name = 'C'
..field.add(makeField('a', '.A'));
FileDescriptorProto fdOtherPackage = FileDescriptorProto()
..name = 'other_package.proto'
..messageType.add(mC)
..dependency.add('a.proto')
..options =
(FileOptions()..setExtension(Dart_options.dartPackage, 'def'));

DescriptorProto mD = DescriptorProto()
..name = 'D'
..field.add(makeField('a', '.A'));
FileDescriptorProto fdNoPackage = FileDescriptorProto()
..name = 'no_package.proto'
..dependency.add('a.proto')
..messageType.add(mD);

DescriptorProto mE = DescriptorProto()
..name = 'E'
..field.add(makeField('d', '.D'));
FileDescriptorProto fdImportNoPackage = FileDescriptorProto()
..name = 'import_no_package.proto'
..dependency.add('no_package.proto')
..messageType.add(mE);

void testFile(FileDescriptorProto fileDescriptor,
List<FileDescriptorProto> dependencies, Uri expectedFilePath) {
final options = parseGenerationOptions(
CodeGeneratorRequest(), CodeGeneratorResponse());
final fileGenerators =
dependencies.map((dep) => FileGenerator(dep, options)).toList();
final fg = FileGenerator(fileDescriptor, options);
fileGenerators.add(fg);
link(options, fileGenerators);
final file = fg.outputFile(DefaultOutputConfiguration(), '.pb.dart');
expectMatchesGoldenFile(
fg.generateMainFile().toString(), 'test/goldens/${file.path}.golden');
expect(file, expectedFilePath);
}

testFile(fdSamePackage, [fda], Uri.file('same_package.pb.dart'));
testFile(fdOtherPackage, [fda], Uri.file('other_package.pb.dart'));
testFile(fdNoPackage, [fda], Uri.file('no_package.pb.dart'));
testFile(fdImportNoPackage, [fdNoPackage, fda],
Uri.file('import_no_package.pb.dart'));
});
}
46 changes: 46 additions & 0 deletions protoc_plugin/test/goldens/no_package.pb.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
///
// Generated code. Do not modify.
// source: no_package.proto
//
// @dart = 2.3
// ignore_for_file: camel_case_types,non_constant_identifier_names,library_prefixes,unused_import,unused_shown_name,return_of_invalid_type

import 'dart:core' as $core;

import 'package:protobuf/protobuf.dart' as $pb;

import 'package:abc/a.pb.dart' as $3;

class D extends $pb.GeneratedMessage {
static final $pb.BuilderInfo _i = $pb.BuilderInfo('D', createEmptyInstance: create)
..aOM<$3.A>(1, 'a', subBuilder: $3.A.create)
..hasRequiredFields = false
;

D._() : super();
factory D() => create();
factory D.fromBuffer($core.List<$core.int> i, [$pb.ExtensionRegistry r = $pb.ExtensionRegistry.EMPTY]) => create()..mergeFromBuffer(i, r);
factory D.fromJson($core.String i, [$pb.ExtensionRegistry r = $pb.ExtensionRegistry.EMPTY]) => create()..mergeFromJson(i, r);
D clone() => D()..mergeFromMessage(this);
D copyWith(void Function(D) updates) => super.copyWith((message) => updates(message as D));
$pb.BuilderInfo get info_ => _i;
@$core.pragma('dart2js:noInline')
static D create() => D._();
D createEmptyInstance() => create();
static $pb.PbList<D> createRepeated() => $pb.PbList<D>();
@$core.pragma('dart2js:noInline')
static D getDefault() => _defaultInstance ??= $pb.GeneratedMessage.$_defaultFor<D>(create);
static D _defaultInstance;

@$pb.TagNumber(1)
$3.A get a => $_getN(0);
@$pb.TagNumber(1)
set a($3.A v) { setField(1, v); }
@$pb.TagNumber(1)
$core.bool hasA() => $_has(0);
@$pb.TagNumber(1)
void clearA() => clearField(1);
@$pb.TagNumber(1)
$3.A ensureA() => $_ensure(0);
}

46 changes: 46 additions & 0 deletions protoc_plugin/test/goldens/other_package.pb.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
///
// Generated code. Do not modify.
// source: other_package.proto
//
// @dart = 2.3
// ignore_for_file: camel_case_types,non_constant_identifier_names,library_prefixes,unused_import,unused_shown_name,return_of_invalid_type

import 'dart:core' as $core;

import 'package:protobuf/protobuf.dart' as $pb;

import 'package:abc/a.pb.dart' as $3;

class C extends $pb.GeneratedMessage {
static final $pb.BuilderInfo _i = $pb.BuilderInfo('C', createEmptyInstance: create)
..aOM<$3.A>(1, 'a', subBuilder: $3.A.create)
..hasRequiredFields = false
;

C._() : super();
factory C() => create();
factory C.fromBuffer($core.List<$core.int> i, [$pb.ExtensionRegistry r = $pb.ExtensionRegistry.EMPTY]) => create()..mergeFromBuffer(i, r);
factory C.fromJson($core.String i, [$pb.ExtensionRegistry r = $pb.ExtensionRegistry.EMPTY]) => create()..mergeFromJson(i, r);
C clone() => C()..mergeFromMessage(this);
C copyWith(void Function(C) updates) => super.copyWith((message) => updates(message as C));
$pb.BuilderInfo get info_ => _i;
@$core.pragma('dart2js:noInline')
static C create() => C._();
C createEmptyInstance() => create();
static $pb.PbList<C> createRepeated() => $pb.PbList<C>();
@$core.pragma('dart2js:noInline')
static C getDefault() => _defaultInstance ??= $pb.GeneratedMessage.$_defaultFor<C>(create);
static C _defaultInstance;

@$pb.TagNumber(1)
$3.A get a => $_getN(0);
@$pb.TagNumber(1)
set a($3.A v) { setField(1, v); }
@$pb.TagNumber(1)
$core.bool hasA() => $_has(0);
@$pb.TagNumber(1)
void clearA() => clearField(1);
@$pb.TagNumber(1)
$3.A ensureA() => $_ensure(0);
}

46 changes: 46 additions & 0 deletions protoc_plugin/test/goldens/same_package.pb.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
///
// Generated code. Do not modify.
// source: same_package.proto
//
// @dart = 2.3
// ignore_for_file: camel_case_types,non_constant_identifier_names,library_prefixes,unused_import,unused_shown_name,return_of_invalid_type

import 'dart:core' as $core;

import 'package:protobuf/protobuf.dart' as $pb;

import 'a.pb.dart' as $3;

class B extends $pb.GeneratedMessage {
static final $pb.BuilderInfo _i = $pb.BuilderInfo('B', createEmptyInstance: create)
..aOM<$3.A>(1, 'a', subBuilder: $3.A.create)
..hasRequiredFields = false
;

B._() : super();
factory B() => create();
factory B.fromBuffer($core.List<$core.int> i, [$pb.ExtensionRegistry r = $pb.ExtensionRegistry.EMPTY]) => create()..mergeFromBuffer(i, r);
factory B.fromJson($core.String i, [$pb.ExtensionRegistry r = $pb.ExtensionRegistry.EMPTY]) => create()..mergeFromJson(i, r);
B clone() => B()..mergeFromMessage(this);
B copyWith(void Function(B) updates) => super.copyWith((message) => updates(message as B));
$pb.BuilderInfo get info_ => _i;
@$core.pragma('dart2js:noInline')
static B create() => B._();
B createEmptyInstance() => create();
static $pb.PbList<B> createRepeated() => $pb.PbList<B>();
@$core.pragma('dart2js:noInline')
static B getDefault() => _defaultInstance ??= $pb.GeneratedMessage.$_defaultFor<B>(create);
static B _defaultInstance;

@$pb.TagNumber(1)
$3.A get a => $_getN(0);
@$pb.TagNumber(1)
set a($3.A v) { setField(1, v); }
@$pb.TagNumber(1)
$core.bool hasA() => $_has(0);
@$pb.TagNumber(1)
void clearA() => clearField(1);
@$pb.TagNumber(1)
$3.A ensureA() => $_ensure(0);
}