Source code generation in Flutter & Dart (Part 1): Reflection and code gen

If you have worked on Flutter projects (or you're a Java developer), you may be familiar with the Mockito library.

Mockito allows developers to create mock objects from existing classes, stub their methods and verify or assert their behavior during tests.

If you have used Mockito for Flutter or Dart projects, you will find the following in their documentation:

To use Mockito's generated mock classes, add a build_runner dependency in your package's pubspec.yaml file, under dev_dependencies; something like build_runner: ^1.11.0.

Some of the code examples will show how to mock an existing class like Cat as follows:

@GenerateMocks([ Cat ])

Then it expects you to run the following command:

flutter pub run build_runner build

The build_runner build command creates a file called *.mocks.dart. What exactly is happening here?

Reflection (and the lack of it)

Flutter's FAQ contains the following question:

***Does Flutter come with a reflection / mirrors system?*** No. Dart includes dart:mirrors, which provides type reflection. But since Flutter apps are pre-compiled for production, and binary size is always a concern with mobile apps, this library is unavailable for Flutter apps. Using static analysis we can strip out anything that isn’t used (“tree shaking”)... This guarantee is only secure if Dart can identify the code path at compile time. To date, we’ve found other approaches for specific needs that offer a better trade-off, such as code generation.

According to Wikipedia, Reflection is:

In computer science, reflective programming or reflection is the ability of a process to examine, introspect, and modify its own structure and behavior

In other languages like Java, libraries like Mockito use reflection to get information at run-time about the classes they are modifying. For instance, a Java function can get all the available methods for a class with the following code:

Class c = Class.forName("java.lang.String");
Method m[] = c.getDeclaredMethods();

In this example, the function gets an array containing all the functions that are part of the java.lang.String class.

We can see how this could be practical for libraries like Mockito: It can find all the methods of a given class and provide stub implementations based on their return and parameter types. All in runtime.

A common approach is to mark classes with annotations, then find those classes through reflection and do something with them:

• Create mock implementations.
• Instantiate singletons for the class.
• Create and inject instances of other classes into the constructor (like the Spring framework does).
• Decorate classes to provide behavior like JSON serialization or String representations, and so on.

A good example on how reflection is leveraged is the Java snippet we included in the previous post "Easy parallelism and multi-threading with Java's CompletableFuture" (updated to remove generic classes for clarity):

private static <Book> List<Book> parseJSON(String textResponse) {
    final ObjectMapper objectMapper = new ObjectMapper();
    List<T> objects = new ArrayList<>();
    try {
      objects =
        objectMapper.readValue(textResponse, new TypeReference<List<Book>>() {});
    } catch (JsonMappingException e) {
      // TODO: Do something with the error
    } catch (JsonProcessingException e) {
      // TODO: Do something with the error
      e.printStackTrace();
    }
    return objects;
  }

In this code example, the Jackson mapper uses reflection to find out the type of class it should use to de-serialize the JSON string:

List<T> objects = objectMapper.readValue(textResponse, new TypeReference<List<Book>>() {});

The parameter new TypeReference<List<Book>>() {} creates a class reference that lets Jackson'sObjectMapperknow that it should parse the JSON and create an instance ofList<Book>.

However, we cannot do this kind of JSON de-serialization in Flutter. Without reflection, we need to consider other approaches for these same tasks. For instance, the package json_serializable uses source code generation for decorating classes with methods to serialize and de-serialize JSON strings.

Source code generation

When we don't have access to metadata about classes in run-time, one alternative is to do this introspection at compile time.

While Flutter limits reflection at run-time, tools like source_get rely on two low-level Dart packages:

• build: "Defines the basic pieces of how a build happens and how they interact."
• analyzer: "This package provides a library that performs static analysis of Dart code. It is useful for tool integration and embedding."

Build provides an interface to implement builders: classes that provide an entry point to the compilation process, allowing us to create files as a result of our source code.

The analyzer package provides tools to retrieve information about our classes: The methods and variables they contain and the annotations they use, among others.

The tool build_runner (which is mentioned in Mockito's documentation) uses these two low-level packages to generate source code.

High-level process: json_serializable

At a high level, the json_serializable package performs the following steps:

• Have developers annotate classes with @JsonSerializable. The annotated class must conform to a set of guidelines for the process to work correctly.
• Have developers run the build_runner tool to generate the code that allows these classes to serialize and de-serialize JSON strings.

At a lower level, what the source code generation process is doing is the following:

• Developers execute the build_runner tool and, in turn, build_runner executes json_serializable builder classes.
  • The builder uses the build to define what files it will read from the source code, and what files it will generate.
  • The builder uses analyzer to find all classes annotated with @JsonSerializable.
  • The builder uses analyzer to find all the attributes and constructors for the serializable classes.
  • The builder creates Dart classes for serializing and deserializing JSON strings into the attributes it found with analyzer.
  • The builder cleans and puts these new classes in the same package as the annotated classes as .g.dart files.
  • These .g.dart files are parts, which allow you to split a class into multiple files.

Following the example from json_serializable's docs, we annotate the following class contained in example.dart:

@JsonSerializable()
class Person {
  /// The generated code assumes these values exist in JSON.
  final String firstName, lastName;

  /// The generated code below handles if the corresponding JSON value doesn't
  /// exist or is empty.
  final DateTime? dateOfBirth;

  Person({required this.firstName, required this.lastName, this.dateOfBirth});

  /// Connect the generated [_$PersonFromJson] function to the `fromJson`
  /// factory.
  factory Person.fromJson(Map<String, dynamic> json) => _$PersonFromJson(json);

  /// Connect the generated [_$PersonToJson] function to the `toJson` method.
  Map<String, dynamic> toJson() => _$PersonToJson(this);
}

After running build_runner, the following file example.g.dart is created:

part of 'example.dart';

Person _$PersonFromJson(Map<String, dynamic> json) => Person(
      firstName: json['firstName'] as String,
      lastName: json['lastName'] as String,
      dateOfBirth: json['dateOfBirth'] == null
          ? null
          : DateTime.parse(json['dateOfBirth'] as String),
    );

Map<String, dynamic> _$PersonToJson(Person instance) => <String, dynamic>{
      'firstName': instance.firstName,
      'lastName': instance.lastName,
      'dateOfBirth': instance.dateOfBirth?.toIso8601String(),
    };

All we have to do is import example.g.dart into example.dart and we will have access to the private functions _$PersonFromJson and _$PersonToJson.

Advantages and disadvantages

The lack of reflection forces developers to be explicit about the types and use cases they need to support. However, we can still rely on generics to create reusable code.

Generating source code reduces the number of manual work developers need to do for repetitive tasks. Without source_gen and json_serializable, we would have to manually create toJson and fromJson functions listing every attribute for each class. There are alternatives for this too, using generics and inheritence. However, it still requires a good amount of manual work.

While generating source code reduces this manual work, it introduces a layer of complexity to our project: Now we have a big chunk of source code that will not be available for inspection until we run the build tool. We can always check-in the generated code into version control, helpings better track changes done by re-running the build tool.

But the main disadvantage is the added complexity. Source code generators depend on the code having very specific conditions (e.g. having public constructors, follow name conventions) which may cause the build process to fail when not met. It's easy to find ourselves spending hours of our day just to figure out that the builder wasn't configured correctly.

What's next

In the next blog post, we will create a simple plugin to build source code for Flutter projects. We will start deep-diving into the build process and how to create reusable modules that can be imported by other projects, just like json_serializable or mockito do.