nx+NestJS+MikroORM. Migrations

I recently worked on a project that involved NestJS with MikroORM running in an nx monorepository with multiple packages.

As much as I love NestJS, and MikroORM is one of my top, if not the top, favorite ORMs of all time, and I definitely see a lot of reasons in favor of nx, I'm not really into nx's setup for NestJS.

But for my debut here at dev.to, I guess I could share my hack(?)--honestly, I have no idea if it's a hack or not, but it works--for dealing with MikroORM migrations.

Yes, I am frighteningly stressed and absolutely sure that there is error upon error here. Nothing like confidence ;)

Step One: nx

I will show you how to do everything from scratch, starting with setting up nx, because this is a decent tutorial.

You can use:

npx create-nx-workspace --pm yarn

to set up a fresh nx workspace. In the example above, I chose yarn as the package manager because I like it, but it's no sin to use another, and pnpm is recommended by nx.

In the following prompts, I specified a path to my new nx workspace, and selected no stack (none):

Then I selected Package-based Monorepo because that's the type I work with most often:

I also skipped all cloud caching.

Step Two: NestJS

At https://nx.dev/nx-api/nest you can find an nx plugin for NestJS, and here you can learn that I could have set up nx with NestJS preinstalled as a default. Well.

Anyway, by running:

nx add @nx/nest 

and then

nx g @nx/nest:app my-nest-app

we will (first) install the @nx/nest plugin, and (then) create a new NestJS application in our workspace.

The Webpack configuration that comes with @nx/nest is dramatically slow compared to NestJS running with Turborepo or even Lerna, let alone standalone. Let's agree that at the very moment that's no worry, and continue with MikroORM...

...but, let's clean up first

The nx generator for NestJS created us my-nest-app and my-nest-app-e2e folders in the workspace root, which is something I don't really like, so I'm going to use a fancy nx generator and move them under the /app folder for better clarity:

nx g @nx/workspace:move --project my-nest-app --destination apps/my-nest-app

and same for the e2e application, of course.

Much better:

Step Three: MikroORM

Since this is a tutorial project, I'm going to use good old sqlite with the better-sqlite driver as described on https://mikro-orm.io/docs/quick-start#installation and https://mikro-orm.io/docs/usage-with-nestjs#installation:

yarn add @mikro-orm/core @mikro-orm/nestjs @mikro-orm/sqlite 

NestJS Configuration

I like to keep things tidy, at least when it comes to my code (as my ADHD does everything it can to keep my life under a constant spatial hurricane), so I will put the MikroOrmModule configuration in a separate directory, but first I will install @nestjs/config:

yarn add @nestjs/config 

I usually create a directory under src/config/, and now I will create a file called sqlite.config.ts:

import { BetterSqliteDriver } from '@mikro-orm/better-sqlite';
import { registerAs } from '@nestjs/config';
import * as path from 'path';

const dbPathFromRepositoryRoot = path.join(process.cwd(), 'apps', 'my-nest-app', 'src', 'common', 'sqlite', 'my-nest-app.sqlite3');

export default registerAs('better-sqlite', () => ({
  driver: BetterSqliteDriver,
  dbName: dbPathFromRepositoryRoot,
  debug: process.env.NODE_ENV !== 'production',
  autoLoadEntities: true,
  allowGlobalContext: true,
}));

and then, configure the module as following:

import sqliteConfig from './config/sqlite.config';
import { MikroOrmModule } from '@mikro-orm/nestjs';
import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';

@Module({
  imports: [
    MikroOrmModule.forRootAsync({
      imports: [ConfigModule.forFeature(sqliteConfig)],
      useFactory: (config: ConfigService) => config.get('better-sqlite'),
      inject: [ConfigService],
    }),
  ],
  providers: [],
})
export class MainModule {}

Some entity

I'm going to create (yes, Ordnung!) an AppModule, and a SomethingModule inside of it:

// src/app/app.module.ts

import { Module } from '@nestjs/common';

import { SomethingModule } from './something/something.module';

@Module({
  imports: [SomethingModule],
})
export class AppModule {}

Since I'm a real fan of keeping my code as clean as possible, I'm going to create an ISomething interface, such as:

// src/app/something/entities/something.interface.ts

export interface ISomething {
  id: number;
  nameOfSomething: string;
  isSomethingNice: boolean;
  createdAt: Date;
  updatedAt: Date;
}

and a Something entity, of course:

// src/app/something/entities/something.ts

import { Entity, EntityRepositoryType, PrimaryKey, Property } from '@mikro-orm/core';

import { ISomething } from '../dto/something.interface';
import { SomethingRepository } from '../repositories/something.repository';

@Entity({ repository: () => SomethingRepository })
export class Something implements ISomething {
  [EntityRepositoryType]?: SomethingRepository;

  @PrimaryKey()
  id!: number;

  @Property()
  nameOfSomething!: string;

  @Property()
  isSomethingNice!: boolean;

  @Property()
  createdAt = new Date();

  @Property({
    onUpdate: () => new Date(),
  })
  updatedAt = new Date();
}

I set up a repository for Something entity, as described here: https://mikro-orm.io/docs/usage-with-nestjs#using-custom-repositories:

import { RepositoryBase } from '../../../shared/db/repository/repository.base';
import { Something } from '../entities/something';

export class SomethingRepository extends RepositoryBase<Something> {}

The RepositoryBase is an official hack described on https://mikro-orm.io/docs/repositories#removed-methods-from-entityrepository-interface to keep the old repository methods MikroORM no longer keeps in repositories.

Of course, we need to load our Something entity into the SomethingModule, like:

// src/app/something/something.module.ts

import { MikroOrmModule } from '@mikro-orm/nestjs';
import { Module } from '@nestjs/common';

import { Something } from './entities/something';

@Module({
  imports: [MikroOrmModule.forFeature([Something])],
})
export class SomethingModule {}

We don't need to register our repository explicitly, as it will be registered via the entity.

Step Four: MikroORM+NestJS migrations

There isn't much written about handling MikroORM migrations with NestJS, but this part of the MikroORM documentation - https://mikro-orm.io/docs/migrations#using-the-migrator-programmatically - suggests a thing or two.

That's why I thought it would be best to have a kinder-Nest running as a migrator, so that before any nx serve my-nest-app, a migrator would have to run first.

NestJS has this feature to handle multi-application ecosystems (see https://docs.nestjs.com/cli/monorepo#cli-properties for example), but I thought combining nx with NestJS monorepo to have a small migrator application would be overkill. That's why I wrote two files:

// src/migrator/migrator.module.ts

import { MikroOrmModule } from '@mikro-orm/nestjs';
import { Logger, Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';

import sqliteConfig from '../config/sqlite.config';
import { SomethingModule } from '../app/something/something.module';

@Module({
  imports: [
    ConfigModule.forRoot(),
    SomethingModule,
    MikroOrmModule.forRootAsync({
      imports: [ConfigModule.forFeature(sqliteConfig)],
      useFactory: (config: ConfigService) => config.get('better-sqlite')!,
      inject: [ConfigService],
    }),
  ],
  providers: [Logger],
})
export class MigratorModule {}

which is a very stripped down version of the MainModule, but with all modules that register entities in the IoC of NestJS. Without this, we won't be able to generate migrations for all entities, especially with autoLoadEntities set to true.

The second file is a main.ts equivalent:

// src/migrator/migrator.ts

async function bootstrap() {
  const app = await NestFactory.create(MigratorModule);

  const logger = app.get(Logger);

  const orm = app.get(MikroORM);
  const migrator = orm.getMigrator();

  await migrator.createMigration();
  logger.debug(`Migration created (or not if no need to)`);

  await migrator.up();
  logger.debug(`Migrated!`);

  // Important! Otherwise, the migrator app won't exit, and the actual app will never start
  process.exit(0);
}

bootstrap();

And it's heavily based on the migration documentation you can find at https://mikro-orm.io/docs/migrations#using-the-migrator-programmatically.

I also had to make some changes to the sqlite.config.ts file:

import { Migrator, TSMigrationGenerator } from '@mikro-orm/migrations';

export default registerAs('better-sqlite', () => ({
  // existing config
  extensions: [Migrator],
  migrations: {
    tableName: 'mikro_orm_migrations', // name of database table with log of executed transactions
    path: dbPathFromRepositoryRoot, // path to the folder with migrations
    pathTs: dbPathFromRepositoryRoot, // path to the folder with TS migrations (if used, you should put path to compiled files in `path`)
    glob: '!(*.d).{js,ts}', // how to match migration files (all .js and .ts files, but not .d.ts)
    transactional: true, // wrap each migration in a transaction
    disableForeignKeys: true, // wrap statements with `set foreign_key_checks = 0` or equivalent
    allOrNothing: true, // wrap all migrations in master transaction
    dropTables: true, // allow to disable table dropping
    safe: false, // allow to disable table and column dropping
    snapshot: true, // save snapshot when creating new migrations
    emit: 'ts' as any, // migration generation mode
    generator: TSMigrationGenerator, // migration generator, e.g. to allow custom formatting
  },
};

and, obviously, install the:

yarn add @mikro-orm/migrations

I didn't want to mess with the standard MikroORM configuration, it worked, but I guess you can do wonders by playing around with these config settings.

Step Five: project.json.

Actually, this might be the trickiest part, because we can't just run the nest CLI. We can do this by running our /src/migrator/migrator.ts, but with ts-node and taking into account custom paths (with tsconfig-paths, if we specified any in our nx workspace (such as @marek/my-nest-app to point to the NestJS application I just created).

So we should install the following:

yarn add ts-node tsconfig-paths

(We don't need to put this in devDependencies since we are using the monorepository root, and there will be no real case where it matters whether we use dependencies or devDependencies).

Also, we should add the migrate target to our NestJS project.json workspace configuration:

{
  "targets": {
    "migrate": {
      "command": "TS_NODE_PROJECT='apps/my-nest-app/tsconfig.app.json' node --require ts-node/register -r tsconfig-paths/register apps/my-nest-app/src/migrator/migrator.ts"
    }
  }
}

Now, if we run:

nx run my-nest-app:migrate

the standalone NestJS migrator application will check for new migrations that need to be created and run, and--due to the process.exit(0)--it will exit. So we can prepare a pipeline like this

{
  "targets": {
    "serve": {
      "dependsOn": ["my-nest-app:migrate"]
    }
  }
}

And... it should work!

At least it works for me :)

Comments

[Johan Vrolix]

How do you handle relationships between entities across different libraries? I am specifically hinting to the fact you might have multiple libraries, which you don’t necessarily use in all applications, so you might end up with including an entity with a relationship through it’s library, but not including the inverse entity of a relationship of the included entity

[Marek Kapusta-Ognicki]

Well, not necessarily like that, at least to my mind. If a library has become a library, it means it has (or should have!) well-defined boundaries of its context. With a reusable UserModule we have a very strictly defined scope of what it does. Now, if we want to make users the owners of the cats, all it takes is to define the relationship in the CatEntity. First, because we don't need, at any cost, the inversion of relationship. Second, logically, UserModule should not cover any logic or scenarios involving users as the owners of the cats, because it's the app-specific domain.
So, in a hypothetical scenario where we would want to implement the logic for a user shelters a cat scenario, that scenario should probably be handled by neither UserModule, nor CatModule, but by a CatOwnerModule, with a service that first fetches the user, then fetches (or creates) the cat entity, assigns to the latter a relationship to the former, and persists the data. As such, the boundaries between contexts (user, cat, cat owner) are not leaking.
Actually, I'm rather inclined to think that if there's a matter of whether we need to define the relationship inversion on some another module, or not, it most likely means, we need another context.
Besides, the very structure of NestJS, broke down into modules, works pretty much similar to the modules as separate libraries. There's hardly any, if any at all, argument for extending in-app UserModule to handle the scenarios which are of another module's interest.

[Marek Kapusta-Ognicki]

On a margin note, in real life, it's not being a person that makes us benefit from the cats we have, or gives us some responsibility. It's the abstract notion of the ownership of a cat. We are accompanied by our cats, because we own them, not because we have two legs, eyes, passport, and a birth certificate. The cats are not petted or fed because they have whiskers, and a specific painting. It's because of a relation between them, and a human feeder, let's say that's the ownership I'm mentioning. So the ownership is something aside from being a person, and aside from being a cat, and both people and cats can function without it, they won't lose their identity, they won't become unable to live, or to produce the ID upon police officer request ;)