Support for Apollo Federation and Microservice Architecture

[johnkm516]

This is more of a question than an issue. I couldn't find a discussion board or slack, sorry if this is the wrong place to ask.

This repo is exactly the starter-kit I was looking for as it minimizes writing repetitive code that is defined in the SDL anyways, and makes full use of all generated Prisma Inputs in the resolvers.

However I am wondering how scalable is the current architecture of this project? Right now all the generated schemas are input into one application so it looks like it's a monolith. Would it be possible to separate services into different app folders, and federate them with Apollo Federation using Apollo Router? Looking over the gulpfile and the relevant code-gen I definitely think it's possible, but I was wondering if it is worth the effort, as I might be missing something (the current architecture might not be a monolith at all).

[ZenSoftware]

Hello @johnkm516. I was planning on setting up a Discord at some point, but currently here is the best place to discuss things. Yes, I was trying to reduce all the most redundant tasks possible by unifying as many systems as possible. I'm glad you see the value in it 😁.

In terms of architecture, it is a sort of monolith in that the API was intended to be a single endpoint to give access to Prisma via GraphQL. I have been looking into Apollo Federation lately and have been mulling over how it would best fit into this project. I'm trying to gain a bit more experience with federating multiple graphs to get a handle of the technology first. I am completely open to re-architecting things if it aids in making things more modular.

The original aim was to have a single javascript import endpoint in which you could access the API. This ended up as an Nx Angular library named @zen/graphql, which is where all the front-end services are being generated. It really simplifies access to the API when there is only one place to look. Though, I was worried about bloating the front-end with unused GraphQL documents that were being code generated. Fortunately there is a way to do tree shaking via webpack magic comments. So only the code that is being used is being included in the final bundle.

You can see the configuration that is generating PURE magic comments here:

zen/apollo-angular.yml

Line 12 in a1abedf

pureMagicComment: true

Again, I am a bit inexperienced with GraphQL federation and not sure what would be a generalized solution that would work well. I'd be happy to work on this issue with you. Could you elaborate a bit more on how you were envisioning the services to be separated?

[johnkm516]

Hi, @ZenSoftware,

To be honest with you, I'm a complete newbie myself. I've spent the past few months trying to figure out all the bits and pieces for a new backend architecture that my organization can adopt (my org currently uses a monolith), and it has been really really difficult trying to understand and fit pieces like Prisma, Nexus, Apollo, together. I want to use GraphQL and the power it brings with Apollo Studio and introspection, but there was so much boilerplate code compared to normal CRUD APIs I had to find a solution that auto-generated much of the boilerplate code. After cloning literally dozens of repos trying to figure all this out your repository was the closest one I came across where it's an schema first approach (where the prisma schema is the single source of truth) and generates all the typedefs and resolvers (using Prisma was important to me).

While I'm not going to be much help on the development front (at least right now as I'm still working to find a solution while simultaneously learning), I did learn a lot in the architecture side of things and the best practices on this end. The architecture I'm envisioning looks like this :

• A monorepo with a prisma folder in the root folder of the project, and subschema microservice app folders. There would be several schema.prisma files located in each subfolder of the same name of the microservice app inside the root prisma folder, and a script where building from root copies the schema.prisma files to its respective app folder and auto-generates code for each app as this repo does currently. Each of the subschema microservices would use a ApolloFederationDriver. I just tested this on your repo and it seems the buildSubgraphSchema accepts the GraphQL schema generated in app/graphql/resolvers/index.ts, then all I did was switch out the driver to ApolloFederationDriver and gql config to ApolloFederationDriverConfig and commented out the subscriptions portion (I got an error here saying ApolloFederationDriverConfig currently does not support subscriptions). While the single graphQL subschema currently works I haven't tested whether or not the federation part with another subschema actually works, I'm sure there needs to be some modification to the code generation to fit Apollo's subschema conventions. There would of course be a federation gateway service that acts as the single point of API access and handles authentication and authorization, and the supergraph schema would be a managed federation. In terms of Angular, I didn't really consider the code generation and how it would generate the the front-end services as I'm currently solely focused on the backend currently and considering the frontend as well is currently out of scope for me. I did look into it a little bit however, and perhaps https://github.com/jamiepine/prismix could be the key here? Since prismix combines the multiple schemas into one schema, the build process would be something like :

1. Copy all subschemas to their respective app prisma folders.
2. Run typedefs and resolver code generation for each of the respective apps
3. Run prismix to combine all the subschemas into a single schema
4. Run any frontend code generation that currently relies on a single schema

• The main problem with microservices and separating schemas is that there are many cases where a service has to get data from another service. My company is an apparel manufacturer, and pretty much everything revolves around purchase orders from our buyers. For example, a purchase order contains several orders with very large amounts of data, from style # (the specific apparel's code), purchase order #, all the different sizes, all the different colors, all the different quantities at different points of the manufacturing process (spreading, cutting, sewing, QA, packaging), ship dates, and we also have to keep track of TNA such as all the logistics and import of raw materials, the export of the product, and so much more (you can see why separation of concerns and modularity with a federated supergraph is extremely important for my company). So I looked into interprocess communication with Kafka, or gRPC, event driven architectures, and honestly all of this makes things too complicated with even more boilerplate code. So here's what I'm thinking :

1. Maintain separation of concerns and model out schemas using best practices of what an Apollo subgraph should look like
2. Code generate all the queries and mutations from the prisma schemas as this repo currently does. Do not put any extra logic in the resolvers or anything, the GraphQL API would only serve to represent and serve data (obviously there could be extra resolver logic here as long as it only refers to data within the bounds of the subschema itself with any basic logic within the bounds of the subschema).
3. Use a microservice orchestration solution like Prefect for all the non-CRUD logic and have it serve as the "brain" of all the main business workflows. That way there is no circular dependencies, no extra latency with hops between microservices, and no hassle in generating protobuf for gRPC or trying to manage non-graphql transporters and pub/sub logic. The orchestration solution would call the gateway and therefore the supergraph, and all the important processes would be done in the orchestration solution (which handles workers as well in a serverless manner and highly scalable), which in turn would also call back to the gateway and mutate the data of all the subgraphs simultaneously through the gateway. Debugging would also be very intuitive and business workflows will be far easier to understand, as solutions like Prefect show a visual rundown of the workflow : https://www.prefect.io/guide/blog/introducing-radar/

The microservice orchestration portion would belong to a different repo for main business processes and is outside the scope of this boilerplate project but just explaining what I'm thinking here.

Transitioning from a monolith to new technology like GraphQL and microservices and trying to decide what the "right" architecture is has been extremely difficult. There's so many different opinions and different implementations and so many different frameworks, libraries / modules after learning and researching so many different stacks and services I still haven't gotten to actually coding (It's always like, oh what's this? Is this better than the other thing? Can this fit into the stack I am currently thinking of?). While I've learned a lot the past months and formed my own opinions on what the backend architecture should look like I don't know for certain if the architecture I describe above is "correct". I'd love to discuss this with you further either here or somewhere else like discord or slack if you'd like. Let me know what you think, or my ideas are even feasible in terms of actually implementing things.

[johnkm516]

Still researching on solutions in terms of implementation, and found this :
https://github.com/0xR/graphql-transform-federation

Looks like we can apply Middleware while leaving the current code generation as is to apply Apollo directives such as @key and make all the types extendable so that other subgraphs may extend the types, after the fact.

Edit :

Found a gateway solution that comes with a whole range of features including auth and subscription and can unify not just apollo federated graphql but basically every other API as well, and composes during compile time a "virtual graph" : https://docs.wundergraph.com/docs/use-cases/api-composition-and-integration#docker-but-for-ap-is

[ZenSoftware]

I am going to rework the dev ops so it will be possible to provide arguments that specify the schema.prisma paths and out directories as arguments. It will modularize things and turn the tool from a monolith, into a more generalized tool.

I'd like to bring up some complexities in how to get the tooling for auto-completion to work with multiple GraphQL schemas within the VSCode. There will need to be multiple apollo.config.js files. I actually submitted a feature request for this a while back. This used to be pretty buggy in the past, but hopefully the team working on their extension has ironed everything out with the issues with monorepos and multiple GraphQL schemas.

I'm going to play with some of the tools that you have linked. Thanks for sharing your investigations with me. Give me a little bit to work on this and I will get back to you with a more comprehensive response. All of this should be do-able.

I will be working on this at federation branch currently. I plan to merge it into the base branch and get eventually get rid of the branch.

Here is the latest commit that contains the baseline changes that integrates ApolloFederationDriver.
e5fcdda

[johnkm516]

@ZenSoftware
I've done some further investigating.
https://github.com/0xR/graphql-transform-federation
currently is out of date and does not work. Furthermore, Wundergraph while it has a lot of features out of the box, currently does not support Apollo Federation 2.

I'm playing around with the graphql-transform-federation in an effort to update the libraries to the latest versions and adapt the code accordingly.
The reason why I think Federation 2 is a must is because of Federation 2's features; specifically, the difference in how subgraph entities are managed and how it simplifies code generation rules for our Prisma schemas. In Federation 1, there is an "originating" subgraph, and other graphs may extend the originating subgraph. In Federation 2, there is no originating subgraph, Apollo Gateway will automatically find entities of the same type and merge them.

This very much simplifies code generation, we can simply add the @key directive to every entity object that is code generated from schema. Furthermore, any schemas containing the key only (stubs that are used only to refer to other subgraphs rather than extend any additional data of the entity) can additionally have a code generation rule to add the "resolvable : false" directive :

I'm currently working on a brand new project from scratch that adds pal.js code generation and transforms the graphql schema object using my updated graphql-transform-federation. I'm ironing out all the errors currently occurring from updating all the libraries to latest. I'll let you know if I get it working.

[ZenSoftware]

Hi John. Thanks for sharing your progress. I'm glad you are investigating solutions into getting Apollo Federation v2 working for us. I agree with your analysis that we would want to be v2 compatible. I have worked tirelessly to get all of the dependencies within the project to the latest versions possible myself. We are currently blocked on upgrading to Prisma v4 due to PalJS. I was thinking of directly pulling out the Apollo SDL generation of PalJS into something we would have more control over as well. In particular we seem to need the ability to inject the appropriate @key directives to the GraphQL SDL types of the subgraphs. I'm glad you are ahead of me on this.

As I mentioned previously. I am reworking the dev ops currently. I am ripping out Gulp and refactoring things to simply use ts-node to execute .ts scripts. In the process of redoing things, I am going to rework the code generation that I am producing to be more modular. I think it might help to give you a bit of an overview of the tools currently being used to spit out code for us.

• PalJS code at prisma-tools/packages/generator/src/sdl needs to be extracted out and it would be nice to upgrade it to be Prisma v4 compatible.
• My templates zen/tools/graphql-codegen & the zen/tools/gulpfile.ts file that is soon to be refactored.
• @graphql-codegen/typescript-apollo-angular is used to produce @zen/graphql front-end code.

Of course other front-end technologies like React will be able to use our web APIs, but because there is so much technological investment in @zen/auth to handle the authorization for Angular. The first step would be to get the current auth system working with Apollo supergraphs & subgraphs to ensure correctness first.

Auth seems to touch everything, and I believe I will need to extract out the Nest modules under zen/apps/api/src/app into their own standalone node libraries that multiple Nest apps can import & configure.

Let me first get most of the dev ops reworked before I extract the Nest modules into their own standalone libraries. I will need to workout the details on how to best model authorization in our sort of federated model. I haven't worked out all the details out on this just yet. I'll report back once I have made more progress on this. 🎐

[ZenSoftware]

@johnkm516
I overhauled the dev ops. I completely scrubbed out Gulp from the project and now the build scripts are pure Node scripts written in TS. Furthermore, instead of calling the PalJS CLI, I directly import from @paljs/generator and execute the generator from there. I refactored things under generate.ts at the root of the project. All the dead code has been scrubbed out of the codebase and now it should be much clearer what is being generated.

The package.json scripts have been renamed:

• apollo:gen is now frontend:gen
• gql:gen is now api:gen

I managed to reduce the total size of the docker image as well by going over all the packages with a fine tooth comb and removing all unused packages. You can pull the latest changes from the base branch. There is more work to be done, but now at least a single application's API can be generated using the generate.ts script. We can start modularizing things now that a single application is stable.

[johnkm516]

@ZenSoftware
I updated deps of https://github.com/0xR/graphql-transform-federation and forked into my own repo. You can find it at
https://github.com/johnkm516/graphql-transform-federation

I also created another repo https://github.com/johnkm516/maple off of another template to test if the SDL transform works, and it does (querying the SDL from the Apollo sandbox shows the directive and resolve reference correctly). This portion of the code transforms the already built schema to add the key directive to the User model and resolveReference for findUniqueUser :

const federationSchema = transformSchemaFederation(schemaWithoutFederation, {
  User: {
    keyFields: ['id'],

    resolveReference(reference, context: { [key: string]: any }, info) {
      console.log(info.returnType.toString());
      return delegateToSchema({
        schema: info.schema,
        operation: OperationTypeNode.QUERY,
        fieldName: 'findUniqueUser',
        args: {
          id: (reference as UserWhereUniqueInput).id,
        },
        context,
        info,
      });
    },
  },
});

However I question the usefulness of this as this part also has to be code-generated, for all the different queries and mutations. I think it would be far more useful to code-generate the federation subschema from the get-go from paljs by extending paljs with a federation : true config and generating the resolveReferences and directives accordingly.

I'll clone your federation branch and also look into changing paljs directly. I'll be honest with you, while I've been looking into NestJS for quite a while I never actually got around to coding in it (in fact I never touched typescript before), so as I mentioned before I'm a huge newbie so my code might look weird. I like to think I'm a fast learner so I'll just dive in anyway and hopefully be of some use. I'll again update on any progress I have if I have any.

[ZenSoftware]

@johnkm516
I'm always happy to see others put in the effort to grow. How you improve as a developer is to solve challenging problems. This is certainly one of them. You seem to be competent with systems integrations, which in itself is a highly valuable skillset and carries over well to programming. Fortunately Nest really couldn't make things easier to build and run a web server. It is actually the ideal platform for newcomers. You can achieve a tremendous amount with very little code.

I changed the output directory for PalJS generated assets from /generated to /paljs. There should be zero ambiguity as to who is generating what now. You can see my little TS script that will generate the varying assets for a single Nest API.

generate.ts

const generator = new ZenGenerator({
  palConfig: require('./pal.js'),
  apiOutPath: path.join(__dirname, 'apps/api/src/app/graphql'),
  caslOutFile: path.join(__dirname, 'apps/api/src/app/auth/casl/generated.ts'),
  frontend: {
    outPath: path.join(__dirname, 'libs/graphql/src/lib'),
  },
});

await generator.run();

We can reuse this script for any number of schema.prisma files & apps. It is currently configured to output to the api application for demonstration purposes.

On another note, for any given Prisma schema there is a lot that PalJS is generating for us. It generates both the TypeScript types under resolversTypes.ts and the GraphQL SDL types within ../typeDefs.ts files. There is so much that it is taking care of for us that it feels a bit daunting to extract out the relevant bits of code from a fork. I'm concerned about losing out on the benefits of getting future upgrades as well.

Instead, maybe it would be best to keep PalJS within the pipeline and do some sort of post-processing step after PalJS that would modify the typeDefs.ts files. You can see the GraphQL SDL type of the Prisma model being defined there. A simple experiment would be to manually add @key directives to the typeDefs.ts files PalJS generates and see if it does what we think it should do.

Before I start experimenting with Apollo Federation v2 myself, I need to extract out the authorization module for Nest and turn it into a standalone library. Multiple Nest modules will be able to import and configure their own @casl/prisma abilities for their own respective Prisma schema that way. That should move us much closer to making everything fully modular.

Let me know if you run into any issues or have any questions. Cheers!

[johnkm516]

@ZenSoftware

I read through the PalJS code generation yesterday and it's actually quite easy to understand. I created my own fork and added the

extend schema
    @link(url: "https://specs.apollo.dev/federation/v2.0", import: ["@key", "@shareable"])

for federation 2 header in typedefs, as well as adding the "federation : boolean" in config of the generator class.
You can read over my changes in my fork. Given how relatively simple the code generation is (and my changes to it), I don't think my changes would be difficult to merge when PalJS eventually gets upgraded with Prisma V4, since all I'm doing is adding some extra string concatenation.

While I agree with you that our changes should be some sort of post-processing to change the generated files after they're created to not modify PalJS directly, as Apollo gets updated with new directives and Prisma also gets updated, we will have to change our post-processing accordingly; whereas modifying the PalJS generator directly allows us to use @paljs/schema and all the data models that it uses to generate. While my fork of graphql-transform-federation works after updating the dependencies, the way it's implemented it's not easy to add new directives like "@Shareable" that's currently not in the implementation, while modifying the generation from the get-go in PalJS it's a simple string concatenation so I don't have to worry about inputting a GraphQLSchema object, reading and modelling the schema to modify it, then build the modified output GraphQLSchema. So I am more leaning toward merging my changes with any future PalJS upgrades manually since my changes won't be too extensive.

Furthermore, the SDL generation actually only uses a few files and has very little cross-dependencies. It's very much self contained and the code looks extremely simple to extract. We could just extract the SDL generator, upgrade the dependencies to latest including Prisma v4, and modify it for our own use. All the files that are relevant to the SDL generation that we use is contained within :

with some utility libraries like Prettier, fs, and some custom type objects from @paljs/types.
How the SDL generation works is it takes the DMMF from @prisma/sdk (now @prisma/internals in Prisma v4) and uses it to concatenate strings and output to the files. Very simple, no other dependencies than the ones I listed. In fact I'm going to try and upgrade the deps to Prisma v4 now.

Let me know what you think.

[johnkm516]

@ZenSoftware

I upgraded all the deps to latest including to Prisma v4, and tested SDL generation. Works fine.

[johnkm516]

@ZenSoftware

I have completed modifications to the PalJS code to generate a federation subschema. I haven't tested through a gateway yet but testing the _entities(...): [Entity]! in Apollo Sandbox to emulate a federation gateway call to a subschema it seems to work. Please have a look over at my fork, npm pack it and replace the palJS with mine and try it out. Let me know what you think.

Edit : Ok never mind I'm having issues when I tried building everything from a clean proj. Need to fix deps and build issues. I'll let you know when I get everything working properly.

Edit 2 : I think it works now?
You have to npm pack the /types and /generator and add them to a local lib folder and change the package.json accordingly.

[ZenSoftware]

@johnkm516

Well done my friend. I have made substantial progress myself. Let me share with you my findings.

According to the Apollo Federation subgraph docs the signature for the __resolveReference function should take 2 paramaters.

1. Reference object
2. Context

Therefore the code should look something like this.

paljs/User/resolvers.ts

const resolvers: Resolvers = {
  // ...
  User: {
    __resolveReference(reference, { prisma }) {
      const [field, value] = Object.entries(reference).find(e => e[0] !== '__typename');
      return prisma.user.findUnique({ where: { [field]: value } });
    },
  },
};

There might be a better way to do this, but notice how I extract the @key fields and do the lookup with Prisma directly. This is how we could resolve our Prisma models given a reference object. But there may be some limitations with this approach. Let me mull this one over for a bit and I will get back to you. But the code above should function for us for now.

Furthermore, the TS type can simply be set to any for now.

resolversTypes.ts

export interface User {
  [key: string]: Resolver<any, any, any>;
  id?: Resolver<Client.User, {}, number>;
  // ...

  __resolveReference?: any
}

Because there is an index signature type of [key: string]: Resolver<any, any, any>, it is making it difficult to explicitly type the __resolveReference function without breaking other types. It will be fine to set it to any.

The __resolveReference generated will be consumed by the Nest GraphQL resolver like so. Reference the Nest Federation docs on the @ResolveReference decorator for more info.
graphql/resolvers/User.ts

@ResolveReference()
resolveReference(@Parent() reference, @Context() ctx: IContext) {
  return resolvers.User.__resolveReference(reference, ctx);
}

My code gen templates will need to include the above code for us when generating the Nest resolvers. I'll be sure to do that once we figure out our situation with the PalJS fork.

As you suggested, I used npm pack to install a local package and then changed the imports within tools/zen-generator.ts to utilize your forked PalJS packages. Then I updated the pal.js config at the root to include your newly added federation option you have implemented. Testing things out, your modified packages seem to compile and execute just fine.

pal.js

module.exports = {
  schema: 'prisma/schema.prisma',
  backend: {
    generator: 'sdl',
    output: 'apps/api/src/app/graphql/paljs',
    doNotUseFieldUpdateOperationsInput: true,
    federation: true,
  },
};

Furthermore, I thought that you should also know that I added the GraphQL SDL definitions for the @key and @link directives. The global-schema.gql.ts has been updated within the federation branch to include the new federation directives that we are using, specifically @key and @link. I found the standard SDL signatures at Apollo Federation subgraph specification and copied them over so that Nest GraphQL schema validator is aware of them.

By the way, there is a small bug at johnkm516/prisma-tools/packages/generator/src/sdl/index.ts#L123
whereby it is outputting an uppercase model name. It should be producing the lowercase variant here. You may need to move the code out of createFiles(...) and into a spot where you will have access to the model.name.

You can pull the latest changes from the federation branch. It now contains the compatibility changes needed. Do let me know if you need clarification on anything here.

[johnkm516]

@ZenSoftware
I made all the modifications you listed.

• Commented out the Fed2 directives header in typedefs
• Changed the interface to __resolveReference?: any
• Changed the code generation to

const resolvers: Resolvers = {
  // ...
  User: {
    __resolveReference(reference, { prisma }) {
      const [field, value] = Object.entries(reference).find(e => e[0] !== '__typename');
      return prisma.user.findUnique({ where: { [field]: value } });
    },
  },
};

I'm currently away on holiday (it's a Thanksgiving-like long weekend in my country) so I won't be able to work until the 13th.
Let me know if you want to change anything with the __resolveReference if you have any better ideas on how we should go about it. During some downtimes over the long weekend I'm going to do some research on auth and how the auth with the federated services and gateway should be handled. I am also thinking about some very granular permissions capabilities; here's what I'm thinking about the design :

At the very top level is the Tenant. Separating the databases even further from DB per service to DB per (tenant + service), it would be easier to also have something like a test tenant when the data gets more and more complicated.

The second level would be Role. Superadmin would have access to all the tenants and all permissions, and admin with all permissions to a specific tenant, and a variety of user Roles such as Finance, Sales, etc.

The third level will be Organizational Group. in a hierarchical tree structure as all organizations would have. For example, if I belong in the X Organization, in the IT department, and in team A of the IT department, I should have all the permissions that belong to X Organization, the IT department, and team A. Team B of the IT department will have all the permissions of X Organization and the IT department, but not necessarily permissions specific to team A, and vice versa. I plan on creating a separate service that syncs the organizational hierarchy with the company's Active Directory later down the road.

Let me know what you think about this data model. If you agree with it I'll make a pull request to this repo later when I start work on it. Figuring out auth comes first though.

[ZenSoftware]

@johnkm516
The latest commit c010092 is setup to utilize your changes. I've tested it and everything seems to work as expected. Nice work.

I would suggest reading up on the Casl docs and familiarize yourself with the ability builder. From there you can play with casl-ability.factory.ts to learn how to grant/deny access. The way I plan to refactor the Nest auth module would be such that other applications can provide their own CaslAbilityFactory. The auth solution I have created for Nest pretty much encapsulates everything that you could possibly need to describe granular permissions. Here are my docs on auth if you haven't seen it yet.

As for how this will work in our federated model, I was hoping we would be able to just forward the HTTP Authorization header that contains the JWT bearer token to the next Nest api (subgraph). That subgraph will itself create a Casl ability for the user that was extracted from the JWT, and then run their own Casl ability over the user. Architecturally, this is how I have set things up to work already. So I don't believe there is much work to be done. It should be possible to implement your security model that you have described with my auth directives. It's just a matter of testing all of this to iron out any of the kinks.

I'll respond back with more details on auth once I work more on it myself. Have a great long weekend. Things are in a very good place for us thus far. We have the core of the functionality working! 🥂

Edit
Please read up on Nest Guards as a prerequisite to everything. You'll be able to understand how the directives actually are achieving their magic that way. I configured the repo to work with VSCode breakpoints, so I would suggest setting breakpoints in casl-ability.factory.ts and then running queries to see if your auth logic is doing what you expect it to. Here is how you pass the Authorization header via Apollo Studio.