02/20/2019 | News release | Archived content
Microservices architecture is very popular these days, and is coming into increasing use at Credit Karma. While there are many reasons to develop microservices, their use comes with a number of potential pitfalls. This article describes a couple of those drawbacks and what we're doing about them.
Our first microservices presented REST APIs. REST provides a uniform interface, statelessness and cacheability - all valuable things in a microservice. The main drawback of REST services concerns their brittleness. It's difficult to upgrade and version them without breaking other services. Mechanisms like RAML and Swagger patch over the problem, but don't really provide any guarantees. Most of the solutions I saw are limited to describing your existing API, which does not help with its development and maintenance.
One of the advantages of microservices is that they're loosely coupled, which makes development and deployment more flexible, but also impacts reliability. In order to mitigate this problem, our engineering teams are following a design-by-contract approach by building Thrift microservices and providing a GraphQL API for our front-end applications. So how does that help make the architecture more reliable?
Let's introduce the actors in the diagram below:
Consumer requests to providers are mediated by contracts. A contract is a strongly typed interface between consumer and provider; it describes all possible request/response transactions. While our company is using Thrift and GraphQL, those are not the only possible options.
What do you get by following this approach?
Contracts provide compelling advantages for integration testing. As a matter of fact, you might enjoy J.B. Rainsberger's awesome talk, 'Integrated Tests Are a Scam,' in which he claims that an 'integrated test with real services is not only ineffective, but rather harmful for your project.'
This is where contracts come to the rescue - rather than testing against actual services, you can test your service against its contracts. This is reminiscent of test-driven and behavior-driven development. Integration tests, which respond with predefined data constrained by the contracts, can mock the client. But what should you do if you don't have the time or resources to create and test the mocks before you can use them to test your code?
That's when you can use Mimic to have the convenience of contract-based development coupled with generated test data.
What exactly is Mimic? It's an open-source tool that I created as a member of the Developer Efficiency team at Credit Karma. It started as a POC to help our engineers be more productive by replacing service dependencies. It's currently used on a daily basis by multiple teams and we're ready to share it with the world. Technically, Mimic is a set of NPM libraries, a CLI tool, and a desktop application built around the idea of faking a real service by implementing its contract. Currently, Mimic supports GraphQL, Thrift and REST services.
So why would you need this tool in the first place? Though faking services is pretty trivial, you still need to provide responses for every endpoint. Mimic can read your service definition and automatically generate responses for you. It does this by evaluating the type structure of the responses, and at the scalar leaves of the structure, it generates default values based on the declared type of the data.
For instance, you might use Mimic …
Let's see it in action. We'll start by defining a new service.
Let's try a GraphQL service
Choose a .graphql file with your schema definition. You can also select multiple files and directories. In this case, Mimic will recursively process all definitions and combine them into a single contract definition.
Name your service and choose a port number.
When the service is created, it appears under the GraphQL Services menu and can be turned on or off. The page will display the full service documentation and configuration.
By default, Mimic will generate valid responses for you, but if you want to define your own, just click the 'Add' button below the Responses table.
You can choose between the JSON and Tree views. The Tree view provides dropdowns with all available options for the current node. In the JSON view, you can change the response and it will be immediately validated against the contract.
Let's test a response.
You can also see this request in the Requests view
After you have all services and responses defined, you can export and share them with your co-worker or use them for Continuous Integration.
Choose the services you want and click 'Export'
Mimic will generate a .mimic file with the selected service definitions, their state and their defined responses. This file is in JSON format, so you can view and edit it as needed. You can share this file and import it via the File/Import Services menu, or you can use it in the Mimic CLI, which represents the headless part of the Mimic application.
To install the CLI version, run 'npm install -g @creditkarma/mimic-cli'. You can then run the mimic command against the exported file. It will fake the exported services and report all of the request and response data in JSON format.
While Mimic doesn't have features to cover every use case, we are working to make it more useful with every commit. If you are interested, please fork the github repo and get involved.