Multi-Marketplace Integration
1 Summary
The goal of this paper is to provide tips and good practices on how to perform a multi-marketplace integration with Maestrano, i.e be able to handle multiple Maestrano-powered marketplaces including Maestrano.com.
The Maestrano delivery network is essentially composed of two types of marketplaces:
- Enterprise Network: these enterprise networks run the Maestrano infrastructure in their private cloud. In order to publish your application to these marketplaces you will need to
- have an agreement with them (which we can facilitate),
- have your app listed on their marketplace (Maestrano Hub),
- link your production environment to their marketplace.
- Express Network: these marketplaces run their own platform using the Marketplace as a Service (MaaS) offering powered by Maestrano's cloud. Being an app provider on maestrano.com makes you automatically available on these marketplaces - meaning you do not need to do anything on your side. Please note that the marketplace owners may choose to restrict their offering to a limited number of apps.
The diagram below provides a general overview of the Maestrano Enterprise Delivery Network:
2 Principles
When doing multi-marketplace integrations, three concepts need to be considered and properly scoped: Configuration, Routes and Models.
Configuration scoping
Your application should retrieve the automatic configuration from the developer platform configuration endpoint, providing its environments credentials. Your application should then be able to use the right marketplace API configuration (e.g.: which REST API host to use) based on the context of a controller action and/or database model. Multi-marketplace configuration is achieved by associating a configuration preset to a "marketplace key" (unique identifier, as an example: 'maestrano') which will then be used to tag models and to parameterize routes.
Route scoping
One way of understanding the context of incoming traffic is to properly scope urls using the "marketplace key" mentioned above. E.g.: Webhooks coming from Maestrano should hit a url like "/maestrano/some-webhook" while webhooks coming from another marketplace should look like "/somemarketplace/some-webhook". In a more general manner, multi-marketplace URLs should be parameterized using a marketplace key parameter: "/#{marketplace}/some-webhook" or "/some-webhook?marketplace=#{marketplace}"
Model scoping
When storing records related to Maestrano and/or Maestrano Enterprise Marketplaces, your application should be able to keep track of which marketplace and which marketplace customer account this record is related to. This is usually done by storing a marketplace key - and marketplace customer account id - on the record or in an identity map table (see section 4).
3 Example of multi-marketplace integration (pseudo code)
The example below gives a high level overview of what your code may look like in the context of a multi-marketplace integration. If you have already done the integration to maestrano.com the code below will look familiar and essentially intends to show how you can "extend" your current integration to make it multi-marketplace.
The code samples show you the general steps involved in getting a basic multi-marketplace setup for Single Sign-On, Single Billing and Connec data sharing. Note that for this example, we assume the use of one of Maestrano's SDKs - available in Ruby, Java, PHP and .NET.
3.1 Configuration
The configuration step involves declaring multiple marketplace configurations, parameterizing the routes that will be used by our controllers (SSO, Connec etc.) and adding a metadata endpoint to expose our marketplace specific configuration.
# This piece of code should be put in an initializer. # # With the Maestrano SDKs, you have the ability to fetch as many marketplace configurations your environment is listed on. # # Fetch the automatic configuration # # Content of your ENV variables: # export MNO_DEVPL_HOST=<developer platform host> # export MNO_DEVPL_API_PATH=<developer platform path> # export MNO_DEVPL_ENV_NAME=<your environment nid> # export MNO_DEVPL_ENV_KEY=<your environment key> # export MNO_DEVPL_ENV_SECRET=<your environment secret> # Maestrano.autoConfigure
# # Let's create parameterized routes # # The single sign-on routes will be used by enterprise marketplaces to trigger and complete SSO handshakes route "/mno-enterprise/:marketplace_key/saml/initialize" to "SamlSsoController" on action "initialize" route "/mno-enterprise/:marketplace_key/saml/consume" to "SamlSsoController" on action "consume" # The Account Webhook routes notify you of groups being removed or users being removed from groups route "/mno-enterprise/:marketplace_key/account/group/:id" to "AccountWebhookController" on action "destroy_group" route "/mno-enterprise/:marketplace_key/account/group/:group_id/user/:id" to "AccountWebhookController" on action "remove_user" # The Connec!™ webhook route will be used by enterprise marketplaces to POST data sharing notifications route "/mno-enterpise/:marketplace_key/connec/receive" to "ConnecWebhookController" with action "receive"
3.2 Single Sign-On
OpenID also available!
The example below assumes you are using our SDK for Single Sign-On which is based on SAML 2.0.
# # This controller handles the Single Sign-On handshake # class SamlSsoController # The 'initialize' controller action responds the following route # GET /mno-enterprise/:marketplace_key/saml/initialize # # The goal of this action is to trigger the Single Sign-On handshake # between the marketplace platform and your application function initialize # Retrieve the marketplace key from the URL parameters marketplace_key = params['marketplace_key'] redirect_to MaestranoSamlRequest.with(marketplace_key).new(params).redirect_url end # The 'consume' controller action responds to the following route # POST /mno-enterprise/:marketplace_key/saml/consume function consume # Retrieve the marketplace key from the URL parameters marketplace_key = params['marketplace_key'] # Process the response saml_response = Maestrano::Saml::Response.with(marketplace_key).new(params[:SAMLResponse]) # Reject if invalid unless saml_response.is_valid? redirect_to "/some/error/path" end # Extract information from the response user_attributes = Maestrano::SSO::BaseUser.new(marketplace_key, saml_response).to_hash_or_associative_array group_attributes = Maestrano::SSO::BaseGroup.new(marketplace_key, saml_response).to_hash_or_associative_array # Find/create the user and the organization # The creation or retrieval of records should be scoped to a specific provider (marketplace_key) user = User.find_or_create_for_maestrano_marketplace(user_attributes, marketplace_key) organization = Organization.find_or_create_for_maestrano_marketplace(group_attributes, marketplace_key) # Add user to the organization if not there already unless organization.has_member?(user) organization.add_member(user) end # Sign the user in and redirect to application root # To be customised depending on how you handle user # sign in and sign_user_in(user) redirect_to "/some/post-login/path" end end
3.3 Account Webhooks
# # This controller handles notification of people leaving a group (remove_user action) or companies # cancelling their subscription to your service (destroy_group) class WebhookAccountController # The 'destroy_group' controller action responds the following route # DESTROY /mno-enterprise/:marketplace_key/account/group/:id function destroy_group # Retrieve the request parameters marketplace_key = params[:marketplace_key] group_uid = params[:id] # Authenticate request as usual unless Maestrano.with(marketplace_key).authenticate(http_basic['login'],http_basic['password']) render json: "Unauthorized, code: '401' end # Retrieve the group/company organization = Organization.find_by_marketplace_and_uid(marketplace_key,group_uid) # Destroy it organization.destroy end # The 'destroy_group' controller action responds the following route # DESTROY /mno-enterprise/:marketplace_key/account/group/:group_id/user/:id function remove_user # Retrieve the request parameters marketplace_key = params[:marketplace_key] group_uid = params[:group_id] user_uid = params[:id] # Authenticate request as usual unless Maestrano.with(marketplace_key).authenticate(http_basic['login'],http_basic['password']) render json: "Unauthorized, code: '401' end # Retrieve the group/company as well as the user organization = Organization.find_by_marketplace_and_uid(marketplace_key,group_uid) user = User.find_by_marketplace_and_uid(marketplace_key,user_uid) # Remove the user organization.remove_user(user) end end
3.4 Billing
The code below shows how to bill an organization which has been tagged with a marketplace key. All Maestrano SDKs offer the ability to scope REST calls with a marketplace key. See the documentation of the relevant SDK for more details.
class MonthlyBillingJob # Run the billing job function run foreach organization in Organization.all() if organization.maestrano_marketplace_key != null # Use Maestrano's billing API Bill.with(organization.maestrano_marketplace_key).create(amount: $100, group_id: organization.maestrano_uid) else # For your own customers, just charge as usual organization.charge_credit_card($100) end end end end
3.5 Connec Data Sharing
The code sample below shows an example of model automatically forwarding a notification to Connec! upon save, in a multi-marketplace way.
class InvoiceModel function save return false unless this.save_to_db if this.maestrano_uid client = MaestranoConnecClient.with(this.maestrano_marketplace_key).new(this.maestrano_group_uid) client.post('/invoices', this.to_maestrano_json) end end end
3.6 Connec Webhooks
# This controller processes any data sharing notifications sent by marketplaces via # Connec! # E.g.: I receive a new invoice from Connec!™ that was created in another application class WebhookConnecController # The 'receive' controller action responds to the following route # POST /mno-enterprise/:marketplace/connec/receive function receive # Retrieve the marketplace key from the URL parameters marketplace_key = params['marketplace_key'] # Authenticate request as usual unless Maestrano.with(marketplace_key).authenticate(http_basic['login'],http_basic['password']) render json: "Unauthorized, code: '401' end # Finally, process the request for a specific marketplace MyConnecWrapperClass.process_invoice_updates(params['invoices'],marketplace_key) end end
4 Good practice: handling objects connected to third parties
You may have - or anticipate to have - a couple of integrations with platforms like Maestrano, QuickBooks, Xero, SalesForce etc. which will require you to synchronize objects using their APIs. For each of these objects, you will need to keep track of the foreign ids associated with the objects you've remotely created and/or received ("connected objects").
Question is: how to properly handle the linking of remote objects to your application models?
Maestrano's approach and recommended way of doing this is to use an identity map table (or "id map"). An id map is a join table allowing you to link any kind of model on your side to remote entities on the other side.
The id map identifies models/records in your application using the following fields:
- ID: the id of the record in your model table (e.g.: invoices table)
- Class Name: the name of the class for your model (e.g.: Invoice)
On the other side, remote entities are identified using the following fields:
- Provider: the key identifying the provider for the remote entity (e.g.: Maestrano, Acme Corp, Xero, QuickBooks etc.). For Maestrano Enterprise marketplaces, this is the marketplace key.
- Realm: the ID of the customer account (organization/company or user) who owns the remote entity. For Maestrano and Maestrano Enterprise marketplaces, this is the group id.
- Entity: the name of the remote entity. You should be able to derive the path of the entity API endpoint from this name.
- RID: the ID of the remote entity.
The diagram below summarizes the concept:
The ID map makes it easy to write reusable code for connected objects. The pseudo code below shows how one could right a module, composing trait, interface or abstract class handling all the logic.
# # This class define the actual model mapped to the identity map table # @table='id_map' class IdMapModel inherits MyFavoriteOrmBaseClass integer id string class_name string provider string realm string entity string rid # Return the resource URL of the remote entity function remote_url manager = MyApiConfigManager.get_config(this.provider) api_host = manager.api_host api_path = manager.api_root_path return host + api_path + "/" + this.entity.downcase + "/" + this.rid end end
module ConnectedObjectModule function id_maps IdMapModel.where(id: this.id, class_name: this.class.to_string) end function id_map_for(provider,realm) IdMapModel.where(id: this.id, class_name: this.class.to_string, provider: provider, realm: realm).first end end
# # Add connected object behaviour to an existing model # class MyInvoiceModel inherits MyFavoriteOrmBaseClass extend_module ConnectedObjectModule end # Retrieve a model inv = MyInvoiceModel.find(123) # Get the URL of a linked resource inv.id_map_for("maestrano","a-group-id").resource_url
Do as you see fit
The example above shows one way of handling connected objects and does not pretend to be a silver bullet. Your application may have different integration requirements. Do not hesitate to change, extend or simply adopt other patterns based on what you need to do!