Now that you're all setup with Maestrano, it's time to take a look at the external application you are integrating with. Hopefully it has one or several gems for both the authentication process and the API calls. Understanding the API you integrate with is very important, so you need to have a read through their documentation.
You will probably have to request API keys and adds them to the application.yml alongside the Maestrano's ones.
Authentication
You need to implement an authentication process to get a token and be able to do API calls on behalf of the user.
The connector engine is thought to be able to use oauth authentication processes, and an oauth_controller.rb is provided as an example. Please note that it is only an example and you will have to implements most of it, as well as create the needed routes, and use them in the provided view.
For more detailed information, visit: Authentication strategies
If all went well, you should now be able to use the 'Link this company to...' link on the home page. Congrats!
Preparing synchronizations
The aim of the connector is to perform synchronizations between Connec!™ and the external application, meaning fetching data on both ends, process them, and push the result to the relevant parties. The Connec!™ part and the synchronization process itself is handled by the framework, so all you have to do is to implement some methods to bridge the calls to the external application.
External.rb
The first file to look for is models/maestrano/connector/rails/external.rb
. It contains three methods that you need to implement:
external_name, which is used for logging purpose only, and should only return the name of the external application, e.g.
def self.external_name
'This awesome CRM'
end |
get_client, which should return either the external application gem api client, or, in the worst case, a custom API client. e.g
See the section below for a general overview and guidelines
def self.get_client(organization)
MyAppAPIManager.new(organization)
end |
entities_list that defines the list of all the entities synchronized by the connector
def self.entities_list
%w(item person payment order)
end |
Entity.rb
The second file is models/maestrano/connector/rails/entity.rb
. It contains some methods to retrieve and send data to the external application API, as well as some helpers methods.
The details of each methods are explained in the entity.rb file provided, and you can have a look at the example section.
FAQ
If creation timestamps are available they can be used as a fallback. In the worst case scenario use Time.now |
If update timestamps are available they can be used as a fallback. In the worst case scenario use Time.now |
If the API does not allow filtering on timestamps or does not expose them, there is no way to filter only updated entities during a synchronization process. In this case, all entities will be re-imported which may impact performances. |
Entities shared and naming conventions
When creating your files, some guidelines can be followed in order to keep consistency and avoid confusion.
When naming your ruby files, the convention is to use the Connec! endpoint name. I.e. if you are mapping People (Connec!) to Contacts (YourApplication) the ruby file will be named person.rb following Rails conventions. Your class skeleton would then look like this: class Entities::Person < Maestrano::Connector::Rails::Entity
def self.connec_entity_name
'Person'
end
def self.external_entity_name
'Contact'
end
def self.mapper_class
PersonMapper
end
# Rest of the code
end
|
Please refer to Maestrano's open source repositories for more examples. Snippets are taken from the BaseCRM repository
If you need to implement complex entities in YourApplication, you can follow a similar process. I.e. if you are mapping People and Organizations (Connec!) to Contacts (YourApplication) the ruby file for the complex entity would be named person_and_organization.rb . class Entities::PersonAndOrganization < Maestrano::Connector::Rails::ComplexEntity
def self.connec_entities_names
%w(Person Organization)
end
def self.external_entities_names
%w(Contact)
end
# Rest of the code
end
|
You will then have three sub-entities named person.rb , organization.rb and contact.rb respectively. The two mapper files for sub-entities will be named person_mapper.rb and organization_mapper.rb class Entities::SubEntities::Person < Maestrano::Connector::Rails::SubEntityBase
def self.external?
false
end
def self.entity_name
'Person'
end
# Rest of the code
end
|
class Entities::SubEntities::Organization < Maestrano::Connector::Rails::SubEntityBase
def self.external?
false
end
def self.entity_name
'Organization'
end
# Rest of the code
end |
class Entities::SubEntities::Contact < Maestrano::Connector::Rails::SubEntityBase
def self.external?
true
end
def self.entity_name
'Contact'
end
def self.mapper_classes
{
'Person' => Entities::SubEntities::PersonMapper,
'Organization' => Entities::SubEntities::OrganizationMapper
}
end
def self.references
{
'Person' => Entities::SubEntities::PersonMapper.person_references,
'Organization' => Entities::SubEntities::OrganizationMapper.organization_references
}
end
def self.object_name_from_external_entity_hash(entity)
if entity['is_organization']
entity['name']
else
"#{entity['first_name']} #{entity['last_name']}"
end
end
end |
|
API Client
If YourApp does not provide a Gem to query the API you would need to create your own.
GUIDELINES
Most APIs support pagination options. The following examples assume you will be using RestClient [https://github.com/rest-client/rest-client]
The typical and easiest way to implement a pagination option in your client is possible when the response provided by the API implements '[prev_page]' and '[next_page'] fields: entities = results_of_your_first_call
while your_saved_response['next_page'] # or ['some_nested_field']['next_page']
response = RestClient.get "#{your_saved_response['next_page']}", your_headers
entities.concat YourDataParser.parse(response.body) # JSON.parse or equivalent if the API returns json
raise 'No response received while fetching subsequent page' unless response && !response.body.blank?
break if last_synchronization_date && entities.last['updated_at'] < last_synchronization_date
your_saved_response = JSON.parse(response)['the_field_containing_next_page'] # The loop will break when '[next_page]' is nil
end
entities |
If YourApp API does not provide meta fields for pagination or uses a different standard you will have to implement a similar logic: # Assuming a first successefull response and data model:
{
"Pagination": {
"NumberOfItems": 14,
"PageSize": 200,
"PageNumber": 1,
"NumberOfPages": 1
},
"Items": [
{1},
{2}
]
}
# Also assuming the API call to get a specific page is:
{YourAppUrl}/{entity_name}/{page_number}
data = JSON.parse(response.body)
pagination = data['Pagination']
entities = data['Items']
if pagination
while pagination['NumberOfPages'] > pagination['PageNumber']
next_page = pagination['PageNumber'] + 1
response = RestClient.get("base_url/#{entity_name}/#{next_page}", query)
data = JSON.parse(response.body)
pagination = data['Pagination']
entities.concat data['Items']
end
end
entities |
|
The Connector Framework supports two options that enable batched calls: __skip and __limit They can be used to implement specific queries: batched_call = opts[:__skip] && opts[:__limit]
# You can assign the values based on YourApp API standards (i.e. top/skip)
query_params = {top: opts[:limit], skip: opts[:skip] }
response = RestClient.get "https://base_url/#{entity_name}?#{query_params.to_query}", headers_get
|
It is possible to set the batch_size for the first synchronization in settings.yml first_sync_batch_size: 25
|
|
Two custom classes are available at the moment to handle specific errors: Maestrano::Connector::Rails::Exceptions::EntityNotFoundError
# and
Maestrano::Connector::Rails::Exceptions::ExternalAPIError
|
Use ::EntityNotFoundError if the external API does not provide web-hooks for deletion and entities are not set to INACTIVE (or similar) but deleted entirely An attempted update on a non existent entity will have to raise this specific error. I.e.: rescue => e
if e.class == RestClient::ResourceNotFound
raise Maestrano::Connector::Rails::Exceptions::EntityNotFoundError.new "The record has been deleted in MyApp"
else
Maestrano::Connector::Rails::ConnectorLogger.log('warn', @organization, "Error sending entities", {external_entity_name: external_entity_name, external_id: external_id, message: e.message, backtrace: e.backtrace})
end
|
Use ::ExternalAPIError for all the errors returned by the external API. |
When implementing the OAuth WebServer flow, you can use this simple implementation to make sure your token is refreshed when needed:
# In your API client
def refresh_token(organization)
# the logic here would be ideally extracted to another Object/Service
body = {
"grant_type" => "refresh_token",
"refresh_token" => "#{organization.refresh_token}"
}
headers = {
"Content-Type" => "application/json",
"Authorization" => "Basic #{ENV['your_api_key']}:#{ENV['your_client_secret']}}"
}
response = RestClient::Request.execute method: :post, url: "https://your_app_url/oauth2/token",
payload: body.to_json, headers: headers
parsed_response = JSON.parse(response)
organization.update(
oauth_token: parsed_response['access_token'],
refresh_token: parsed_response['refresh_token']
)
end
def with_token_refresh
yield
rescue
refresh_token(organization)
yield
end
|
Now it is sufficient to wrap your API calls in a block:
def get_entities(entity_name, opts = {}, last_synchronization_date = nil)
with_token_refresh do
# Your API call and logic
end
end
|
|
In the connector framework last_synchronization_date is the parameter used to implement date filtering: Your client should provide a way to limit the calls using the date provided and retrieve the entities that have been modified only after the specified date: def get_entities(entity_name, opts = {}, last_synchronization_date = nil)
with_token_refresh do
# Usually, if last_synchronization_date is not nil you should use the API filtering capabilities (if provided)
# i.e. add to the query string "update_at=last_synchronization_date"
# If the API does not support filtering by date, you might fetch the data ordered by updated_at(descending) and limit the number of records.
# i.e. Break when entities.last['updated_at'] <= last_synchronization_date
end
end
|
|