Read actions

A read action reads data from your customer’s SaaS and send them to Destinations that you define.

​

Defining reads

To read an object, you need to specify:

• objectName: to indicate which object you’d like to read. This should match the name of the object in the official documentation for the SaaS API.
• destination: the name of the destination that you’ve defined
• schedule (optional): how frequently the read should happen. This value must be a schedule in cron syntax, and can be as frequent as every 10 minutes. If you do not define a schedule, you can explicitly trigger reads via API.
• backfill (optional): whether Ampersand should read historical data when a customer installs an integration. See Backfill behavior for details. If you omit this, it means that a backfill won’t happen at the time of installation, but you can still trigger a backfill later via API.

...
     objects:
        - objectName: lineItem
          destination: lineItemWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          backfill:
            defaultPeriod:
              days: 30
          ...

​

Required fields and optional fields

Fields can be either be required or optional. If a field is required, then all users who install this integration will need to give your app read access to that field. If a field is optional, then users can choose whether they’d like your app to read that field. For these fields, you will specify:

• fieldName: the name of the field from the official SaaS API documentation.

   objects:
        - objectName: contact
          destination: contactWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          requiredFields:
          - fieldName: firstname
          - fieldName: lastname
          - fieldName: email
          optionalFields:
          - fieldName: company

​

Allow users to pick from all fields

If you want to give your user the option to pick from any of the fields in their object, use optionalFieldsAuto: all. The UI component will populate a list of all the fields pulled from that object (including custom fields) and allow them to pick which ones your app will be able to read.

      objects:
        - objectName: contact
          destination: contactWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          requiredFields:
          - fieldName: firstname
          - fieldName: lastname
          - fieldName: email
          # All other fields are optional
          optionalFieldsAuto: all

​

Object and field mappings

You might want to ask your users during the set up of the integration to map a field (standard or custom) to a concept in your product, because your various customers might be using different fields for the same purpose. You can also predefine object and field mappings that do not involve user interaction. Learn more in Object and Field Mapping.

​

Backfill behavior

Backfill behavior describes whether Ampersand will do an initial read of your customer’s historic data when they connect their SaaS instance, and how far back data will be read. For example, if your integration reads a customer’s contacts stored in their CRM, you can configure whether you want to only read new and updated contacts going forward, or if you also want to do an initial backfill of the pre-existing contacts in their CRM.

​

No backfill

If you only want to read new and updated records moving forward and do not wish to read any pre-existing records, then you can simply omit the backfill key in the integration definition, or you can write 0 days as the default period.

      objects:
        - objectName: contact
          backfill:
            defaultPeriod:
              days: 0 # Omitting backfill object will also default to 0 days.
          ...

​

Full historical backfill

If you want to do a full backfill of all the existing records when a customer connects their SaaS instance, set fullHistory to true. If you have customers that have large SaaS instances, please ensure that your webhook endpoint can handle a high number of messages in quick succession. You may find it helpful to use a webhook gateway solution like Hookdeck.

      objects:
        - objectName: contact
          backfill:
            defaultPeriod:
              fullHistory: true
          ...

​

Limited time backfill

You can select a specific time frame for backfill, such as “the last 30 days” or “the last 90 days”. Here’s an example of how to do so:

      objects:
        - objectName: contact
          backfill:
            defaultPeriod:
              days: 30 # Backfill the last 30 days of data
          ...

​

Trigger a read

You can call the trigger read API when you want Ampersand to read data for a particular customer. The data will be sent asynchronously to your destination. You can either use this API alongside a scheduled read (defined by the schedule field in the amp.yaml), or you can exclusively use the trigger read API without defining a schedule.

There are 2 types of reads that can be triggered:

• Full historic read

  {
    groupRef: "xye23r3df",
    mode: "async"
  }

• Read all records from a specific timestamp to now

  {
    groupRef: "xye23r3df",
    mode: "async",
    sinceTimestamp: "2012-04-23T18:25:43.511Z"
  }

​

Delivery modes

Ampersand sends data to destinations that you specify in your manifest file. Please refer to webhook destinations for more information on the data schema of the webhooks.

You can define the delivery mode within the manifest file.

​

Auto

This is the default delivery mode if you do not specify one. When you set the delivery mode to auto, Ampersand will automatically send results to your destination as it reads new data from the SaaS instance.

specVersion: 1.0.0
integrations:
  - name: salesforce-buffered-read
    provider: salesforce
    read:
      objects:
        - objectName: contact
          destination: contactWebhook
          schedule: "*/30 * * * *" # Data will be read every 30 minutes
          delivery:
            mode: auto # Data is delivered as soon as it is available

​

On request

When you set the delivery mode to onRequest, Ampersand will not send webhooks as it reads new data, but only when you request for more results. This is useful when you want to control the rate at which you receive data. For precise control over how much data you receive each time, you can configure the page size, which specifies the number of records to send in each webhook payload.

Please note that the page size is a maximum, and the actual number of records in each payload may be less if there are fewer records available in the SaaS instance. Currently, may configure a page size between 50 and 500. Please reach out to us if you need this to be lower or higher.

Here’s an example of how to configure the delivery mode to onRequest for a Salesforce Contact object, with a max page size of 50 records per webhook message:

specVersion: 1.0.0
integrations:
  - name: salesforce-buffered-read
    provider: salesforce
    read:
      objects:
        - objectName: contact
          destination: contactWebhook
          schedule: "*/30 * * * *" # Data will be read every 30 minutes
          delivery:
            mode: onRequest
            pageSize: 50

​

Requesting results

When you are ready to receive & process webhook messages, you can call the deliver results endpoint, which will asynchronously send the stored results to the configured destination.

For example, if you are ready to receive and process a maximum of 300 records, and you have a pageSize of 50 in your amp.yaml, you should request 6 pages (300 divided by 50).

​

What happens if you request for more pages than are available?

If you request for more pages than are available, Ampersand will first send you all the available records at the time of your request. Then, as it reads new data, it will send you the new records as they become available until you have received your requested number of pages.

For example, if you request for 10 pages of results, but there are only 4 pages available at the time of your request, Ampersand will send you the 4 pages immediately. If Ampersand reads 20 pages of results in the next scheduled read, Ampersand will send you 6 pages more, resulting in a total of 10 pages of results being delivered. Ampersand will then hold the next 14 pages until you request for more results.

​

Full example

specVersion: 1.0.0
integrations: 
  - name: readSalesforce
    provider: salesforce
    read:
      objects:
      
        - objectName: account
          destination: accountWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          # Read all accounts when integration is installed
          backfill:
            defaultPeriod:
              fullHistory: true
          requiredFields:
            - fieldName: id
            - fieldName: name
            - fieldName: industry
          optionalFields:
            - fieldName: annualrevenue
            - fieldName: website
                
        - objectName: contact
          destination: contactWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          # Read contacts from the last 30 days when integration is installed
          backfill:
            defaultPeriod:
              days: 30
          requiredFields:
            - fieldName: id
            - fieldName: firstname
            - fieldName: lastname
            - mapToName: pronoun
              mapToDisplayName: Pronoun
              prompt: We will use this word in emails we send out.
          # All other field are optional
          optionalFieldsAuto: all