A read action reads data from your customer’s SaaS and send them to Destinations that you define.

​
Defining reads

To read an object, you need to specify:

  • objectName: to indicate which object you’d like to read. This should match the name of the object in the official documentation for the SaaS API.
  • destination: the name of the destination that you’ve defined
  • schedule: how frequently the read should happen. This value must be a schedule in cron syntax
  • backfill (optional): whether Ampersand should read historical data when a customer installs an integration. If omitted, then Ampersand will only incrementally read data that has been created or updated after that point. See Backfill behavior for details.
YAML
...
     objects:
        - objectName: lineItem
          destination: lineItemWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          backfill:
            defaultPeriod:
              days: 30
          ...

On the roadmap

The ability for your users to define their own sync schedules.

​
Required fields and optional fields

Fields can be either be required or optional. If a field is required, then all users who install this integration will need to give your app read access to that field. If a field is optional, then users can choose whether they’d like your app to read that field. For these fields, you will specify:

  • fieldName: the name of the field from the official SaaS API documentation.
YAML
   objects:
        - objectName: contact
          destination: contactWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          requiredFields:
          - fieldName: firstname
          - fieldName: lastname
          - fieldName: email
          optionalFields:
          - fieldName: company

Embeddable UI produced by the above config

​
Allow users to pick from all fields

If you want to give your user the option to pick from any of the fields in their object, use optionalFieldsAuto: all. The UI component will populate a list of all the fields pulled from that object (including custom fields) and allow them to pick which ones your app will be able to read.

YAML
      objects:
        - objectName: contact
          destination: contactWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          requiredFields:
          - fieldName: firstname
          - fieldName: lastname
          - fieldName: email
          # All other fields are optional
          optionalFieldsAuto: all

​
Field mappings

You might want to ask your users during the set up of the integration to map a field (standard or custom) to a concept in your product, because your various customers might be using different fields for the same purpose. Field mappings can either be inside requiredFields or optionalFields.

For these fields, you’ll specify:

  • mapToName: when Ampersand delivers the data from this field to you, this is the name that will be used to identify the field.
  • mapToDisplayName: the text to display to the user in the UI component when asking them to select a custom field (e.g. For the yaml example below, the text will say, “Which of your custom fields map to Contact Notes?“)
  • default (optional): the default field, you should only use standard fields as defaults.
  • prompt (optional): additional context that you want to show your user in the UI Component about this field. This should be a full sentence.
YAML
      objects:
        - objectName: contact
          destination: contactWebhook
          optionalFields:
          # field that your app created
          - fieldName: myAppPriorityScore
          # field that your user created
          - mapToName: notes
            mapToDisplayName: Contact Notes
            prompt: These are notes that you would like to surface in our app. 
            ...

Embeddable UI produced by the above config

​
Backfill behavior

Backfill behavior describes whether Ampersand will do an initial read of your customer’s historic data when they connect their SaaS instance, and how far back data will be read. For example, if your integration reads a customer’s contacts stored in their CRM, you can configure whether you want to only read new and updated contacts going forward, or if you also want to do an initial backfill of the pre-existing contacts in their CRM.

​
No backfill

If you only want to read new and updated records moving forward and do not wish to read any pre-existing records, then you can simply omit the backfill key in the integration definition, or you can write 0 days as the default period.

YAML
      objects:
        - objectName: contact
          backfill:
            defaultPeriod:
              days: 0 # Omitting backfill object will also default to 0 days.
          ...

​
Full historical backfill

If you want to do a full backfill of all the existing records when a customer connects their SaaS instance, set fullHistory to true. If you have customers that have large SaaS instances, please ensure that your webhook endpoint can handle a high number of messages in quick succession. You may find it helpful to use a webhook gateway solution like Hookdeck.

yaml
      objects:
        - objectName: contact
          backfill:
            defaultPeriod:
              fullHistory: true
          ...

​
Limited time backfill

You can select a specific time frame for backfill, such as “the last 30 days” or “the last 90 days”. Here’s an example of how to do so:

yaml
      objects:
        - objectName: contact
          backfill:
            defaultPeriod:
              days: 30 # Backfill the last 30 days of data
          ...

​
Full example

yaml
specVersion: 1.0.0
integrations: 
  - name: readSalesforce
    provider: salesforce
    read:
      objects:
      
        - objectName: account
          destination: accountWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          # Read all accounts when integration is installed
          backfill:
            defaultPeriod:
              fullHistory: true
          requiredFields:
            - fieldName: id
            - fieldName: name
            - fieldName: industry
          optionalFields:
            - fieldName: annualrevenue
            - fieldName: website
                
        - objectName: contact
          destination: contactWebhook
          schedule: "*/10 * * * *" # every 10 minutes
          # Read contacts from the last 30 days when integration is installed
          backfill:
            defaultPeriod:
              days: 30
          requiredFields:
            - fieldName: id
            - fieldName: firstname
            - fieldName: lastname
            - mapToName: pronoun
              mapToDisplayName: Pronoun
              prompt: We will use this word in emails we send out.
          # All other field are optional
          optionalFieldsAuto: all

​
Webhook messages

Ampersand will read data on the schedule you specify in amp.yaml. If there are any new or updated records since our last read, Ampersand will send you webhook messages. Each webhook message contains 1 or more records, and the maximum size of a webhook message is 300KB.

​
When a record is too big

Ampersand attaches results to a the webhook message in two ways:

  1. Inline: In most cases, the result is directly included in the webhook’s result field.
  2. URL: If a single record is over 300KB, it becomes too big to be sent over a webhook. In this case, the result won’t be included inline. Instead, the webhook message will contain a signed download URL in resultInfo.downloadUrl. The URL expires after 15 minutes.

resultInfo.type indicates how the result is attached to the message:

If type is inline, the full result is in the result field. If type is url, you can fetch the full result by making a GET request to resultInfo.downloadUrl.

In both cases, resultInfo.numRecords tells you how many records are available in the result.

​
Example webhook message (inline)

Here’s a sample webhook message for a Salesforce Contact object:

JavaScript
{
  "projectId": "ampersand-project-id",
  "provider": "salesforce",
  "groupRef": "xyz-company",
  "groupName": "XYZ Company",
  "installationId": "installation-id",
  "objectName": "Contact",
  // Only certain providers will have workspace available.
  "workspace": "salesforce-subdomain",
  "resultInfo": {
    "type": "inline",
    "numRecords": 2
  },
  "result": [
    {
      "fields": {
        "id": "001Dp00000P8QurIAF",
        "firstname": "Sally",
        "lastname": "Jones"
      },
      "mappedFields": {
        "pronoun": "she"
      },
      "raw": {
        "id": "001Dp00000P8QurIAF",
        "firstname": "Sally",
        "lastname": "Jones",
        "pronoun_custom_field": "she"
        // ... other fields for this record
      }
    },
    {
      "fields": {
        "id": "001Dp00000P9BusEIS",
        "firstname": "Taylor",
        "lastname": "Lao"
      },
      "mappedFields": {
        "pronoun": "they"
      },
      "raw": {
        "id": "001Dp00000P9BusEIS",
        "firstname": "Taylor",
        "lastname": "Lao",
        "pronoun_custom_field": "they"
        // ... other fields for this record
      }
    }
  ]
}

​
Example webhook message (URL)

JavaScript
{
  "projectId": "ampersand-project-id",
  "provider": "salesforce",
  "groupRef": "xyz-company",
  "groupName": "XYZ Company",
  "installationId": "installation-id",
  "objectName": "Contact",
  // Only certain providers will have workspace available.
  "workspace": "salesforce-subdomain",
  "resultInfo": {
    "type": "url",
    "downloadUrl": "https://storage.googleapis.com/....",
    "numRecords": 1
  },
}

​
OpenAPI spec

Check out our OpenAPI spec for the full webhook message schema and more details on the data structure. You can use this to generate types for your language and easily parse the webhook message.

​
Handling webhook results

Here’s some pseudo-code to illustrate how you can get parse result from a webhook:

go
// Generated from the OpenAPI spec
type WebhookMessage struct {
	Action string `json:"action"`
	GroupName string `json:"groupName"`
	GroupRef string `json:"groupRef"`
     ...
}

func handleWebhookEndpointMessage(message WebhookMessage) { 
  // Assumes that you have already parsed the webhook message into the openapi.WebhookMessage type 

  var results map[string]any

  switch message.ResultInfo.Type {
    case "url":
      // If result type is "url", download the data
      res, err := http.Get(message.ResultInfo.DownloadUrl)
      if err != nil {
        // handle error
      }

      results = parseBodyIntoMap(res.Body)
    case "inline":
      // If result type is "inline", data is directly in the message
      results = message.Result

    default:
      // This should not happen - contact us if it does!
  }

  fmt.Printf("Received %d records", message.ResultInfo.NumRecords) 

  // Process results as needed
}

Was this page helpful?

OverviewWrite Actions