Syncing data to cloud

Once you're happy with your application, you can start syncing with the cloud by provisioning a backend from your project. DataStore can connect to remote backend and automatically sync all locally saved data using GraphQL as a data protocol.

Setup cloud sync

Synchronization between offline and online data can be tricky. DataStore's goal is to remove that burden from the application code and handle all data consistency and reconciliation between local and remote behind the scenes, while developers focus on their application logic. Up to this point the focus was to setup a local data store that works offline and has all the capabilities you would expect from a data persistence framework.

The next step is to make sure the locally saved data is synchronized with a cloud backend powered by AWS AppSync.

Push the backend to the cloud

By now you should have a backend created with conflict detection enabled, as described in the Getting started guide.

Check the status of the backend to verify if it is already provisioned in the cloud.

amplify status

You should see a table similar to this one.

| Category | Resource name     | Operation | Provider plugin   |
| -------- | ----------------- | --------- | ----------------- |
| Api      | amplifyDatasource | No Change | awscloudformation |

In case Operation says Create or Update you need to push the backend to the cloud.

amplify push

Existing backend

DataStore can connect to an existing AWS AppSync backend that has been deployed from another project, no matter the platform it was originally created in. In these workflows it is best to work with the CLI directly by running an amplify pull command from your terminal and then generating models afterwards, using the process described in the Getting started guide.

For more information on this workflow please see the Multiple Frontends documentation.

Distributed data

When working with distributed data, it is important to be mindful about the state of the local and the remote systems. DataStore tries to make that as simple as possible for you; however, some scenarios might require some consideration.

For instance, when updating or deleting data, one has to consider that the state of the local data might be out-of-sync with the backend. This scenario can affect how conditions should be implemented.

Update and delete with predicate

For such scenarios both the save() and the delete() APIs support an optional predicate which will be sent to the backend and executed against the remote state.

There's a difference between the traditional local condition check using if/else constructs and the predicate in the save() and delete() APIs as you can see in the example below.

Conflict detection and resolution

When concurrently updating the data in multiple places, it is likely that some conflict might happen. For most of the cases the default Auto-merge algorithm should be able to resolve conflicts. However, there are scenarios where the algorithm won't be able to be resolved, and in these cases, a more advanced option is available and will be described in detail in the conflict resolution section.

Clear local data

Amplify.DataStore.clear() provides a way for you to clear all local data if needed. This is a destructive operation but the remote data will remain intact. When the next sync happens, data will be pulled into the local storage again and reconstruct the local data.

One common use for clear() is to manage different users sharing the same device or even as a development-time utility.

Hub.listen('auth', async (data) => {
  if (data.payload.event === 'signOut') {
    await DataStore.clear();
  }
});

This is a simple yet effective example. However, in a real scenario you might want to only call clear() when a different user is signedIn in order to avoid clearing the database for a repeated sign-in of the same user.

Selectively syncing a subset of your data

By default, DataStore fetches all the records that you’re authorized to access from your cloud data source to your local device. The maximum number of records that will be stored locally is configurable here.

You can utilize selective sync to persist a subset of your data instead.

Selective sync works by applying predicates to the base and delta sync queries, as well as to incoming subscriptions.

import { DataStore, syncExpression } from 'aws-amplify/datastore';
import { Post, Comment } from './models';

DataStore.configure({
  syncExpressions: [
    syncExpression(Post, () => {
      return (post) => post.rating.gt(5);
    }),
    syncExpression(Comment, () => {
      return (comment) => comment.status.eq('active');
    })
  ]
});

When DataStore starts syncing, only Posts with rating > 5 and Comments with status === 'active' will be synced down to the user's local store.

Reevaluate expressions at runtime

Sync expressions get evaluated whenever DataStore starts. In order to have your expressions reevaluated, you can execute DataStore.clear() or DataStore.stop() followed by DataStore.start().

If you have the following expression and you want to change the filter that gets applied at runtime, you can do the following:

let rating = 5;

DataStore.configure({
  syncExpressions: [
    syncExpression(Post, () => {
      return (post) => post.rating.gt(rating);
    })
  ]
});

async function changeSync() {
  rating = 1;
  await DataStore.stop();
  await DataStore.start();
}

Calls to start, query, save, delete, observe, and observeQuery will only start DataStore and reevaluate sync expressions if DataStore is not already started.

In the example above, changeSync() invokes DataStore.start() after DataStore.stop() has completely finished. This causes sync expressions to be reevaluated. Subsequently, when the sync expression is reevaluated, the rating variable has been changed to 1, so all Posts with rating > 1 will get synced down.

Keep in mind: DataStore.stop() will retain the local store's existing content. Run DataStore.clear() to clear the locally-stored contents.

async function changeSync() {
  rating = 8;
  await DataStore.clear();
  await DataStore.start();
}

This will clear the contents of your local store, reevaluate your sync expressions and re-sync the data from the cloud, applying all of the specified predicates to the sync queries.

You can also have your sync expression return Predicates.ALL in order to remove any filtering for that model. This will have the same effect as the default sync behavior.

let rating = null;

DataStore.configure({
  syncExpressions: [
    syncExpression(Post, () => {
      if (rating) {
        return (post) => post.rating.gt(rating);
      }

      return Predicates.ALL;
    })
  ]
});

Async expressions

You can pass a Promise to syncExpression:

DataStore.configure({
  syncExpressions: [
    syncExpression(Post, async () => {
      const ratingValue = await getRatingValue();
      return (post) => post.rating.gt(ratingValue);
    })
  ]
});

DataStore will wait for the Promise to resolve before applying the expression to the sync. Async expressions can also be reevaluated at runtime, just like synchronous expressions (see previous section).

Shorthand

If you don't need to add any logic to your syncExpression, you can use the following shorthand, returning the predicate directly:

DataStore.configure({
  syncExpressions: [syncExpression(Post, (post) => post.rating.gt(5))]
});

Advanced use case - Query instead of Scan

You can configure selective sync to retrieve items from DynamoDB with a query operation against a GSI. By default, the base sync will perform a scan. Query operations enable a highly efficient and cost-effective data retrieval for customers running DynamoDB at scale. Learn about creating GSIs with the @index directive here.

In order to do that, your syncExpression should return a predicate that maps to a query expression.

For example, for the following schema:

type User @model {
  id: ID!
  firstName: String!
  lastName: String! @index(name: "byLastName", sortKeyFields: ["createdAt"])
  createdAt: AWSDateTime!
}

Both of these sync expressions will result in AWS AppSync retrieving records from Amazon DynamoDB via a query operation:

DataStore.configure({
  syncExpressions: [
    syncExpression(User, () => {
      const lastName = await getLastNameForSync();
      return user => user.lastName.eq(lastName)
    })
  ]
});

// OR

DataStore.configure({
  syncExpressions: [
    syncExpression(User, () => {
      const lastName = await getLastNameForSync();
      return user => user.and(user => [
        user.lastName.eq(lastName),
        user.createdAt.gt('2020-10-10')
      ])
    })
  ]
});

To construct a query expression, return a predicate with the primary key of the GSI. You can only use the eq operator with this predicate.

For the schema defined above user => user.lastName.eq('Bobby') is a valid query expression.

Optionally, you can also add the sort key to this expression with an and operator and any of the following operators: eq | ne | le | lt | ge | gt | beginsWith | between.

E.g., user => user.and(user => [user.lastName.eq('Bobby'), user.createdAt.gt('2020-10-10')]).