Syncing data to cloud

Once you're happy with your application, you can start syncing with the cloud by provisioning a backend from your project. DataStore can connect to remote backend and automatically sync all locally saved data using GraphQL as a data protocol.

Setup cloud sync

Synchronization between offline and online data can be tricky. DataStore's goal is to remove that burden from the application code and handle all data consistency and reconciliation between local and remote behind the scenes, while developers focus on their application logic. Up to this point the focus was to setup a local data store that works offline and has all the capabilities you would expect from a data persistence framework.

The next step is to make sure the locally saved data is synchronized with a cloud backend powered by AWS AppSync.

Add the API plugin

Although DataStore presents a distinct API, its cloud synchronization functionality relies on the underlying API category. Therefore, you will still be required to incorporate the API plugin when working with DataStore.

Make sure you have the following plugin dependency in your pubspec.yaml.

amplify_api: ^1.0.0

Locate your Amplify initialization code, and add an AmplifyAPI() plugin. Your initialization code should already include an AmplifyDataStore() plugin from previous steps. Note the new import statement for API towards the top of the file.

Be sure to import your API library first:

// import the Amplify API plugin
import 'package:amplify_api/amplify_api.dart';
import 'package:amplify_datastore/amplify_datastore.dart';
import 'package:amplify_flutter/amplify_flutter.dart';

import 'amplifyconfiguration.dart';
import 'models/ModelProvider.dart';

Then update your configuration function, in this example it is _configureAmplify function, by adding AmplifyAPI like mentioned:

void _configureAmplify() async {
  final datastorePlugin = AmplifyDataStore(
    modelProvider: ModelProvider.instance,
  );
  // Add the following line and update your function call with `addPlugins`
  final api = AmplifyAPI();
  await Amplify.addPlugins([datastorePlugin, api]);
  try {
    await Amplify.configure(amplifyconfig);
  } on AmplifyAlreadyConfiguredException {
    print('Tried to reconfigure Amplify; this can occur when your app restarts on Android.');
  }
}

Push the backend to the cloud

By now you should have a backend created with conflict detection enabled, as described in the Getting started guide.

Check the status of the backend to verify if it is already provisioned in the cloud.

amplify status

You should see a table similar to this one.

| Category | Resource name     | Operation | Provider plugin   |
| -------- | ----------------- | --------- | ----------------- |
| Api      | amplifyDatasource | No Change | awscloudformation |

In case Operation says Create or Update you need to push the backend to the cloud.

amplify push

Existing backend

DataStore can connect to an existing AWS AppSync backend that has been deployed from another project, no matter the platform it was originally created in. In these workflows it is best to work with the CLI directly by running an amplify pull command from your terminal and then generating models afterwards, using the process described in the Getting started guide.

For more information on this workflow please see the Multiple Frontends documentation.

Distributed data

When working with distributed data, it is important to be mindful about the state of the local and the remote systems. DataStore tries to make that as simple as possible for you; however, some scenarios might require some consideration.

For instance, when updating or deleting data, one has to consider that the state of the local data might be out-of-sync with the backend. This scenario can affect how conditions should be implemented.

Update and delete with predicate

For such scenarios both the save() and the delete() APIs support an optional predicate which will be sent to the backend and executed against the remote state.

import 'package:amplify_flutter/amplify_flutter.dart';

import 'models/ModelProvider.dart';

Future<void> savePredicate(Post post) async {
  final post = ...; // get post using the query API
  final updatedPost = post.copyWith(title: '[Amplified]');
  try {
    // if the post title has changed to something else other than
    // a string that starts with "[Amplify]", the save will be rejected
    await Amplify.DataStore.save(
      updatedPost,
      where: Post.TITLE.beginsWith("[Amplify]"),
    );
  } on DataStoreException catch (e) {
    safePrint('Could not update post: $e');
  }
}

There's a difference between the traditional local condition check using if/else constructs and the predicate in the save() and delete() APIs as you can see in the example below.

// Tests only against the local state
Future<void> savePredicateLocally(Post post) async {
  if (post.title.startsWith('[Amplify]')) {
      await Amplify.DataStore.save(post);
  }
}

// Only applies the update if the data in the remote backend satisfies the criteria
Future<void> savePredicateRemotely(Post post) async {
  try {
    await Amplify.DataStore.save(
      post,
      where: Post.TITLE.beginsWith('[Amplify]'),
    );
  } on DataStoreException catch (e) {
    ...
  }
}

Conflict detection and resolution

When concurrently updating the data in multiple places, it is likely that some conflict might happen. For most of the cases the default Auto-merge algorithm should be able to resolve conflicts. However, there are scenarios where the algorithm won't be able to be resolved, and in these cases, a more advanced option is available and will be described in detail in the conflict resolution section.

Clear local data

Amplify.DataStore.clear() provides a way for you to clear all local data if needed. This is a destructive operation but the remote data will remain intact. When the next sync happens, data will be pulled into the local storage again and reconstruct the local data.

One common use for clear() is to manage different users sharing the same device or even as a development-time utility.

import 'dart:async';

import 'package:amplify_flutter/amplify_flutter.dart';

final hubSubscription =
    Amplify.Hub.listen(HubChannel.Auth, (AuthHubEvent hubEvent) async {
  if (hubEvent.eventName == 'SIGNED_OUT') {
    try {
      await Amplify.DataStore.clear();
      safePrint('DataStore is cleared as the user has signed out.');
    } on DataStoreException catch (e) {
      safePrint('Failed to clear DataStore: $e');
    }
  }
});

This is a simple yet effective example. However, in a real scenario you might want to only call clear() when a different user is signedIn in order to avoid clearing the database for a repeated sign-in of the same user.

Selectively syncing a subset of your data

By default, DataStore fetches all the records that you’re authorized to access from your cloud data source to your local device. The maximum number of records that will be stored locally is configurable here.

You can utilize selective sync to persist a subset of your data instead.

Selective sync works by applying predicates to the base and delta sync queries, as well as to incoming subscriptions.

void _configureAmplify() async {
  // Update AmplifyDataStore instance like below
  final datastorePlugin = AmplifyDataStore(
    modelProvider: ModelProvider.instance,
    syncExpressions: [
      DataStoreSyncExpression(Post.classType, () => Post.RATING.gt(5)),
      DataStoreSyncExpression(
        Comment.classType,
        () => Comment.POST.beginsWith('This'),
      )
    ],
  );
  ...
}

When DataStore starts syncing, only Posts with rating > 5 and Comments with POST id begins with "This" will be synced down to the user's local store.

Reevaluate expressions at runtime

Sync expressions get evaluated whenever DataStore starts. In order to have your expressions reevaluated, you can execute Amplify.DataStore.clear() or Amplify.DataStore.stop() followed by Amplify.DataStore.start().

If you have the following expression and you want to change the filter that gets applied at runtime, you can do the following:

int rating = 5;

Future<void> initialize() async {
  final dataStorePlugin = AmplifyDataStore(
    modelProvider: ModelProvider.instance,
    syncExpressions: [
      DataStoreSyncExpression(
        Post.classType,
        () => Post.RATING.gt(rating),
      ),
    ],
  );

  await Amplify.addPlugin(dataStorePlugin);
}

Future<void> changeSync() async {
  rating = 1;
  try {
    await Amplify.DataStore.stop();
  } catch(error) {
    print('Error stopping DataStore: $error');
  }

  try {
    await Amplify.DataStore.start();
  } on Exception catch (error) {
    print('Error starting DataStore: $error');
  }
}

Each time DataStore starts (via start or any other operation: query, save, delete, or observe), DataStore will reevaluate the syncExpressions.

In the above case, the predicate will contain the value 1, so all Posts with rating > 1 will get synced down.

Keep in mind: Amplify.DataStore.stop() will retain the local store's existing content. Run Amplify.DataStore.clear() to clear the locally-stored contents.

Future<void> changeSync() async {
  rating = 8;
  try {
    await Amplify.DataStore.clear();
  } on Exception catch (error) {
    print('Error clearing DataStore: $error');
  }

  try {
    await Amplify.DataStore.start();
  } on Exception catch (error) {
    print('Error starting DataStore: $error');
  }
}

This will clear the contents of your local store, reevaluate your sync expressions and re-sync the data from the cloud, applying all of the specified predicates to the sync queries.

You can also have your sync expression return QueryPredicateConstant.all in order to remove any filtering for that model. This will have the same effect as the default sync behavior.

int rating = 5;

Future<void> initialize() async {
  var dataStorePlugin = AmplifyDataStore(
    modelProvider: ModelProvider.instance,
    syncExpressions: [
      DataStoreSyncExpression(
        Post.classType,
        () {
          if (rating > 0) {
            return QueryPredicate.all;
          }

          return Post.RATING.gt(rating);
        },
      ),
    ],
  );

  await Amplify.addPlugin(dataStorePlugin);
}

Advanced use case - Query instead of Scan

By default, Datastore sync performs a Scan on DynamoDB tables, which is less efficient when a table contains a large set of data. You can use syncExpression when configuring Datastore to filter synced data to improve performance using Query rather than Scan.

You can enable Query during the sync by following the below steps:

1. Create a Global Secondary Index (GSI) for your schema, e.g. in the below schema a GSI will be created using field lastName as the GSI primary key, and createdAt as the GSI sort key. Learn about creating GSIs with the @index directive here.

type User @model {
  id: ID!
  firstName: String!
  lastName: String! @index(name: "byLastName", sortKeyFields: ["createdAt"])
  createdAt: AWSDateTime!
}

2. Configure DataStore with syncExpression returning a predicate that maps to a query expression.

To construct a query expression, return a predicate with the primary key of the GSI. You can only use the eq operator with this predicate.

For the schema defined above User.LASTNAME.eq("Doe") is a valid query expression.

Optionally, you can also chain the sort key to this expression, using any of the following operators: eq | ne | le | lt | ge | gt | beginsWith | between.

E.g., User.LASTNAME.eq("Doe").and(User.CREATEDAT.gt("2020-10-10")).

Both of these sync expressions will result in AWS AppSync retrieving records from Amazon DynamoDB via a query operation:

Future<void> initializeSingleEquals() async {
  // Using eq operator with the primary key of the GSI
  final singleEqualsStore = AmplifyDataStore(
    modelProvider: ModelProvider.instance,
    syncExpressions: [
      DataStoreSyncExpression(
        User.classType,
        () => User.LASTNAME.eq("Doe"),
      ),
    ],
  );

  await Amplify.addPlugin(singleEqualsStore);
}

Future<void> initializeChainedEquals() async {
  // Using eq operator with the primary key of the GSI and
  // chaining the gt operator with the sort key
  final chainedEqualGtStore = AmplifyDataStore(
    modelProvider: ModelProvider.instance,
    syncExpressions: [
      DataStoreSyncExpression(
        User.classType,
        () => User.LASTNAME.eq("Doe").and(User.CREATEDAT.gt("2020-10-10")),
      ),
    ],
  );

  await Amplify.addPlugin(chainedEqualGtStore);
}