Offline Setup Guide

Overview

This guide provides step-by-step instructions for converting a non-offline Formbird environment into a fully functional offline environment. Starting from a standard Formbird installation without offline capabilities, this guide will walk you through all the necessary configuration steps to enable offline functionality.

The setup process includes: 1. Setting up the offline key system 2. Configuring user accounts for offline access 3. Configuring the application configuration file with offline settings 4. Adding offline keys to all application documents (templates, rulesets, components, etc.) 5. Configuring file attachments for offline access (optional) 6. Configuring offline map caching (optional)

Table of Contents

• Prerequisites
• Step 1: Set Up Offline Key System
• Step 2: Configure User Account Control Documents
• Step 3: Configure Application Configuration File
• Step 4: Add Offline Keys to Application Documents
• 4.1 Templates
• 4.2 Client-Side Rulesets
• 4.3 Ruleset Includes
• 4.4 Components
• 4.5 Vendor Libraries
• 4.6 Error Document Template
• 4.7 Account
• 4.8 Account Control
• 4.9 offlineIndex
• Step 5: Configure File Attachments for Offline Access (Optional)
• Step 6: Configure Offline Map Caching (Optional)
• Important Warning: Standalone Offline Keys
• Enabling Offline on a Device
• Setup Checklist
• Related Documentation

Prerequisites

Before beginning the offline setup, ensure you have: - A working Formbird application environment - Access to modify the application configuration document - Administrative access to create and modify user account control documents - Access to modify templates, rulesets, and ruleset includes - Have imported the offline official release @Release Offline System Setup 202602A - https://ruleset-dev.formbird.com/form/c3fb021d-5d8b-491d-810d-5a22276ac38c

Step 1: Set Up Offline Key System

The offline key system controls which documents, templates, and rulesets are available for offline caching. At its most basic level, you can use a simple string key such as "offline" to mark content for offline access.

Understanding Offline Keys

• Key Types: Offline keys can be either simple strings (e.g., "offline") or UUIDs (e.g., "your-offline-uuid-key"). Both formats work identically. It's recommended to use a uuid key for improved security.
• Existing Keys: You can use existing keys that are already in your system and simply add the 'offline' right to them in user account control documents.
• New Keys: Alternatively, you can create a new dedicated offline key (either a string or UUID).

Choosing a Key Strategy

Option 1: Use Existing Keys - If you already have a key system in place, you can add the 'offline' right to existing keys in user account control documents. - This approach maintains consistency with your existing access control system.

Option 2: Create a Dedicated Offline Key

• Create a new key specifically for offline functionality (e.g., "offline" or a UUID).
• This approach provides clear separation between offline access and other access rights.

Example - Simple String Key:

"offline"

Example - UUID Key:

"your-offline-uuid-key"

Note: Replace your-offline-uuid-key with your own generated UUID in your implementation.

Step 2: Configure User Account Control Documents

For users to access offline functionality, their account control documents must include an offline key with the "Offline" right. Users who have "Offline" in their accessKeys.rights will see the relevant offline cache options in the application.

Note: The "Offline" right is case-sensitive and must be capitalized.

Adding Offline Right to Existing Keys

If you're using an existing key and adding the offline capability:

{
  "accessKeys": [
    {
      "keyId": "your-existing-key-id",
      "rights": ["Create", "Update", "Read", "Offline"]
    }
  ]
}

Adding a New Offline Key

If you're creating a new dedicated offline key:

{
  "accessKeys": [
    {
      "keyId": "offline",
      "rights": ["Offline"]
    },
    {
      "keyId": "your-existing-read-key",
      "rights": ["Read"]
    }
  ]
}

Important: When using a standalone offline key (one that doesn't include CRUDE - Create, Read, Update, Delete, Edit - rights), you must also ensure users have an additional key with "Read" rights so that user groups can read the documents. See the warning section below for more details.

Multiple Keys Example

A user account control document can have multiple keys with different rights:

{
  "accessKeys": [
    {
      "keyId": "project-key-1",
      "rights": ["Create", "Update", "Read", "Delete", "Offline"]
    },
    {
      "keyId": "project-key-2",
      "rights": ["Read", "Offline"]
    },
    {
      "keyId": "offline",
      "rights": ["Offline"]
    }
  ]
}

Step 3: Configure Application Configuration File

Add the required offline configuration to your application's configuration document. This configuration is added to the clientConfiguration section of the configuration document.

Note: The application configuration file does not require an offline key - it is cached automatically.

Complete Configuration Structure

Add the following configuration to the clientConfiguration object in your application configuration document:

{
  "clientConfiguration": {
    "maxUploadSize": "12MB",
    "unsyncedRecordCheckTime": 5000,
    "offline": {
      "syncher": "rust",
      "dexieAddons": [
        {
          "fileName": "https://cdn.formbird.com/componentsdexie-addon/v1-0-3/dexie-elasticsearch-addon.js",
          "exportName": "ElasticIndexedDBAddon"
        }
      ],
      "searchTimeout": 90000,
      "syncPageSize": 10000,
      "android": {
        "enableSharedWorkers": false
      }
    },
    "offlineSearchOptions": {
      "geohashPrecision": {
        "boundingBox": 8
      },
      "indexedDBIndexesPriority": [
        "documentId",
        "*actionRel____documentId",
        "*siteRel____name",
        "*siteRel____documentId",
        "*orderRel____documentId",
        "*truckRel____documentId",
        "*parentRel____documentId",
        "*primaryVehicleRel____documentId",
        "*primaryVehicleRel____name",
        "*secondaryVehicleRel____documentId",
        "*secondaryVehicleRel____name",
        "*localeRel____documentId",
        "*localeRel____name",
        "*shiftRel____documentId",
        "*accountTypeRel____documentId",
        "*customerRel____documentId",
        "systemHeader.templateId",
        "*appTags",
        "status",
        "startDate",
        "siteType",
        "systemHeader.summaryName",
        "systemHeader.systemType",
        "systemHeader.currentVersion"
      ]
    },
    "cachingEnableTemplate": [
      {
        "documentId": "9ed4c3fb-1a59-49dc-be26-7e40e95ab194",
        "name": "Enabling Caching"
      }
    ],
    "cachingNotAllowed": false
  }
}

Key Configuration Properties

See 109-Configuration-Reference.md for complete property documentation.

IndexedDB Indexes Priority

The indexedDBIndexesPriority array defines which fields should have indexes created in the local IndexedDB store. This is critical for query performance.

Key Points:

• Index 0 is highest priority in the array
• If indexedDBIndexesPriority is not set, the default priority is the order of the offlineIndex document indexedDBIndexes field
• If a queried field has no index, a full table scan occurs (slow)
• Place common system fields (systemHeader.templateId, *appTags, systemHeader.systemType, systemHeader.currentVersion) at the end as they are present on every document
• Keep this array in sync with the offlineIndex indexedDBIndexes document

Best Practices for Index Priority:

Prioritize indexes by the power of each term to minimize query result size, not by query frequency. The highest priority indexed term that matches a query will have the biggest impact on performance, so a term that returns a smaller subset of documents is most desirable.

• High priority: documentId - returns a single document, very efficient
• Medium priority: Relationship fields like *siteRel____documentId - typically return small subsets
• Low priority: *appTags, systemHeader.templateId - many documents share the same values, returning large result sets

Even a less frequently queried term that returns only one document should be a higher priority than a frequently queried term that returns hundreds of documents.

Index Naming Quick Reference:

• Simple field: "status", "documentId"
• Nested property: "systemHeader.templateId"
• Array field: "*appTags" (prefix with * to index each array value individually)
• Array subfield: "*siteRel____documentId" (the ____ extracts properties from objects in an array into a new flattened array property, and * indexes each value individually)

How Array Subfield Indexing Works:

When a document has a relationship array like:

{
  "siteRel": [
    { "documentId": "abc-123", "name": "Site A" },
    { "documentId": "def-456", "name": "Site B" }
  ]
}

The index "*siteRel____documentId" works in two steps: 1. ____ (four underscores) - During indexing, extracts the documentId property from each object in siteRel and creates a new property on the stored document: json "siteRel____documentId": ["abc-123", "def-456"] 2. * (asterisk prefix) - Tells IndexedDB to flatten the array and store an index entry for each value, enabling queries to match any individual value in the array

This enables efficient querying like "find all documents where siteRel contains documentId abc-123".

Note: You can index an array field without the * prefix, but this stores the entire array as a single index key. If you want to query for individual items within an array, you must use the * prefix.

See 109-Configuration-Reference.md for detailed index naming conventions and examples.

Create the offlineIndex Document (Required)

You must create an offlineIndex document for offline functionality. This document stores the IndexedDB indexes and must contain the same keys as indexedDBIndexesPriority.

{
  "documentId": "your-offline-index-uuid",
  "indexedDBIndexes": [
    "documentId",
    "*actionRel____documentId",
    "status",
    "systemHeader.templateId",
    "*appTags",
    "systemHeader.systemType",
    "systemHeader.currentVersion"
  ],
  "appTags": ["offline", "system"],
  "systemHeader": {
    "systemType": "offlineIndex",
    "summaryName": "Offline Index",
    "keyIds": ["your-offline-key"]
  }
}

Note: The order of indexes in indexedDBIndexes does not matter (unlike indexedDBIndexesPriority where order determines query priority).

Step 4: Add Offline Keys to Application Documents

This section provides a comprehensive overview of all document types that require offline keys to be added to their systemHeader.keyIds array. Each document type is detailed in the subsections below.

Quick Reference: Document Types Requiring Offline Keys

Standard Offline Key Configuration

For all document types, the offline key is added to the systemHeader.keyIds array:

{
  "systemHeader": {
    "systemType": "<document-type>",
    "keyIds": ["your-offline-key"]
  }
}

If the document already has existing keys, add the offline key to the array:

{
  "systemHeader": {
    "keyIds": ["existing-key-1", "existing-key-2", "your-offline-key"]
  }
}

4.1 Templates

For a template to be available offline, it must have an offline key added to its systemHeader.keyIds field.

Template Configuration

Add the offline key to the template's systemHeader.keyIds array:

{
  "documentId": "template-uuid-here",
  "name": "My Template",
  "systemHeader": {
    "systemType": "template",
    "keyIds": ["your-offline-uuid-key"]
  },
  ...
}

Note: If the template already has keys in keyIds, add the offline key to the existing array:

{
  "systemHeader": {
    "keyIds": ["existing-key-1", "existing-key-2", "your-offline-uuid-key"]
  }
}

4.2 Client-Side Rulesets

All client-side rulesets must have an offline key added to their systemHeader.keyIds field to be available offline.

{
  "documentId": "ruleset-uuid-here",
  "name": "My Ruleset",
  "systemHeader": {
    "systemType": "ruleset",
    "keyIds": ["your-offline-uuid-key"]
  },
  ...
}

4.3 Ruleset Includes

Ruleset includes must also have an offline key added to their systemHeader.keyIds field.

{
  "documentId": "ruleset-include-uuid-here",
  "name": "My Ruleset Include",
  "systemHeader": {
    "systemType": "rulesetInclude",
    "keyIds": ["your-offline-uuid-key"]
  },
  ...
}

4.4 Components

Custom components used in offline templates must have an offline key added to their systemHeader.keyIds field.

{
  "documentId": "component-uuid-here",
  "name": "My Custom Component",
  "systemHeader": {
    "systemType": "component",
    "keyIds": ["your-offline-uuid-key"]
  },
  ...
}

Note: Only add offline keys to components that are actually used in templates that will be accessed offline.

4.5 Vendor Libraries

Vendor libraries (third-party JavaScript libraries) used in offline templates must have an offline key added to their systemHeader.keyIds field.

{
  "documentId": "vendor-library-uuid-here",
  "name": "My Vendor Library",
  "systemHeader": {
    "systemType": "vendorLibrary",
    "keyIds": ["your-offline-uuid-key"]
  },
  ...
}

Note: Only add offline keys to vendor libraries that are required by templates or rulesets used offline.

4.6 Error Document Template

The error document template is displayed when an error occurs during offline operation. This template must have an offline key to be available offline.

{
  "documentId": "error-template-uuid-here",
  "name": "Error Document Template",
  "systemHeader": {
    "systemType": "template",
    "keyIds": ["your-offline-uuid-key"]
  },
  ...
}

Note: Ensure your application's error handling template is marked with an offline key so users see proper error messages when offline.

4.7 Account

User account documents must have an offline key to allow the account information to be cached for offline access.

{
  "documentId": "account-uuid-here",
  "systemHeader": {
    "systemType": "account",
    "keyIds": ["your-offline-uuid-key"]
  },
  ...
}

Note: The account document stores user-specific information needed for offline authentication and identification.

4.8 Account Control

Account control documents define user permissions and access rights. *Offline keys are not required* to be added to the systemHeader.keyIds field for account control documents.

A subset of the user's account control is automatically cached for offline access. The system handles this automatically when caching is enabled.

{
  "documentId": "account-control-uuid-here",
  "systemHeader": {
    "systemType": "accountControl",
    "keyIds": [...]
  },
  "accessKeys": [
    {
      "keyId": "your-offline-uuid-key",
      "rights": ["Offline", "Read"]
    }
  ],
  ...
}

*Note:* The account control document must still contain the accessKeys array with the Offline right as described in Step 2 for the user to have offline access.

4.9 offlineIndex

The offlineIndex document stores the IndexedDB index configuration and must have an offline key to be cached.

{
  "documentId": "your-offline-index-uuid",
  "indexedDBIndexes": [
    "documentId",
    "*actionRel____documentId",
    "status",
    "systemHeader.templateId",
    "*appTags",
    "systemHeader.systemType",
    "systemHeader.currentVersion"
  ],
  "appTags": ["offline", "system"],
  "systemHeader": {
    "systemType": "offlineIndex",
    "summaryName": "Offline Index",
    "keyIds": ["your-offline-uuid-key"]
  }
}

Note: The offlineIndex document is also configured in Step 3. Ensure the indexedDBIndexes array matches the indexedDBIndexesPriority in your application configuration.

Step 5: Configure File Attachments for Offline Access (Optional)

If your application uses file attachments that need to be available offline, you must configure the fileReference template and ensure new file references receive offline keys.

5.1 Configure the fileReference Template

The fileReference template must have an offline key in its systemHeader.keyIds:

{
  "systemHeader": {
    "systemType": "template",
    "keyIds": ["your-offline-uuid-key"]
  }
}

5.2 Add Offline Keys to New File References

New fileReference documents need offline keys to be cached. Configure one of the following:

Option A: Using attachKeys on the Template

Add attachKeys to the fileReference template so new documents automatically receive the offline key:

{
  "attachKeys": ["your-offline-uuid-key"]
}

Option B: Using a preSaveClient or preSaveOffline Rule

Create a ruleset that adds the offline key to new fileReference documents before saving. This approach is useful when you need conditional logic for which files should be available offline.

• preSaveClient - Runs on all client saves
• preSaveOffline - Runs only when saving while offline

See 110-File-and-Attachment-Handling.md for more details on offline file handling.

Step 6: Configure Offline Map Caching (Optional)

If your application uses maps that need to be available offline, configure the openLayers.mapViewsToCache array in clientConfiguration.

Example Configuration

{
  "clientConfiguration": {
    "openLayers": {
      "mapViewsToCache": [
        {
          "name": "Vicmaps Colour",
          "styleLocationUrls": [
            "https://vicmap.land.vic.gov.au/hosting/rest/services/Hosted/Vicmap_Vector_Tile_Basemap_Colour_WM_Hosted/VectorTileServer/resources/styles/root.json"
          ],
          "minZoom": 1,
          "maxZoom": 15,
          "dataBounds": [
            {
              "topLeftCoord": [144.173, -37.209],
              "bottomRightCoord": [144.965, -37.959]
            }
          ],
          "url": "https://vicmap.land.vic.gov.au/hosting/rest/services/Hosted/Vicmap_Vector_Tile_Basemap_Colour_WM_Hosted/VectorTileServer/tile/{z}/{y}/{x}.pbf"
        }
      ]
    }
  }
}

Key Properties: - name - Display name for the cached map view - dataBounds - Geographic areas to cache (topLeftCoord/bottomRightCoord as [longitude, latitude]) - minZoom/maxZoom - Zoom level range to cache - url - Tile URL pattern with {z}, {y}, {x} placeholders

Note: Caching large areas at high zoom levels requires significant storage. Balance coverage area against zoom detail.

See 109-Configuration-Reference.md for complete OpenLayers configuration options.

Important Warning: Standalone Offline Keys

Critical: If you are using a standalone 'offline' key (a key that only has the 'offline' right and does not include CRUDE access rights like 'Create', 'Read', 'Update', 'Delete', or 'Edit'), you must add an additional key to allow user groups to read the documents.

Why This Is Necessary

When a document only has a standalone offline key without read rights, user groups may not have permission to read the document, even though it's marked for offline access. This can cause issues when users try to access cached documents offline.

Correct Configuration Example

If using a standalone offline key, ensure users also have a key with read access:

{
  "accessKeys": [
    {
      "keyId": "your-offline-uuid-key",
      "rights": ["Offline"]
    },
    {
      "keyId": "read-access-key",
      "rights": ["Read"]
    }
  ]
}

And ensure documents have both keys:

{
  "systemHeader": {
    "keyIds": ["your-offline-uuid-key", "read-access-key"]
  }
}

Alternative: Combined Key Approach

To avoid this issue, you can use a single key that includes both offline and read rights:

{
  "accessKeys": [
    {
      "keyId": "offline-access",
      "rights": ["Read", "Offline"]
    }
  ]
}

Enabling Offline on a Device

Once all configuration steps are complete, users can enable offline caching on their device:

1. Login to the environment with offline configuration and keys set up
2. Open the account menu (user dropdown) - if configured correctly, a "Cache" option will be visible
3. Click "Cache" → "Enable"
4. Wait on the caching template until the "Cached successful" message displays

Note: Users must stay on the caching progress template until caching completes. Navigating away may interrupt the caching process.

If the "Cache" option does not appear in the account menu: - Verify the user's account control document has an access key with the "Offline" right - Verify cachingNotAllowed is not set to true in clientConfiguration

Setup Checklist

Use this checklist to ensure all steps are completed:

• [ ] Step 1: Set up offline key system
• [ ] Decided on key strategy (existing key vs. new key)

• [ ] Created or identified offline key to use

• [ ] Step 2: Configured user account control documents

• [ ] Added offline key to accessKeys with 'offline' right
• [ ] Verified users with offline right will see cache options

• [ ] If using standalone offline key, added additional read key

• [ ] Step 3: Added offline configuration to application configuration file

• [ ] Added offline object with dexieAddons (required)
• [ ] Added offlineSearchOptions.indexedDBIndexesPriority
• [ ] Added cachingEnableTemplate linking to progress template

• [ ] Created offlineIndex document with matching indexedDBIndexes

• [ ] Step 4: Added offline keys to application documents

• [ ] 4.1 Templates: Added offline key to all required templates' systemHeader.keyIds
• [ ] 4.2 Client-Side Rulesets: Added offline key to all client-side rulesets' systemHeader.keyIds
• [ ] 4.3 Ruleset Includes: Added offline key to all ruleset includes' systemHeader.keyIds
• [ ] 4.4 Components: Added offline key to custom components used offline
• [ ] 4.5 Vendor Libraries: Added offline key to vendor libraries used offline
• [ ] 4.6 Error Document Template: Added offline key to error handling template
• [ ] 4.7 Account: Added offline key to user account documents
• [ ] 4.8 Account Control: Verified accessKeys has Offline right (no systemHeader.keyIds needed - auto-cached)

• [ ] 4.9 offlineIndex: Added offline key to offlineIndex document

• [ ] Step 5: Configured file attachments for offline access (if needed)

• [ ] Added offline key to fileReference template's systemHeader.keyIds

• [ ] Configured attachKeys on template OR preSaveClient rule to add keys to new files

• [ ] Step 6: Configured offline map caching (if needed)

• [ ] Added openLayers configuration to clientConfiguration
• [ ] Configured mapViewsToCache with appropriate bounds and zoom levels

• [ ] Verified tile server URLs are accessible

• [ ] Verification: Tested offline functionality

• [ ] Verified "Cache" option appears in account menu
• [ ] Enabled caching and received "Cached successful" message
• [ ] Verified templates are available offline
• [ ] Verified documents can be accessed offline
• [ ] Verified standalone key configuration (if applicable)

Related Documentation

• 100-Formbird-Offline-Mode.md - General offline mode concepts and usage
• 109-Configuration-Reference.md - Detailed configuration reference for all offline settings
• 101-Offline-Architecture.md - Technical architecture of the offline system
• 102-IndexedDB-and-Dexie.md - IndexedDB and Dexie configuration details
• 104-Offline-Syncing.md - Offline syncing implementation details
• JayRules: Setting Rules for Offline Processing - Extra ruleset fields to control when rules run offline