Connectors
A connector is a technology-specific implementation of a well-defined capability interface. Connectors form the infrastructure adapter layer of the platform, translating abstract, domain-oriented contracts into concrete interactions with storage backends, cloud services, messaging systems, distributed ledger networks, and other external systems. By separating the capability interface from its implementation, the platform allows any combination of backends to be selected and swapped at deployment time without modifying application code.
Why Pluggable Connectors
The connector architecture solves a fundamental problem in distributed systems: operational flexibility. Once applications are built on this platform, they should be able to evolve their infrastructure without being forced to rewrite their application code or switch their underlying technology stack.
This means deployments can run on any infrastructure, whether that is a cloud provider such as AWS, Azure, or GCP; a private on-premise data centre; or a hybrid combination with some services cloud-hosted and others running locally. The same application code works everywhere because the stack does not depend on a specific cloud vendor's APIs or proprietary services.
Similarly, organisations can start with one backend technology and migrate to another without stopping the application. A deployment might begin with PostgreSQL for entity storage during initial development, migrate to DynamoDB as it scales cloud-first, and later add a separate ScyllaDB cluster for a specific entity type without changing a single line of application code. Only the deployment configuration changes, allowing the infrastructure team to optimise for cost, performance, compliance, or operational expertise without coupling those decisions to service implementation.
The connector model also enables vendor independence. Rather than binding the platform to a specific cloud provider's ecosystem or a particular database technology, the abstraction allows teams to make infrastructure decisions based on long-term organisational strategy rather than architectural lock-in. This is especially important in evolving landscapes where technologies, pricing models, and service availability change rapidly.
The Connector Interface
Every connector in the platform implements IComponent, an interface defined in the @twin.org/core package. The contract is deliberately minimal, consisting of four members: a required className() method that returns a stable string identifier for the implementing class, and three optional lifecycle methods named bootstrap, start, and stop. Each lifecycle method accepts an optional nodeLoggingComponentType string that allows log output generated during startup to be routed correctly through the logging subsystem before it is fully initialised.
The separation between bootstrap and start allows connectors with ordering dependencies to signal their readiness in stages. A connector may use bootstrap to perform initialisation tasks that must complete before any other connector starts, and start to begin normal operation once all bootstrapped dependencies are available. The stop method provides a structured shutdown path, allowing connections to be closed and in-flight operations to complete cleanly before the process exits.
Capability Interfaces
Each operational domain within the platform defines a connector interface that extends IComponent and adds the methods specific to that capability. These interfaces are defined in the corresponding -models package of each repository and are the canonical contracts against which all implementations are written. Consumers of connectors depend only on these interfaces, never on the concrete implementing classes.
IEntityStorageConnector provides a generic, schema-aware interface for persistent storage. Its type parameter T represents the entity type being stored, and the interface defines set, get, remove, and query methods. The query method supports conditional filtering using an EntityCondition<T> expression, multi-property sorting, property projection for partial reads, cursor-based pagination, and configurable result limits.
ILoggingConnector defines a required log method accepting an ILogEntry value, and an optional query method for connectors that persist log entries and support retrieval. The optional nature of query means that a simple console connector need not implement it, whilst a database-backed connector can offer full querying capabilities without breaking the common interface.
IVaultConnector provides a comprehensive interface for secret and cryptographic key management. In addition to the expected create, add, get, rename, and remove operations on named keys, the interface includes sign, verify, encrypt, and decrypt methods that operate on named keys, and a separate group of setSecret, getSecret, and removeSecret methods for arbitrary opaque secret storage.
IIdentityConnector covers the full DID (Decentralised Identifier) document lifecycle, including creation, resolution, update, and removal of DID documents, as well as verifiable credential and verifiable presentation issuance, verification, and revocation.
IBlobStorageConnector, IEventBusConnector, INftConnector, IWalletConnector, IFaucetConnector, IMessagingEmailConnector, IMessagingSmsConnector, and IMessagingPushNotificationConnector follow the same pattern of extending IComponent with the members needed for their respective domains.
Concrete Implementations
For each capability interface there are one or more concrete connector packages, each targeting a specific backend technology. Entity storage alone has nine concrete implementations: an in-memory variant for testing and lightweight deployments, a file-based variant for local persistence, and cloud-provider-specific variants for MongoDB, MySQL, PostgreSQL, ScyllaDB, AWS DynamoDB, Azure CosmosDB, and GCP Firestore. A synchronised storage variant adds cross-replica consistency guarantees on top of any underlying entity storage connector, making it suitable for active-active deployments where multiple nodes write to a shared dataset.
Logging has implementations: a console connector that formats and writes entries to standard output with ANSI colour coding, configurable level filtering, and optional group suppression; and an entity-storage-backed connector that persists log entries through any configured entity storage backend, enabling structured log queries, retention policies, and integration with the telemetry subsystem.
The vault domain offers a local entity-storage-backed connector suitable for development and testing contexts, and an integration with HashiCorp Vault for production deployments that require enterprise-grade secret management with audit trails, access policies, and high availability.
Blob storage is served by connectors targeting IPFS, AWS S3, Azure Blob Storage, GCP Storage, and an in-memory option for testing, plus a file-system-backed variant for local development.
The identity, wallet, NFT, and verifiable storage domains each have IOTA Tangle-backed connectors for production use on a distributed ledger, and entity-storage-backed connectors that provide the same interface against a local database. This dual availability makes it straightforward to build and test full application workflows without requiring ledger connectivity, test tokens, or a running IOTA node.
The Factory System
Each connector domain's -models package exports a named factory singleton created through Factory.createFactory<IXxxConnector>, a generic utility from @twin.org/core. The factory stores named generator functions and holds lazily created instances. Because it is backed by a SharedStore, the same singleton is accessible across all module boundaries within a running process without any direct import coupling between the modules involved. A connector registered by the engine initialisation code is immediately available to any service that holds a reference to the same factory, even if the two modules have no knowledge of each other.
Connector instances are registered by name using a call such as EntityStorageConnectorFactory.register('my-store', () => instance). Consuming code retrieves instances by the same name using EntityStorageConnectorFactory.get('my-store'), which throws an informative error if the requested name has not been registered. The hasName method can be used to check availability before attempting retrieval. The factory name used for each registry, for example 'entity-storage' or 'logging', scopes the instances to their specific connector interface type, preventing name collisions across domains.
Connector Factory: Registration and Resolution
Connector Composition
Connectors can be built on top of other connectors, allowing complex capabilities to be assembled from simpler primitives without duplicating infrastructure code. EntityStorageVaultConnector is the primary example: it implements the full IVaultConnector interface but delegates all persistence to two instances of IEntityStorageConnector, one for cryptographic key material and one for opaque secrets. At construction time it retrieves both entity storage connectors from EntityStorageConnectorFactory by name, which means it can work with any configured entity storage backend simply by adjusting the instance names in the engine configuration. Swapping from an in-memory store to a PostgreSQL cluster requires only a configuration change rather than any code modification.
The same principle applies to EntityStorageIdentityConnector and EntityStorageWalletConnector, which are implemented on top of entity storage rather than requiring a distributed ledger. This approach makes feature-complete development and testing environments possible without network connectivity, test account balances, or external service dependencies.
Connector Composition: EntityStorageVaultConnector
Engine Configuration
Connector selection and configuration at runtime is managed through IEngineConfig. The configuration object contains a types map whose keys correspond to connector and component domains, and whose values are arrays of typed configuration entries. Each entry carries a discriminated union type field that identifies which concrete implementation to use, followed immediately by the implementation-specific options needed to construct that class. For example, an entity storage entry of type 'scylladb' would carry a ScyllaDB connection options object, whilst a type of 'memory' requires no additional options at all.
Multiple entries for the same domain are supported and serve different purposes. One entry can be marked with isDefault: true to indicate which instance services should use when no specific connector name is requested. Additional entries can be given individual instance names through overrideInstanceType to permit multiple named connectors of the same type to coexist. This is particularly useful when different entity schemas should be stored in different database clusters, or when a secondary logging connector should write to a different destination than the primary one.
In a node deployment, configuration is driven by environment variables. The engineEnvBuilder in the node-core package translates variables such as ENTITY_STORAGE_CONNECTOR_TYPE, VAULT_CONNECTOR_TYPE, and LOGGING_CONNECTOR_TYPE into the corresponding IEngineConfig.types entries, with further variables supplying the backend-specific connection details. The builder also resolves file-path roots, state filenames, and feature flags that control which optional components are active.
Type initialisers reside in engine/packages/engine-types/src/components/ with one file per connector domain. When EngineCore.start() runs, it iterates the registered initialisers in order, constructs the appropriate class based on the configuration entry, registers the instance in the domain factory, and adds it to context.componentInstances for lifecycle management. Because all registration happens in this single startup pass, the full connector graph is consistently wired before any service resolves its dependencies.
Lifecycle Management
Once every connector has been instantiated and registered, the engine calls bootstrap() on all entries in context.componentInstances in registration order. Connectors that return false from bootstrap() signal that initial setup was not fully successful, and the engine records the outcome without aborting the startup sequence. After all bootstrap calls complete, start() is called on every instance in the same order, signalling that normal operation can begin. When the process shuts down, stop() is called in reverse registration order so that dependent connectors are given the opportunity to finalise their own state before the connectors they depend on are torn down.
Connectors that do not require any of these lifecycle phases simply omit the optional methods. The in-memory entity storage connector, for example, requires no external connection setup and therefore implements neither bootstrap nor start. At the other end of the spectrum, a connector that opens a connection pool, migrates a database schema, and subscribes to change notifications during startup would implement all three lifecycle methods with clearly separated responsibilities.
Available Connectors
The table below lists the connector packages available across the platform. Package names follow the naming convention @twin.org/<domain>-connector-<backend>.
| Domain | Backend | Package |
|---|---|---|
| Entity Storage | In-memory | @twin.org/entity-storage-connector-memory |
| Entity Storage | File system | @twin.org/entity-storage-connector-file |
| Entity Storage | MongoDB | @twin.org/entity-storage-connector-mongodb |
| Entity Storage | MySQL | @twin.org/entity-storage-connector-mysql |
| Entity Storage | PostgreSQL | @twin.org/entity-storage-connector-postgresql |
| Entity Storage | ScyllaDB | @twin.org/entity-storage-connector-scylladb |
| Entity Storage | AWS DynamoDB | @twin.org/entity-storage-connector-dynamodb |
| Entity Storage | Azure CosmosDB | @twin.org/entity-storage-connector-cosmosdb |
| Entity Storage | GCP Firestore | @twin.org/entity-storage-connector-gcp-firestore |
| Entity Storage | Synchronised | @twin.org/entity-storage-connector-synchronised |
| Blob Storage | IPFS | @twin.org/blob-storage-connector-ipfs |
| Blob Storage | AWS S3 | @twin.org/blob-storage-connector-aws-s3 |
| Blob Storage | Azure Blob | @twin.org/blob-storage-connector-azure |
| Blob Storage | GCP Storage | @twin.org/blob-storage-connector-gcp |
| Blob Storage | In-memory | @twin.org/blob-storage-connector-memory |
| Blob Storage | File system | @twin.org/blob-storage-connector-file |
| Logging | Console | @twin.org/logging-connector-console |
| Logging | Entity Storage | @twin.org/logging-connector-entity-storage |
| Vault | Entity Storage | @twin.org/vault-connector-entity-storage |
| Vault | HashiCorp Vault | @twin.org/vault-connector-hashicorp |
| Identity | IOTA | @twin.org/identity-connector-iota |
| Identity | Entity Storage | @twin.org/identity-connector-entity-storage |
| Identity | Universal Resolver | @twin.org/identity-connector-universal |
| Wallet | IOTA | @twin.org/wallet-connector-iota |
| Wallet | Entity Storage | @twin.org/wallet-connector-entity-storage |
| NFT | IOTA | @twin.org/nft-connector-iota |
| NFT | Entity Storage | @twin.org/nft-connector-entity-storage |
| Verifiable Storage | IOTA | @twin.org/verifiable-storage-connector-iota |
| Verifiable Storage | Entity Storage | @twin.org/verifiable-storage-connector-entity-storage |
| Attestation | NFT-based | @twin.org/attestation-connector-nft |
| Attestation | Open Attestation | @twin.org/attestation-connector-open-attestation |
| Event Bus | In-process | @twin.org/event-bus-connector-local |
| Telemetry | Entity Storage | @twin.org/telemetry-connector-entity-storage |
| Messaging | AWS | @twin.org/messaging-connector-aws |
| Messaging | Entity Storage | @twin.org/messaging-connector-entity-storage |