forked from github-mirrorer/taskchampion-sync-server
Finish building Postgres support (#133)
This includes: - Building a Docker image for Postgres as well as SQLite - Fuller instructions for usage of the package, including the Postgres builds. A few related things changed here: - `.env` is not used anymore -- the defaults in the Dockerfiles are sufficient - The Rust version in the Dockerfiles is increased to match the MSRV, and with it the Alpine version bumped to one built with that Rust version. - Cargo dependencies on native-tls and openssl updated to include only the `vendored` feature, so as not to require a system openssl installation. - Two GitHub jobs are set up, to build the two different Docker images - The documentation incorrectly suggested using `DELETE .. CASCADE` to delete clients. This syntax does not exist, as the cascading delete is configured in the schema.
This commit is contained in:
Dustin J. Mitchell
committed by
GitHub
GitHub
parent
820aaf363c
commit
ab6df362bf
@ -2,4 +2,10 @@
|
||||
|
||||
- [Introduction](./introduction.md)
|
||||
- [Usage](./usage.md)
|
||||
- [Docker Compose](./usage/docker-compose.md)
|
||||
- [Docker Images](./usage/docker-images.md)
|
||||
- [Binaries](./usage/binaries.md)
|
||||
- [Integration](./integration.md)
|
||||
- [Pre-built Images](./integration/pre-built.md)
|
||||
- [Rust Crates](./integration/crates.md)
|
||||
- [Sync Protocol Implementation](./integration/protocol-impl.md)
|
||||
|
||||
@ -1,3 +1,16 @@
|
||||
# Integration
|
||||
|
||||
TBD (pending Postgres support)
|
||||
Taskchampion-sync-server can be integrated into larger applications, such as
|
||||
web-based hosting services.
|
||||
|
||||
- Most deployments can simply use the pre-built Docker images to implement the
|
||||
sync protocol, handling other aspects of the application in separate
|
||||
containers. See [Pre-built Images](./integration/pre-built.md).
|
||||
|
||||
- More complex deployments may wish to modify or extend the operation of the
|
||||
server. These can use the Rust crates to build precisely the desired
|
||||
functionality. See [Rust Crates](./integration/crates.md).
|
||||
|
||||
- If desired, an integration may completely re-implement the [sync
|
||||
protocol](https://gothenburgbitfactory.org/taskchampion/sync.html). See [Sync
|
||||
Protocol Implementation](./integration/protocol-impl.md).
|
||||
|
||||
17
docs/src/integration/crates.md
Normal file
17
docs/src/integration/crates.md
Normal file
@ -0,0 +1,17 @@
|
||||
# Rust Crates
|
||||
|
||||
This project publishes several Rust crates on `crates.io`:
|
||||
|
||||
- [`taskchampion-sync-server-core`](https://docs.rs/taskchampion-sync-server-core)
|
||||
implements the core of the protocol
|
||||
- [`taskchampion-sync-server-storage-sqlite`](https://docs.rs/taskchampion-sync-server-storage-sqlite)
|
||||
implements an SQLite backend for the core
|
||||
- [`taskchampion-sync-server-storage-postgres`](https://docs.rs/taskchampion-sync-server-storage-postgres)
|
||||
implements a Postgres backend for the core
|
||||
|
||||
If you are building an integration with, for example, a custom storage system,
|
||||
it may be helpful to use the `core` crate and provide a custom implementation
|
||||
of its `Storage` trait.
|
||||
|
||||
We suggest that any generally useful extensions, such as additional storage
|
||||
backends, be published as open-source packages.
|
||||
40
docs/src/integration/pre-built.md
Normal file
40
docs/src/integration/pre-built.md
Normal file
@ -0,0 +1,40 @@
|
||||
# Pre-built Images
|
||||
|
||||
The pre-built Postgres Docker image described in [Docker
|
||||
Images](../usage/docker-images.md) may be adequate for a production deployment.
|
||||
The image is stateless and can be easily scaled horizontally to increase
|
||||
capacity.
|
||||
|
||||
## Database Schema
|
||||
|
||||
The schema defined in
|
||||
[`postgres/schema.sql`](https://github.com/GothenburgBitFactory/taskchampion-sync-server/blob/main/postgres/schema.sql)
|
||||
must be applied to the database before the container will function.
|
||||
|
||||
The schema is stable, and any changes to the schema will be made in a major
|
||||
version with migration instructions provided.
|
||||
|
||||
An integration may:
|
||||
|
||||
- Add additional tables to the database
|
||||
- Add additional columns to the `clients` table. If those columns do not have
|
||||
default values, ensure the server is configured with `CREATE_CLIENTS=false` as
|
||||
described below.
|
||||
- Insert rows into the `clients` table, using default values for all columns
|
||||
except `client_id` and any application-specific columns.
|
||||
- Delete rows from the `clients` table. Note that this table is configured to
|
||||
automatically delete all data associated with a client when the client's row is
|
||||
deleted.
|
||||
|
||||
## Managing Clients
|
||||
|
||||
By default, taskchampion-sync-server creates a new, empty client when it
|
||||
receives a connection from an unrecognized client ID. Setting
|
||||
`CREATE_CLIENTS=false` disables this functionality, and is recommended in
|
||||
production deployments to avoid abuse.
|
||||
|
||||
In this configuration, it is the responsibility of the integration to create
|
||||
new client rows when desired, using a statement like `INSERT into clients
|
||||
(client_id) values ($1)` with the new client ID as a parameter. Similarly,
|
||||
clients may be deleted, along with all stored task data, using a statement like
|
||||
`DELETE from clients where client_id = $1`.
|
||||
10
docs/src/integration/protocol-impl.md
Normal file
10
docs/src/integration/protocol-impl.md
Normal file
@ -0,0 +1,10 @@
|
||||
# Sync Protocol Implementation
|
||||
|
||||
The [sync protocol](https://gothenburgbitfactory.org/taskchampion/sync.html) is
|
||||
an open specification, and can be re-implemented from that specification as
|
||||
desired. This specification is not battle-tested, so refer to
|
||||
taskchampion-sync-server's implementation to resolve any ambiguities, and
|
||||
please create pull requests to resolve the ambiguity in the specification.
|
||||
|
||||
We suggest that new implementations be published as open-source packages where
|
||||
possible.
|
||||
@ -1,11 +1,30 @@
|
||||
# Introduction
|
||||
|
||||
Taskchampion Sync-Server is an implementation of the TaskChampion [sync
|
||||
Taskchampion-sync-server is an implementation of the TaskChampion [sync
|
||||
protocol][sync-protocol] server. It supports synchronizing Taskwarrior tasks
|
||||
between multiple systems.
|
||||
|
||||
[sync-protocol]: https://gothenburgbitfactory.org/taskchampion/sync.html
|
||||
|
||||
The project provides both pre-built images for common use-cases (see
|
||||
[usage](./usage.md)) and Rust libraries that can be used to build more
|
||||
sophisticated applications ([integration](./integration.md)).
|
||||
|
||||
It also serves as a reference implementation: where the
|
||||
[specification][sync-protocol] is ambiguous, this implementation's
|
||||
interpretation is favored in resolving the ambiguity. Other implementations of
|
||||
the protocol should interoperate with this implementation.
|
||||
|
||||
## Sync Overview
|
||||
|
||||
The server identifies each user with a client ID. For example, when
|
||||
syncing Taskwarrior tasks between a desktop computer and a laptop, both systems
|
||||
would use the same client ID to indicate that they share the same user's task data.
|
||||
|
||||
Task data is encrypted, and the server does not have access to the encryption
|
||||
secret. The server sees only encrypted data and cannot read or modify tasks in
|
||||
any way.
|
||||
|
||||
To perform a sync, a replica first downloads and decrypts any changes that have
|
||||
been sent to the server since its last sync. It then gathers any local changes,
|
||||
encrypts them, and uploads them to the server.
|
||||
|
||||
[sync-protocol]: https://gothenburgbitfactory.org/taskchampion/sync.html
|
||||
|
||||
@ -1,87 +1,22 @@
|
||||
# Usage
|
||||
|
||||
## Running the Server
|
||||
This repository is flexible and can be used in a number of ways, to suit your
|
||||
needs.
|
||||
|
||||
The server is a simple binary that serves HTTP requests on a TCP port. The
|
||||
server does not implement TLS; for public deployments, the recommendation is to
|
||||
use a reverse proxy such as Nginx, haproxy, or Apache httpd.
|
||||
- If you only need a place to sync your tasks, using cloud storage may be
|
||||
cheaper and easier than running taskchampion-sync-server. See
|
||||
[task-sync(5)](http://taskwarrior.org/docs/man/task-sync.5/) for details on
|
||||
cloud storage.
|
||||
|
||||
### Using Docker-Compose
|
||||
- If you have a publicly accessible server, such as a VPS, you can use `docker
|
||||
compose` to run taskchampion-sync-server as pre-built docker images. See
|
||||
[Docker Compose](./usage/docker-compose.md).
|
||||
|
||||
Every release of the server generates a Docker image in
|
||||
`ghcr.io/gothenburgbitfactory/taskchampion-sync-server`. The tags include
|
||||
`latest` for the latest release, and both minor and patch versions, e.g., `0.5`
|
||||
and `0.5.1`.
|
||||
- If you would like more control, such as to deploy taskchampion-sync-server
|
||||
within an orchestration environment such as Kubernetes, you can deploy the
|
||||
docker images directly. See [Docker Images](./usage/docker-images.md).
|
||||
|
||||
The
|
||||
[`docker-compose.yml`](https://raw.githubusercontent.com/GothenburgBitFactory/taskchampion-sync-server/refs/tags/v0.6.1/docker-compose.yml)
|
||||
file in this repository is sufficient to run taskchampion-sync-server,
|
||||
including setting up TLS certificates using Lets Encrypt, thanks to
|
||||
[Caddy](https://caddyserver.com/).
|
||||
|
||||
You will need a server with ports 80 and 443 open to the Internet and with a
|
||||
fixed, publicly-resolvable hostname. These ports must be available both to your
|
||||
Taskwarrior clients and to the Lets Encrypt servers.
|
||||
|
||||
On that server, download `docker-compose.yml` from the link above (it is pinned
|
||||
to the latest release) into the current directory. Then run
|
||||
|
||||
```sh
|
||||
TASKCHAMPION_SYNC_SERVER_HOSTNAME=taskwarrior.example.com \
|
||||
TASKCHAMPION_SYNC_SERVER_CLIENT_ID=your-client-id \
|
||||
docker compose up
|
||||
```
|
||||
|
||||
The `TASKCHAMPION_SYNC_SERVER_CLIENT_ID` limits the server to the given client
|
||||
ID; omit it to allow all client IDs.
|
||||
|
||||
It can take a few minutes to obtain the certificate; the caddy container will
|
||||
log a message "certificate obtained successfully" when this is complete, or
|
||||
error messages if the process fails. Once this process is complete, configure
|
||||
your `.taskrc`'s to point to the server:
|
||||
|
||||
```none
|
||||
sync.server.url=https://taskwarrior.example.com
|
||||
sync.server.client_id=your-client-id
|
||||
sync.encryption_secret=your-encryption-secret
|
||||
```
|
||||
|
||||
The docker-compose images store data in a docker volume named
|
||||
`taskchampion-sync-server_data`. This volume contains all of the task data, as
|
||||
well as the TLS certificate information. It will persist over restarts, in a
|
||||
typical Docker installation. The docker containers will start automatically on
|
||||
system startup. See the docker-compose documentation for more information.
|
||||
|
||||
### Running the Binary
|
||||
|
||||
The server is configured with command-line options. See
|
||||
`taskchampion-sync-server --help` for full details.
|
||||
|
||||
The `--listen` option specifies the interface and port the server listens on.
|
||||
It must contain an IP-Address or a DNS name and a port number. This option is
|
||||
mandatory, but can be repeated to specify multiple interfaces or ports. This
|
||||
value can be specified in environment variable `LISTEN`, as a comma-separated
|
||||
list of values.
|
||||
|
||||
The `--data-dir` option specifies where the server should store its data. This
|
||||
value can be specified in the environment variable `DATA_DIR`.
|
||||
|
||||
By default, the server will allow all clients and create them in the database
|
||||
on first contact. There are two ways to limit the clients the server will
|
||||
interact with:
|
||||
|
||||
- To limit the accepted client IDs, specify them in the environment variable
|
||||
`CLIENT_ID`, as a comma-separated list of UUIDs. Client IDs can be specified
|
||||
with `--allow-client-id`, but this should not be used on shared systems, as
|
||||
command line arguments are visible to all users on the system. This convenient
|
||||
option is suitable for personal and small-scale deployments.
|
||||
|
||||
- To disable the automatic creation of clients, use the `--no-create-clients`
|
||||
flag or the `CREATE_CLIENTS=false` environment variable. You are now
|
||||
responsible for creating clients in the database manually, so this option is
|
||||
more suitable for large scale deployments.
|
||||
|
||||
The server only logs errors by default. To add additional logging output, set
|
||||
environment variable `RUST_LOG` to `info` to get a log message for every
|
||||
request, or to `debug` to get more verbose debugging output.
|
||||
- For even more control, or to avoid the overhead of container images, you can
|
||||
build and run the taskchampion-sync-server binary directly. See
|
||||
[Binaries](./usage/binaries.md).
|
||||
|
||||
|
||||
49
docs/src/usage/binaries.md
Normal file
49
docs/src/usage/binaries.md
Normal file
@ -0,0 +1,49 @@
|
||||
# Binaries
|
||||
|
||||
Taskchampion-sync-server is a single binary that serves HTTP requests on a TCP
|
||||
port. The server does not implement TLS; for public deployments, the
|
||||
recommendation is to use a reverse proxy such as Nginx, haproxy, or Apache
|
||||
httpd.
|
||||
|
||||
One binary is provided for each storage backend:
|
||||
|
||||
- `taskchampion-sync-server` (SQLite)
|
||||
- `taskchampion-sync-server-postgres` (Postgres)
|
||||
|
||||
### Running the Binary
|
||||
|
||||
The server is configured with command-line options or environment variables.
|
||||
See the `--help` output for full details.
|
||||
|
||||
For the SQLite binary, the `--data-dir` option or `DATA_DIR` environment
|
||||
variable specifies where the server should store its data. For the Postgres
|
||||
binary, the `--connection` option or `CONNECTION` environment variable
|
||||
specifies the connection information, in the form of a [LibPQ-style connection
|
||||
URI](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS).
|
||||
The remaining options are common to all binaries.
|
||||
|
||||
The `--listen` option specifies the interface and port the server listens on.
|
||||
It must contain an IP-Address or a DNS name and a port number. This option is
|
||||
mandatory, but can be repeated to specify multiple interfaces or ports. This
|
||||
value can be specified in environment variable `LISTEN`, as a comma-separated
|
||||
list of values.
|
||||
|
||||
By default, the server will allow all clients and create them in the database
|
||||
on first contact. There are two ways to limit the clients the server will
|
||||
interact with:
|
||||
|
||||
- To limit the accepted client IDs, specify them in the environment variable
|
||||
`CLIENT_ID`, as a comma-separated list of UUIDs. Client IDs can be specified
|
||||
with `--allow-client-id`, but this should not be used on shared systems, as
|
||||
command line arguments are visible to all users on the system. This convenient
|
||||
option is suitable for personal and small-scale deployments.
|
||||
|
||||
- To disable the automatic creation of clients, use the `--no-create-clients`
|
||||
flag or the `CREATE_CLIENTS=false` environment variable. You are now
|
||||
responsible for creating clients in the database manually, so this option is
|
||||
more suitable for large scale deployments. See [Integration](../integration.md)
|
||||
for more information on such deployments.
|
||||
|
||||
The server only logs errors by default. To add additional logging output, set
|
||||
environment variable `RUST_LOG` to `info` to get a log message for every
|
||||
request, or to `debug` to get more verbose debugging output.
|
||||
43
docs/src/usage/docker-compose.md
Normal file
43
docs/src/usage/docker-compose.md
Normal file
@ -0,0 +1,43 @@
|
||||
# Docker Compose
|
||||
|
||||
The
|
||||
[`docker-compose.yml`](https://raw.githubusercontent.com/GothenburgBitFactory/taskchampion-sync-server/refs/tags/v0.6.1/docker-compose.yml)
|
||||
file in this repository is sufficient to run taskchampion-sync-server,
|
||||
including setting up TLS certificates using Lets Encrypt, thanks to
|
||||
[Caddy](https://caddyserver.com/). This setup uses the SQLite backend, which is
|
||||
adequate for one or a few clients.
|
||||
|
||||
You will need a server with ports 80 and 443 open to the Internet and with a
|
||||
fixed, publicly-resolvable hostname. These ports must be available both to your
|
||||
Taskwarrior clients and to the Lets Encrypt servers.
|
||||
|
||||
On that server, download `docker-compose.yml` from the link above (it is pinned
|
||||
to the latest release) into the current directory. Then run
|
||||
|
||||
```sh
|
||||
TASKCHAMPION_SYNC_SERVER_HOSTNAME=taskwarrior.example.com \
|
||||
TASKCHAMPION_SYNC_SERVER_CLIENT_ID=your-client-id \
|
||||
docker compose up
|
||||
```
|
||||
|
||||
The `TASKCHAMPION_SYNC_SERVER_CLIENT_ID` limits the server to the given client
|
||||
ID; omit it to allow all client IDs. You may specify multiple client IDs
|
||||
separated by commas.
|
||||
|
||||
It can take a few minutes to obtain the certificate; the caddy container will
|
||||
log a message "certificate obtained successfully" when this is complete, or
|
||||
error messages if the process fails. Once this process is complete, configure
|
||||
your `.taskrc`'s to point to the server:
|
||||
|
||||
```none
|
||||
sync.server.url=https://taskwarrior.example.com
|
||||
sync.server.client_id=your-client-id
|
||||
sync.encryption_secret=your-encryption-secret
|
||||
```
|
||||
|
||||
The docker-compose images store data in a docker volume named
|
||||
`taskchampion-sync-server_data`. This volume contains all of the task data, as
|
||||
well as the TLS certificate information. It will persist over restarts, in a
|
||||
typical Docker installation. The docker containers will start automatically
|
||||
when the Docker dameon starts. See the docker-compose documentation for more
|
||||
information.
|
||||
57
docs/src/usage/docker-images.md
Normal file
57
docs/src/usage/docker-images.md
Normal file
@ -0,0 +1,57 @@
|
||||
# Docker Images
|
||||
|
||||
Every release of the server generates Docker images. One image is produced for
|
||||
each storage backend:
|
||||
- `ghcr.io/gothenburgbitfactory/taskchampion-sync-server` (SQLite)
|
||||
- `ghcr.io/gothenburgbitfactory/taskchampion-sync-server-postgres` (Postgres)
|
||||
|
||||
The image tags include `latest` for the latest release, and both minor and
|
||||
patch versions, e.g., `0.5` and `0.5.1`.
|
||||
|
||||
## Running the Image
|
||||
|
||||
At startup, each image applies some default values and runs the relevant binary
|
||||
directly. Configuration is typically by environment variables, all of which are
|
||||
documented in the `--help` output of the binaries. These include
|
||||
|
||||
- `RUST_LOG` - log level, one of `trace`, `debug`, `info`, `warn` and `error`.
|
||||
- `DATA_DIR` (SQLite only; default `/var/lib/taskchampion-sync-server/data`) -
|
||||
directory for the synced data.
|
||||
- `CONNECTION` (Postgres only) - Postgres connection information, in the form
|
||||
of a [LibPQ-style connection
|
||||
URI](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING-URIS).
|
||||
- `LISTEN` (default `0.0.0.0:8080`) - address and port on which to listen for
|
||||
HTTP requests.
|
||||
- `CLIENT_ID` - comma-separated list of client IDs that will be allowed, or
|
||||
empty to allow all clients.
|
||||
- `CREATE_CLIENTS` (default `true`) - if true, automatically create clients on
|
||||
first sync. If this is set to false, it is up to you to initialize clients in
|
||||
the DB.
|
||||
|
||||
### Example
|
||||
|
||||
```shell
|
||||
docker run -d \
|
||||
--name=taskchampion-sync-server \
|
||||
-p 8080:8080 \
|
||||
-e RUST_LOG=debug \
|
||||
-v /data/taskchampion-sync-server:/var/lib/taskchampion-sync-server/data \
|
||||
taskchampion-sync-server
|
||||
```
|
||||
|
||||
### Image-Specific Setup
|
||||
|
||||
The SQLite image is configured with `VOLUME
|
||||
/var/lib/taskchampion-sync-server/data`, persisting the task data in an
|
||||
anonymous Docker volume. It is recommended to put this on a named volume, or
|
||||
persistent storage in an environment like Kubernetes, so that it is not
|
||||
accidentally deleted.
|
||||
|
||||
The Postgres image does not automatically create its database schema. See the
|
||||
[integration section](../integration/pre-built.md) for more detail. This
|
||||
implementation is tested with Postgres version 17 but should work with any
|
||||
recent version.
|
||||
|
||||
Note that the Docker images do not implement TLS. The expectation is that
|
||||
another component, such as a Kubernetes ingress, will terminate the TLS
|
||||
connection and proxy HTTP traffic to the taskchampion-sync-server container.
|
||||
Reference in New Issue
Block a user