Partial first draft of a manual, generated using Daux

.gitignore

@@ -3,6 +3,7 @@
 /vendor/
 /vendor-bin/*/vendor
 /documentation/
+/manual/
 /tests/coverage/
 /arsse.db*
 /config.php

RoboFile.php

@@ -17,7 +17,7 @@ class RoboFile extends \Robo\Tasks {
      * Please see the PHPUnit documentation for available options.
     */
     public function test(array $args): Result {
-        return $this->runTests("php", "typical", $args);
+        return $this->runTests(escapeshellarg(\PHP_BINARY), "typical", $args);
     }
 
     /**
@@ -27,7 +27,7 @@ class RoboFile extends \Robo\Tasks {
      * See help for the "test" task for more details.
     */
     public function testFull(array $args): Result {
-        return $this->runTests("php", "full", $args);
+        return $this->runTests(escapeshellarg(\PHP_BINARY), "full", $args);
     }
 
     /**
@@ -36,7 +36,7 @@ class RoboFile extends \Robo\Tasks {
      * See help for the "test" task for more details.
     */
     public function testQuick(array $args): Result {
-        return $this->runTests("php", "quick", $args);
+        return $this->runTests(escapeshellarg(\PHP_BINARY), "quick", $args);
     }
 
     /** Produces a code coverage report
@@ -161,7 +161,7 @@ class RoboFile extends \Robo\Tasks {
             $dir."robo.bat",
         ]);
         // generate a sample configuration file
-        $t->taskExec("php arsse.php conf save-defaults config.defaults.php")->dir($dir);
+        $t->taskExec(escapeshellarg(\PHP_BINARY)." arsse.php conf save-defaults config.defaults.php")->dir($dir);
         // package it all up
         $t->taskPack($archive)->addDir("arsse", $dir);
         // execute the collection
@@ -170,4 +170,9 @@ class RoboFile extends \Robo\Tasks {
         $this->_exec("git worktree prune");
         return $out;
     }
+
+    public function manual(array $args): Result {
+        $execpath = escapeshellarg(realpath(self::BASE."vendor/bin/daux"));
+        return $this->taskExec($execpath)->arg("generate")->option("-d", self::BASE."manual")->args($args)->run();
+    }
 }

docs/config.json

@@ -0,0 +1,11 @@
+{
+    "title": "The Arsse",
+    "tagline": "The clean & modern RSS server that doesn't give you any crap.",
+    "languages": {
+        "en": "English"
+    },
+    "html": {
+        "float": false,
+        "toggle_code": false
+    }
+}

docs/en/010_About.md

@@ -0,0 +1,6 @@
+The Arsse is a news aggregator server which implements multiple synchronization protocols, including [version 1.2](https://github.com/nextcloud/news/blob/master/docs/externalapi/Legacy.md) of [NextCloud News](https://github.com/nextcloud/news)' protocol and the [Tiny Tiny RSS](https://git.tt-rss.org/git/tt-rss/wiki/ApiReference) protocol (details below). Unlike most other aggregator servers, The Arsse does not include a Web front-end (though one is planned as a separate project), and it relies on existing protocols to maximize compatibility with existing clients.
+
+At present the software should be considered in an "alpha" state: though its core subsystems are covered by unit tests and should be free of major bugs, not everything has been rigorously tested. Additionally, many features one would expect from other similar software have yet to be implemented. Areas of future work include:
+
+- Providing more sync protocols (Google Reader, Fever, others)
+- Better packaging and configuration samples

docs/en/020_Installation/010_Requirements.md

@@ -0,0 +1,11 @@
+The Arsse has the following requirements:
+
+- A Linux server utilizing systemd and Nginx (tested on Ubuntu 16.04)
+- PHP 7.0.7 or later with the following extensions:
+    - [intl](http://php.net/manual/en/book.intl.php), [json](http://php.net/manual/en/book.json.php), and [hash](http://php.net/manual/en/book.hash.php)
+    - [dom](http://php.net/manual/en/book.dom.php), [simplexml](http://php.net/manual/en/book.simplexml.php), and [iconv](http://php.net/manual/en/book.iconv.php) (for picoFeed)
+    - One of:
+        - [sqlite3](http://php.net/manual/en/book.sqlite3.php) or [pdo_sqlite](http://php.net/manual/en/ref.pdo-sqlite.php) for SQLite databases
+        - [pgsql](http://php.net/manual/en/book.pgsql.php) or [pdo_pgsql](http://php.net/manual/en/ref.pdo-pgsql.php) for PostgreSQL 10 or later databases
+        - [mysqli](http://php.net/manual/en/book.mysqli.php) or [pdo_mysql](http://php.net/manual/en/ref.pdo-mysql.php) for MySQL 8.0.7 or later databases
+- Privileges to create and run daemon processes on the server

docs/en/020_Installation/040_Database_Setup/000_SQLite.md

@@ -0,0 +1,3 @@
+SQLite requires very little setup. By default the database will be created at the root of The Arsse's program directory (e.g. `/usr/share/arsse/arsse.db`), but this can be changed with the [`dbSQLite3File` setting](/en/Configuring_The_Arsse#dbSQLite3File). 
+
+Regardless of the location chosen, The Arsse **must** be able to both read from and write to the database file, as well as create files in the directory containing it. This is because SQLite also creates a write-ahead log file and a shared-memory file during operation. 

docs/en/020_Installation/040_Database_Setup/index.md

@@ -0,0 +1,13 @@
+The Arsse supports the following database backends:
+
+- SQLite 3.8.3 and later
+- PostgreSQL 10 and later
+- MySQL 8.07 and later
+
+All of the above are supported both via their PDO driver extensions as well as their native PHP extensions. One or the other is selected based on availability in your PHP installation.
+
+Functionally there is no reason to prefer either SQLite or PostgreSQL over the other. SQLite is significantly simpler to set up in most cases, requiring only read and write access to a containing directory in order to function; PostgreSQL may perform better than SQLite when serving hundreds of users or more, though this has not been tested.
+
+MySQL, on the other hand, is **not recommended** due to its relatively constrained index prefix limits which may cause some newsfeeds which would otherwise work to be rejected. If using MySQL, special care should also be taken when performing schema upgrades, as errors during the process can leave the database in a half-upgraded state which The Arsse cannot itself recover from.
+
+Note that MariaDB is not compatible with The Arsse: its support for common table expressions is, as of this writing, not sufficient for our needs.

docs/en/050_Configuring_The_Arsse.md

@@ -0,0 +1,368 @@
+## General settings
+
+### lang
+
+| Type   | Default |
+|--------|---------|
+| string | `"en"`  |
+
+The default language locale, mostly used when reporting errors on the command line or in logs. Currently only `"en"` (English) is available.
+
+## Database Settings
+
+### dbDriver
+
+| Type   | Default     |
+|--------|-------------|
+| string | `"sqlite3"` |
+
+The database driver to use. The following values are understood:
+
+- `"sqlite3"` for SQLite databases
+- `"postgresql"` for PostgreSQL databases
+- `"mysql"` for MySQL databases
+
+It is also possible to specify the fully-qualified name of a class which implements the database driver interface. For example, specifying `"JKingWeb\Arsse\Db\SQLite3\PDODriver"` would use the PDO driver for SQLite 3.
+
+### dbAutoUpdate
+
+| Type    | Default |
+|---------|---------|
+| boolean | `true`  |
+
+Whether to attempt to automatically upgrade the database schema when upgrading The Arsse to a new version with schema changes.
+
+If set to `false`, the database schema must be manually upgraded. Schema files can be found under `sql/<backend>/#.sql`; the `UPGRADING` file will advise when a schema upgrade is required.
+
+### dbTimeoutConnect
+
+| Type     | Default |
+|----------|---------|
+| interval | `5.0`   |
+
+The number of seconds to wait before returning a timeout error when connecting to a database server.
+
+Consult [PostgreSQL's documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-CONNECT-TIMEOUT) for details on how the timeout is interpreted by PostgreSQL. This setting does not apply to SQLite.
+
+### dbTimeoutExec
+
+| Type     | Default |
+|----------|---------|
+| interval | `0.0`   |
+
+The number of seconds to wait before returning a timeout error when executing a database operation (i.e. computing results).
+
+PostgreSQL and MySQL both interpret `0.0` as "wait forever". This setting does not apply to SQLite.
+
+## Database Settings Specific to SQLite 3
+
+### dbSQLite3File
+
+| Type           | Default |
+|----------------|---------|
+| string or null | `null`  |
+
+The full path and file name of SQLite database. The special value `null` evaluates to a file named `"arsse.db"` in the directory where The Arsse is installed.
+
+### dbSQLite3Key
+
+| Type   | Default |
+|--------|---------|
+| string | `""`    |
+
+The key used to encrypt/decrypt the SQLite database. This is only relevant if using the [SQLite Encryption Extension](https://www.sqlite.org/see/).
+
+### dbSQLite3Timeout
+
+| Type     | Default |
+|----------|---------|
+| interval | `60.0`  |
+
+The number of seconds for SQLite to wait before returning a timeout error when trying to acquire a write lock on the database file. Setting this to a low value may cause operations to fail with an error.
+
+Consult [SQLite's documentation](https://sqlite.org/c3ref/busy_timeout.html) for more details.
+
+## Database Settings Specific to PostgreSQL
+
+### dbPostgreSQLHost
+
+| Type   | Default |
+|--------|---------|
+| string | `""`    |
+
+The host name, address, or socket path of the PostgreSQL database server.
+
+Consult [PostgreSQL's documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNECT-HOST) for more details.
+
+### dbPostgreSQLUser
+
+| Type   | Default   |
+|--------|-----------|
+| string | `"arsse"` |
+
+The log-in user name for the PostgreSQL database server.
+
+### dbPostgreSQLPass
+
+| Type   | Default |
+|--------|---------|
+| string | `""`    |
+
+The log-in password for the PostgreSQL database server.
+
+### dbPostgreSQLPort
+
+| Type    | Default |
+|---------|---------|
+| integer | `5432`  |
+
+The TCP port on which to connect to the PostgreSQL database server, if connecting via TCP.
+
+### dbPostgreSQLDb
+
+| Type   | Default   |
+|--------|-----------|
+| string | `"arsse"` |
+
+The name of the database used by The Arsse on the PostgreSQL database server.
+
+### dbPostgreSQLSchema
+
+| Type   | Default |
+|--------|---------|
+| string | `""`    |
+
+The name of the schema used by The Arsse on the PostgreSQL database server. When not set to the empty string, the PostgreSQL search path is modified to consist of the specified schema with a fallback to the public schema.
+
+Consult [PostgreSQL's documentation](https://www.postgresql.org/docs/current/ddl-schemas.html) for more details.
+
+### dbPostgreSQLService
+
+| Type   | Default |
+|--------|---------|
+| string | `""`    |
+
+A PostgreSQL service file entry to use *instead of* the above configuration; if using a service entry all above PostgreSQL-specific parameters except schema are ignored.
+
+Consult [PostgreSQL's documentation](https://www.postgresql.org/docs/current/libpq-pgservice.html) for more details.
+
+## Database Settings Specific to MySQL
+
+### dbMySQLHost
+
+| Type   | Default       |
+|--------|---------------|
+| string | `"localhost"` |
+
+The host name or address of the MySQL database server. The values `"localhost"` and `"."` are special.
+
+Consult [MySQL's documentation](https://dev.mysql.com/doc/refman/8.0/en/connecting.html) for more details.
+
+### dbMySQLUser
+
+| Type   | Default   |
+|--------|-----------|
+| string | `"arsse"` |
+
+The log-in user name for the MySQL database server.
+
+### dbMySQLPass
+
+| Type   | Default |
+|--------|---------|
+| string | `""`    |
+
+The log-in password for the MySQL database server.
+
+### dbMySQLPort
+
+| Type    | Default |
+|---------|---------|
+| integer | `3306`  |
+
+The TCP port on which to connect to the MySQL database server, if connecting via TCP.
+
+### dbMySQLDb
+
+| Type   | Default   |
+|--------|-----------|
+| string | `"arsse"` |
+
+The name of the database used by The Arsse on the MySQL database server.
+
+### dbMySQLSocket
+
+| Type   | Default |
+|--------|---------|
+| string | `""`    |
+
+A Unix domain socket or named pipe to use for the MySQL database server when not connecting via TCP.
+
+## User Management Settings
+
+### userDriver
+
+| Type   | Default      |
+|--------|--------------|
+| string | `"internal"` |
+
+The user management driver to use. Currently only `"internal"` is available, which stores user IDs and hashed passwords in the configured database.
+
+It is also possible to specify the fully-qualified name of a class which implements the user management driver interface. For example, specifying `"JKingWeb\Arsse\User\Internal\Driver"` would use the internal driver.
+
+### userPreAuth
+
+| Type    | Default |
+|---------|---------|
+| boolean | `false` |
+
+Whether users are authenticated by the Web server before requests are passed to The Arsse. If set to `true` The Arsse will perform no HTTP-level authentication and assume that the user ID supplied in either the `REMOTE_USER` CGI variable or the `Authorization` HTTP header-field (if `Basic` authentication was used) is authentic.  
+
+For synchronization protocols which implement their own authentication (such as Tiny Tiny RSS), this setting may or may not affect how protocol-level authentication is handled; consult the section on The Arsse's [supported protocols](/en/Supported_Protocols) for more information.
+
+If the user has not previously logged in, an entry is created for them in the database automatically. If the Web server uses `Basic` HTTP authentication and passes along the `Authorization` field, a hash of the user's password will also be stored in The Arsse's database.
+
+### userHTTPAuthRequired
+
+| Type    | Default |
+|---------|---------|
+| boolean | `false` |
+
+Whether to require successful HTTP authentication before processing API-level authentication, for protocol which implement their own authentication.
+
+### userSessionEnforced
+
+| Type    | Default |
+|---------|---------|
+| boolean | `true`  |
+
+Whether invalid or expired API session tokens should prevent logging in when HTTP authentication is used, for protocol which implement their own authentication.
+
+### userSessionTimeout
+
+| Type     | Default   |
+|----------|-----------|
+| interval | `"PT24H"` |
+
+The period of inactivity after which log-in sessions should be considered invalid. Session timeouts should not be made too long, to guard against session hijacking.
+
+### userSessionLifetime
+
+| Type     | Default   |
+|----------|-----------|
+| interval | `"P7D"`   |
+
+The maximum lifetime of log-in sessions regardless of recent activity. Session lifetimes should not be made too long, to guard against session hijacking.
+
+### userTempPasswordLength
+
+| Type    | Default |
+|---------|---------|
+| integer | `20`    |
+
+The desired length in characters of randomly-generated user passwords. When [adding users](/en/Command-Line_Functionality), omitting a desired password generates a random one; this setting controls the length of these passwords.
+
+## Newsfeed Fetching Service Settings
+
+### serviceDriver
+
+| Type   | Default        |
+|--------|----------------|
+| string | `"subprocess"` |
+
+The newsfeed fetching service driver to use. The following values are understood:
+
+- `"serial"`: Fetches feeds and processed them one at a time. This is the slowest method, but is simple and reliable.
+- `"subprocess"`: Fetches and processes multiple feeds concurrently by starting a separate process for each feed using PHP's [`popen`](https://php.net/manual/en/function.popen.php) function. This uses more memory and processing power, but takes less total time.
+
+It is also possible to specify the fully-qualified name of a class which implements the service driver interface. For example, specifying `"JKingWeb\Arsse\Service\Serial\Driver"` would use the serial driver.
+
+### serviceFrequency
+
+| Type     | Default  |
+|----------|----------|
+| interval | `"PT2M"` |
+
+The interval the newsfeed fetching service observes between checks for new articles. Note that requests to foreign servers are not necessarily made at this frequency: each newsfeed is assigned its own time at which to be next retrieved. This setting instead defines the length of time the fetching service will sleep between periods of activity.
+
+Consult "[Newsfeed Refresh Schedule](/en/Installation/Using_the_Daemon#Newsfeed_Refresh_Schedule)" for details on how often newsfeeds are fetched.
+
+### serviceQueueWidth
+
+| Type    | Default |
+|---------|---------|
+| integer | `5`     |
+
+The maximum number of concurrent feed updates to perform, if a concurrent service driver is used. 
+
+### fetchTimeout
+
+| Type     | Default |
+|----------|---------|
+| interval | `10.0`  |
+
+The maximum number of seconds to wait for data when fetching newsfeeds from foreign servers.
+
+### fetchSizeLimit
+
+| Type    | Default           |
+|---------|-------------------|
+| integer | `2 * 1024 * 1024` |
+
+The maximum size, in bytes, of data to accept when fetching a newsfeed. Newsfeeds larger than this will be rejected to guard against denial-of-servioce attacks.
+
+The default value is equal to two megabytes.
+
+### fetchEnableScraping
+
+| Type    | Default |
+|---------|---------|
+| boolean | `true`  |
+
+Whether to allow the possibility of fetching full article contents from an article's source, if a newsfeed only provides excerpts. Whether fetching will actually happen is governed by a per-newsfeed toggle (defaulting to `false`) which currently can only be changed by manually editing the database.
+
+### fetchUserAgentString
+
+| Type           | Default |
+|----------------|---------|
+| string or null | `null`  |
+
+The [user agent](https://tools.ietf.org/html/rfc7231#section-5.5.3) The Arsse will identify as when fetching newsfeeds. The special value null will use an identifier similar to the following:
+
+```
+Arsse/0.6.0 (Linux 4.15.0; x86_64; https://thearsse.com/)
+```
+
+## Housekeeping settings
+
+### purgeFeeds
+
+| Type             | Default   |
+|------------------|-----------|
+| interval or null | `"PT24H"` |
+
+How long to keep a newsfeed and its articles in the database after all its subscriptions have been deleted. Specifying `null` will retain unsubscribed newsfeeds forever, whereas an interval evaluating to zero (e.g. `"PT0S"`) will delete them immediately.
+
+Note that articles of orphaned feeds are still subject to the `purgeArticleUnread` threshold below.
+
+### purgeArticlesRead
+
+| Type             | Default |
+|------------------|---------|
+| interval or null | `"P7D"` |
+
+How long to keep a an article in the database after all users subscribed to its feed have read it. Specifying `null` will retain articles up to the `purgeArticlesUnread` threshold below, whereas an interval evaluating to zero (e.g. `"PT0S"`) will delete them immediately. 
+
+If an article is starred by any user, it is retained indefinitely regardless of this setting.
+
+This setting also governs when an article is hidden from a user after being read by that user, regardless of its actual presence in the database.
+
+### purgeArticlesUnread
+
+| Type             | Default  |
+|------------------|----------|
+| interval or null | `"P21D"` |
+
+How long to keep a an article in the database regardless of whether any users have read it. Specifying `null` will retain articles forever, whereas an interval evaluating to zero (e.g. `"PT0S"`) will delete them immediately. 
+
+If an article is starred by any user, it is retained indefinitely regardless of this setting.