- Update Docopt
- Switch toin-house Picofeed branch for now
- Update composer-bin
- Update php-cs-fixer
Daux has been left as-is for now even though we're using an old version
The Feed class sets dynamic properties on Picofeed classes; this will
need to be completely rewritten. Version 1.2 of the Laminas XML class
also uses a deprecated function, but upgrading it to 1.3 will require
PHP 7.3.
For now this only adds convenience wrappers around Guzzle to somewhat
emulate Diactoros (albeit with a different API). Code and tests will be
adjusted in due course.
Daux uses Cmmonmark, which does not support indention, required for
proper formatting of manual pages. Consequently, the manul page will
instead be standalone.
@ -8,7 +8,7 @@ Information on how to install and use the software can be found in [the manual](
The main repository for The Arsse can be found at [code.mensbeam.com](https://code.mensbeam.com/MensBeam/arsse/), with a mirror also available [at GitHub](https://github.com/mensbeam/arsse/). The GitHub mirror does not accept bug reports, but the two should otherwise be equivalent.
[Composer](https://getcomposer.org/) is required to manage PHP dependencies. After cloning the repository or downloading a source code tarball, running `composer install` will download all the required dependencies, and will advise if any PHP extensions need to be installed. If not installing as a programming environment, running `composer install --no-dev` is recommended.
[Composer](https://getcomposer.org/) is required to manage PHP dependencies. After cloning the repository or downloading a source code tarball, running `composer install` will download all the required dependencies, and will advise if any PHP extensions need to be installed. If not installing as a programming environment, running `composer install -o --no-dev --no-scripts` is recommended.
# Repository structure
@ -34,13 +34,15 @@ Also necessary to the functioning of the application is the `/vendor/` directory
The `/locale/` and `/sql/` directories contain human-language files and database schemata, both of which are occasionally used by the application in the course of execution. The `/www/` directory serves as a document root for a few static files to be made available to users by a Web server.
The `/dist/` directory, on the other hand, contains samples of configuration for Web servers and init systems. These are not used by The Arsse itself, but are merely distributed with it for reference.
The `/dist/` directory, on the other hand, contains general and system-specific build files, manual pages, and samples of configuration for Web servers and other system integration. These are not used by The Arsse itself, but are used during the process of preparing new releases for supported operating systems.
## Documentation
The source text for The Arsse's manual can be found in `/docs/`, with pages written in [Markdown](https://spec.commonmark.org/current/) and converted to HTML [with Daux](#building-the-manual). If a static manual is generated its files will appear under `/manual/`.
In addition to the manual the files `/README.md` (this file), `/CHANGELOG`, `/UPGRADING`, `/LICENSE`, and `/AUTHORS` also document various things about the software, rather than the software itself.
The Arsse also has a UNIX manual page, also written in Markdown, which can be found under `/manpages/`. [Pandoc](https://pandoc.org/) is needed to convert it to the appropriate format, with the results stored under `/dist/man/`.
In addition to the manuals the files `/README.md` (this file), `/CHANGELOG`, `/UPGRADING`, `/LICENSE`, and `/AUTHORS` also document various things about the software, rather than the software itself.
## Tests
@ -50,7 +52,7 @@ The `/tests/` directory contains everything related to automated testing. It is
@ -62,19 +64,19 @@ PHPUnit's configuration can be customized by copying its configuration file to `
The `/vendor-bin/` directory houses the files needed for the tools used in The Arsse's programming environment. These are managed by the Composer ["bin" plugin](https://github.com/bamarni/composer-bin-plugin) and are not used by The Arsse itself. The following files are also related to various programming tools:
| `/.gitattributes`| Git settings for handling files |
| `/.gitignore`| Git file exclusion patterns |
| `/.php-cs-fixer.dist.php` | Configuration for [php-cs-fixer](https://cs.symfony.com) |
| `/.php-cs-fixer.cache` | Cache for php-cs-fixer |
| `/composer.json`| Configuration for Composer |
| `/composer.lock`| Version synchronization data for Composer |
| `/RoboFile.php`| Task definitions for [Robo](https://robo.li/) |
| `/robo`| Simple wrapper for executing Robo on POSIX systems |
| `/robo.bat`| Simple wrapper for executing Robo on Windows |
In addition the files `/package.json`, `/yarn.lock`, and `/postcss.config.js` as well as the `/node_modules/` directory are used by [Yarn](https://yarnpkg.com/) and [PostCSS](https://postcss.org/) when modifying the stylesheet for The Arsse's manual.
In addition the files `/package.json` and `/postcss.config.js` as well as the `/node_modules/` directory are used by [Yarn](https://yarnpkg.com/) and [PostCSS](https://postcss.org/) when modifying the stylesheet for The Arsse's manual.
# Common tasks
@ -88,11 +90,11 @@ There is also a `test:quick` Robo task which excludes slower tests, and a `test:
### Test coverage
Computing the coverage of tests can be done by running `./robo coverage`, after which an HTML-format coverage report will be written to `/tests/coverage/`. Either [PCOV](https://github.com/krakjoe/pcov), [Xdebug](https://xdebug.org), or [phpdbg](https://php.net/manual/en/book.phpdbg.php) is required for this. PCOV is generally recommended as it is faster than Xdebug; phpdbg is faster still, but less accurate. If using either PCOV or Xdebug, the extension need not be enabled globally; PHPUnit will enable it when needed.
Computing the coverage of tests can be done by running `./robo coverage`, after which an HTML-format coverage report will be written to `/tests/coverage/`. Either [PCOV](https://github.com/krakjoe/pcov) or [Xdebug](https://xdebug.org) is required for this. PCOV is generally recommended as it is faster than Xdebug. Neither extension need be enabled globally; Robo will enable it when needed.
## Enforcing coding style
The [php-cs-fixer](https://cs.symfony.com) tool, executed via `./robo clean`, can be used to rewrite code to adhere to The Arsse's coding style. The style largely follows [PSR-2](https://www.php-fig.org/psr/psr-2/) with some exceptions:
The [php-cs-fixer](https://cs.symfony.com) tool, executed via `./robo clean`, can be used to rewrite code to adhere to The Arsse's coding style. The style largely follows [PSR-12](https://www.php-fig.org/psr/psr-12/) with some exceptions:
- Classes, methods, and functions should have their opening brace on the same line as the signature
- Anonymous functions should have no space before the parameter list
@ -107,13 +109,11 @@ The manual employs a custom theme derived from the standard Daux theme. If the s
## Packaging a release
Producing a release package is done by running `./robo package`. This performs the following operations:
Producing release packages is done by running `./robo package`. This performs the following operations:
- Duplicates a working tree with the commit (usually a release tag) to package
- Generates the manual
- Duplicates a [Git](https://git-scm.com/) working tree with the commit (usually a release tag) to package
- Generates the HTML manual
- Installs runtime Composer dependencies with an optimized autoloader
- Deletes numerous unneeded files
- Exports the default configuration of The Arsse to a file
- Compresses the remaining files into a tarball
Due to the first step, [Git](https://git-scm.com/) is required to package a release.
/** Returns the necessary shell arguments to print error output or all output to the bitbucket
*
* @param bool $all Whether all output (true) or only error output (false) should be suppressed
*/
protected function blackhole(bool $all = false): string {
$hole = IS_WIN ? "nul" : "/dev/null";
return $all ? ">$hole 2>&1" : "2>$hole";
}
/** Executes PHPUnit, used by the test and coverage tasks.
*
* This also executes the built-in PHP Web server, which is required to fetch some newsfeeds during tests
*
* @param string $executor The path to the PHP binary to execute with any required extra arguments. Normally this is either "php" or the result of findCoverageEngine()
* @param string $set The set of tests to run, either "typical" (excludes redundant tests), "quick" (excludes redundant and slow tests), "coverage" (excludes tests not needed for coverage), or "full" (all tests)
* @param array $args Extra arguments passed by Robo from the command line
*/
protected function runTests(string $executor, string $set, array $args): Result {
switch ($set) {
case "typical":
@ -145,9 +149,38 @@ class RoboFile extends \Robo\Tasks {
@ -314,6 +454,15 @@ class RoboFile extends \Robo\Tasks {
return $t->run();
}
/** Parses the contents of the CHANGELOG file into an array structure
*
* This is done line-by-line and tends to be quite strict.
* The parsed output can be used to generate changelogs in other formats,
* such as a Debian changelog or RPM changelog.
*
* @param string $text The text of the CHANGELOG file
* @param string $targetVersion The x.y.z version number of the latest release. This is used to check that version numbers and dates have been updated when preparing a release
*/
protected function changelogParse(string $text, string $targetVersion): array {
$lines = preg_split('/\r?\n/', $text);
$version = "";
@ -323,9 +472,9 @@ class RoboFile extends \Robo\Tasks {
$expected = ["version"];
for ($a = 0; $a <sizeof($lines);){
$l = rtrim($lines[$a++]);
if (in_array("version", $expected) && preg_match('/^Version (\d+(?:\.\d+)*) \(([\d\?]{4}-[\d\?]{2}-[\d\?]{2})\)\s*$/', $l, $m)) {
if (in_array("version", $expected) && preg_match('/^Version ([\d\?]+(?:\.[\d\?]+)*) \(([\d\?]{4}-[\d\?]{2}-[\d\?]{2})\)\s*$/D', $l, $m)) {
$version = $m[1];
if (!preg_match('/^\d{4}-\d{2}-\d{2}$/', $m[2])) {
if (!preg_match('/^\d{4}-\d{2}-\d{2}$/D', $m[2])) {
// uncertain dates are allowed only for the top version, and only if it does not match the target version (otherwise we have forgotten to set the correct date before tagging)
if (!$out && $targetVersion !== $version) {
// use today's date; local time is fine
@ -338,7 +487,7 @@ class RoboFile extends \Robo\Tasks {
$out .= "\n -- The Arsse team <no-contact@code.mensbeam.com> ".\DateTimeImmutable::createFromFormat("Y-m-d", $entry['date'], new \DateTimeZone("UTC"))->format("D, d M Y")." 00:00:00 +0000\n\n";
$out .= "\n -- $authorName <$authorMail> ".\DateTimeImmutable::createFromFormat("Y-m-d", $entry['date'], new \DateTimeZone("UTC"))->format("D, d M Y")." 00:00:00 +0000\n\n";
}
return $out;
}
/** Produces a Debian "source control" file from various bits of data
*
* As with a Debian changelog, the output is of a very exacting format,
* and this function should be modified with care.
*
* @param string $dir The path to Debian-specific files, with trailing slash
* @param string $version The Debian version string, in the format x.y.z-a
* @param array $tarballs An array of paths to the "orig" and "debian" tarball files
*/
protected function generateDebianSourceControl(string $dir, string $version, array $tarballs): string {
// read in control file
if (!$control = @file_get_contents($dir."control")) {
throw new \Exception("Unable to read Debian control file");
}
// read the format
if (!$format = @file_get_contents($dir."source/format")) {
throw new \Exception("Unable to read source format in Debian files");
}
// read the binary packages from the control file
if (preg_match_all('/^Package:\s*(\S+)/m', $control, $m)) {
$binary = [];
foreach ($m[1] as $pkg) {
$binary[] = $pkg;
}
} else {
throw new \Exception("No packages defined in Debian control file");
}
// read the package architectures from the control file
if (preg_match_all('/^Architecture:\s*(\S+)/m', $control, $m) || sizeof($m[1]) != sizeof($binary)) {
$architecture = [];
foreach ($m[1] as $pkg) {
$architecture[] = preg_replace('/\s/', "", $pkg);
}
} else {
throw new \Exception("Number of architectures defined in Debian control file does not match number of packages");
}
// read the package sections from the control file
if (preg_match_all('/^Section:\s*(\S+)/m', $control, $m) || sizeof($m[1]) != sizeof($binary)) {
$section = [];
foreach ($m[1] as $pkg) {
$section[] = $pkg;
}
} else {
throw new \Exception("Number of sections defined in Debian control file does not match number of packages");
}
// read the package priorities from the control file
if (preg_match_all('/^Priority:\s*(\S+)/m', $control, $m) || sizeof($m[1]) != sizeof($binary)) {
$priority = [];
foreach ($m[1] as $pkg) {
$priority[] = $pkg;
}
} else {
throw new \Exception("Number of priorities defined in Debian control file does not match number of packages");
}
// read simple metadata from the control file
$metadata = [];
foreach (["Source", "Maintainer", "Homepage", "Standards-Version", "Vcs-Browser", "Vcs-Git"] as $meta) {
if (preg_match('/^'.$meta.':\s*(.+)/m', $control, $m)) {
$metadata[$meta] = $m[1];
} else {
throw new \Exception("$meta is not defined in Debian control file");
}
}
// read build dependencies from control file
if (preg_match('/(?:^|\n)Build-Depends:\s*((?:[^\n]|\n(?= ))+)/s', $control, $m)) {
$buildDepends = preg_replace('/\s/', "", $m[1]);
} else {
$buildDepends = "";
}
// trim format
$format = trim($format);
// consolidate binaries and package list
$packageList = [];
for ($a = 0; $a <sizeof($binary);$a++){
$packageList[] = "$binary[$a] deb $section[$a] $priority[$a] arch=$architecture[$a]";
