Speed up unit tests

jking commented

2017-09-29 13:57:49 -04:00

Owner

By now, the whole test suite takes a non-trivial amount of time to run. Partly this is because code coverage is always computed (assuming XDebug is installed, which is itself a source of slowdown), but tests which make use of password_hash() and password_verify()---as well as tests which parse feeds---incur significant delays from sha-256 hashing. Tearing down the database schema and re-creating it for each and every database test probably contributes as well (though this is probably offset somewhat by using an in-memory database).

All these problems can be worked around, but would require increasingly esoteric PHP extension dependencies for anyone wishing to hack on the server. It would also require time and effort to profile and actually tune the slower tests.

By now, the whole test suite takes a non-trivial amount of time to run. Partly this is because code coverage is always computed (assuming XDebug is installed, which is itself a source of slowdown), but tests which make use of `password_hash()` and `password_verify()`---as well as tests which parse feeds---incur significant delays from sha-256 hashing. Tearing down the database schema and re-creating it for each and every database test probably contributes as well (though this is probably offset somewhat by using an in-memory database). All these problems can be worked around, but would require increasingly esoteric PHP extension dependencies for anyone wishing to hack on the server. It would also require time and effort to profile and actually tune the slower tests.

jking added this to the Future milestone

2017-09-29 13:57:49 -04:00

jking added the

enhancement

testing

help wanted

labels

2017-09-29 13:57:49 -04:00

jking commented

2017-10-12 10:36:30 -04:00

Author

Owner

Another way to speed up unit tests would be to use atoum (which can run tests in parallel) rather than PHPUnit. This would mean rewriting most of our ~500 tests, however.

Another way to speed up unit tests would be to use [atoum](http://atoum.org/) (which can run tests in parallel) rather than PHPUnit. This would mean rewriting most of our ~500 tests, however.

jking referenced this issue from a commit

2017-12-08 14:37:57 -05:00

Use Robo for programming task execution The plan had originally been to use Bldr, but Bldr's features were not particularly compelling, and it has not been maintained recently. By contrast Robo has very useful features while still meeting my needs. In addition to migrating the build process, the current Robo file also runs unit tests, with or without coverage reporting. Fixes #116; improves #113

jking referenced this issue from a commit

2017-12-16 17:21:32 -05:00

Fix feed timeout test; improves #113

jking commented

2017-12-16 17:47:36 -05:00

Author

Owner

For various reasons, Atoum seems not to be any faster than PHPUnit, and is actually slower by a significant amount in a limited test run.

Thus far testing times have gone from:

With coverage: 1:13
Without coverage: 0:27
With fixed timeout test: 0:23

Disabling Xdebug entirely seems to cut off another 10s, but disabling it for testing and enabling it for coverage may get a bit hairy. It may be worth doing either way, as it would be useful to fully control loaded extensions during testing.

For various reasons, Atoum seems not to be any faster than PHPUnit, and is actually slower by a significant amount in a limited test run. Thus far testing times have gone from: - With coverage: 1:13 - Without coverage: 0:27 - With fixed timeout test: 0:23 Disabling Xdebug entirely seems to cut off another 10s, but disabling it for testing and enabling it for coverage may get a bit hairy. It may be worth doing either way, as it would be useful to fully control loaded extensions during testing.

jking referenced this issue from a commit

2017-12-16 20:03:09 -05:00

Grouped slow tests; improves #113

jking referenced this issue from a commit

2017-12-17 10:27:39 -05:00

Remove reliance on xdebug; fixes #113 Code coverage information is now gathered via phpdbg (a separate executable) by if available rather than xdebug, as the latter is hard to turn on and off. A "test:quick" task has also been added to Robo, which excludes 31 tests which together account for almost two thirds of the test run time. This should pave the way for testing to be added as a commit hook for Git.

jking closed this issue

2017-12-17 10:27:39 -05:00

jking modified the milestone from Future to 0.3.0

2018-01-01 09:38:40 -05:00

Rows
Columns

Speed up unit tests #113