WIP: External tests
-
Review changes -
-
Download -
Patches
-
Plain diff
Some API changes and (upcoming) patches to automate the testing of Pd-l2ork externals.
"External tests" might be a bit of a misnomer as the tools should be usable for testing internal classes as well.
Ideally, the tests would do the following:
- load all libs in the relevant libdir
- fetch the list of new creator names that came from loading the libraries in the libdir
- try to create an object using each new creator name (with required args if necessary-- unfortunately some externals fail)
- for each successfully instantiated object, enumerate the methods and call each one with whatever args are required.
- check the output against the output we saved from the previous release (of course this implies saving changed state in each release as necessary)
- for each dsp method, output a single block of audio using
[switch~]
(probably want to check against known good state, too, but that may create a lot of state) - report any new method and new creators
The nice thing is that we can use a nonzero arg to "menuquit" in order to signal to CI that there was a failure in any of these steps.
This will allow us to more easily make changes to the codebase. Not only external libraries but also the build system and core, as any soft failures (of which there are many in the build system) that screw up something in an external lib will (eventually) trigger a hard failure in this test framework.
Difficulties
- State-saving and other side-effect-based methods. For now we might have to exclude certain methods like "write" or "read" that are more difficult to test using such an automated framework.
- "helper" creators and methods. Some objects have "proxy" objects with their own class names. However, I think such objects have their constructors disabled. It may be possible to add a check for that in
[classinfo]
and thus skip these. (Not sure what to do about "helper" methods, though. - Probably lots of other problems I'm not thinking of yet...
Merge request reports
Compare and
Show latest version
- version 15d4119d6a
- version 14b3dda6b4
- version 13147ef8ae
- version 12070594c4
- version 1186dae341
- version 10fc0822a0
- version 9b02fceba
- version 87cfac026
- version 713dc22ec
- version 6533888d2
- version 5d8dfb24e
- version 4fab23d62
- version 31d02d9af
- version 249a66be1
- version 1dbb1f218
- master (base)
- latest version91c256f433 commits,
- version 15d4119d6a30 commits,
- version 14b3dda6b429 commits,
- version 13147ef8ae28 commits,
- version 12070594c427 commits,
- version 1186dae34125 commits,
- version 10fc0822a024 commits,
- version 9b02fceba23 commits,
- version 87cfac02621 commits,
- version 713dc22ec19 commits,
- version 6533888d218 commits,
- version 5d8dfb24e16 commits,
- version 4fab23d6210 commits,
- version 31d02d9af6 commits,
- version 249a66be14 commits,
- version 1dbb1f2183 commits,
Compare changes
- Side-by-side
- Inline
Files
7Loading