WIP: External tests
Some API changes and (upcoming) patches to automate the testing of Pd-l2ork externals.
"External tests" might be a bit of a misnomer as the tools should be usable for testing internal classes as well.
Ideally, the tests would do the following:
- load all libs in the relevant libdir
- fetch the list of new creator names that came from loading the libraries in the libdir
- try to create an object using each new creator name (with required args if necessary-- unfortunately some externals fail)
- for each successfully instantiated object, enumerate the methods and call each one with whatever args are required.
- check the output against the output we saved from the previous release (of course this implies saving changed state in each release as necessary)
- for each dsp method, output a single block of audio using
[switch~]
(probably want to check against known good state, too, but that may create a lot of state) - report any new method and new creators
The nice thing is that we can use a nonzero arg to "menuquit" in order to signal to CI that there was a failure in any of these steps.
This will allow us to more easily make changes to the codebase. Not only external libraries but also the build system and core, as any soft failures (of which there are many in the build system) that screw up something in an external lib will (eventually) trigger a hard failure in this test framework.
Difficulties
- State-saving and other side-effect-based methods. For now we might have to exclude certain methods like "write" or "read" that are more difficult to test using such an automated framework.
- "helper" creators and methods. Some objects have "proxy" objects with their own class names. However, I think such objects have their constructors disabled. It may be possible to add a check for that in
[classinfo]
and thus skip these. (Not sure what to do about "helper" methods, though. - Probably lots of other problems I'm not thinking of yet...
Merge request reports
Activity
added 1 commit
- 49a66be1 - force all iemlib classes to create without args
added 6 commits
- f2f04f3d - make all cyclone classes instantiate when no arguments are provided
- 7e3c3f7d - remove unnecessary pd files from fluid~
- d47295f5 - remove duplicated abstraction from non-default library
- f94ff9eb - use [output~] in fluid~ library, remove some aliased abstractions
- d6d32a03 - remove a bunch of state files that somehow got added to the repo at the outset
- d8dfb24e - triage iem16's lack of shared lib with statically declared copy/pasta functions
Toggle commit listadded 1 commit
- 13dc22ec - allow all bin_ambi objects to instantiate without any arguments
added 1 commit
- fc0822a0 - allow iem_spec2 objects to instantiate without arguments
added 1 commit
- 86dae341 - remove nameclashing classes from non-default loaded abstraction libraries
added 1 commit
- b3dda6b4 - remove line referencing non-existent help files
added 1 commit
- d4119d6a - remove references to helpers in fluid~ that got removed