Commit 77c0e8f9 authored by Jonathan Wilkes's avatar Jonathan Wilkes
Browse files

Reformat

parent b03950de
......@@ -2,42 +2,43 @@ Automated Testing Framework
---------------------------
### Goal
build a testing framework that covers most of the methods of all internal
and external classes that ship with Purr Data. This should include the "dsp"
Build a testing framework that covers most methods of internal
and external classes. This should include the "dsp"
method of all signal-based externals, so that we can compare the block output
of each across commits to serve as both a regression and code coverage test.
Benefits: this will make it easier for newcomers to make fixes and improvements
to both the core of Purr Data and its many external libraries. Clear errors
from the CI will help them catch bugs earlier in the development process,
### Benefits
This will help developers catch bugs earlier in the development process,
leading to overall cleaner code in merge requests and less disruption to
users from regressions.
Difficulty level: Moderate. One of the challenges here is that not all classes
have sane defaults for their methods. For the most obvious example, the internal
### Difficulty level
Moderate. One challenge is that not all classes
have sane defaults for their methods. For example, the internal
class [until] will iterate in an infinite loop when the "bang" method is
invoked. Even with external libraries, such poor defaults are entrenched. A
robust testing system must therefore work around these challenges.
Another challenge is that a substantial number of the external libraries have
never been tested at all. Many are not 64-bit clean. With this in mind, there
will probably be an initial high number of crashes with any testing system.
Another challenge is that many external libraries have
never been tested at all. Many are not 64-bit clean.
A maintainable and sensible testing system will therefore take an incremental
approach, starting with the most well-tested and popular libraries and
branching out from there to the more obscure ones.
Languages: thanks to some introspection classes availabe in Purr Data, the
testing framework can be fully implemented in Purr Data itself. However, some
### Languages
Testing framework can be fully implemented in Purr Data itself. However, some
familiarity with shell scripting tools and the platforms supported by
Purr Data will be useful (i.e., Windows, OSX, and GNU/Linux).
Change to Double Precision Floating Format
------------------------------------------
Goal: change Purr Data's numeric data type from single-precision floating point
### Goal
Change Purr Data's numeric data type from single-precision floating point
to double-precision floating point.
Benefits: Purr Data has a single numeric type to represent numeric data.
### Benefits
Purr Data has a single numeric type to represent numeric data.
Changing this type to double-precision has many benefits:
* increased precision in the internal sample format used in the audio engine
* increases the maximum index into an array of audio data without losing
......@@ -54,7 +55,8 @@ example, the upcoming MIDI revision uses 32-bit integers which can simply
be converted to double (while single-precision would lose data and require a
more complicated approach).
Details: this requires some changes to the core, leveraging Katja Vetter's
### Details
This requires some changes to the core, leveraging Katja Vetter's
previous work to find performant replacements for the core DSP classes. It also
requires changes to some of the external libraries-- such as freeverb and
others-- which either have separate APIs based on numeric precision or rely on
......@@ -66,7 +68,8 @@ in output of the "dsp" methods of external classes. Developers can then use
the output of those tests to get a better sense of the scope of the work and
prioritize which libraries to refactor first.
Difficulty: Hard. This is mitigated somewhat by the prior work of Katja Vetter
### Difficulty
Hard. This is mitigated somewhat by the prior work of Katja Vetter
and others who have done double-precision implementations for most of the
core audio engine. Still, this change touches many different parts of the code
and will no doubt result in some insidious crashes and runtime errors that
......@@ -74,20 +77,23 @@ will require a well-rounded knowledge of the C programming language, floating
point type standards, and Purr Data's internals including the core DSP
API and algorithms.
Languages: C, some C++ (for some of the external libraries coded in C++),
### Languages
C, some C++ (for some of the external libraries coded in C++),
plus basic shell scripting familiarity (both `find` and `grep` will come in
handy here).
API for HTML5 Web Apps
----------------------
Goal: create an API and simple user interface for loading a web app
that is linked to a Pd patch.
### Goal
Create an API and simple user interface for loading a web app
that can communicate with a Pd patch.
Details: Currently Pd patches are essentially just an HTML page with
an inline SVG which displays an editable diagram. It should be possible
for the user to instead open an arbitrary HTML document that contains
a web app which communicates using a javascript API with a Pd patch.
### Details
Currently Pd patches are essentially just an HTML5 page with
a window/application menu. It should be possible
for the user to instead open an arbitrary HTML document that can communicate
through a javascript interface with a Pd patch.
Implementation suggestion: since the current pdgui.js module already has
an interface for sending messages to the Pd backend, we should probably
......@@ -109,7 +115,8 @@ the web app (or perhaps using Pd's search path).
* the Pd patch we wanted to open has loaded
* the Pd patch we loaded has closed
Languages: Javascript. Some basic knowledge of C will be helpful if we need
### Languages
Javascript. Some basic knowledge of C will be helpful if we need
to add a method in the Purr Data engine. However, such methods will probably
be quite simple and won't require implementing any complex algorithms in the
C language.
......@@ -117,9 +124,11 @@ C language.
K12 Mode
--------
Goal: Port the K12 Mode that Pd-l2ork introduced.
### Goal
Port the K12 Mode that Pd-l2ork introduced.
Details: Currently we ship a lot of the K12 abstractions with Purr Data.
### Details
Currently we ship a lot of the K12 abstractions with Purr Data.
However, I never ported the special mode that starts the program up in a
"K12" mode that replaces the window/application menubar with a more
user-friendly, clickable object menu.
......@@ -141,23 +150,27 @@ Either way, this feature vastly improves the usability of Purr Data for new
users, giving them a way to quickly generate sounds and experiment with the
interface.
Difficulty: Hard. A lot of the initial implementation of K12 mode was made
### Difficulty
Hard. A lot of the initial implementation of K12 mode was made
in the core engine in C. This will require at least reading a lot of the
source code and making sense of the changes made for K12 mode, plus possibly
refactoring some of that code so that it can be handled in the GUI instead.
On the other hand, there is a working version of the old K12 mode which can
be consulted as a reference implementation.
Languages: Javascript, C, CSS, HTML.
### Languages
Javascript, C, CSS, HTML.
Profile and Optimize Purr Data for Realtime Safety
--------------------------------------------------
Goal: Find the "pain points" in Purr Data's core message dispatcher and
### Goal
Find the "pain points" in Purr Data's core message dispatcher and
audio engine, and optimize the code where possible to improve the realtime
scheduling of DSP.
Details: The core DSP algorithms of Purr Data tend to avoid system calls,
### Details
The core DSP algorithms of Purr Data tend to avoid system calls,
unnecessary branching, and other calls which would make the performance of
the audio process unreliable. However, there are many areas of Purr Data
which are not optimized for realtime scheduling.
......@@ -203,24 +216,28 @@ Note: There may be some overlap with the other profiling idea listed below.
Developers for both ideas may therefore benefit by periodically sharing their
work with each other.
Difficulty: Moderate to hard. The initial profiling will take some time but
### Difficulty
Moderate to hard. The initial profiling will take some time but
isn't particularly challenging. Making changes to the core audio engine,
however, will require some knowledge of Linux system interfaces and some of
Purr Data's internals. Properly assessing and testing any threading techniques
in C is also frought with peril and will require extreme care in order to
keep the code maintainable and avoid insidious bugs.
Languages: basic to advanced shell scripting, C, plus familiarity with
### Languages
Basic to advanced shell scripting, C, plus familiarity with
profiling tools like gprof and others.
ASCII art to Purr Data diagram conversion
-----------------------------------------
Goal: make a GUI interface where the user can either type or paste in
### Goal
Make a GUI interface where the user can either type or paste in
ASCII art for a Purr Data diagram and have it converted to a floating selection
in the current Purr Data diagram.
Details: Often Purr Data users rely on ASCII art in forums, mailing lists
### Details
Often Purr Data users rely on ASCII art in forums, mailing lists
and other documentation to give examples of Purr Data programs. This is
convenient because it's quicker than taking a screenshot of the Purr Data
GUI and takes up less space, too.
......@@ -230,7 +247,8 @@ selection of a Purr Data object chain would be useful. This would let users
paste ASCII art from the mailing list directly into the interface. It would
also make it easier to create Purr Data diagrams using only the keyboard.
Bonus Goal: make it possible to convert a subset of Purr Data diagrams into
### Bonus Goal
Make it possible to convert a subset of Purr Data diagrams into
ASCII art.
Bonus Goal Details: Unfortunately Purr Data allows arbitrary positioning
......@@ -249,7 +267,8 @@ could disallow objects to overlap with each other. Still, the connections
among the various objects (i.e., the arcs which connect the nodes) could
overlap in such a way that makes unambiguous ASCII art impossible.
Difficulty: Easy to hard. This idea runs the gamut-- a simple GUI-side
### Difficulty
Easy to hard. This idea runs the gamut-- a simple GUI-side
parser for simple ASCII art is a quick exercise. Handling some of the more
complicated art with multiple and/or crossed connections is moderate. Scanning
the mailing lists for examples and covering the majority of them is hard.
......@@ -261,16 +280,19 @@ OCR and some of the current work converting Photoshop mockups to HTML pages.
However, Purr Data diagrams tend to be much simpler than arbitrary Photoshop
art, so the problem may be more tractable here than in other fields.
Languages: Javascript if converting ASCII art to Purr Data diagram, C if
### Languages
Javascript if converting ASCII art to Purr Data diagram, C if
attempting to convert from graphical diagram *back* to ASCII art.
Navigation of "Wireless" Objects
--------------------------------
Goal: make it possible for the user to navigate among all extant wireless
### Goal
Make it possible for the user to navigate among all extant wireless
objects that are bound to a particular symbol.
Details: Purr Data diagrams typically consist of objects-- English words in
### Details
Purr Data diagrams typically consist of objects-- English words in
boxes-- connected by Bezier curves. In a complex diagram, however, the
Bezier curves can end up obscuring the flow of the data in the diagram. For
these instances, "wireless" objects like `[send]` and `[receive]` may be
......@@ -291,7 +313,8 @@ of hyperlinks to the console. When the user clicks any hyperlink, the
corresponding object can be highlighted (or the relevant diagram brought up
with the object in it highlighted).
Difficulty: Easy to moderate. Purr Data's bindelem and pd_bind API are a little
### Difficulty
Easy to moderate. Purr Data's bindelem and pd_bind API are a little
tricky if one hasn't used them before. However, once that interface is
understood it is all that is needed to return a complete list of all
nonlocal objects bound to a particular symbol.
......@@ -300,15 +323,18 @@ The remainder of the problem is easy and can be done in Javascript. However,
this problem "upgrades" gracefully. Once the initial UI is done, it can be
tested on users and iteratively improved from there.
Languages: Javascript, C.
### Languages
Javascript, C.
Terminal REPL
-------------
Goal: make a little REPL interface with which the user can interact with
### Goal
Make a little REPL interface with which the user can interact with
Purr Data programs and program state.
Details: Purr Data is being used in situations where the hardware is an embedded
### Details
Purr Data is being used in situations where the hardware is an embedded
device. While the current GUI runs on most common hardware including the RPI,
there are situations where it would be more convenient to simply interact using
a text interface (locally or over ssh).
......@@ -327,19 +353,23 @@ and well-documented/specified. This way Purr Data users can always interact
with and create programs easily on any embedded device, even if there is
no direct display. (This would also be very handy for debugging purposes.)
Difficulty: Moderate. An initial REPL can be created with the current Purr
### Difficulty
Moderate. An initial REPL can be created with the current Purr
Data API, but it won't be particularly user-friendly. To achieve that requires
more work and an understanding of Purr Data's message dispatching system.
Languages: C, some shell scripting.
### Languages
C, some shell scripting.
Core Accessibility
------------------
Goal: ensure that Purr Data is accessible by coupling accessibility
### Goal
Ensure that Purr Data is accessible by coupling accessibility
with the core UX
Details: especially because Purr Data is a graphical environment, it's
### Details
Especially because Purr Data is a graphical environment, it's
important to make sure the core functionality is accessible. Rather than
tack on accessibility as an afterthought, Purr Data should have a UX that
makes accessibility features a generally useful part of the programming
......@@ -357,13 +387,15 @@ Note: there may be some overlap with the REPL idea above, as the REPL could
provide a sensible way for a user to traverse the diagram as an alternative
to using the GUI.
Difficulty: Moderate. For example, it will be necessary to study the current
### Difficulty
Moderate. For example, it will be necessary to study the current
GUI implementation to figure out how to extend it to add keyboard navigation.
It will also be necessary to study pre-existing approaches to making SVG
diagrams accessible and study the current state of HTML5 tools that facilitate
this.
Languages: Javascript, HTML, CSS. Some basic C knowledge may be required to
### Languages
Javascript, HTML, CSS. Some basic C knowledge may be required to
send a richer set of data about each object from the core to the GUI. However,
there is already an interface that can do this-- it just needs to be hooked
into the GUI.
......@@ -371,10 +403,12 @@ into the GUI.
Purr Data Message and DSP Profiler
----------------------------------
Goal: measure the time it takes for each object in a Purr Data diagram to
### Goal
Measure the time it takes for each object in a Purr Data diagram to
process its data and display the results in the diagram.
Details: Purr Data users would benefit greatly from the ability to profile
### Details
Purr Data users would benefit greatly from the ability to profile
their programs while they are running. This is easy to do for the program
as a whole, but challenging to do per-object.
......@@ -399,37 +433,91 @@ Note: There may be overlap with the other profiling idea listed above, as
developers on both ideas will probably be using the same tools and can
therefore benefit by periodically sharing their work with each other.
Bonus goal: Figure out a way to meaningfully profile DSP objects. DSP objects
### Bonus goal
Figure out a way to meaningfully profile DSP objects. DSP objects
typically process data at a high sample rate (44,100 is common) so displaying
the data in a user-friendly and meaningful way is tricky.
Difficulty: Moderate to Hard. This feature touches the main artery of the
### Difficulty
Moderate to Hard. This feature touches the main artery of the
message dispatching system, and the bonus goal would touch the main DSP
routine. In both cases realtime scheduling deadlines must be taken into
account by careful profiling.
Languages: C for the profiling business logic, HTML5 for displaying the
### Languages
C for the profiling business logic, HTML5 for displaying the
results in the GUI.
Streamlining Purr Data GUI-Pd communication
------------------------------------------
The Pd GUI is heavily entangled with the Pd audio backend. In fact, most of the "gestures" performed on the GUI are passed straight to the Pd engine for processing. The GUI gestures are then "analyzed" by the audio thread, which may respond with triggering a GUI action, changing the state of an object, or nothing.
For instance, each mouse move triggers a `motion` message to the Pd backend, handled by `canvas_motion()` in `g_editor.h`. This calls `canvas_doclick(... doit = false)`, which in turn iterates through all the objects on the patch and asks each of them "does the cursor happen to be on top of you?" (`canvas_findhitbox()`/`canvas_hitbox()`), calling a callback function (`w_getrectfn()`) for each of those objects.
Now, most of the time the cursor is not on an object (or patch cable) and the calls to `w_getrecfn()` have no effect, except for wasting CPU power. There are two notable exceptions:
a) when the mouse pointer is on top of an object, or one of its inlets or outlets, or on top of a patch cord, or on top of a GUI object, the mouse pointer may change, plus, e.g.: flickering inlets/outlets.
b) some objects use the calls to `w_getrecfn()` to track mouse position (e.g.: [mousestate] from cyclone).
The above results in a plethora of CPU cycles being wasted, which may cause dropouts when using small blocksizes and/or embedded platforms. Besides - and perhaps most importantly - it seems the wrong approach that some GUI-specific actions (like the ones at a) above) have to be processed and validated by the audio engine, within the audio thread.
We could therefore think of an improvement to the Purr-data architecture, where the GUI stuff (e.g.: point a) above) is delegated uniquely to the GUI, which makes for lower CPU usage and potentially a more responsive GUI. For instance, the GUI could be designed to only send `motion` messages when the mouse is on top of an object and it could send alongside with it the Pd "tag" of the object, so that `w_getrectfn()` can be called only for the relevant object).
The optimal approach would involve handling all the graphics effects (in/outlet animation, mouse pointers) directly within the GUI, and only sending `motion` messages when something relevant to the Pd engine is _actually_ happening (e.g.: when connecting objects).
Additionally, and looking forward, in order to address point b), objects that need to track mouse position should declare this at initialization and should be kept in a dedicated list, so that the `motion` messages from the GUI can be delivered only to them with minimal CPU waste.
An alternative - and probably worse - approach to the problem, which could reduce peak CPU usage, would be for the Pd audio engine to maintain a "rasterized" cached map of the patch (e.g.: by calling `w_getrecfn()` for each object at each pixel). This way, it could simply look up the cached map in response to each `motion` message. The cache could be recomputed in a separate thread every time after a new object or patch cord is created. Threading issues may arise here, in case one of the objects is deleted while the cached map is being built.
This project comes with a number of challenges, including: potential threading issues between the engine and the GUI, the necessity to re-write the C code of some objects, providing complete documentation for creators of externals, maintaining - where possible (e.g.: excluding objects that track mouse position) - backwards compatibility with Pd.
More details on a previous attempt at addressing the problem can be found [here]( http://disis.music.vt.edu/pipermail/l2ork-dev/2017-June/001383.html).
### Goal
Move some of the GUI callbacks out of Purr Data's audio engine so that GUI
interaction is less likely to cause dropouts.
### Details
The Pd GUI is heavily entangled with the Pd audio backend. In fact, most of the
"gestures" performed on the GUI are passed straight to the Pd engine for
processing. The GUI gestures are then "analyzed" by the audio thread, which may
respond with triggering a GUI action, changing the state of an object, or
nothing.
For instance, each mouse move triggers a `motion` message to the Pd backend,
handled by `canvas_motion()` in `g_editor.h`. This calls
`canvas_doclick(... doit = false)`, which in turn iterates through all the
objects on the patch and asks each of them "does the cursor happen to be on top
of you?" (`canvas_findhitbox()`/`canvas_hitbox()`), calling a callback function
(`w_getrectfn()`) for each of those objects.
Now, most of the time the cursor is not on an object (or patch cable) and the
calls to `w_getrecfn()` have no effect, except for wasting CPU power. There are
two notable exceptions:
a) when the mouse pointer is on top of an object, or one of its inlets or
outlets, or on top of a patch cord, or on top of a GUI object, the mouse
pointer may change, plus, e.g.: flickering inlets/outlets.
b) some objects use the calls to `w_getrecfn()` to track mouse position
(e.g.: [mousestate] from cyclone).
The above results in a plethora of CPU cycles being wasted, which may cause
dropouts when using small blocksizes and/or embedded platforms. Besides - and
perhaps most importantly - it seems the wrong approach that some GUI-specific
actions (like the ones at a) above) have to be processed and validated by the
audio engine, within the audio thread.
We could therefore think of an improvement to the Purr-data architecture, where
the GUI stuff (e.g.: point a) above) is delegated uniquely to the GUI, which
makes for lower CPU usage and potentially a more responsive GUI. For instance,
the GUI could be designed to only send `motion` messages when the mouse is on
top of an object and it could send alongside with it the Pd "tag" of the object,
so that `w_getrectfn()` can be called only for the relevant object).
The optimal approach would involve handling all the graphics effects (in/outlet
animation, mouse pointers) directly within the GUI, and only sending `motion`
messages when something relevant to the Pd engine is _actually_ happening
(e.g.: when connecting objects).
Additionally, and looking forward, in order to address point b), objects that
need to track mouse position should declare this at initialization and should
be kept in a dedicated list, so that the `motion` messages from the GUI can be
delivered only to them with minimal CPU waste.
An alternative - and probably worse - approach to the problem, which could
reduce peak CPU usage, would be for the Pd audio engine to maintain a
"rasterized" cached map of the patch (e.g.: by calling `w_getrecfn()` for each
object at each pixel). This way, it could simply look up the cached map in
response to each `motion` message. The cache could be recomputed in a separate
thread every time after a new object or patch cord is created. Threading issues
may arise here, in case one of the objects is deleted while the cached map is
being built.
### Difficulty: Moderate. This project comes with a number of
challenges, including: potential threading issues between the engine and the
GUI, the necessity to re-write the C code of some objects, providing complete
documentation for creators of externals, maintaining - where possible (e.g.:
excluding objects that track mouse position) - backwards compatibility with Pd.
More details on a previous attempt at addressing the problem can be found
[here]( http://disis.music.vt.edu/pipermail/l2ork-dev/2017-June/001383.html).
### Languages
Javascript and C.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment