README.md 7.72 KB
Newer Older
Jonathan Wilkes's avatar
Jonathan Wilkes committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Automated Testing Framework
---------------------------

Goal: build a testing framework that covers most of the methods of all internal
and external classes that ship with Purr Data. This should include the "dsp"
method of all signal-based externals, so that we can compare the block output
of each across commits to serve as both a regression and code coverage test.

Benefits: this will make it easier for newcomers to make fixes and improvements
to both the core of Purr Data and its many external libraries. Clear errors
from the CI will help them catch bugs earlier in the development process,
leading to overall cleaner code in merge requests and less disruption to
users from regressions.

Change to Double Precision Floating Format
------------------------------------------

Goal: change Purr Data's numeric data type from single-precision floating point
to double-precision floating point.

Benefits: Purr Data has a single numeric type to represent numeric data.
Changing this type to double-precision has many benefits:
* increased precision in the internal sample format used in the audio engine
* increases the maximum index into an array of audio data without losing
precision. Current single-point floating point indicies severely limit
array indexing, requiring the user to manage both an index and an onset into
the array to retain precision.
* creates a consistent interface between the core audio engine and the HTML5
GUI which itself uses double-precision floating point for represented numbers.
* makes it easier for cases where users want integers, as a double-precision
floating point type can itself contain a 32-bit integer. (Single-precision
floating point cannot.)
* makes it easier for third-party developers to interface with Purr Data. For
example, the upcoming MIDI revision uses 32-bit integers which can simply
be converted to double (while single-precision would lose data and require a
more complicated approach).

Details: this requires some changes to the core, leveraging Katja Vetter's
previous work to find performant replacements for the core DSP classes. It also
requires changes to some of the external libraries-- such as freeverb and
others-- which either have separate APIs based on numeric precision or rely on
the numeric type being single-precision.

This work will benefit immensely from the automated testing framework listed
above. That framework will immediately reveal many crashers and discrepancies
in output of the "dsp" methods of external classes. Developers can then use
the output of those tests to get a better sense of the scope of the work and
prioritize which libraries to refactor first.
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

API for HTML5 Web Apps
----------------------

Goal: create an API and simple user interface for loading a web app
that is linked to a Pd patch.

Details: Currently Pd patches are essentially just an HTML page with
an inline SVG which displays an editable diagram. It should be possible
for the user to instead open an arbitrary HTML document that contains
a web app which communicates using a javascript API with a Pd patch.

Implementation suggestion: since the current pdgui.js module already has
an interface for sending messages to the Pd backend, we should probably
leverage that to communicate from the web app to Pd. Then it's only
necessary to add a feature that allows a Pd Patch to send messages back
to the web app.

The user should then be able to load an HTML file using the existing
File->Open dialog. Then the web app will need something like the following
interface:

* ability to send messages to Pd (already exists)
* ability to tell Pd to load a particular patch in the same directory as
the web app (or perhaps using Pd's search path).
* ability to close a Pd patch
* ability to specify whether the patch we're loading should default to
  being visible or hidden
* ability to set callbacks for the web app to receive the following events:
  * the Pd patch we wanted to open has loaded
  * the Pd patch we loaded has closed
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99

K12 Mode
--------

Goal: Port the K12 Mode that Pd-l2ork introduced.

Details: Currently we ship a lot of the K12 abstractions with Purr Data.
However, I never ported the special mode that starts the program up in a
"K12" mode that replaces the window/application menubar with a more
user-friendly, clickable object menu.

That mode makes it easier for beginners to get started. In porting it, we
can take advantage of the new HTML5 GUI:
* The menu itself can leverage HTML/CSS to be more responsive, autohide, etc.
* The entire K12 Mode could itself just be a web-app separate from normal
  Pd app operation.

To figure out which is best, it's probably helpful to know more about the
history of K12 mode, and whether the ability to also open and interact with
normal Pd objects turned out to be an important feature.
Jonathan Wilkes's avatar
Jonathan Wilkes committed
100
101
102
Ivica Bukvic has a lot of experience using this mode when teaching
kids, so we can look to him for some more insight into the right path to
take.
103
104
105
106

Either way, this feature vastly improves the usability of Purr Data for new
users, giving them a way to quickly generate sounds and experiment with the
interface.
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152

Profile and Optimize Purr Data for Realtime Safety
--------------------------------------------------

Goal: Find the "pain points" in Purr Data's core message dispatcher and
audio engine, and optimize the code where possible to improve the realtime
scheduling of DSP.

Details: The core DSP algorithms of Purr Data tend to avoid system calls,
unnecessary branching, and other calls which would make the performance of
the audio process unreliable. However, there are many areas of Purr Data
which are not optimized for realtime scheduling.

The first order of business would be to profile Purr Data in several areas
to see where problems may be. Some likely culprits are the following:

* parse time for incoming messages from GUI to the audio/messaging engine
* overhead of sending messages to the GUI, especially for visual arrays and
  drawing instructions for data structures
* overhead of sending streams of "motion" messages from GUI to Pd to track
  mouse position
* overhead of walking the linked list of objects in order to find the object
  under the mouse/mouseclick
* overhead of tracking mouse position over a visual array
* overhead of calculating bboxes in the audio process as opposed to GUI
* overhead of calculating text positioning in audio process as opposed to GUI
* overhead of counting UTF-8 code points in audio process to calculate
  line breaks
* overhead of handling incoming/outgoing socket traffic in the same thread
  as the audio scheduler
* probably other areas

Once we get a sense of the pain points, we can tackle the problem of how
to optimize in a maintainable manner. Some references for this:

* [Guilio Moro's work](https://github.com/giuliomoro/purrdata/commit/9dc3223ece79be5f60a6a629450b52a79b9e050c) on using a separate thread for the socket connections from GUI to the core audio engine (plus all the other socket connections
like netsend, netreceive, etc.)
* Guilio Moro's work on a threaded microsleep for the event loop.
* [Guilio Moro's work](https://github.com/giuliomoro/purrdata/commits/simpler-motion) to simplify GUI communication by handling more of the mouse motion/click
  logic in the GUI. This results in fewer messages from GUI to audio engine,
  but still requires a linked-list walk in the audio engine to find the
  relevant object.
* Matt Barber's link to a new cosine wave generator algorithm that may be
  more performant than the current implementation. (Not so important for
  current performance, but this may become more relevant once we switch to
  double-precision for block samples.)
153
154
155
* Possibility to vectorize DSP algos using SIMD. Also more crude experiments
  by just hand-unrolling one or two classes when N=64 (i.e., the most common
  block size) and measuring the performance impact (if any).