Commit 86534ddd authored by Jonathan Wilkes's avatar Jonathan Wilkes
Browse files

revise two project ideas to fit the reduced GSoC program

parent afa3f64f
......@@ -60,117 +60,38 @@ You can find more information about our progress and TODOs from
### Languages
Javascript, HTML, CSS, and experience with SVG will be plus
Completed Projects From Previous Years
* Add Double Precision Floating Point Format. Pranay Gupta.
* ASCII Art interpreter. Aayush Surana.
* Purr Data Web App Frontend. Hugo Carvalho.
* Purr Data Web App Backend. Zack Lee.
* Canvas-private abstractions, automated encapsulation and abstraction saving.
Guillem Bartrina.
Inject Purr Data Directly into the Web
Profile and Purr Data CPU Usage in Realtime
### Goal
Add a WebAssembly target and pure HTML5 GUI framework for running Purr
Data as a web app.
### Benefits
Installing Purr Data is often the biggest hurdle for newcomers. Running
it in a web browser simplifies this process and allows Purr Data to be run
on any device, with any architecture, as long as it has a web browser installed.
Give users the ability to visually and/or programmatically measure which DSP
objects are using the most CPU.
### Details
For the core, we want to produce a WebAssembly binary that can load a patch
and produce audio using the Web Audio API. If possible, we want to also
leverage Chromium's newer callback-based interface to compute realtime audio
as efficiently as possible.
Purr Data has a set of built-in objects for measuring performance for an object
or objects which are not doing any DSP computation.
For the GUI, we need a framework that can display and edit a Pd patch
within a container in a web page. It should be possible to run the core
WebAssembly binary with or without the GUI displayed.
However, for the DSP objects, the user has no easy way to gain insight into
how much CPU each DSP object is using. There are a few clever hacks that users
can employ to achieve this, but they are eithr time-consuming or obscure.
The challenge with the GUI is that Purr Data currently uses multiple
toplevel windows to display a patch and its visible subcanvases, and
properties windows for certain widgets. The framework needs to allow
viewing all of these in a container within a *single* browser window.
### Challenges
There has been some prior art for building a WebAssembly binary
for the core audio engine. However, the difficulty will lie in building
the GUI framework that has a workable UX within a single browser window.
It would be much better if there were a system-wide feature to give visual
feedback to the user so they can find out which DSP objects are the most
### Languages
C, HTML5, and Emscripten
Profile and Optimize Purr Data for Realtime Safety
### Goal
Find the "pain points" in Purr Data's core message dispatcher and
audio engine, and optimize the code where possible to improve the realtime
scheduling of DSP.
### Details
The core DSP algorithms of Purr Data tend to avoid system calls,
unnecessary branching, and other calls which would make the performance of
the audio process unreliable. However, there are many areas of Purr Data
which are not optimized for realtime scheduling.
The first order of business would be to profile Purr Data in several areas
to see where problems may be. Some likely culprits are the following:
* parse time for incoming messages from GUI to the audio/messaging engine
* overhead of sending messages to the GUI, especially for visual arrays and
drawing instructions for data structures
* overhead of sending streams of "motion" messages from GUI to Pd to track
mouse position
* overhead of walking the linked list of objects in order to find the object
under the mouse/mouseclick
* overhead of tracking mouse position over a visual array
* overhead of calculating bboxes in the audio process as opposed to GUI
* overhead of calculating text positioning in audio process as opposed to GUI
* overhead of counting UTF-8 code points in audio process to calculate
line breaks
* overhead of handling incoming/outgoing socket traffic in the same thread
as the audio scheduler
* probably other areas
Once we get a sense of the pain points, we can tackle the problem of how
to optimize in a maintainable manner. Some references for this:
* [Guilio Moro's work]( on using a separate thread for the socket connections from GUI to the core audio engine (plus all the other socket connections
like netsend, netreceive, etc.)
* Guilio Moro's work on a threaded microsleep for the event loop.
* [Guilio Moro's work]( to simplify GUI communication by handling more of the mouse motion/click
logic in the GUI. This results in fewer messages from GUI to audio engine,
but still requires a linked-list walk in the audio engine to find the
relevant object.
* Matt Barber's link to a new cosine wave generator algorithm that may be
more performant than the current implementation. (Not so important for
current performance, but this may become more relevant once we switch to
double-precision for block samples.)
* Possibility to vectorize DSP algos using SIMD. Also more crude experiments
by just hand-unrolling one or two classes when N=64 (i.e., the most common
block size) and measuring the performance impact (if any).
Note: There may be some overlap with the other profiling idea listed below.
Developers for both ideas may therefore benefit by periodically sharing their
work with each other.
C for the backend, and HTML5 for the simple visualizations to show the user
the results of performance measurements.
### Challenges
The initial profiling will take some time but
isn't particularly challenging. Making changes to the core audio engine,
however, will require some knowledge of Linux system interfaces and some of
Purr Data's internals. Properly assessing and testing any threading techniques
in C is also frought with peril and will require extreme care in order to
keep the code maintainable and avoid insidious bugs.
Purr Data's internals.
### Languages
Basic to advanced shell scripting, C, plus familiarity with
profiling tools like gprof and others.
C, plus some HTML5 for simple visualization of the performance metrics
Terminal REPL
......@@ -183,7 +104,7 @@ Purr Data programs and program state.
Purr Data is being used in situations where the hardware is an embedded
device. While the current GUI runs on most common hardware including the RPI,
there are situations where it would be more convenient to simply interact using
a text interface (locally or over ssh).
a text interface (either locally or over ssh).
The user can already communicate with Purr Data's audio engine over a socket
connection. So the "read" and "evaluate" part already exists. However, Purr
......@@ -207,6 +128,17 @@ more work and an understanding of Purr Data's message dispatching system.
### Languages
C, some shell scripting.
Completed Projects From Previous Years
* Add Double Precision Floating Point Format. Pranay Gupta.
* ASCII Art interpreter. Aayush Surana.
* Purr Data Web App Frontend. Hugo Carvalho.
* Purr Data Web App Backend. Zack Lee.
* Canvas-private abstractions, automated encapsulation and abstraction saving.
Guillem Bartrina.
Core Accessibility
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment