Gem on the Mac
Ok, here's my first go at it. This includes the necessary changes to build a working Gem library on macOS. The build works with both MacPorts and Homebrew now, but prefers Homebrew if it is installed. (As a working libquicktime is only available on Homebrew, the QT4L backends are disabled in the MacPorts build for now, but they work -- kind of -- in the Homebrew build.) All in all, this works quite well actually, but there are some known quirks, see below.
I also updated the Homebrew instructions in the toplevel README. And the MR contains a backport from upstream (vanilla) which improves diagnostics of library loading. While this is unrelated, it helped a lot to diagnose library issues which prevented this to work at first.
Known quirks:
-
The gemmacoswindow (which is the default on macOS) doesn't work at present, at least not on anything that's more recent than Sierra. This is a known bug, see https://github.com/umlaeute/Gem/issues/257. The window backend which worked best for me (on Mojave at least) is the SDL2 window, so we (build and) default to that. Of course, this means that SDL2 must be installed (it's listed in the updated README). The SDL2 window does have some quirks of its own on the Mac, though. You can't close it, but you can open different patches in succession and have them render to the same window.
-
Video loading and recording via QT4L works (in the Homebrew build), but the options are extremely limited. For one thing, it seems that there's no mp4v codec in the libquicktime from Homebrew, so no MPEG4 playback. In fact, there doesn't seem to be much of any of the video formats that people commonly use. Maybe we can fix that by building our own libquicktime or port one of the existing versions that are known to work, such as the one from Ubuntu. But that's something for another day.
On the bright side, basic OpenGL functionality, images, 3D models, and video capture all seem to work fine, so it's certainly usable (and better than not having Gem supported on the Mac at all).
I tested this with both Homebrew and MacPorts, both builds work fine for me. Test builds on Linux and Windows also continue to work.
Merge request reports
Activity
requested review from @jwilkes
assigned to @jwilkes
I forgot to mention that I also took the opportunity to clean out some old Fink-related cruft, that's in rev. 5eb251d7. This does add some noise to the changeset, but I felt that it was necessary to reduce noise in the build logs.
Maybe the libquicktime from Homebrew would work better if rebuilt against some optional dependencies, possibly after modifying the formula? Currently we only have lame and vorbis installed (per the Homebrew instructions) IIRC, so that may explain the lack of video codecs in libquicktime. I'm looking for advice on this.
I just noticed another issue with libquicktime: For recording, it depends on some codecs available as shared libs (so files) in /usr/local/opt/libquicktime/lib/libquicktime. These seem to be loaded dynamically, they're not linked into libquicktime.dylib; it looks for them under the Homebrew prefix. This is apparent from Gem's terminal output during startup:
dylib loading file '/Applications/Pd-l2ork.app/Contents/Resources/app.nw/extra/Gem/gem_recordQT4L.so'! [codecinfo] Error: Cannot open plugin directory /usr/local/Cellar/libquicktime/1.2.4_5/lib/libquicktime (forgot make install?)Obviously, just copying the so files to /Applications/Pd-l2ork.app/Contents/lib/libquicktime doesn't help with this, we'd need a custom build of libquicktime to patch up the load path. Bummer.
Loading doesn't fare much better. I can load a few files, such as alea.avi, but playback is sluggish and sometimes has artifacts. That might be due to my 10 yr old MacBook Pro, though, which isn't exactly a monster gaming machine by today's standards.
BTW, I'm having exactly the same issues with Pd-0.51-4 and the Gem from Deken, so this isn't a problem with the present MR per se. Either there are issues with libquicktime itself (which I doubt because it works great on Ubuntu 22.04 at least), Homebrew's libquicktime version (possible), or the Gem QT4L backends (less likely, because again, everything works in Ubuntu 22.04).
I'd still like to include the QT4L backends in the Homebrew build, though, if only to achieve feature (and bug) parity with Vanilla+Deken.
Jonathan, what's your take on this?
Note to self: To make the bottled libquicktime from Homebrew happy, we also need to have faac and jpeg installed; add those to the Homebrew instructions. libfaac has MPEG2-AAC and MPEG4-AAC, jpeg the JPEG image and movie codecs. Unfortunately, these aren't in the libquicktime dependencies, but corresponding plugins are included in the libquicktime package.
Now let me try whether I can patch up that load path in libquicktime so that everything will just work OOTB (TM).
Now let me try whether I can patch up that load path in libquicktime so that everything will just work OOTB (TM).
Well, I got libquicktime from git (which amazingly has commits from 2021; I thought that it was unmaintained) to compile, but it doesn't really work much better than the version from Homebrew. While digging through the source, though, I found that there's an environment variable,
LIBQUICKTIME_PLUGIN_DIR, that can be set to point libquicktime to its plugin directory. So that problem appears to be solvable.I've also taken another look at the vlc plugin, but it's only for video capture. It's unfortunate that Gem basically relies on a largely unmaintained library for loading and saving the lion's share of useful video file formats, but that's just the way things stand right now, AFAICT.
That was easy enough. libquicktime plugins are now included, as well as their dependencies (dylibs), so that everything just works without having to install libquicktime from Homebrew.
I think that I'm finished with this for now. It works well enough, and given the limitations of Homebrew's libquicktime, which doesn't include libavcodec from ffmpeg, it's all the video support that we're going to get right now. And actually it works better than on Windows; I can play at least some videos with it.
I also understand the choice of backends in Homebrew's package. They included what (mostly) works, and nothing else. Specifically, libavcodec59 broke the API big time, and libquicktime just hasn't been updated to work with that version. It seems to work alright with libavcodec58 from ffmpeg 4.4, though; that's what Ubuntu 22.04 has. So if we can build our own libquicktime package which links against this version, we'll hopefully have much better video loading and recording capabilities in the future. But I estimate that this will be more than what I can do on a rainy afternoon or two, so I'd say that we'll leave it for another day.
Ready to be merged when you are.
BTW, libquicktime was also the cause of my Arch woes with Gem video playback and recording. I gave the libquicktime from the AUR the boot, built the one from Debian instead (using all the patches from libquicktime_1.2.4-12, which is exactly the same patched version that the Homebrew package uses), made sure to forcibly link it against avcodec58 from ffmpeg 4.4 (which is still in the official Arch repos), and installed it manually.
What can I say, everything works again, just as it does in Ubuntu 22.04. Now I only need to create a PKGBUILD for that version and upload it to the AUR, since the versions currently available there are all broken.
Of course, this means that we could just take the Homebrew package and rebuild it from source against ffmpeg 4.4. And yes, that version is also available in Homebrew (ffmpeg@4). I just didn't dare to install it yet, since last time when I tried installing ffmpeg (the most recent package), it pulled in a gazillion of dependencies including the entire Guile interpreter and Rust compiler and wanted to build all that from source.
Anyway, while working on this MR, I learned a lot more about libquicktime and Gem's video support than I ever wanted to know. And I think that having Gem included in the Mac build is a fairly substantial advancement, which will give Purr Data on the Mac a major boost.
libquicktime_1.2.4-12, which is exactly the same patched version that the Homebrew package uses
Too good to be true. :) Actually, the Homebrew package only applies the CVE patches from Debian, so we still have to build our own which includes them all. For the time being, I just unpack libquicktime-1.2.4.tar.gz and the patches folder from libquicktime_1.2.4-12.debian.tar.xz side by side, then apply the patches:
cd libquicktime-1.2.4 for x in ../patches/*.patch; do patch -p1 -i $x; doneffmpeg@4 needs to be installed from Homebrew, which takes a loooong time... It also needs to be linked explicitly, as it is a "keg-only" package (whatever that means):
brew install ffmpeg@4 brew link ffmpeg@4Then configure libquicktime as follows (snitched from libquicktime.rb):
./configure --disable-dependency-tracking --enable-gpl --without-doxygen --without-gtk --without-xThen do the usual
make && make install. (Make sure that you have libquicktime from brew uninstalled or at least unlinked first.)Then just rebuild the purr-data package to have all the shiny new libquicktime plugins included, done!
I thought that I'd document this here in case someone wants to follow my steps, or I need to look it up myself later. Hopefully, I can soon figure out how to adjust libquicktime.rb and build the Homebrew package locally, this will make things a little easier.
Hopefully, I can soon figure out how to adjust libquicktime.rb and build the Homebrew package locally, this will make things a little easier.
This wasn't hard, and I needed to do it anyway, because our embed-MacOSX-dependencies.sh script didn't detect the custom-built version. I'm attaching my libquicktime-ffmpeg.rb formula below. Note that you'll have to copy this to /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula, then you can install it with
brew install --build-from-source libquicktime-ffmpeg. (This will pull in fftw@4, which, along with its dependencies, may take a looong time to compile, unless your macOS version happens to have a bottled version.)I'm actually getting quite fond of brew, it's been growing on me. It's a great package manager, and creating packages with it is as easy (if not easier) than with pacman. I might even start using it on Linux.
Edited by Albert Gräfadded 1 commit
- 9ef88402 - Check for a custom libquicktime-ffmpeg package.
Still had to massage the embed-MacOSX-dependencies.sh script a little, so that it detects the libquicktime-ffmpeg package if it is installed, and finds dependencies in the cellar (probably due to ffmpeg@4 being keg-only).
This is working much better now. I can record and play back video files now, just like on Linux. There's still an odd issue with the colors I get from the libquicktime ffmpeg plugin, though:
Note the blueish cast there. While that looks cool, that's not how it's supposed to be.
For anyone who wants to give it a whirl, you can find the purr-data package I just built on my Google drive, click here.
Anyway, that's strictly an issue with the libquicktime-ffmpeg package, and not this MR, which is now finished.
Edited by Albert GräfImportant general question:
- does all this build 100% from homebrew with no extra work outside of homebrew-based build business? E.g., no "download this old binary from geocities.com/quicktimeMuseum and beat the final boss of the 2d scroller to unlock the keycracker"
Less important, uninformed question:
- why are we dealing with quicktime in 2022? Don't we just need h264, avi, png? And can't we just get that through Bellard's ffmpeg?
does all this build 100% from homebrew with no extra work outside of homebrew-based build business?
Yes. It's only that now, if /usr/local/opt doesn't exist, then it will fall back to MacPorts in /opt/local, if that exists. That's for the benefit of users who may already have MacPorts on their system and don't want to also install Homebrew (such as I did until now).
The added check in the last commit, rev. 9ef88402, only checks to see if my custom libquicktime-ffmpeg package is installed (which is simply a variant of libquicktime which also includes the ffmpeg@4 plugin). Otherwise it uses the stock libquicktime package from Homebrew.
why are we dealing with quicktime in 2022?
That also confused me to no end at first.
I don't know why the Gem devs opted for libquicktime and not ffmpeg to provide video decoding and encoding capabilities, so I can only speculate. Nowadays ffmpeg is of course the 800 pound gorilla in the room, but that wasn't always the case. AFAICT from the logs, Gem's QT4L backend goes back to at least 2003, when both libquicktime and ffmpeg were still nascent. I still remember these days; non-linear video editing was almost unheard of in the Linux world back then, and video formats were still a patent minefield, so there was only very little FLOSS software support available in this realm. Maybe libquicktime was more capable at the time, more convenient, or the Gem devs simply knew it better, who knows? You know how these decisions happen. And lateron there is nobody around to write a new backend until the preferred alternative (libquicktime in this case) has finally died from bitrot.
