Friday, January 19, 2007

RFC: MetaFFRend / FF meta-plugin authoring

Request for Comments

OK, now that FFRend 1.3 is out the door, it's time to get serious about MetaFFRend. I hope to spark a bit of pre-coding discussion here, so that once the coding starts we can get it mostly right the first time. Here are some issues that have been bumping around in my head.

Summary

This is a Request For Comments regarding the VJ Forums proposal to turn FFRend into a plugin-authoring tool, somewhat similar to Pete Warden's FreeChain application. The idea is basically to export a FFRend project as a Freeframe plugin. This "meta-plugin" could be loaded into a Freeframe host software, and would behave just as if you were running the equivalent project in FFRend.

The meta-plugin may contain links to the plugins it requires, or alternatively, the plugins may be "bundled" into the meta-plugin. In the latter case, it's easier to distribute the meta-plugin, because there's no dependence on external files. Specifically, a meta-plugin will consist of FFRend's rendering engine, links to (or compressed copies of) the necessary plugins, and the project data (parameter settings, automations, etc).

Linked plugins and bundling

A meta-plugin contains links to other plugins. This works fine as long as the meta-plugin stays on the PC where it was created, but as soon as you distribute it, you have problems. There's very little chance that the absolute paths will happen to be correct on someone else's PC. IMO the solution has two parts. The first part is, the meta-plugin should search for its plugins in an intelligent way, e.g. the following steps, in order:

1) Try the absolute paths from the project data first.
2) Then look for an INI file, e.g. My Documents\FFRend\MetaFFRend.ini, which could contain a plugin path, e.g. PluginPath=C:\whatever;C:\foo;
3) Next, try My Documents\FFRend\Plugins.
4) Finally, look in the same folder the meta-plugin was loaded from.
5) If none of the above works, add an error message to a log file, e.g. My Documents\FFRend\MetaFFRend.log

The second part of the solution is, there should be an option to bundle all the plugins that the meta-plugin requires into the meta-plugin. This could make the meta-plugin quite large, so it probably makes sense to compress the plugins, e.g. using zlib which is open-source. The first time you load a bundled meta-plugin, it will unpack itself to the folder specified in MetaFFRend.ini, or if that fails, to My Documents\FFRend\Plugins. On subsequent loads it will detect that the plugins are already there, and skip the unpack. This means the meta-plugin could be slow to load the first time, but subsequent loads will be fast.

Meta-parameters

Like any other FF plugin, a meta-plugin can expose parameters to the host. These parameters are referred to as "meta-parameters". The consensus on this forum was that each meta-parameter should be able to control multiple things. When I say "things" I mean control targets; the potential targets in FFRend are not only FF parameters, but also modulator settings (frequency, waveform, etc.), bypass, and other properties. The reason why this is such an important capability, is that many hosts severely limit the number of FF parameters a plugin can expose. By allowing "grouping", the meta-plugin can make better use of a limited number of parameters.

Anatomy of a Meta-Plugin

The base MetaFFRend DLL (i.e. without any project data embedded in it) lives in the same folder as FFRend. It has a tag at the end of the DLL file, consisting of 16 characters: BASEMETAFFREND01

When FFRend creates a new meta-plugin, it starts with the base plugin. It removes the end-of-DLL tag and replaces it with the project data, followed by a different tag, also 16 bytes, like so:

char Id[10]; // must contain characters MetaFFRend
WORD Version; // version number
DWORD DataOfs; // offset of project data from end of DLL file

What's nice about this scheme is it's easy to read the project data from the DLL: you just read the last 16 bytes, and now you know whether it's a valid meta-plugin, and if so, where to find the project data. Project data is a CArchive, and starts with a CMetaPlugin header:

CString m_PluginName; // plugin name, 16 characters maximum
CString m_Description; // a description of the plugin
CString m_AboutText; // author and license information
int m_BitDepthMask; // mask of supported bit depths
int m_PluginMajorVersion; // number before decimal point
int m_PluginMinorVersion; // number after decimal point
int m_NumInputFrames; // number of input frames plugin expects
CDWordArray m_InpTargetIdx; // for each input frame, index of target sub-plugin, or -1 for default
bool m_IsBundled; // true if sub-plugins are bundled into meta-plugin
bool m_IsCompressed; // true if bundled sub-plugins are compressed
CMetaParamArray m_Param; // information about each meta-parameter

class CMetaParam:

CString m_ParamName; // parameter name, 16 characters maximum
float m_Val; // parameter's initial value
CMetaParamTargetArray m_Target; // array of parameter targets

class CMetaParamTarget:

int m_PlugIdx; // index of destination plugin, or -1 for misc. property
int m_ParamIdx; // index of destination parameter, or -1 for plugin property
int m_PropIdx; // index of property
float m_RangeStart; // start of parameter range
float m_RangeEnd; // end of parameter range

Saturday, January 13, 2007

Heap-trashing bug also in RadialBlur, SpiralBlur & TimeBlur

I previously reported finding a bug in Pete Warden's Mixer plugin. As it turns out, this bug also occurs in three other Pete plugins: RadialBlur, SpiralBlur, and TimeBlur. The original bug report for PeteMixer is here.

PeteMixer, PeteRadialBlur, PeteSpiralBlur, and PeteTimeBlur all have the same bug: they use MMX movq (64-bit move) in situations where movd (32-bit move) was intended. In all cases the results are the same: the two bytes immediately following the end of the output buffer get trashed, AKA munged, hosed, stomped, toasted, vaporized, etc.

Since the output buffer is almost certainly on the heap, the consequences depend entirely on how the host uses the heap. This probably isn't deterministic, so depending on the circumstances the bug could cause strange and wonderful behavior, or crash the host, or have no effect all.

I searched the plugin sources for other instances of movq, and found no other instances except in a few support modules. I haven't had time to wade through those modules yet, but my guess is they won't have the bug, since they weren't cut from the same cloth, as it were. The support modules that use movq are:

BoxFilter
Radiant
ImageMath

I have built UNOFFICIAL patched versions of the buggy plugins. You can download the binaries here, and the patched source files are here. PeteMixer was patched previously, but I included it in the above downloads for completeness.

I diffed the code carefully, and found no differences between my versions and the original that aren't "good" differences. For testing, I ran the original and the patched plugin side-by-side in FFRend, sent their outputs into a mixer, and A/B'd them to ensure that their output was identical.

Here are the relevant MD5 checksums:

BUGGY VERSIONS:
PeteMixer BUGGY.dll 5d7f65d48627e065f218bab5ee1d42a4
PeteRadialBlur BUGGY.dll 04551870addf5b3c8c14740e1aa7d316
PeteSpiralBlur BUGGY.dll b1e66597fbebeac46e59a208fd893aba
PeteTimeBlur BUGGY.dll c636cb633bc404f050497c4369f70241

PATCHED VERSIONS:
PeteMixer.dll a4c78bfa1a34f895264c9e20e3c6b035
PeteRadialBlur.dll 93592d40a4c2b9470f601c0e20042c1b
PeteSpiralBlur.dll f2f9b4f8d0740d170f87b173d9b1f83f
PeteTimeBlur.dll 031cc8426accc21df0989b34fe884f69

Tuesday, January 09, 2007

Adapting CSizingControlBar to work without idle time

Since MFC generally relies on the existence of idle time, an MFC app that doesn't have idle time is arguably incorrectly designed. However, the theoretical multithreaded alternative is so difficult to implement correctly, that in practice, timer-driven MFC apps often do real work in their message loops, potentially using up all their idle time. My app (FFRend) is an example of this: since it processes video frames in OnTimer, if the processing becomes complex enough, idle time drops to zero.

This approach causes serious problems for Cristi Posea's otherwise delightful CSizingControlBar. It also causes problems elsewhere in the app, but these are minor issues related to the UpdateCmdUI mechanism, or the status bar message line. In an app without idle time, CSizingControlBar exhibits the following undesirable behaviors:

1. While the bar is docked, resizing it doesn't work: the bar is not repainted. This occurs because the sizing bar's implementation uses DelayRecalcLayout, which helps prevents flicker, but also relies on idle processing.

2. When the bar is floated, the main frame's layout is not updated. Again, the cause is DelayRecalcLayout. Note that this problem also occurs with other control bars, not only the sizing bar.

3. When the bar is floating, its Close button doesn't work: the bar remains visible, and left-clicking the edge of its frame causes the bar to shrink to a tiny rectangle.

4. The docked bar's close button doesn't react to mouse-overs.

I have found simple solutions for the first three problems. The last problem is still unsolved, but luckily it's the least serious one.

Problems 1 and 2 can both be solved by conditionally calling RecalcLayout from main frame's OnTimer handler. The condition is a simple test for pending idle layout, specifically, if (m_nIdleFlags & idleLayout) is true. I discovered this technique quite by accident in the "Professional UI Solutions" support forum.
 
void CMainFrame::OnTimer(UINT nIDEvent)
{
Sleep(40); // emulate some work that consumes all our idle time

#if ENABLE_NO_IDLE_FIXES // ck
// CSizingControlBar uses DelayRecalcLayout, in order to avoid flicker when
// resizing docked bars. DelayRecalcLayout requires idle processing, which
// normally isn't a problem, but this app typically doesn't have idle time.
// Our solution is to periodically test the idle flags, and if there's idle
// layout pending, call RecalcLayout. This also ensures that the layout is
// updated when controls bars are floated (any bars, not just sizing ones).
if (m_nIdleFlags & idleLayout)
RecalcLayout();
#endif

CFrameWnd::OnTimer(nIDEvent);
}

Problem 3 is solved by customizing CMiniDockFrameWnd to handle WM_SYSMESSAGE. In OnSysMessage, if the ID is SC_CLOSE, hide the dock frame, via ShowWindow(FALSE).

void CSizingDockFrame::OnSysCommand(UINT nID, LPARAM lParam)
{
CMiniDockFrameWnd::OnSysCommand(nID, lParam);
// in the default implementation, if there's no idle time, the close button
// doesn't work: the bar remains visible, and left-clicking the edge of its
// frame causes the bar to shrink to a tiny rectangle
if (nID == SC_CLOSE)
ShowWindow(SW_HIDE); // but this does work
}

Note that the derived dock frame must be installed in CMainFrame::OnCreate, by setting the CFrameWnd member m_pFloatingFrameClass, immediately after the call to EnableDocking.

EnableDocking(CBRS_ALIGN_ANY);

// use a custom dock frame that can handle zero idle time
m_pFloatingFrameClass = RUNTIME_CLASS(CSizingDockFrame);

With these two simple, lightweight fixes, CSizingControlBar works quite happily in an app without idle time. Ever now and then there's a happy ending. Enjoy!

Saturday, January 06, 2007

MetaFFRend : FFRend as a plugin authoring tool

I plan to re-purpose FFRend, from an effects renderer, to a plugin authoring tool, somewhat like Pete Warden's discontinued FreeChain project. This idea was proposed by Leo Mayberry (AKA KillingFrenzy) on VJ Forums, and seconded by many others. FFRend will be able to output a freeframe "meta-plugin" that behaves identically to a given FFRend project. A meta-plugin contains *links* to other plugins. A meta-plugin is a DLL containing FFRend's rendering engine, plus some project data (which plugins to load, parameter and automation settings, signal routing, etc).

The project is in development, and about 50% done. I already have proof of concept: I can create a meta-plugin that runs a FFRend project from within any freeframe host. The main limitations are, 1) it's hard-coded which project the meta-plugin loads, and 2) no parameters are exposed to the host.

The biggest headache is creating the GUI within FFRend for deciding which parameters the meta-plugin should expose to the host (I call these "meta-parameters"). Ideally a meta-parameter should be able to control different types of properties, not only plugin parameters, but also oscillator settings and so forth, as MIDI already does. Also, it would be nice if a single meta-parameter could control multiple properties (i.e. grouping). This is important because many hosts severely limit the number of freeframe parameters.

The structure of the project data needs to be finalized soon. I'm working on that now. Meanwhile I'm also working on some ease-of-use features for FFRend: a dockable file browser with tabs for Projects, Plugins, and Clips, and a dockable preview window. Both features are commonly found in VJ software.