Continuing with my reply . . .TIPS AND RULES[
NOTE: I am using "DP7" to indicate "Digital Performer 7.24" . . . ]
(14) You need to set the DP7 hardware drive to use the minimal practical buffering that your Mac can handle for DP7, which for the 2.8-GHz 8-core Mac Pro here in the sound isolation studio is shown in the following screen capture image . . .

I did a lot of experiments to determine the optimal values, and this configuration works nicely . . .
The Mac Pro here in the sound isolation studio has 8GB of memory, but both NOTION 3 and DP7 are 32-bit applications, and for the most part they are single threaded, so what happens is that perhaps 10 to 15 percent of the processing power of the Mac Pro is used, which also is the case for nearly everything, except QuickTime when it is exporting video, which uses all 8 cores at nearly 100 percent, but otherwise everything else uses perhaps 10 to 15 percent of the available processing power, memory, and so forth of the Mac Pro, which on one hand is a bit annoying but on the other hand is nice, since it maps to the Mac Pro being able to handle a lot more work when everything moves to 64-bits , even though I am running Mac OS X 10.7.3 (Lion) in 64-bit mode . . .
(15) If you are using and VSTi virtual instruments or VST effects plug-ins, you need to ensure that they are set to do "highest quality" audio generation, if this option is available, which is the case for the IK Multimedia VSTi virtual instruments . . .
(16) The information for doing ReWire with Digital Performer at the Propellerhead Software website and the MOTU website suggest using an Auxiliary track in DP7, but I use stereo tracks, because they have better controls and stuff . . .
The key is to set the Input buttons for the DP7 tracks, which turns them light blue or whatever, and this is required to be able to hear the NOTION 3 generated audio being sent from NOTION 3 to DP7 via the ReWire channels . . .
Once the NOTION 3 generated audio is recorded in DP7 as soundbites, you need to unclick the Input buttons to hear the recorded soundbites . . .
But when you are recording the NOTION 3 generated audio, you also need to click the "rec" buttons for the DP7 tracks . . .
(17) Depending on the total number of staves or tracks in your NOTION 3 score, you might have exceeded the bounds of what NOTION 3 can handle, even though NOTION 3 does not crash . . .
As you approach the upper limit of what NOTION 3 can handle within its 32-bit application workspace, some strange things start happening, and while NOTION 3 might not lock-up or crash, it will behave erratically . . .
The strategy I use here in the sound isolation studio is to keep the total number of instruments in the range of 20 to 25, where I only use VSTi virtual instruments most of the time, although occasionally I will use the NOTION 3 guitar, since the guitar notation stuff is very nice . . .
If I need more instruments, then I clone the original NOTION 3 score and use the clone for more VSTi virtual instruments, always keeping the total in the range of 20 to 25 instruments, where a key part of this strategy is to have perhaps 5 common instruments (kick drum, bass, chords, simple melody, and whatever), which provide aural references to what is happening in the song, since by the time I start working with clones the structure of the song is precisely defined and I know enough about it to need only a few common reference instruments . . .
Similarly, you want to keep whatever DP7 is doing within its reasonable limits, which with video also happening might require a bit of experimenting . . .
(18) You might want to get an external SSD drive and use the Thunderbolt port on the Mac mini. A smaller SSD drive might be sufficient, and it will be considerably faster for anything which currently is read from the internal hard drive in real-time, which is something I am pondering doing here in the sound isolation studio, except that the 2.81-GHz 8-core Mac Pro does not have a Thunderbolt port, but there is another way to run an external SSD drive . . .
THOUGHTSThis is all that comes to mind at present, so do all this stuff, and let me know what happens . . .
However, it is useful to know that Apple has lowered the price of Logic Pro 9 when it is purchased via the Mac App Store, where it is $199.99 (US) plus sales tax . . .
There advantages to using Logic Pro 9, since it is a 64-bit application when you configure it accordingly, and it has true stereo panning controls like the NOTION 3 Mixer . . .
I like DP7, but it is a 32-bit application, and I am not certain how it works when you are doing (a) ReWire with NOTION 3 and (b) video . . .
If doing the ReWire work and video simultaneously is too much for DP7 to handle, then I would try Logic Pro 9, which I am planning to do sooner or later, since I need true stereo panning controls rather than the stereo balance controls that DP7 currently has, where the difference is that with a true stereo panning control you can move the left side stuff entirely to the right side and vice-verse, including all points in between, but with a stereo balance control, all you can do is increase or decrease the relative volume of the left and right channels respectively, which is
not panning . . .
There are workarounds for the panning stuff in DP7, but for all practical purposes it requires routing a stereo track to two monaural tracks, which is a hassle and is visually distracting . . .
In contrast, the NOTION 3 Mixer provides elegant true stereo panning controls, where you will notice that the "panning control" actually is a control within a control, visually, which is quite elegant and easy to use, where there are three tiny "dots", and if you move the left "dot" to the right, this pans the left channel to the right channel, and so forth and so on, which maps to being able to locate specific instruments very precisely, which is another discussion, but the simple version is that panning generally is logarithmic, hence it works a bit differently than intuition suggests, because in addition to the location aspect, there also are relative volume aspects, which maps to far-left to top-center being logarithmic rather than linear, as is top-center to far-right, where for example sounds near top-center need to be lower in volume, because you hear it with both ears, while sounds more to the sides need to be louder, where the volume is logarithmic, which makes it like a strange type of clock where locating an instrument at 10:00 requires a bit of finessing, while putting the instrument at 9:00 is easy, as is putting it 12:00, with 11:00 being more difficult, and so forth and so on . . .
Lots of FUN! 