Still a little ways to go
Kenn:
While the technology we saw was incredibly impressive, there were still in beta, and thus there were still a few problems that needed to be hammered out.
Separation Boundary
This was the most notable issue, and one which would definitely be fixed by the time any product shipped. What did it look like? Down the middle of the screen, you could see the delineation of the two cards - a slight tearing effect where a few pixels were slightly misaligned. It really wasn't too noticeable, except where a number of effects such as alpha transparencies were stacked on top of each other. The problem is simply one of alignment however, and "it'll be perfect by shipping time," Metabyte claims.
The following is a simulated image I cooked up in Photoshop to demonstrate the effect. The important things to remember are: 1) it's not real, and 2) it should be a non-issue as Metabyte tweaks and perfects their hardware and software.

This picture is fake,
but the effect is representative
Thresh:
Just to reiterate how strongly they feel about this, Metabyte is positive that this will be fixed; they say it's a very easy issue to take care of. Syncing the top and bottom half should be a lot less trouble than aligning every other scan line in Voodoo2 SLI, so we're taking that statement at face value.
Unbalanced Load
This is an issue first brought to my attention by Alan Dang from 3DGaming. In certain situations you'll have much more complex geometry on one half of the screen, especially when the image is separated into vertical halves. If the top half of the screen is filled with sky while all of the terrain is on the bottom, the load is going to be unevenly distributed to the bottom card, and your extra 3D card will sit there stalling. Needless to say, there goes your speed advantage.
As reported by Sharky, Metabyte is working on a way to distribute the load between the two cards unevenly, in a process called load balancing. When we asked about this, Metabyte wasn't ready to disclose any details about it, but they acknowledged that this was a solution they were developing, as well as several others. They cited being able to send more information to the AGP card (if you run one AGP card with one PCI card in tandem) in such a case, but I don't see why it wouldn't be possible to distribute the load between two PCI cards as well.
My personal take on the matter is that it might be more balanced to split the screen vertically instead of horizontally (or provide an option to choose from either of them). Since most of today's 3D games are still very much based on increasing geometry on the Z axis to make larger, sprawling levels (as evidenced in Tribes, Unreal, Half Life, any racing game, etc.), the polygon distribution would be more symmetrical if divided vertically. Of course, there are instances where you're halfway hidden against a doorframe trying to peek into the adjoining room, but that's what you get with full-freedom 3D.
Metabyte's response to this is that horizontally, you're working with individual scan lines. Vertically, you'd have to be splitting individual pixels, which will incur much more overhead. Considering how 3D cards perform final rendering to screen, this makes a lot of sense, but the potential problem still exists.
Software Overhead
As a largely software modification, there has to be considerable overhead in coordinating multiple cards. One of the factors Metabyte claimed had a significant performance hit was rendering the polygons divided in the middle between the two 3d cards. Such polygons will have to be split into smaller polygons and then sent to the appropriate card for rendering. However, the raw power of having two 3D accelerators should shine through most any bottlenecked overhead, and software optimization will play a big role as development continues.
Pending Competition
Thresh:
While Metabyte has been working on their Stepsister technology for over three months now, the very announcement of its possibility has other companies scrambling off of their asses to implement the same technology. To paraphrase Orson Scott Card, once people know it can be done, it's only a matter of time before it happens.
However, while the competition is going to be there (this technology is way too hot to NOT imitate), it's probably safe to say that Metabyte is way ahead of any potential copycats.