The picture above shows (a rather grainy) image of a 'One For All' AV sender. The SCART plug (for those of you outside of Europe, it's the part with silver pins) simply fits into the back of a TV and the rest of the device bends up until vertical.
In my opinion this is how USB 'dongles' should be made. It makes far more sense than jutting out to the side and needn't get in the way of typing. I'd imagine if done right, it'd be less prone to accidents than one that sticks out to the side. With a hinge (like on the One For All above) it could still be used horizontal if need be.