First off, some of the crazy-mad-insane-omg-forever-awesome effects are actually just out of the anime...
RahXephon was barely known when the video came out so few could really tell which was Koop's editing and what was already in the footage. To this day I haven't seen the anime so I couldn't tell you myself.
I said it once and I'll say it again:
masking. Moving shapes that are set to a certain transparency type can show the footage inside the shape, and can also move whatever direction the shape is going if it's set correctly to do so. Koop made the various circles, rectangles and lines in MS Paint, brought them into Magix and monkeyed with their transparency levels to show different layers of footage on top of each other.
The part near the end where it seems like a line of rectangles circle around the character are just carefully placed rows of rectangles to make it seem that it's circling around the characters. It's actually a really clever technique.