What makes a "good" AMV?
There are as many answers to that question as there are categories of AMV. I guess I would say that it depends on what you want your AMV to do. If you want to be funny, then look up tips on comedy (or start with audio that is already hilarious and find a way to illustrate it). If you want to be thrilling, then look up tips on suspense or action-movie editing.
However, there are a few technical traits that will help almost all AMVs because getting them "right" will avoid distracting from your content. Don't use crappy, low-res video (unless that's the category), don't abuse your favorite effect until even a newbie notices it, don't violate contest rules if you want to enter your AMV for judging...
Structure-wise, it's helpful to set a hook early. Most watchers start at the beginning, and if not "hooked" within ten seconds (maybe less), they'll bail out and go on to something else. No amount of later awesomeness will save you if your audience never sees it. If your audio selection starts slowly, then you might want to chop it. An audience might also postpone judgement somewhat for a "bumper", but that only buys you another 5 or so seconds. When in doubt, cut to the chase. Even adding a little text can work because it engages viewers into reading, and because you can prime your viewers to see what you want them to see in all that follows.
Don't repeat yourself. Just because your music choice repeats its chorus five times down the home stretch doesn't mean you must. If you can tell deliver your message in less, then consider chopping your audio source. Don't use 4:20 to tell a 2:44 joke.
Choose audio that will appeal to your audience. If you want mass appeal, then don't choose something (like death metal) that appeals to a narrow niche while making everyone else feel as if they're being attacked. Your AMV may have a strong impact, but not a positive one. I'm not saying to go to a snooze-fest opposite extreme (i.e. elevator-music), just don't go way outside others' comfort zones unless you're willing to lose them from your audience.
by its simple definition, an AMV is an Anime of choice put to a Japanese song
I don't think that the audio needs to be Japanese, just the anime (or pan & scan manga).
Majority of the scenes synchronize on some level with the song. (Pace, beat ETC)
Synchronization is enjoyable (at least to me), while video-syncopation is uncomfortable. In addition, a human brain has some natural rhythms. Much popular music fits one or more of them (that's one reason that they're popular). Video cuts that match both audio and natural rhythms "feel good".
1. When you start an AMV, how do you go about picking a song? Does the lyrics/message/theme of the song itself have to match the anime you pick to go with it? Basically, what level of relation does the song and anime need to have?
Besides what I said above, your audio choice
must be something that YOU will enjoy hearing 100+ times (because you will). After that, it depends what you're trying to do. If lyrics are crisp (well enunciated), then they can convey a message that you are illustrating. If the lyrics are meaningful but difficult for an unfamiliar listener to parse at full speed, then you may consider adding your own text for key words or phrases (be consistent). Also, when using text (always your own and never left-over fan-subbing), remember that your audience will need time to read it, and some will fatigue if there's too much of it. Like any spice, use it with care. The difference between a cure and a poison is the dosage.
Not all AMVs depend on lyrics. Some use instrumental music to back up mime-quality anime, and others provide their own text around the edges to inject the AMV editor's own meaning into the anime (often to comic effect).
3. What are the general unwritten rules or guidelines for AMVs? Is it better to use 1 anime for the AMV, or can you use multiple animes and still end up with good results? Also, does a good AMV jump around and mix lots of random scenes that look cool, or does it "summarize" 1 particular episode or notable scene?
Again, the answers depend on what you're trying to do. In general, if you're telling us something about one anime's setting or characters, then it's almost always jarring to see a cut from another anime. However, if you want your characters to interact with an interloper, then it can be good fun (i.e. have a purpose for the adulteration). OTOH, an AMV could be about an anime trope, in which case it would make perfect sense to run a sequence of dozens of different anime, each advancing your illustration.
It's like war or business: Choose your objective and then organize around it. AMVs generally aren't long enough to hare off in multiple directions at once (although some deliver what are really sequential anthologies, sometimes with a shaggy-dog ending hearkening back to the opening).
The views above reflect my own personal viewing tastes, which have been shaped somewhat by comments from AMV editors and fans over the years. You may reject them and perhaps still find an audience.
