Well, from my experience, if there is the possibility of hot glue (from a typical hot glue gun) melting through the rubber/plastic material, there is always the option of getting a glue gun that switches the heat setting (or just getting a low heat glue gun). Those kind with the low temp. are meant for delicate materials or substances that would otherwise be melted by the normal temp. setting.
Other than that...I dunno, I haven't dealt too heavily with other forms of glue, but good luck, Tom! Is this a secret cosplay, perhaps?