I only have 6-8 flies that I have tied for my tarpon season. Sure, I mess around on the vise and tie up all sorts of creations that I hope will unlock some aggression in the fish, but the truth is that I use light and dark Keys style flies, light and dark Steve huff Ballyhoo flies, Worm flies and light and dark Toad style flies regularly. With those flies, you can probably get a fish to bite.
Flyfishing for permit is difficult not because the fish are terribly difficult to catch rather that our flies and techniques do not accurately represent what they want to eat. Permit look at a dead crab the same as they look at a fly. The profile is interesting but not really what they had in mind. Flies need to have action resembling the real crabs and until they do, it is my opinion that they will remain exceptionally difficult to catch on fly.