We have recently been having a lot of issues with tags and categories misbehaving in wordpress and I finally figured out the solution. Since it was anything but obvious I thought I share my findings.