Nice post ! I've tried all kinds of system over the years. I started with the simple 1st Ed thing of 1XP per monster hit point or gold piece gained (rapidly moving on from the idea of whoever got the killing blow collects all the XP for a monster !). I've used the 5th Ed milestone system, but that always feels like a lazy cop-out, and I got a bit fed up with my players griping because they felt like a simple encounter should be worth a milestone.

Nowadays, I mostly play D&D 3.5, and use something loosely based on the system in that ruleset. Being a sad anorak, I maintain it all on an Excel spreadsheet. There's a column for each character, and the XP awards section is split into 3 parts; XP for combat, XP for "set piece" non-combat situations (traps, puzzles, situations that require specific RP, etc), and XP for general roleplaying, ability to stay in character, and offline contributions (e.g. I give 100XP to whoever posts a write-up of that week's session).

Although somewhat weighted towards combat, it seems to work better to reward the less front-line characters, and helps reward lateral thinking and roleplaying. When it comes down to it, the RP awards are still arbitrary, but the spreadsheet enables me to compare the characters to see if it just "feels right" for the amount of effort they put in.