The same goes for cooshoos/hyenas/blink dogs and silver wolves/mist wolves/the cute dog.

There are only so many colors, and the l(ook) command makes it easy to figure out what sort of creature you're faced with. I'd find it much more confusing if the long-established colors changed...

It doesn't matter that "look" command exists - players shouldn't waste it every time they encounter a letter that they have already seen hundreds of times.
Only similar monsters should share a letter and color.

If there is such a drastic lack of free colors, Ratling Master Thieves should share color with ordinary Ratling Thieves, and not with completely different Gargantuan Rats - or Gargantuan Rats should share color with ordinary and Giant rats.
Same with Vapor Rats - they share far, far more properties with Chaos Rats than with Giant Raccoons.

Do you have suggestions for a better color scheme for these instances?

1) In my opinion yellow is a perfect color for vapor rats - this would make them unique in their own way since only ratling guardians are represented as yellow "r".
I have frequently ran into gray "r" only to discover they were in fact not raccoons after they landed 2-3 corrupting, PV-ignoring hits.
Yellow is close to gray and should not make it too difficult to get used to the new color.
Yellow also seems to work with lightning - crown of lightning is a yellow crown when unidentified, yellow armor and weapon prefixes grant resistance to lightning, vapor rats are chiefly found in temple of elemental air and seem immune to the charged electricity (though not to lightning attacks if Ancardia Wiki tells the truth).

Another option for vapor rats would be a light pink (corruptor color) or deep violet (orb guardian color) to reflect their chaotic/corrupting nature, but this would make them almost indistinguishable from the significantly less dangerous chaos rats. It's still a viable option though.

2) Ratling master thieves should follow the same color scheme that swordsmen and master swordsmen follow - blue and dark blue. The problem is that blue "r" is already used by ratling traders.
However there is a precedence here since dark blue @ is shared by many monsters - some are hostile, some are neutral, like thugs, master swordsmen, big casino guard and the priest.
People have learned to distinguish between them, mainly because they appear under special circumstances, are limited to 1-2 specific levels or are entirely unique - random thugs are exceptionally rare and otherwise appear only when summoned by shopkeepers.
There is only one priest in the game and he is always in the same place. Only one big casino guard exists as well - he's stationary and occupies only one level.

The same is true for ratling traders - they appear in arena and occasionally in HMV, that's it.
Thus I think giving ratling thieves the same blue color and ratling master thieves dark blue isn't going to cause too much confusion. After all traders are actually just legal thieves ;)
Similar color pattern is noted among dragons - baby/adult dragons have two colors - lighter and darker - and among many other monsters.

Another option is to just give both ratling thief types the same dark blue color, there is a precedence for that as well:
Dopplegangers/lords/kings always have the same color yet the threat they pose can be non-existent, significant or extreme, you can't tell by the color alone.

As noted above - group them based on abilities. Ratling master thieves would have the same color as non-master ratling thieves. Chaos rat spawn and corrupt just like vapor rats, so they are also a good choice to have the same color... Gargantuan rats are just bigger Giant Rats.