A friction mallet is a mallet with a flexible handle and super ball at the end, used to create sound by rubbing on a surface, or creating friction. There isn’t any need for extra notation for this technique, other than a note to specify when to use the mallet. Super balls make for terrible beaters, so performers should assume that it’s being used for friction.
You could, in theory, use a friction mallet on absolutely anything, but some instruments react much better than others. A vibraphone, for instance, would mostly just emit the sound of the mallet being dragged, and almost no pitch. This technique is mostly used on where it works best, which is on a drum head. Check out the video below for a quick demo on some sounds you can get from a friction mallet on a low floor tom. (this video is not sponsored by Evans)
A handful of really cool sounds come out, through a combination of placement, speed, and pressure on the drum head.
A note on orchestration: while this video has some pretty loud samples, the dynamic ceiling of this technique is pretty low, regardless of surface. If the instrument is unamplified, be careful with writing it in an overly dense texture. I’ve had quite a few experiences where I’m trying my hardest to make a loud growl and am hopelessly buried beneath the ensemble.
A quick aside: Sometimes it seems that these mallets are more popular with composers than performers, so it isn’t the worst idea to own some of your own; they’re cheap and easy to make, or you can buy this 3-pack for a reasonable price online. (This is not product placement, I just like these and am sometimes too lazy to make my own).