Shortly after posting I found a fix for the incorrect colors; using YCbCr color space! YCbCr is probably the closest to how VHS represents colors, the results are pretty much perfect without any unwanted artifacts.
This is great, thank you! Not only is SoX faster to use than Audacity, it also has multiple resamplers (or resampler settings) that are much better suited for my needs. One resampler solves my problem with ringing artifacts by toning them down, and another resampler shifts the image slightly and produces natural looking color bleed. What a gold mine.
>>There are no vertical artifacts
>rotate the image and do it again.
I tried that actually, but the results look bad and not like VHS at all. It must be something else...
>You could probably process the image in raw RGB
I'd like to avoid that as much as possible. In my current workflow I don't do any operations in RGB space, everything is done in YCbCr including noise addition. That being said, there's a plugin on Github for a program called AviSynth that creates the "dot crawl" effect and nothing else. I might study it / use it in the future to achieve some interference between the signals.
>There's all sorts of existing audio effects in various forms
Agreed, there's still plenty of room for experimentation. One thing I discovered while playing in Audacity is that running an image through a low pass filter creates a "ghosting" effect, similar to 1910s-1920s video or extremely worn out VHS tapes. It's honestly quite interesting what you can do with images by processing them as audio!