Personally, when I have encountered this same question, I've looked for the soul of the watch. As obscure a concept in watches as in humans.
The question I ask myself is "can I feel the hand of the watchmaker on the watch, the design, the movement, the details, the craftsmanship, the buying experience, the combination?"
Not literally but spiritually, as I know many independents may not have actually assembled the piece themselves - but maybe they did. Or, the "distance" from them doing it themselves is short enough that I can feel their hands in the process.
When a watchmaker has moved too far from the process (due to number of watches made; their focus on the business vs the making; the loss of the spark, or whatever), then the soul of the watch changes.
So things like meeting them in person, a new movement design, the buying experience, a personal accomplishment by the watchmaker all contribute to the soul. So even a piece from a true independent can seem soulless and I'm more likely to pass
One final thought on the buying experience - supporting and collecting time pieces from independent watchmakers is an ongoing learning process. Having access to the latest news and community is part of the thrill for me so I support not only independent watchmakers, but independent representatives of those watchmakers.