Also, even if "people with job X" caught on, that wouldn't actually achieve anything. The impression or image in your head that you get when you hear "people who work in a factory" vs. "factory workers" is the same, even if the wording deliberately makes reference to people.

I appreciate what the intention is behind it, just like with "people with disabilities" vs. "disabled people", but it doesn't address the societal issues that result in the actual discrimination/problematic views.

This is more of a theoretical argument, but from a linguistic perspective there's a common misconception that language shapes our perception of reality, when in fact it's the other way round. If you talk about subject X which is looked down on for whatever reason, the commonly used word for it can take on a derogatory tone, which leads to well-meaning people using a more positive word. This is commonly seen with words for women in patriarchal societies, and it leads to a cycle of neutral words becoming derogatory and polite words becoming the new standard, until that word becomes derogatory and has to be replaced in turn. None of this is to say we shouldn't be careful with the language we use, and in fact it's a good way of signalling respect, but I just want to highlight that adopting new terminology is itself won't achieve much.