I think fundamentally, the concern is misplaced. The fact you need to work for wealth is a convention of our constraints. The change in constraints would lead to other means of distribution. It's easy to see if someone who believes more productivity is good would not see making jobs obsolete a real problem. Thew would see us adapting to the new conditions in a relatively short while.
> The fact you need to work for wealth is a convention of our constraints
The current constraint is "you need to produce to have things".
If one company's AI takes all the jobs, and thus does all the producing-to-have-things, the constraint transforms into "you need that company's permission to have things".
Those who are concerned is implying that any new distribution mechanism is not going to favour them.
And under the capitalist system, if nothing changes, the "new" distribution system is indeed not going to favour them - at best there would be some sort of UBI, and at worst you would be left to starve in the streets.
However, i cannot see how one can transition to a new system, and yet have the existing powers in the current system agree and not be disadvantaged.
Yes, this poses political problems and it will have to be hashed out politically. Again, the problems with these are political not necessarily technological. And they are also tractable. There are many many possible ways this can play out and we should be careful which we choose. I just don't think the conclusion is foregone. There have been people's jobs displaced in the past. And things settled eventually. I won't argue they settled for "better" because many people are unhappy about many things, but I will say... the world is amazing.
At the risk of sounding like a longtermist, I think that when all is done, the result will be a net positive one, but it WILL cause strife for many people - probably me included. But I refuse to keep my childern's future hostage because I might have to reskill.
If you are speaking about the world, hundreds of millions in the next 5 years is probably closer to reality in my opinion. And from your question I think that you already know the answer.