Because the USSR didn’t want them to.
The Soviet Union wanted people to make revolt in their home country, raise the red flag and join the Soviet Union as an entire happy nation, free from exploitation classes and liberal scum.
Some people were too impatient to wait for the rest of their compatriots to free themselves. They traveled to the Soviet border and soon discovered it was guarded better than Fort Knox. Look at the DMZ between North and South Korea to get an idea: that´s us who taught the Norks how to guard borders.
This is what the Soviet border looked like:
To the left, a barbed fence. In the middle, a plowed strip of mud where anyone who crossed it would leave visible footsteps, so that the incursion could be detected and hunted down by patrols who inspected it with assured frequency.
The foreign poor people who somehow managed to find an opening in the fence and didn´t get shot by border guards, would promptly be located inside the USSR thanks to all-encompassing ID checks in every nook and corner. Again, look at the North Korea. They learned from the best.
Thereafter, the people´s court of justice would swiftly get the trespasser a prolonged sentence as a foreign spy. Who else but a cunning spy would have skills and resources to travel half the world and outfox the Soviet border guards? The Soviet Union was known to live in the iron circle of capitalist powers that dreamt of annihilating us; who could blame us for the preference to be rather safe that sorry.
Much more predictable was the legal route through Soviet embassies and consulates around the world. Sure, they had strong preference for people who were not poor, but chances for success were generally good. A German-Jewish idealist Elinor Lipper was one of the lucky applicants, and wrote a book about what happened afterwards.
The number of foreigners who wanted to move to the USSR dropped after WW2. Still, there were some exciting stories. Among them the tale of Lee Harvey Oswald. He found love in our country and got a kid, but later preferred the career of an American president assassin over the safety of life as a floor operator at an assembly plant in Minsk.