Buddhist and popular American positive thinking would say you deserve the trouble because of crimes committed in past lives or negative thoughts in the present one. That is a fanciful, mythological explanation, acceptable if you can't find something better. In fact The Odyssey offers an explanation in which actions taken in the first part of life explain consequences in later life, and that has to be better, because you can look for reasons, particular causes in this world, the earlier life leads to the later.
If you leave home you will be attacked, possessed, threatened by strangers, and sometimes be driven to preemptively protect yourself from attack. Your story telling will also attract to you hospitality, and perhaps you will offer hospitality yourself at your temporary homes.
Assuming you have acted for your own reasons, not by coercion or loyalty to a group, you have made yourself into a special kind of person:
It doesn't need too much thought to see that for the people you are going to pass among on your way home, who are neither home-leavers nor fighters, you present a problem and an opportunity.
You teach another way of life, with its advantages and disadvantages, and this is a good lesson to learn. Some people will want to take advantage of your temporary visit. You can also be harmed by your hosts without affecting in any way their usual lives. Not only because of your physical vulnerability of being alone among strangers, but because being of the type "stranger, home-leaver" you cannot count on the sympathy of strangers, at least not as a group. You left your group.
The form The Odyssey takes is episodes of danger and escape, danger and escape. There is no one conflict over principle, but a sequence of conflicts, each identical in being about the consequences of being a fighter and home-leaver. That is a most important point: there are no significant stories after leaving home. Significant stories are about the relation of individual to society, are about being home. And this is known by the people the fighter and home-leaver travels among in his long return home. The meaning, only meaning, of his story is getting home. He is a bad investment. If you dispute with him you won't learn anything about life. You might as well kill him.
Some of you have read The Odyssey, and know what a fundamental book it is. If such an immensely important book says your life has a meaning, the first part explaining the last, and people you meet with on your journey home tell you your life, this life, has no meaning in itself, you have to look to past lives or negative thoughts for explanation, you know how to choose between explanations.