The world is split into three open world parts you need to fast travel between. (Skellige Islands, Novigrad/no mans land and White Orchard) This is a known fact.
You must have misunderstood something, as Vizima for example is not part of the open world map, but a separate location you can only access via quick travel. Here's the official map of the game for reference: (Novigrad/No mans land is the large map, Skellige islands is the second large open world location you need to fast travel to get there. White Orchard is the lower right small map that is also a separate location you need to fast travel to get to)