Lets go back to 1st principles on the star map front. The overall objective is to produce an interesting future history that can provide a good base situation for a strategic wargame, an SF novel or two, and an RPG campaign, possibly all at once. The secondary objective is to enjoy the journey, which may mean giving the any particular attention that it deserves. This is my hobby, not my job, and playing with this is fun by itself.
The role of the star map in this exercise is to provide the stage upon which the play is acted out. It has to be large enough that, even though the human action is crowded at the centre, there is a sense of awesome extent toward the wings. It should be large enough that other species may have their acts on the same stage without a sense of confinement. In other words it has to be big enough to be empty and impersonal. Not just stated to be, but experienced to be in the sense of "show, don't tell".
So, how do we manage the data if we want to push the boundaries out, say, 150 parsecs to include a round million things? And where do we get the data if we want to do that?
Part of the solution (as I picture it at this moment) is "make it up"; the other is "broad strokes where detail is not needed".
I don't want to contradict reality, but at the same time I am willing to sweep some annoying realities under the carpet. So, lets say we start with a catalog down to 9th magnitude. That particular catalog has just shy of 120,000 objects - it is all of Hipparchus plus all of the Yale Bright Star. If an object is dimmer than about 6th or 7th magnitude the human eye can't see it from Earth.
So what I propose is this (using a competent database such as mySQL or Derby and lots of Java programming):
- Take all of the "good" systems -- those with clean positions -- and build a starting dataset with absolute minimum information compactly encoded with each cataloged system. By compactly encoded I mean about 20 bytes/system -- mostly be relying on putting bulky information in secondary tables that are only used when detailing is needed. This minimal table will start with 120,000 rows. It will end up with lots more.
- Cleanup. The initial data will have real-world grit. Astronomical catalogs are never as simple as the undergrad textbooks let on, especially for entries such as spectral type -- generally because the observation has never been made. This has to be purged so that everything is expressed with an entirely synthetic cleanness that will fit well with a computer implementation of some reasonable random star system generation rule.
- Backfill. We have base object density for nearby objects. As we move out from Sol, more objects will fall below the magnitude limit of the catalog, whatever that is. We make up for it by completely randomly generating objects up to more-or-less the expected density that are dimmer than the required magnitude at this distance. That means that out ratio of made-up to measured entries would go up as the distance from Sol increases. While someone with a better catalog will soon discover the fake bits, it will never contradict what someone (even in a dark place) can see when they look up.
- Big Picture. In an approach familiar to Traveller GMs, identify the star systems that are good candidates for habitable worlds, and randomly generate them along with any key infrastructure. Then identify the systems on the minimum cost jump routes between them and generate big picture items -- in Traveller, for example, the presence of a Gas Giant was always important.
- Make choices and detail where important. Pick major species homeworld and detail those systems, for example.
- Drill down when needed. Until the players or plot are about to enter a system, or the fleet is about to have a battle in it, the details don't matter. Until the players are planning to land on a planet, the weather does not matter very much. ("Lets see, your flight plan calls for a landfall on Earth. Near Winnipeg. In February. Got a parka in the locker?"). Once something is generated, it goes in the database; but we are never going to have to develop complete data for a million stars, 10 million planets, 100 million moons and odd rocks.