Ok, I know you are probably tired of people pointing out new RPG engines to you...
LOL!
To be honest, I already know what I want in a game engine. It just doesn't exist.
Read the 5th book of the Pandora's Box series.
Sounds like a huge game, right?
I'll let you in on a little secret:
We could do it today, and it wouldn't take a supercomputer to do it.
Imagine, if you will, a game where you have several basic environments, already generated. These would correspond to the basic terrain types - boreal, arctic, temperate forest, mountain, desert, etc. Each basic area has several basic variations - rivers, roads, etc. You have several types of basic villages where there's a road running through it, eight more where there's a river, and eight more where there's a road AND a river. Then towns, cities, etc. All are pre-generated, pre-modeled areas. With inhabited areas, you already know what kind of people you need just from the buildings you have in it. merchants, smiths, children, animals, guards, etc. Thus, each inhabited area has a set of attribute variables for the people and animals you'll find in it.
Big, I know. You'd literally have thousands of areas. But, you can trim this down by artificially limiting your environments. Like in DS1 - there's no "tundra", for example. You have snow, desert, jungle (with the expansion), beach, etc. Come up with a series of basic environment types, and terrain that links them together.
Of course, before you can tackle that project, the first priority is to get your artists a set of tools they can all use to produce the environments and test them. We'll presume you did that first. ;-)
Anyway...
Now, visualize a map in your mind - but not a map like humans see it, a map as a computer sees it. Basically, it's a grid. Each grid has a set of coordinates. When a world is generated, the game lays out basic continents, islands, etc. But each square (oh, let's say 1 mile by 1 mile) corresponds to a coordinate. As to what you FIND in that square, well, the map already is generated to know the basic terrain type that's there, whether there's a river going through it, roads, etc. From there, the game uses algorithms to determine exactly what is found there. But it doesn't have to keep it in memory - it's generated from the basic terrain and the coordinate variables. In other words, if there's a forest at coordinates 128x88 and the seed number that generated the map was 1,536,653,505, then there will ALWAYS be a type 1 village there with a road going through it. Only the map the game generates is actually stored - it's just written to a file that's read as needed. NPC's are generated the same way - what's in each square depends on the coordinates and the tile.
Names for NPC's, their posessions, everything - all generated from algorithms, all pre-determined. The game doesn't have to remember anything other than the map. Everything else is generated from that. Each map file contains the generated tile data, and the random seed used to generate the map, which is used as a further seed to make similar maps be different.
To get an idea of how this would work, check out an old game called "Sid Meier's Alpha Centauri." Run the game, then look at the tiles. Realize that you could generate an entire world like that, and each square would be different from it's neighbors. Yes, if you look over the entire world, you're going to find squares that are similar. BUt on a local scale, you won't.
From there, the game takes 'seed' cultures, and plots their locations on the map using simple algorithms. Certain cultures that fight a lot will tend to be next to each other, etc. You have a specific list of cultures and races (preferrably a large list), which are again determined based on the map's seed and the tiles it ended up with. Several variations on capital cities can be made, which appear on the map based on the tiles.
Names for cities, NPC's, and other similar things are generated the same way. The game doesn't have to remember anything, it's all determined by formulas.
NPC interactions can also be done the same way. After all, NPC's that are found in cities are going to spawn in specific spots, with specific roles. When each village or city is created, the artist deliniates interactions, which are stored as part of the tile. Thus, NPC's can appear with full names, backgrounds, and daily routines.
And none of it actually has to be stored in RAM, the game can simply call it up as needed.
Of course, this is the ideal. In practice, you'd probably end up with just a few basic terrain types, a few basic cities, etc. Your "First Edition" maps are going to be pretty simplistic, generating basically a big island in the middle of an unknown ocean.
But, as your artists continue building, testing and releasing new stuff, the game can be constantly expanded. Players themselves could be handed the tools to expand the world, meaning there would be a rapid explosion of new tiles that could be downloaded and the game program would simply plug them into the algorigthms.
The game system itself should be very fixed and unchanging. The more goodies you add onto the game system, the bigger it gets, and the more likely you are to run across bugs. Remember all those old "Hey, How Come I Can't Use X Mod with Y and Z Mod in Dungeon Siege?" posts? Not all gameplay mods the players will come up with will be compatible. Most, in fact, will not. Thus, gameplay and game mechanics have to be something that the players can't change.
Develop a simple combat system for melee and ranged, like what you see in Mageworld. There are a fixed number of weapons, you can't add new ones because that plays into what merchants will have for sale, which is a part of the underlying game algorithms and is something players shouldn't touch. Conversely, players should (like in mageworld) be able to basically make everything in the game, if they have the skills, tools and supplies.
The spell system should work similarly. Lots of spells, but the players have to discover them (or craft them) themselves.
Knowing that you will have certain cultures with certain cities, it becomes possible to generate quests. Quests would work similarly to how they did in the old Microprose game, "Darklands." Talking to NPC's generates the quest, at which point you have to go somewhere - perhaps to talk to another NPC that the game knows is going to exist based on the algorithmic method of world generation, or to go to a small village or dungeon or whatever that appears on the map once the quest is initiated, and disappears after the quest is done (returning to the default state of the tile). The Darklands quest system was very neat, had literally thousands of possible variations, and it ran on machines that were limited to 1 meg of ram. Darklands also had a specific list of weapons and armor that was available in the game, and despite that characters could potentially hold thousands of items of multiple types, character files were simplified because of that.
Multiplayer action becomes very simple. Instead of transferring the entire map between the host and client machines, the host machine only needs to send the seed number that was used to generate the map. Of both players are using the same mods - poof, the client machines spend a few minutes generating the game world, and you're ready to play.
All of the necessary game data (the wilderness tiles, civilization tiles, dungeon tiles, graphics music, etc) would all fit on a single "data" DVD, with the game program itself sitting on the player's hard drive. Mods would go in a special mod directory on the player's hard drive.
Then, after everyone has basically exhausted themselves and the game sales have finally slipped, the company releases one final patch - a patch that allows the players to create mods with new characters, new weapons, new spells, etc. Weapons, armor, spells and such would all be based on editable text files that point to art rescources in the mod directory. New character models and animations would be done similarly. Boom - suddely the game has a new life, as modders pick it up and start modding it into everything imaginable, adding new stuff that nobody ever thought of before.
Then, when game sales slip again...
They release it as Open Source, allowing people to create their own games and sell them using the game engine, on the condition that the original company gets paid royalties.
Development time for the first release would be on the close order of five years, though this could be reduced to less than two years by using isometric graphics (Diablo-style view) and using sprites instead of 3-D models. And sure, everyone is going to notice that the game graphics will very likely be years behind the times. But the moddability will keep people coming back, easily giving the game a lifespan of twenty years before it becomes "Open Source", and is earning the creators royalties for the rest of their natural lives.
All of this is possible today. Right now. We have the computer technology to do it.
But, I don't have the money to start pulling together software engineers to make this, and modern game design is basically turning completely away from anything I would ever be interested in playing, so it remains just a dream.