Quantcast
[opne]Possibility: Different physically SavedVariable files for the different servers - ESOUI
Thread Tools Display Modes
03/16/20, 04:53 AM   #1
Baertram
 
Baertram's Avatar
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 3,050
[opne]Possibility: Different physically SavedVariable files for the different servers

Hi ZOs devs,

it would be very cool if you would give us the possibility to create different SV files for the different servers.
Like <addon_name_eu>.lua and <addon_name_na>.lua if parameter "splitSavedVarsByWorldName" is used in ZO_SavedVars.

Some of the curernt files get really big and slow down the loading of the game a lot, but it's becasue they contain the data for the different servers.
So if I login to EU and only need the EU data it would be great if only those would be loaded.

Please also think about a method to copy the SV data from one SV file to the other SV file (with the same addon name for sure!) so one would still be able to copy SV data between servers somehow.

Thank you very much for considering this.
  Reply With Quote
03/16/20, 06:31 AM   #2
Shinni
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 150
One workaround would be to create two container addons.
So you have one addon "MyMainAddon" which is the actual addon, and two addons "MyAddon'sEUdata" and "MyAddons'USdata".
Store your heavy data in the latter two and disable one of the addons for the respective EU/US characters.
  Reply With Quote
03/16/20, 07:43 AM   #3
sirinsidiator
 
sirinsidiator's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 1,417
I'd rather see them add a proper solution for data heavy addons to efficiently store and access their data. The current Saved Variables format was never intended to contain megabytes of anything as can be seen on the load times.
Adding a way to store raw data could already help in some cases, as it would remove the need to run everything through the Lua interpreter on load. Even better would be a database interface that allows to only fetch the data that is needed when it is used.
__________________
https://www.imperialtradingcompany.eu/ - My Addons - The Vault (Early updates and experimental projects) - My patreon - My blog
>siri.exe MyAddon
Does your addon work? [y/n] n
There is a typo in there.
  Reply With Quote
03/16/20, 12:36 PM   #4
Rhyono
AddOn Author - Click to view addons
Join Date: Sep 2016
Posts: 646
I'd be in favor of something even more basic, such as a flag like SeparateVarsByServer: true in our txt in addons (not a user setting)

No extra lua programming on our end, it would just automatically treat US, EN and PTS as separate saved vars files.

This would allow addons that have some reason for cross-server storage (maybe that's their purpose!) to continue to function the same, but something where the data never should mix would be able to set it differently.
  Reply With Quote
03/16/20, 01:59 PM   #5
Micke2nd
AddOn Author - Click to view addons
Join Date: Mar 2017
Posts: 43
Originally Posted by sirinsidiator View Post
The current Saved Variables format was never intended to contain megabytes of anything as can be seen on the load times.
But even if, it's still only a few MB. In my case 60 MB. How big is your folder ? Even if 600 MB, what's not a big size for a modern drive. So maybe is just your drive the bottleneck ?

Zos is currently updating the multithreading usage. Actually predestined for loading independent resources faster. But especially in this case, the drive has to deliver fast.
  Reply With Quote
03/16/20, 02:34 PM   #6
sirinsidiator
 
sirinsidiator's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 1,417
The bottleneck for saved var loading is the Lua interpreter. Their efforts to make asset loading multi-threaded won't affect that at all.
__________________
https://www.imperialtradingcompany.eu/ - My Addons - The Vault (Early updates and experimental projects) - My patreon - My blog
>siri.exe MyAddon
Does your addon work? [y/n] n
There is a typo in there.
  Reply With Quote
03/16/20, 02:51 PM   #7
Rhyono
AddOn Author - Click to view addons
Join Date: Sep 2016
Posts: 646
Are you thinking something like a SQLite database? That'd probably be far more performant than a 10k row Lua table.
  Reply With Quote
03/17/20, 12:09 PM   #8
Baertram
 
Baertram's Avatar
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 3,050
Originally Posted by sirinsidiator View Post
I'd rather see them add a proper solution for data heavy addons to efficiently store and access their data. The current Saved Variables format was never intended to contain megabytes of anything as can be seen on the load times.
Adding a way to store raw data could already help in some cases, as it would remove the need to run everything through the Lua interpreter on load. Even better would be a database interface that allows to only fetch the data that is needed when it is used.
Also a very welcome alternative/replacement, right.

One workaround would be to create two container addons.
So you have one addon "MyMainAddon" which is the actual addon, and two addons "MyAddon'sEUdata" and "MyAddons'USdata".
Store your heavy data in the latter two and disable one of the addons for the respective EU/US characters.
Yep, the way like some addons do it already.
This wouldn't allow me to share the data between the addons properly as long as they are disabled but would work I guess.

The bottleneck for saved var loading is the Lua interpreter. Their efforts to make asset loading multi-threaded won't affect that at all.
Correct. My SSD is fast enough to handle this But loading times still increase the bigger the files are, independent from any drive speeds!
  Reply With Quote

ESOUI » Developer Discussions » Wish List » [opne]Possibility: Different physically SavedVariable files for the different servers

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off