Quantcast work in progress - libASV (Advanced Saved Variables) - ESOUI
Thread Tools Display Modes
08/12/14, 09:57 AM   #1
zgrssd
AddOn Author - Click to view addons
Join Date: May 2014
Posts: 281
work in progress - libASV (Advanced Saved Variables)

The built in system of accessing the saved variables is a mixed blessing. While it helps with keeping the settings ordered, is relatively simple to use and allows simple versioning/return to default, it has downsides & limitations galore.
Nothing made this as obvious as the broken GetDisplayName function in 1.2.3 and it's fix in 1.3.3 and thier effect on ZO_SavedVars.

In the end a saved variable is nothing but a global variable that get's dumped to disk on UI unload and read in from disk on UI load. So many have started working with them directly. So I got the idea to make a library for that. Trying to get a bit of the simplicity and order of the original, without the limitations.

Advantages:
Unified approach wich simplifies debugging and development.
Ability to override everything - account name and character name - to get everyones data.
A new Scale "OS account wide". A specific area of the saved variable not tied to any account. This is helpfull if you want to share data between accounts, but mostly if you have data that is in no way bound to one account (runtime gathered harvensting nodes or skyshard positions, for example).

Downsides:
No versioning control (so no simple return). This is something I will not be likely to support. I might think of some way to do it later. You have to keep track of the version yourself.
No defaults - this is temporary till I ironed the basic code out.
If you switch from the Built in version to this one, you either have to reset the data or write basic migration code yourself (this should be a one time work. I can give examples).


This is the data structure (wich can be found inside your saved variable file) how it would appear in a multi-zen account, multi Character situation:
Lua Code:
  1. SavedVariableName = {
  2.     ["libASV_Strcuture_1"] = {
  3.         ["libASV_OS_user_wide"] = {
  4.              --data goes here
  5.         },
  6.         ["@acountname1"] = {
  7.             ["libASV_AccountWide"] = {},
  8.             ["Character1"] = {},
  9.             ["Character2"] = {}
  10.         },
  11.         ["@acountname2"] = {
  12.             ["libASV_AccountWide"] = {},
  13.             ["Character 3"] = {}
  14.         }
  15.     }
  16. }
Useage example:
:AccessOSUserWide("SavedVariableName") will return the table at "libASV_OS_user_wide" and create all tables that are missing along the way.
:AccountWide("SavedVariableName", [accountOverride]) gives you the libASV_AccountWide table of your account by default. Can be overridden via optional argument. This is for data not tied to any character.
:AllCharacters("SavedVariableName", [accountOverride]) Gives you a table containing all the Characters for the specified account inlcuding all the subtables. This is everything under the account name, aside from the Accont wide area. Defaults to current account. Usefull if you just want to cross access data stored on the Character level.
:CharacterWide("SavedVariableName", [accountOverride], [characterOverride]) - gives you the table for the specified account, specified character and creates tables along the way. Defaults to current, but can be overridden.
  Reply With Quote
08/13/14, 09:40 AM   #2
zgrssd
AddOn Author - Click to view addons
Join Date: May 2014
Posts: 281
The silence is deafening.

What do you guys think?
Would this be usefull or pointless (as you go full custom way anyway)?
Anything I forgot to cover?
  Reply With Quote
08/13/14, 09:56 AM   #3
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 679
I'm accessing the saved var directly, with a utility function which auto-vivifies deeply nested subtables. That way I have a single point of access to everything, per-character, account-wide and cross-account data. And I think I'll stick with it. Until I shoot myself in the foot by breaking compatibility (due to having no versioning )
  Reply With Quote
08/13/14, 09:59 AM   #4
Randactyl
 
Randactyl's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 253
I'd much rather use this than ZO_SavedVars if it means I'll be less likely to be affected by shenanigans similar to 1.2.

It all sounds great. I'd really like to see version control, since it would be nice if the replacement method didn't have any drawbacks to the default.
__________________
Randactyl | NA Megaserver | Blood for the Pact!

My Addons
Check out my GitHub repos to follow my latest changes and see some addons before they are released.
  Reply With Quote
08/13/14, 12:41 PM   #5
Sasky
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 241
Versioning shouldn't be too hard to implement and I'd think it'd be almost required to implement if looking to replace the ZO_SavedVars. Here's some ideas for doing versioning and default values:

Store version number
I think this would be best done per data group. Account-wide and OS-wide you could put one level up from where the data is stored easily. Per character you either need to put in with the data (and just have the version field be reserved) or put under the account level and prefix with character name.

New format sample:
Lua Code:
  1. SavedVariableName = {
  2.     ["libASV_Strcuture_1"] = {
  3.         libASV_OS_version = 3,
  4.         ["libASV_OS_user_wide"] = {
  5.              --data goes here
  6.         },
  7.         --Showing first option for characters here
  8.         ["@acountname1"] = {
  9.             libASV_version = 1, --For AccountWide variables
  10.             ["libASV_AccountWide"] = {},
  11.             version_Character1 = 1,
  12.             ["Character1"] = {},
  13.             version_Character2 = 1,
  14.             ["Character2"] = {}
  15.         },
  16.         --Showing second option for characters here
  17.         ["@acountname2"] = {
  18.             libASV_version = 1,
  19.             ["libASV_AccountWide"] = {},
  20.             ["Character 3"] = {
  21.                 libASV_version = 1, --Reserved field
  22.                 --Other data
  23.             }
  24.         }
  25.    }
  26. }

Access functions
You'd need to pass in the version number and an upgrade function. You could do a variable-wide upgrade, but that would expose your underlying structure, which is hardly ideal.
:AccessOSUserWide("SavedVariableName", version, [defaults], [upgradeFunction])
:AccountWide("SavedVariableName", version, [defaults], [upgradeFunction], [accountOverride])
:AllCharacters("SavedVariableName", [accountOverride])
:CharacterWide("SavedVariableName", version, [defaults], [upgradeFunction], [accountOverride], [characterOverride])

The function behavior (except AllCharacters) would be:
Code:
IF not initialized THEN
    COPY in default values
    RETURN table
END

IF version different THEN
   IF upgradeFunction THEN
       CALL upgrade function on table
   ELSE
       COPY in default values
   END
END

RETURN table
You'd need to use the deep copy utility function for setting the default variables.

Upgrade function
Parameters:
- existing version
- existing table

Result/return:
Could either have the function modify the table in-place (more efficient) or just return the new table (susceptible to errors if not familiar with tables are reference).

Again, you don't know what to do for upgrade in the default, so you'd just reset to defaults.
  Reply With Quote
08/15/14, 06:18 AM   #6
zgrssd
AddOn Author - Click to view addons
Join Date: May 2014
Posts: 281
Originally Posted by Sasky View Post
Versioning shouldn't be too hard to implement and I'd think it'd be almost required to implement if looking to replace the ZO_SavedVars.
I have several reasons not to do it:
First, it is alot of work and a lot of added complexity.
Second, I also propably need some sub-division (like the namespaces) so the programmer can reset part of the data without affecting others. Doing this account or even Character wide is not granular enough.
Third and most importantly: It is pretty trivial to make versionoing yourself. Once you started making your own (for more advanced cases like migrating applicable settings from previous versions), those extra checks built into the function will be a resource drain.

I might add it later. But I don't see the need to hardcode this personally.
  Reply With Quote
08/15/14, 06:27 AM   #7
QuadroTony
Banned
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 828
can this new idea solve this problem with too large savedvariables file?

http://forums.elderscrollsonline.com...nical-question
  Reply With Quote
08/15/14, 12:20 PM   #8
QuadroTony
Banned
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 828
mby dynamic loading for each zone& to prevent this lags and errors when we have alot of nodes
  Reply With Quote
08/15/14, 01:03 PM   #9
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 679
Originally Posted by QuadroTony View Post
mby dynamic loading for each zone& to prevent this lags and errors when we have alot of nodes
Not possible. The game loads the the file as a whole. You can't control it.
  Reply With Quote
08/15/14, 01:18 PM   #10
QuadroTony
Banned
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 828
nothing "not possible" for a good programmer =)
  Reply With Quote
08/17/14, 01:18 AM   #11
Sasky
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 241
Originally Posted by QuadroTony View Post
nothing "not possible" for a good programmer =)
http://en.wikipedia.org/wiki/Halting_problem
More seriously, we are limited by the API. The game itself loads the whole file, so we can't change that unless we work at Zenimax and can access the game code.

Originally Posted by zgrssd View Post
I have several reasons not to do it:
First, it is alot of work and a lot of added complexity.
Second, I also propably need some sub-division (like the namespaces) so the programmer can reset part of the data without affecting others. Doing this account or even Character wide is not granular enough.
Third and most importantly: It is pretty trivial to make versionoing yourself. Once you started making your own (for more advanced cases like migrating applicable settings from previous versions), those extra checks built into the function will be a resource drain.

I might add it later. But I don't see the need to hardcode this personally.
1. If this is intended to replace, I feel it should at a minimum mimic functionality.
2. I don't see the need for namespaces, as the library doesn't try for namespaces from the API. I can't come up with a decent use case of why that'd be needed either, so guess you'd need to elaborate.
3a. If it's trivial, that seems to me more argument to put it in library to limit code re-use.
3b. It's called once on the library call which should only be done for an individual bucket once per /reloadui, so it's negligible cost on setup. If someone's calling ZO_SavedVars or this every frame update (when it would have an impact), that's bad usage and should be changed regardless.

It almost seems you're trying to build in something more advanced than the ZO_SavedVars versioning, then declaring that's too much and saying no versioning is needed.I was trying to think of how to lay it out clearer, and ended up pretty much dry-coding the whole thing...
Lua Code:
  1. --Register with LibStub
  2. local asv = LibStub:NewLibrary("LibASV-1.0", 0.1)
  3. if not asv then return end  --the same or newer version of this lib is already loaded into memory
  4.  
  5. -- Local constants
  6. local acctWide = "libASV_AccountWide"
  7. local osWide = "libASV_OS_wide"
  8.  
  9. -- Utility functions
  10. ---
  11. -- @param t - Saved variables table
  12. --      Table has reserved field t._version for current version
  13. -- @param version - expected version for the table
  14. -- @param upgradeFunction - If NIL will just copy in default values. Expected f(version, table) and upgrade in-place
  15. -- @param defaults - NOT nillable. Used to reset in case upgradeFunction is NIL
  16. --
  17. local function versionControl(t, version, upgradeFunction, defaults)
  18.     --t is passed in by reference, so it's changed in-place and no return needed
  19.     if version ~= t._version then
  20.         if upgradeFunction then
  21.             upgradeFunction(t._version, t)
  22.             --_version is reserved, so we want to make sure it's a good value
  23.             t._version = version
  24.         else
  25.             ZO_DeepTableCopy(defaults, t)
  26.         end
  27.     end
  28. end
  29.  
  30. --- Utility function to access sub-table
  31. -- @param table
  32. -- @param key
  33. local function accessNonNull(table, key)
  34.     if table[key] == nil then
  35.         table[key] = {}
  36.     end
  37.  
  38.     return table[key]
  39. end
  40.  
  41. -- API functions
  42.  
  43. ---
  44. -- @param savedVar (required) name used for the SavedVariables field in the manifest file
  45. -- @param version (required) version of the saved variable
  46. -- @param defaults (optional) defaults to use on initialization. Defaults to {}
  47. -- @param upgradeFunction (optional) what to call if different saved variable version.
  48. --           Default behavior is replace with defaults
  49. -- @return (table reference) to the OS-wide saved variables
  50. ---
  51. function asv:AccessOSUserWide(savedVar, version, defaults, upgradeFunction)
  52.     defaults = defaults or {}
  53.  
  54.     --Check if initialized
  55.     if not savedVar[osWide] then
  56.         return ZO_DeepTableCopy(defaults, savedVar[osWide])
  57.     end
  58.  
  59.     versionControl(savedVar[osWide], version, upgradeFunction, defaults)
  60.     return t
  61. end
  62.  
  63. ---
  64. -- @param savedVar (required) name used for the SavedVariables field in the manifest file
  65. -- @param version (required) version of the saved variable
  66. -- @param defaults (optional) defaults to use on initialization. Defaults to {}
  67. -- @param upgradeFunction (optional) what to call if different saved variable version.
  68. --           Default behavior is replace with defaults
  69. -- @param accountOverride (optional) which account to use. Defaults to current
  70. -- @param characterOverride (optional) which character to use. Defaults to current
  71. -- @return (table reference) to the saved variables for that character
  72. ---
  73. function asv:AccessCharacterWide(savedVar, version, defaults, upgradeFunction, accountOverride, characterOverride)
  74.     accountOverride = accountOverride or GetDisplayName() --Or the more robust 'GetDisplayName' from DisplayNameFix
  75.     characterOverride = characterOverride or GetUnitName("player")
  76.  
  77.     if not savedVar[accountOverride] then
  78.         savedVar[accountOverride] = {}
  79.     end
  80.  
  81.     local accountTable = savedVar[accountOverride]
  82.  
  83.     if not accountTable[characterOverride] then
  84.         return ZO_DeepTableCopy(defaults, accountTable[characterOverride])
  85.     end
  86.  
  87.     versionControl(accountTable[characterOverride], version, upgradeFunction, defaults)
  88.     return t
  89. end
  90.  
  91. ---
  92. -- @param savedVar (required) name used for the SavedVariables field in the manifest file
  93. -- @param version (required) version of the saved variable
  94. -- @param defaults (optional) defaults to use on initialization. Defaults to {}
  95. -- @param upgradeFunction (optional) what to call if different saved variable version.
  96. --           Default behavior is replace with defaults
  97. -- @param accountOverride (optional) which account to use. Defaults to current
  98. -- @return (table reference) to the saved variables for that account
  99. ---
  100. function asv:AccessAccountWide(savedVar, version, defaults, upgradeFunction, accountOverride)
  101.     return self:AccessCharacterWide(savedVar, version, defaults, upgradeFunction, accountOverride, acctWide)
  102. end
  103.  
  104. ---
  105. -- @param savedVar (required) name used for the SavedVariables field in the manifest file
  106. -- @param accountOverride (optional) which account to use. Defaults to current account
  107. -- @return array list of all characters with saved variables for the account
  108. ---
  109. function asv:AllCharacters(savedVar, accountOverride)
  110.     accountOverride = accountOverride or GetDisplayName() --Or the more robust 'GetDisplayName' from DisplayNameFix
  111.  
  112.     local accountTable = savedVar[accountOverride]
  113.     if not accountTable then
  114.         return {}
  115.     end
  116.  
  117.     local chars = {}
  118.     for k in pairs(accountTable) do
  119.         if k ~= acctWide then
  120.             table.append(chars, k)
  121.         end
  122.     end
  123.  
  124.     return chars
  125. end

Sample upgrade function, showing it upgrade in-place.
Lua Code:
  1. function upgrade(version, table)
  2.     if version < 2 then --Set new field foo to default
  3.         table.foo = 20
  4.     end
  5.     if version < 3 then -- Set new field bar to default
  6.         table.bar = 30   end
  7.     end
  8. end
  9.  
  10. asv:AccessOSUserWide(mySavedVar, 3, { a=1,b=2, foo=20, bar=30 }, upgrade)
  Reply With Quote

ESOUI » AddOns » Alpha/Beta AddOns » work in progress - libASV (Advanced Saved Variables)

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off