Problem Traversing the Saved Variables Table

[Xrystal]

Okay,

I have set up my Saved Variables table as follows:

local function InitSavedVariables(...)
    SVData = ZO_SavedVars:NewAccountWide(addonSV, 1, "Data", SVDefault)
    SVDataIndex = 1
end

It then gets written to as follows:

SVData = SVData or {}
SVData[zone] = SVData[zone] or {}
SVData[zone][name] = SVData[zone][name] or {}
SVDataIndex = #SVData[zone][name] + 1
SVData[zone][name][SVDataIndex] = SVData[zone][name][SVDataIndex] or {}
SVData[zone][name][SVDataIndex]["Action"] = action
SVData[zone][name][SVDataIndex]["X"] = xPos
SVData[zone][name][SVDataIndex]["Y"] = yPos
SVData[zone][name][SVDataIndex]["Date"] = dateValue
SVData[zone][name][SVDataIndex]["Time"] = timeValue

But when I try and traverse the table as follows:

local function RecurseSavedVariables(...)
  -- Zone
  for i,v in pairs(SVData) do
    ChatMsg:AddMessage(string.format("(i,v) - %s : %s (%s)",tostring(i),tostring(v),type(v)))
    if ( type(v) == "table" ) then
      -- Name
      for i2,v2 in pairs(v) do
        ChatMsg:AddMessage(string.format("(i2,v2) - %s : %s (%s)",tostring(i2),tostring(v2),type(v2)))
        if (type(v2) == "table") then
          -- Values
          for i3,v3 in pairs(v2) do
            ChatMsg:AddMessage(string.format("(i3,v3) - %s : %s (%s)",tostring(i3),tostring(v3),type(v3)))
          end
        end
      end
    end
  end
end

It seems to think it has the following data in there:

GetInterfaceForCharacter : function : xxxxxxxx : (function)
default : table : xxxxxxxx : (table)

and despite having a table in there it doesn't traverse it further as expected, to see what else is hiding there.

Is there a special way to traverse the Saved Variables table in ESO ?

[Seerah]

I have written down that the first parameter passed to your function should be a string. So... "addonSV"

[Errc]

Originally Posted by Seerah I have written down that the first parameter passed to your function should be a string. So... "addonSV"

addonSV is likely just a variable for whatever they named SV...

When your saved variables gets created it isn't blank. It stores the default values in a default table and GetInterfaceForCharacter is just a function there is in there. The default table is the same as SVDefault value, not sure what GetInterfaceForCharacter does, but it is in there all the same.

I suggest using Zgoo to examine your saved variables. You can see exactly how it is laid out and see how it reacts when you change values in it.

[Xrystal]

That's the name of my addon saved variable table.

I've been putting this at the top of my addons, to make things a bit easier:
local addonName = "XrysGatherer"
local addonVersion = "0.0.1"
local addonSV = addonName.."_SavedVariables"


In the gather addon it works out as "XrysGatherer_SavedVariables". That's why I don't get, writing to it works fine .. but reading from it seems to be a bit messed up unless you are looking for a specific element like I do to identify an existing element.

[Xrystal]

Originally Posted by Errc addonSV is likely just a variable for whatever they named SV... When your saved variables gets created it isn't blank. It stores the default values in a default table and GetInterfaceForCharacter is just a function there is in there. The default table is the same as SVDefault value, not sure what GetInterfaceForCharacter does, but it is in there all the same. I suggest using Zgoo to examine your saved variables. You can see exactly how it is laid out and see how it reacts when you change values in it.

But the question is why isn't it traversing it properly. Other tables work fine, just the saved variables one is messing up to the point of putting a stumbling block on me writing any addons that would need to access their respective tables.

I guess it's confusing me why SVData is linked to the Data part of the table for writing but not for reading.

The data is stored in the file as follows:

["Default"]
["@ACCOUNT"]
["$AccountWide"]
["Data"]
... data added by addon such as ..
["Khenarthi's Roost"]
["Aspect Rune"]
[1]
xPos = ...
yPos = ...
zone = ...
etc

when I use

for index,value in pairs(SVData) do
CHAT_SYSTEM:AddMessage(index,value,type(value))
end

I am expecting the first element to be "Khenarthi's Roost", xxxxxx, table telling me to traverse down a level to receive the second as "Aspect Rune",xxxxx,table, with the third as "1",xxxx,table, and the fourth displaying the first set of actual values.

However, if I say want to look for a specific item such as :

local zoneData = SVData["Khanarthi's Roost"]
local itemData = zoneData["Aspect Rune"]

CHAT_SYSTEM:AddMessage(itemData[1].xPos,itemData[1].yPos) would display fine ( formatting aside that is )

[Xrystal]

Looking at Esoheads Saved Variable table it looks like they do things differently so will try and do something similar with my addons.


Edit: Well after a lot of trial and error I finally almost had to copy what Esohead did and then adjust it to reflect my table contents.

Ended up changing the layout of the Saved Variables File and spec

local function InitSavedVariables(...)
    SVData = {
        ["History"] = ZO_SavedVars:NewAccountWide("XrysGatherer_SavedVariables", 1, "History", SVDefault)
    }
    SVDataIndex = 1
end

And using the way Esohead traversed through the table as a guide the following works flawlessly.

ChatMsg:AddMessage("Traversing Harvest History")
    for history,sv in pairs(SVData) do
        ChatMsg:AddMessage(history)
        if history == "History" then
            for zone, t1 in pairs(SVData[history]) do
                for item, t2 in pairs(SVData[history][zone]) do
                    for index, t3 in pairs(SVData[history][zone][item]) do
                        local data = SVData[history][zone][item][index]
                        ChatMsg:AddMessage(string.format("%d: %s %s in %s at %0.3f,%0.3f",index,item,tostring(data["Action"]),zone,data["X"],data["Y"]))
                    end
                end
            end
        end
    end

[Xrystal]

Okay, scratch the last reply.

It only works that way if I log in with an EMPTY!!!! Saved Variables table. Which totally defeats the point.


If I log in with an empty table that games session will correctly traverse the table ( maybe without the need for adding the history header table but I am not spending another half hour changing it back now ) but the moment I tell it not to clear the table out the old traverse problem returns.

By the way, what the heck is that version value for ? Should I set that to another value ?

Can some bright spark see what the heck I am doing wrong here or is it a bug in the SavedVariables table?

edit: As you can see this is annoying me to no end as I have wasted my beta weekend writing this addon and haven't been able to finish it because of this problem.

Here is my code in its entirety barring the Update routine being called from the XML as per the norm.

-- Values used often so created variables for them
local addonName = "XrysGatherer"
local addonVersion = "0.0.1"
local addonSV = addonName.."_SavedVariables"
 
-- References to System and Manager objects
local ChatMsg = CHAT_SYSTEM
local EventMgr = EVENT_MANAGER
 
-- References to the Saved Variables Data
local SVDefault = {}
local SVData = {}
local SVDataIndex = 0
 
-- Whether the player is currently interacting with something
local interacting = false
 
-- Whether the player wants to view his history
local showHistoryAtLogin = true
 
-- Initialise the Saved Variables File
-- Found in EsoHead addon ( thanks )
local function InitSavedVariables(...)
    SVData = {
        ["History"] = ZO_SavedVars:NewAccountWide("XrysGatherer_SavedVariables", 1, "History", SVDefault)
    }
    SVDataIndex = 1
end
 
-- Check if the item already exists in the database in that zone and location ( or very close to it )
-- Variation of the test found in Esohead addon ( thanks )
local function SVDataExists(xPos,yPos,zone,name)
    local found = false
    local zoneData = SVData["History"][zone]
    if not zoneData then return false end
    local nameData = zoneData[name]
    if not nameData then return false end
    for i,v in pairs(nameData) do
        local x,y = v["X"],v["Y"]
        if math.abs(xPos - x) < 0.005 and math.abs(yPos - y) < 0.005 then 
            found = true 
            break 
        end     
    end
    if found then return true end
    return false
end
 
-- Update Routine to track harvest interactions which do not have an event at present
function XrysGatherer_Update(...)
 
    -- Is the player interacting with something ?
    local playerInteracting = IsPlayerInteractingWithObject()
    if not playerInteracting then interacting = false return end
 
    -- Check if the Interaction Type is Harvest
    local interactMode = GetInteractionType()
    if interactMode  ~= INTERACTION_HARVEST then return end
 
    -- Check if we are still processing the last interaction update cycle
    if interacting then return end
    interacting = true
 
    -- Get some important information into variables for use later
    local unitID = "player"
    local zone = GetUnitZone(unitID)
    local xPos,yPos,zPos = GetMapPlayerPosition(unitID)
    local action, name, blocked, extra, context = GetGameCameraInteractableActionInfo()
    local dateValue = GetDate()
    local timeValue = GetTimeString()
 
    -- Just in case it thinks crafting is a harvestable object
    if action == "Use" then 
        ChatMsg:AddMessage(string.format("Action: %s Name: %s Mode: %d Extra: %d",action,name,interactMode,extra))
        return 
    end
 
    -- Does the node exist in the database ?
    local nodeExists = SVDataExists(xPos,yPos,zone,name)
    local NewOrOld = "New"
    if nodeExists then NewOrOld = "Old" end
 
    -- Display information on the screen regardless if it is an old or new node
    ChatMsg:AddMessage(string.format("[%s] %s: %s at ( x:%0.3f, y:%0.3f ) in %s",NewOrOld,action,name,xPos,yPos,zone))
 
    -- Only update the saved variable file if the node doesn't exist in there yet
    if not nodeExists then 
        ChatMsg:AddMessage("Adding Harvest Node to Database")
        SVData = SVData or {}
        SVData["History"]                       = SVData["History"] or {}
        SVData["History"]                       = SVData["History"] or {}
        SVData["History"][zone]                 = SVData["History"][zone] or {}
        SVData["History"][zone][name]           = SVData["History"][zone][name] or {}
 
        SVDataIndex = #SVData["History"][zone][name] + 1
 
        SVData["History"][zone][name][SVDataIndex]              = SVData["History"][zone][name][SVDataIndex] or {}
        SVData["History"][zone][name][SVDataIndex]["Action"]    = action
        SVData["History"][zone][name][SVDataIndex]["X"]         = xPos
        SVData["History"][zone][name][SVDataIndex]["Y"]         = yPos
        SVData["History"][zone][name][SVDataIndex]["Date"]      = dateValue
        SVData["History"][zone][name][SVDataIndex]["Time"]      = timeValue
    end
 
    ChatMsg:AddMessage("Traversing Harvest History")
    for history,sv in pairs(SVData) do          
        if history == "History" then
            for zone, t1 in pairs(SVData[history]) do
                for item, t2 in pairs(SVData[history][zone]) do
                    for index, t3 in pairs(SVData[history][zone][item]) do
                        local data = SVData[history][zone][item][index]
                        ChatMsg:AddMessage(string.format("%d: %s %s in %s at %0.3f,%0.3f",index,item,tostring(data["Action"]),zone,data["X"],data["Y"]))
                    end
                end
            end
        end
    end
 
end
 
-- Our Addon is loaded so we can start work
local function AddOnLoaded(eventID,addon)
    if addon ~= addonName then return end
    EventMgr:UnregisterForEvent(addonName,eventID)
    ChatMsg:AddMessage(string.format("%s %s Loaded",addonName,addonVersion))
    InitSavedVariables()
 
        -- Have to have this here at this point to get the table traversing properly that session.   
        -- If I remove this line the function error appears again and it won't traverse.
    SVData["History"] = {}
end
 
-- Track addons loading
EventMgr:RegisterForEvent( addonName ,EVENT_ADD_ON_LOADED , AddOnLoaded )

Here is a screenshot of what is happening when I try and traverse through the table.

If I change the traversing to the following in the hopes of bypassing the other stuff I get the same problem:

for zone, t1 in pairs(SVData[history]) do
                for item, t2 in pairs(SVData[history][zone]) do
                    for index, t3 in pairs(SVData[history][zone][item]) do
                        local data = SVData[history][zone][item][index]
                        ChatMsg:AddMessage(string.format("%d: %s %s in %s at %0.3f,%0.3f",index,item,tostring(data["Action"]),zone,data["X"],data["Y"]))
                    end
                end
            end

Again, only if the data exists when logging in.

[Xrystal]

Hmm, very weird. ..

Thought it was zgoo bugging out as I am starting the addon with what I thought was an empty table but lo and behold I must be clearing it before it is loaded.

However, because I thought it was clearing it the traversing of the table working is only listing the items gathered that session. Not the whole contents of the SavedVariables table.

How do I access the history part of the table and traverse it ? SVData["History"] doesn't seem to work except for the fresh data that session. Guess it's time to tear apart zgoo but at 11pm UK time I probably won't get time to do anything useful if I get to grips with how they do it.

Everything I have done so far that has taken me what seems to be all day but probably only half of it has been to no avail apart from stopping the error message from appearing.

[Xrystal]

Well 2 hrs or so later, finally got it figured... whew. Now I am happy and not the sulky bitch I was before rofl.

This is how I managed to access the saved variables in it's entirety. The local SVData file is only the contents of that session as it only works if you clear the tables when the addon is loaded, but the contents of XrysGatherer_SavedVariables is the whole data.

This means I may be able to go back to the old version and use this code anyway and not have to fix the data everyone has been gathered so far.

Unfortunately at 1:30am I doubt I'll be doing anything else this beta weekend except to test that the original version can traverse properly this way too and when I want to rather than only after it has been updated which is what it is currently doing and upload a working version of that.

ChatMsg:AddMessage("Traversing Saved Variables")
    harvestHistory = {}
    for default,sv in safepairs(XrysGatherer_SavedVariables) do 
        for account,accountv in safepairs(sv) do
            for accountWide,acWideV in safepairs(accountv) do
                for history,historyV in safepairs(acWideV) do
                    if history == "History" then
                        table.insert(harvestHistory,historyV )
                    end
                end
            end
        end
    end
 
    ChatMsg:AddMessage("Traversing Harvest History")
    for historyIndex,historyData in safepairs(harvestHistory) do
        for zoneIndex,zoneData in safepairs(historyData) do
            if zoneIndex ~= "version" then
                for itemIndex,itemData in safepairs(zoneData) do
                    for indexIndex,indexData in safepairs(itemData) do
                        ChatMsg:AddMessage(string.format("%s %s in %s at %0.3f,%0.3f",itemIndex,tostring(indexData["Action"]),zoneIndex,indexData["X"],indexData["Y"]))
                    end
                end
            end
        end
    end

[Xrystal]

Yes I know I'm resurrecting an old thread, but its my own thread and I have resolved the problem I had before ... will mean a data reshuffle in my gatherer addon but I will tie that in with the database site going live.

I finally realised that the secret is to have a multiple level of data in the SVData table.

local SVDefault = {}
local SVDefaultOptions = {
    Filters = {
        [CRAFTING_TYPE_ALCHEMY] = true,
        [CRAFTING_TYPE_BLACKSMITHING] = true,
        [CRAFTING_TYPE_CLOTHIER] = true,
        [CRAFTING_TYPE_ENCHANTING] = true,
        [CRAFTING_TYPE_PROVISIONING] = true,
        [CRAFTING_TYPE_WOODWORKING] = true,
    },
    Verbose = {
        ["New Node"] = true,
        ["Old Node"] = true,
    },
}
SVData = {
  ["NodeData"] = ZO_SavedVars:NewAccountWide("XRGASavedVariables", 2, "NodeData", SVDefault),
  ["ItemData"] = ZO_SavedVars:NewAccountWide("XRGASavedVariables", 2, "ItemData", SVDefault),
  ["Options"] = ZO_SavedVars:New("XRGASavedVariables",2,"Options",SVDefaultOptions),
}

The above will generate NodeData, ItemData and Options as tables but traversing them doesn't work due to the functions and default tables that are sitting in them as well. The secret, and I am sure most of you figured this out ages ago is to add an extra layer to your data. It seems without knowing it I did it for the options table but where I never needed to traverse it I never noticed the solution till now.

local SVDefaultData = { NodeData = {}, ItemData = {} }
local SVDefaultOptions = {
    Filters = {
        [CRAFTING_TYPE_ALCHEMY] = true,
        [CRAFTING_TYPE_BLACKSMITHING] = true,
        [CRAFTING_TYPE_CLOTHIER] = true,
        [CRAFTING_TYPE_ENCHANTING] = true,
        [CRAFTING_TYPE_PROVISIONING] = true,
        [CRAFTING_TYPE_WOODWORKING] = true,
    },
    Verbose = {
        ["New Node"] = true,
        ["Old Node"] = true,
    },
}
SVData = {
  ["Data"] = ZO_SavedVars:NewAccountWide("XRGASavedVariables", 2, "Data", SVDefaultData),
  ["Options"] = ZO_SavedVars:New("XRGASavedVariables",2,"Options",SVDefaultOptions),
}

So now, instead of trying to traverse the old NodeData table, I simply use SVData["Data"]["NodeData"] and traverse that. Works great in my test runs so far.

[Aicam]

the real problem traversing the SavedVars is that the actual data is stored in a metatable.
i use the following (rather long) snipped to auto generate default get/set functions for my SavedVars.

local function Factory( input, output, pre )
  if pre ~= "" then
    pre = pre .. "."
  end
 
  for k, v in pairs( input ) do
    if k ~= "version" and type( v ) ~= "function" then
      output[ k ] = {}
      if type( v ) == "table" then
        Factory( v, output[ k ] , pre..k )
      else
        output[ k ][ "get" ] = function()
          return input[ k ]
        end
 
        output[ k ][ "set" ] = function( new )
          local old = input[ k ]
          input[ k ] = new
          CM:FireCallbacks( "VariableChanged", Addon.id, pre..k, old, new )
        end
 
      end
    end
  end
end
 
local SavedVars, Accessors = ZO_SavedVars:New( Addon.id, Addon.SavedVarsVersion, nil, Addon.Defaults, nil ), {}
 
if t and type( SavedVars ) == "table" then
  local meta = getmetatable( SavedVars )
  if meta and meta.__index and type( meta.__index ) == "table" then
    Factory( meta.__index, Accessors, "" )
  end
end

[Xrystal]

Currently I do the following, and then use the resulting tables for the traversing, its not perfect but for the most part it is called when people first open the history or import file and when they click the refresh button. Finding out that a slight change in data layout will bypass this problem will hopefully cut down on the amount of code I am needing to use. Technically accountWide also includes the characterWide values as well but it makes it clearer that I am only looking at accountWide values.

local nodeInfo = {}
local itemInfo = {}
for default,sv in pairs(XrysGatherer_SavedVariables) do 
    for account,accountv in pairs(sv) do
        for accountWide,acWideV in pairs(accountv) do
            for i,v in pairs(acWideV) do
                if i == "ItemData" then itemInfo = v end
        if i == "NodeData" then nodeInfo = v end
            end
        end
    end
end

[zgrssd]

As I wrote in my update of the Wiki entry on saved variables, # does not work on the SavedVariable access object - it always will return 0, regardless how many int indexed values you have in there.
It does work fully on table you put one level "deeper". The easiest trick is to just add string index and store your table under that string index.

[lyravega]

# should work on saved variables if you construct it like an array. If this isn't the case, it won't work. # only works for proper arrays.

[zgrssd]

Originally Posted by lyravega # should work on saved variables if you construct it like an array. If this isn't the case, it won't work. # only works for proper arrays.

# does not work on saved variables with int indizes. It returns 0, always.
Sub-elements of the saved table work normaly regarding #

Just put it into SavedVar.Data[i] isntead of SavedVar[i] and you can properly itterate over SavedVar.Data using #SavedVar.Data.
It is also better practice to hand normal arrays (instead of Saved Var Acessor objects) to functions.

[lyravega]

Originally Posted by zgrssd # does not work on saved variables with int indizes. It returns 0, always. Sub-elements of the saved table work normaly regarding # Just put it into SavedVar.Data[i] isntead of SavedVar[i] and you can properly itterate over SavedVar.Data using #SavedVar.Data. It is also better practice to hand normal arrays (instead of Saved Var Acessor objects) to functions.

You can insert data at the root of the table... and if you use it like an array, # returns correct stuff.