Thread Tools Display Modes
12/16/20, 01:28 PM   #1
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Append new data to saved vars

I am pretty new to Lua after avoiding it in my job back when we had things using it but I've picked up an old addon and have been trying to get it to do things that I want it to do, the original creator left it a long time back.

It pulls guild bank deposits/withdrawals and dumps it in a Lua, it does the whole pull all in one go when you kick off the scan, which isn't ideal in the first place but I don't mind for now.

The problem I have is it can only use whats in the guild history even if it has older data it seems to just overwrite it and then its lost, so really I'm just looking to see if someone can help show me how I can get it to pull in the current data, append the new stuff to it and remove dupes.

Any help would be appreciated, I've attached the lua just to make it easier for someone to check as I have tried to work out what functions are doing what but my lack of knowledge doesnt help there.
Attached Files
File Type: lua NewGuildBankLedger.lua (21.6 KB, 409 views)
  Reply With Quote
12/16/20, 02:29 PM   #2
Baertram
Super Moderator
 
Baertram's Avatar
WoWInterface Super Mod
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 4,912
A hint:
If you need to read guild history you should change the addon to use the library "LibHistoire" as it was newly designed to get this data performantly.
Manually reading the histry is slower and might be buggy, as well as using older addon code.

In that file at first please change ALL occurrences of LibStub to no LibStub calls as this library is obsolete.
e.g.
local LAM2 = LibStub("LibAddonMenu-2.0")
->
local LAM2 = LibAddonMenu2

Each library has written in it's description what the global variable to use is named, e.g. for LibAddonMenu it's LibAddonMenu2 since version 28 I think.
In the txt file (named "manifest") of the addon please remove all hardcoded lines to something like
/libs/LibAddonMenu-2.0/LibAddonMenu-2.0.lua
/libs/LibAddonMenu-2.0/controls/...

And add this:
## DependsOn: LibAddonMenu2.0>=31

This assures you do not load any included very old libraries and break addons, but it will search for a properly instaleld library "LibAddonMenu-2.0" in version 31 or higher, providing it's own LibAddonMenu-2.0.txt where the ## AddOnVersion: 31 is specified. So no dulicate versions are loaded anymore, what LibStub managed to suppress in the past, but ZOs code does better meanwhile (since Summerset).


The SavedVariables are created here:
Lua Code:
  1. self.savedVariables = ZO_SavedVars:NewAccountWide(
  2.                               "GuildBankLedgerVars"
  3.                             , self.savedVarVersion
  4.                             , nil
  5.                             , self.default
  6.                             )
If you want to add another parameter after the , self.default to make it save the file contents differently for each server you play on:
Lua Code:
  1. self.savedVariables = ZO_SavedVars:NewAccountWide(
  2.                               "GuildBankLedgerVars"
  3.                             , self.savedVarVersion
  4.                             , nil
  5.                             , self.default
  6.                             , GetWorldName()
  7.                             )

GetWorldName() rturns "EU Megaserver" or "NA Megaserver" or "PTS".
The SV file will save a table named "GuildBankLedgerVars" and below you will find all data stored behind the subtable ["xx Megaserver"] -> ["@AccountName"] -> ["$AccountWide"] (because ZO_SavedVars:NewAccountWide was used).


Ziggr then used some OLD API functions to check if there are some guild history events firing and to catch them, e.g.
in

Lua Code:
  1. -- Async poll to fetch ALL guild bank history data from the ESO server
  2. -- Calls ServerDataComplete() once all data is loaded.
  3. function GuildBankLedger:ServerDataPoll(guild_index)

It was checking via DoesGuildHistoryCategoryHaveMoreEvents for more events to come in (as the server decces WHEN and IF the events come in! That's why youd efinately should switch to LibHistoire and NOT use this anymore).

As his comment says alread: In the end it calls function
GuildBankLedger:ServerDataComplete

And in there it uses the function RecordEvent to add some data to internal addon's table
self.fetched_str_list

But where is the SavedVars connection?
In function GuildBankLedger:ServerDataComplete it was assigned:

Lua Code:
  1. self.savedVariables.history[guild_name] = self.fetched_str_list[guild_index]

In the SV file you will find a subtable history with the subtables per guild_name -> having all the recorded event data from self.fetched_str_list of the [guild_index]

Hope this helps as overview. But please consider changing this to use "LibHistoire" if you want to work on it!
Or you might run into either server performance trouble and or other old prolems which were solved if you sue this lib.
It was build to make several addons work togeher AND NOT query the same guild history data again and again... This was a big performance prolem, and still is, on the server. So pelase support the better approach.

Last edited by Baertram : 12/16/20 at 02:46 PM.
  Reply With Quote
12/16/20, 05:36 PM   #3
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Thanks for your pointers, I've got it working using libhistoire now which makes it a lot faster to run the save and I understand a bit more about how the whole script works as well as lua, so that was a huge help!

I'll see what happens with all the data now, I'm hoping that this will help it grab more than just the 10 days that are in history which is one of the end goals for this, although I would then need to find a way to limit the records as it could get Very BIG very quick.
  Reply With Quote
12/16/20, 06:04 PM   #4
Baertram
Super Moderator
 
Baertram's Avatar
WoWInterface Super Mod
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 4,912
Yw.
Your uploaded patch file got some problems.
Check the comments section of your patch please.
  Reply With Quote
12/17/20, 11:57 AM   #5
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Okay so all that is updated and I did a whole other bunch of work on it.

Is there some way to get it to pull historical events or add some sort of time frame (maybe using libhistoire?) just so that I can have more than 10 days worth of events exported?
  Reply With Quote
12/17/20, 12:43 PM   #6
Baertram
Super Moderator
 
Baertram's Avatar
WoWInterface Super Mod
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 4,912
Not sure why there should be a cap at 10 days? Where is that information written?

I think you are able to fecth all guild history data from the past as far as the server is able to retrieve them.
You need to manually update them at the guild history page, using the E keybind several times, like MasterMerchant 3 describes in it's sticky post -> links with howto update data.

But also check the library's description:

Originally Posted by Histy
Unlike other addons, the library won't request the full range of history provided by the server when you first use it. Instead it will immediately force a link and start collecting data from there. If you want to collect older history in that situation, you will need to manually request it and then hit the rescan button for that guild and category.

Last edited by Baertram : 12/17/20 at 12:46 PM.
  Reply With Quote
12/17/20, 01:09 PM   #7
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Yeah I think the issue might be that the addon is trying to pull data from the server still and not from libhistoire. As I have a whole bunch of stored data in libhistoire due to it having been running for a while now, so the link on there shows okay but this addon is pulling from the wrong place.

I'm trying to work out how to stop it doing that now, and I think when thats fixed it will solve my issue!
  Reply With Quote
12/17/20, 02:49 PM   #8
Baertram
Super Moderator
 
Baertram's Avatar
WoWInterface Super Mod
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 4,912
Well, the addon's old source was pulling from the server, correct. Via DoesGuildHistoryCategoryHaveMoreEvents etc.
I did not check yur new code bu if you did no remove this and did not change the SV data so that LibHistoire's SV is used instead, nothing changed
This needs to be stripped and totally re-coded to ONLY use LibHistoire as the source.

Never worked with that lib but the description got the "For Developers" and "API" section.
There also are examples how to iterate over the stored events of the lib, so you could fully stip the event collection AND storage in the addon's own SV data, and just switch to use LibHistoire instead.
  Reply With Quote
12/17/20, 03:08 PM   #9
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Yeah I realised after it was running that all I had managed to do was remove the pause and the force checking for new events, which wasnt too helpful as the actual data it collects still looks at the guild history.

I'm looking into it now but it does look like it needs starting from scratch so I guess I'll wait until I have some time off to look into that as I'll need to look at how its currently collecting data and then doing whatever else with it to then write it to use the lib.

Well guess thats my next project, thanks for your help.
  Reply With Quote
12/18/20, 07:10 PM   #10
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Do you know why I get a nil value for the guildID in this part of my script? (Line 2)

Lua Code:
  1. function GuildBankLedger:SetupListener(guildID)
  2.   self.LibHistoireListener[guildID] = LGH:CreateGuildHistoryListener(guildID, GUILD_HISTORY_BANK)

It should pull it from the command that calls it right? which is this one.

Lua Code:
  1. for i = 1, GetNumGuilds() do
  2.         local guildID = GetGuildId(i)
  3.         GuildBankLedger:SetupListener(guildID)
  4.     end

I feel like im just not understanding lua very well.
  Reply With Quote
12/18/20, 09:41 PM   #11
Sharlikran
 
Sharlikran's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 626
You would set up the listener like that. You will have to make sure the variables are correct for the SavedVariables and the defaults.
Attached Files
File Type: zip GuildBankLedger_3_2.zip (9.2 KB, 420 views)

Last edited by Sharlikran : 12/18/20 at 09:45 PM.
  Reply With Quote
12/19/20, 12:54 PM   #12
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Thanks for the reply and the edits on the file.

When testing I think my issue has been that it wasnt waiting for the callback from LibHistoire. So I've added this in now using
Lua Code:
  1. LGH:RegisterCallback(LibHistoire.callback.INITIALIZED, function()
  2.     ....code here
  3.     end)

But it doesnt seem like it ever actually does anything further, I have a few bits in there that should write debug text during the saving of the events and such but none of that happens.
Does it just take a while to do all of that process the first time? I have quite a few stored events but I can't think it would be that bad?
  Reply With Quote
12/19/20, 08:06 PM   #13
Sharlikran
 
Sharlikran's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 626
Nope not needed and I won't know what that will do or if it will break anything.

MM uses what I edited. Once the event comes in, you will have to determine what kind of bank event it is, build the proper event for it, and then add that event to the individual table of different bank events that the original mod used.

I could even see in the old code there were different tables for different kinds of bank events. You should probably use the old table style as the event and not my example.

You might even want to start over. Update what is needed for libaddonmenu or remove libstub. Just the minimum. Then add the two routines for the listener. Then look at the tables and the old code to see which table was used for the events. Make the listener use the old table style. Then after you now have each different type of event set up depending on the bank event just send the event to the mod like the old event was doing.

Last edited by Sharlikran : 12/19/20 at 09:41 PM.
  Reply With Quote
12/20/20, 11:38 AM   #14
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Oh okay, so if I take that out I end up back at where I was, where for some reason having [guildID] in the listener command causes it to error on seeing a nil value, if I remove that it moves onto the next command that also uses [guildID]

Lua Code:
  1. GuildBankLedger.LibHistoireListener[guildID] = LGH:CreateGuildHistoryListener(guildID, GUILD_HISTORY_BANK)

I added a debug message to tell me what that variable was at the time and it does come back with an actual ID so not sure where its going wrong.
I had been using the code from MM to try and get this to work originally, as its the only addon I knew that did it, seeing your edits tells me im on the right path at least but not sure whats going on with that variable.

I've attached the whole file that im working with at the moment.
Attached Files
File Type: lua GuildBankLedger3.3.lua (15.8 KB, 417 views)
  Reply With Quote
12/20/20, 12:04 PM   #15
Sharlikran
 
Sharlikran's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 626
Lua Code:
  1. function MasterMerchant:SetupListener(guildID)
  2.   -- listener
  3.   MasterMerchant.LibHistoireListener[guildID] = LGH:CreateGuildHistoryListener(guildID, GUILD_HISTORY_STORE)

If you remove the [guildID] from there you will have issues. I don't understand what you mean by with it there you have errors. You could have issues other places due to other things. Although I didn't test that code I don't believe.
  Reply With Quote
12/20/20, 12:17 PM   #16
Baertram
Super Moderator
 
Baertram's Avatar
WoWInterface Super Mod
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 4,912
You did not initialize GuildBankLedger.LibHistoireListener as a table isn't it?

Lua Code:
  1. function GuildBankLedger:Initialize()
  2. self.LibHistoireListener = {}
  3. ...
  4. end

That's why it says nil error on that line. You want to add [guilID] to the table GuildBankLedger.LibHistoireListener but it is not a table.

And next time please post the whole lua errors (instead of only saying there is a nil error) so we are able to easily see where the nil happens. It tells you in that messages if you read it correctly :-) It also shows you the contents of variables and tables in that error message if you expand it.
And if you use sirinsdiators browser plugin for LibDebugLogger it even generates you an even better overview here adirectly inside the browser at www.esoui.com -> e.g. via the Greasemonkey or Tampermonkey browser plugins/addons.

https://www.esoui.com/forums/showthread.php?t=9113

Last edited by Baertram : 12/20/20 at 12:20 PM.
  Reply With Quote
12/20/20, 01:12 PM   #17
Sharlikran
 
Sharlikran's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 626
The way it's set up it already is initialized in the beginning of the file. Unless he didn't copy part of it.
  Reply With Quote
12/20/20, 03:51 PM   #18
Baertram
Super Moderator
 
Baertram's Avatar
WoWInterface Super Mod
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 4,912
I said: The table is missing, not the init of whatever.

This is where "GuildBankLedger.LibHistoireListener" is mentioned the 1st time:
Lua Code:
  1. function GuildBankLedger:SetupListener(guildID)
  2. ...
  3.   GuildBankLedger.LibHistoireListener[guildID] = LGH:CreateGuildHistoryListener(guildID, GUILD_HISTORY_BANK)
It is using the table "GuildBankLedger.LibHistoireListener" with the index "[guildID]".
But where is the table "GuildBankLedger.LibHistoireListener" defined?

At least not in the file which was attached (searched for GuildBankLedger.LibHistoireListener and self.LibHistoireListener and LibHistoireListener, and there is not any line with = {} or = function which would create that table).

So either add the table init line at the beginning of the file or inside the function itsself like this:
Lua Code:
  1. function GuildBankLedger:SetupListener(guildID)
  2. --LGH:RegisterCallback(LibHistoire.callback.INITIALIZED, function()
  3.   -- listener
  4.   d("Starting SetupListenerFunction " .. guildID)
  5.  
  6.   --This was missing: Create the table GuildBankLedger.LibHistoireListener ( = {}) or re-use it if it already exists
  7.   GuildBankLedger.LibHistoireListener = GuildBankLedger.LibHistoireListener or {}
  8.   GuildBankLedger.LibHistoireListener[guildID] = LGH:CreateGuildHistoryListener(guildID, GUILD_HISTORY_BANK)

Last edited by Baertram : 12/20/20 at 03:54 PM.
  Reply With Quote
12/20/20, 04:01 PM   #19
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
Ah that will probably be it then as I am missing that piece.

Yeah I had planned to I just wasn't in game at the time, I'm a lot more used to writing things in PowerShell than lua sadly.

Thank you for all your help I am most of the way there with this now, just sorting out getting events to write the way I need them to.
  Reply With Quote
12/20/20, 04:46 PM   #20
myristican
AddOn Author - Click to view addons
Join Date: Dec 2020
Posts: 10
One other question, as ive just realised it has a problem.

How do I make sure that when it writes events it adds them to what is already there?
Some new events came in when I was logged in, but the older ones had already been written the time before and so it wrote the new events and then cleared all the old ones.

EDIT: Think I figured it out... existing addon grabbed this data from the last "saved run" as this is no longer the case it just blanks the value instead of pulling it from the existing saved variables.

Last edited by myristican : 12/20/20 at 05:16 PM.
  Reply With Quote

ESOUI » Developer Discussions » Lua/XML Help » Append new data to saved vars

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off