New Guild History API

[nightstrike2]

Originally Posted by sirinsidiator You should not use the low-level api directly

Is the low level API the list of functions on the first page?

Also, to Imp's point, the API that you linked similarly doesn't have an overview, it's just a list of functions (and their implementation, obviously)

[Sharlikran]

Known issues with Update 41

* Mod authors cannot check duplicate sales between old and new server data
* Item links from the server do not contain extended information
* Guild history categories may become unusable

To sum up the known issues, mod authors can not cross reference any previous sales data with the current sales data from the server and look for duplicate sales. Item links don't contain any extended information such as writ vouchers, potion effects, and possibly effects for glyphs. In addition to that you may have all green bars but switch characters and any Trader category could become stuck and you won't be able to use the forward arrow key or E Show More to request more data.

[Baertram]

Originally Posted by nightstrike2 Is the low level API the list of functions on the first page? Also, to Imp's point, the API that you linked similarly doesn't have an overview, it's just a list of functions (and their implementation, obviously)

Both correct, as far as I can say.

We do not have any detail docu, thus reading the ZOs code is how we do it (similar in addons then) as it shows how it should work.

[ImpOfThePerverse]

Originally Posted by nightstrike2 Is the low level API the list of functions on the first page? Also, to Imp's point, the API that you linked similarly doesn't have an overview, it's just a list of functions (and their implementation, obviously)

UESP has a diff of the current API version versus the last version that lists functions that were added and removed.

https://esodata.uesp.net/current/apidiff.txt

There are quite a few more changes to the guild history API than were listed in ZOS's API patch notes. The first thing I did when trying to debug the addon I'm trying to update was to get VS Code set up for Lua as per this thread:

https://www.esoui.com/forums/showthread.php?t=9875

The function signatures found here are out of date, so I went through and deleted the functions that look guild history related that were listed as removed in the UESP diff. That at least points out which parts of the addon I'm updating are broken.

The full, updated API is linked below, but there's no explanation of the functions or any kind of overview of the guild history API:

https://www.esoui.com/forums/attachm...3&d=1709571451

[Baertram]

Originally Posted by ImpOfThePerverse The function signatures found here are out of date

Thanks for the reminder, updated it now.

[ImpOfThePerverse]

Originally Posted by Baertram Thanks for the reminder, updated it now.

Awesome! That will help considerably.


Where I'm getting stuck is in determining when a guild history request has been completed. I have a test addon that generates a request for 1 hour's worth of data when I enter a chat command. I can then check the status of the request or view the results via additional chat commands.

When I check status, DoesGuildHistoryHaveOutstandingRequest will return false, which makes it sound like all guild history requests are complete, but doesn't seem like a good indicator because presumably it will return true if another addon has outstanding requests.

IsGuildHistoryRequestComplete returns false, and GetGuildHistoryRequestFlags returns GUILD_HISTORY_REQUEST_FLAG_CREATED_FROM_ADDON as the only positive flag, which makes it sound like the request is not complete.

If I check the results it lists events as having been returned, which, if results are cached in chunks of 500, should mean the request is complete, unless there are some multiple of 500 results, in which case there could be additional chunks incoming. I'm getting just 1 result currently though, so that should mean the request is complete.

I see there's an event, EVENT_GUILD_HISTORY_CATEGORY_UPDATED, that I could register for. Does it get sent when a new chunk gets delivered in that category? Not sure what the flags it returns mean though so I'm not sure how to use it.


I did see this post over on the elder scrolls online forums:

https://forums.elderscrollsonline.co...ry-time#latest

It makes me wonder what happens when you create a request from some timestamp up until the current timestamp. Do new guild history events take some time register on the server? If so, does the server wait until they've registered before returning guild history request results, or could a guild history request possibly return incomplete results if a request is made before all of the events within its time window have registered?

[Baertram]

Originally Posted by ImpOfThePerverse I see there's an event, EVENT_GUILD_HISTORY_CATEGORY_UPDATED, that I could register for. Does it get sent when a new chunk gets delivered in that category? Not sure what the flags it returns mean though so I'm not sure how to use it.

From ZOS code:
Event gets registered here: https://github.com/esoui/esoui/blob/...0C123-L150C140
And calls self: OnCategoryUpdated(guildId, eventCategory, flags)
https://github.com/esoui/esoui/blob/...nager.lua#L154

It creates categoryData here:
local categoryData = guildData:GetEventCategoryData(eventCategory)

Which should be an object of ZO_GuildHistoryEventCategoryData
Which should then use (via this call: https://github.com/esoui/esoui/blob/...nager.lua#L158) this function here:
ZO_GuildHistoryEventCategoryData:OnCategoryUpdated(flags)

https://github.com/esoui/esoui/blob/..._data.lua#L365
-> ZO_FlagHelpers.MaskHasFlag(flags, GUILD_HISTORY_CATEGORY_UPDATE_FLAG_REFRESHED)
GUILD_HISTORY_CATEGORY_UPDATE_FLAG_NEW_INFO = 2
GUILD_HISTORY_CATEGORY_UPDATE_FLAG_REFRESHED = 4
GUILD_HISTORY_CATEGORY_UPDATE_FLAG_RESPONSE_RECEIVED = 1


Hope this helps with finding what the flags are and do.

For every other question you go there, I geuss somene more experienced with that guild history needs to step in here.

[ImpOfThePerverse]

Originally Posted by Baertram From ZOS code...

I'm still not 100% sure about how to interpret all of the flags, but the event seems to fire with the NEW_INFO flag when a new guild history event shows up in the guild history on the server. That could be used to trigger a new guild history request if you require up to the minute results in your cache, but I don't think it has anything to do with the requests themselves. I'm also not sure if it fires for each new guild history event, or for batches of events.

[ImpOfThePerverse]

Originally Posted by Baertram For every other question you go there, I geuss somene more experienced with that guild history needs to step in here.

I've finally gotten a request to complete. I tried requesting a day's worth of data from 1 week ago, and from 24 hours ago, and both requests completed pretty much immediately (IsGuildHistoryRequestComplete returns true, GUILD_HISTORY_REQUEST_FLAG_COMPLETE is the only positive flag, and there are results in the cache for that time window.)

Requests for more recent data produce results in the cache but the requests are never flagged as complete. My guess is it's a server syncing issue? The server can't guarantee that all of the latest results are in, so the request isn't complete. I haven't narrowed down how recent a request can be and still complete, if that's what's going on, but there probably won't be a reliable number if it's tied to server loads and whatnot.

[ZOS_DanBatson]

Whether or not a request is complete or complete-able is not based on server load. If you create a request for a range, it should be marked as complete if one of two things happens:
1. We already have events with no gaps that cover the entire time range of the request
2. We get a response from the server and it is empty, does not specify a next id, or the next id is an id we already have

As for the flags, GUILD_HISTORY_CATEGORY_UPDATE_FLAGs:
RESPONSE_RECEIVED refers to getting a response back from a client request (as opposed to the server pushing a new event)
NEW_INFO either means we received new events we didn't have before, or the category is now considered "up to date" which can happen if we requested latest info and got nothing new back but even knowing that is new info
REFRESHED refers to all the data needs to be reevaluated, right now only happening when permissions change

and GUILD_HISTORY_REQUEST_FLAG:
COMPLETE - We have gotten all events there are to get for the given time range and category
RESPONSE_PENDING - A request message from the client is in flight being processed by the server, waiting for a response
RESPONSE_PENDING_FROM_ADDON - Pairs with RESPONSE_PENDING, denoting that it was an add-on request. Not really used for anything atm tbh.
CHUNKS_DIRTY - Used internally to track updating and saving ranges
QUEUED - Request was made while on cooldown and allowed to queue, so it's in the queue waiting for cooldown to end so it can send to the server. Mutually exclusive with RESPONSE_PENDING
QUEUED_FROM_ADDON - Pairs with QUEUED, used internally to ensure requests from add-ons use add-on cooldown. Will be set if either the request was created by an add-on or if the call to RequestMoreEvents was fired from an add-on.

Note that DoesGuildHistoryHaveOutstandingRequest() specifically returns true if a request had an in flight message, i.e. RESPONSE_PENDING. Queued requests do not count as Outstanding. Requests that aren't complete do not count as Outstanding.

Also note that whenever we talk about gaps and ranges, the reason for the seemingly convoluted setup is because it facilitates being able to request events out of order. We use ranges and gaps to know where the wholes are in our requests so we don't miss events. In the old version of history, you could only request starting from now and going back in time. There was no way to get a specific range of time, which was what made it so annoying for add-ons like MM. Now we have a lot more control, but with that control comes more complication. We're working through those growing pains now. I appreciate your patience while we iron out the kinks. There are a lot of fixes coming thanks to the help of Siri and Sharlikran working with us over the last several weeks.