ESOUI

ESOUI (https://www.esoui.com/forums/index.php)
-   General Authoring Discussion (https://www.esoui.com/forums/forumdisplay.php?f=174)
-   -   New Guild History API (https://www.esoui.com/forums/showthread.php?t=10724)

cyxui 03/11/24 07:47 PM

Played around with the new api and it seems like a hot mess. The request created from CreateGuildHistoryRequest seems to persist after reloadUI. Not sure if this will eat up the queue CD or not. if someone reloads UI a lot then tons of requests will get created. Not to mention how slow the requests are.

I started a request for one day worth of data (GetTimesStamp(), GetTimeStamp() - 60 * 60 * 24) for a small guild. It has been more than 30min already and its still running. Amount of history should be < 500. Performance is abysmal.

Sharlikran 03/11/24 08:38 PM

What I am noticing is that when mods are all fighting for data it seems to cause the UI to freeze. It's been explained to me as on cooldown but however you want to say it. Nothing works when the mods are enabled. For example if I turn all the mods off, I can go get 150 pages of sales data. Once I reload I will have one page of data and all I see is this.



Every category, for every guild is frozen. Nothing works and if I want to get data manually, by pressing E then I get this.



When I haven't requested anything and even if I wait 5 seconds I can't request data.

So I'm not convinced that mods should use the API directly and they should all be using LibHistorie so that if it wants data it can request it, then transmit that to all the mods that are waiting for the data.

If not users are going to be super confused thinking something is broken.

Sharlikran 03/11/24 08:41 PM

Quote:

Originally Posted by cyxui (Post 49462)
Played around with the new api and it seems like a hot mess. The request created from CreateGuildHistoryRequest seems to persist after reloadUI. Not sure if this will eat up the queue CD or not. if someone reloads UI a lot then tons of requests will get created. Not to mention how slow the requests are.

I started a request for one day worth of data (GetTimesStamp(), GetTimeStamp() - 60 * 60 * 24) for a small guild. It has been more than 30min already and its still running. Amount of history should be < 500. Performance is abysmal.

I still think there is something to be said about trying to use proactive measures to only request what you need. Tracking how long the person has been offline, checking if the sales exist, and then process the data. If someone is using MM they will already have data so TTC doesn't need to request it. It will already exist in the binary cache.

Regardless of any UI issues currently, when the data is in the cache, it stays there and can be obtained. This isn't like in the past where once you logged out then you had to request 10 days of data again. Once you have the 10 days of data, it's there and it's not going anywhere.

nightstrike2 03/11/24 09:23 PM

Quote:

Originally Posted by sirinsidiator (Post 48710)
Whatever you choose. You can set it to a 100 years if you so wish.

Now that the update has arrived, I can't see any obvious way to set this retention information. Where do you configure it for more than 30 days?

cyxui 03/11/24 10:41 PM

Quote:

Originally Posted by Sharlikran (Post 49464)
I still think there is something to be said about trying to use proactive measures to only request what you need. Tracking how long the person has been offline, checking if the sales exist, and then process the data. If someone is using MM they will already have data so TTC doesn't need to request it. It will already exist in the binary cache.

Regardless of any UI issues currently, when the data is in the cache, it stays there and can be obtained. This isn't like in the past where once you logged out then you had to request 10 days of data again. Once you have the 10 days of data, it's there and it's not going anywhere.

In theory thats what SHOULD happen. But it aren't. At least with my experience. Say I requests for 1 day worth of data. 30min later and its done (Seriously? 30min?). Then I log off + login and requests for 1 day worth of data again. Theoretically the game should be smart enough to know that I am only requesting for 5min worth of data since everything between 1 day ago and 5min ago are there. But nope. The game goes on and run the 30min request once more. I would totally expect 5 min worth of data to take minutes not half an hour.

cyxui 03/11/24 10:42 PM

Quote:

Originally Posted by Sharlikran (Post 49463)
What I am noticing is that when mods are all fighting for data it seems to cause the UI to freeze. It's been explained to me as on cooldown but however you want to say it. Nothing works when the mods are enabled. For example if I turn all the mods off, I can go get 150 pages of sales data. Once I reload I will have one page of data and all I see is this.



Every category, for every guild is frozen. Nothing works and if I want to get data manually, by pressing E then I get this.



When I haven't requested anything and even if I wait 5 seconds I can't request data.

So I'm not convinced that mods should use the API directly and they should all be using LibHistorie so that if it wants data it can request it, then transmit that to all the mods that are waiting for the data.

If not users are going to be super confused thinking something is broken.

You dont even need multiple addons to break this. Just reload UI twice and boom. The whole thing is locked up.

Sharlikran 03/12/24 01:15 AM

Sufice it to say I manipulated the guild history from chat to repair it when it stopped working for me.

1 out of two guilds was repaired, the other I had to clear the cache and then request the data again.

I did this from chat with no mods. So I have simply reported the guild history as broken.

I have a backup of the files so when it breaks they can be compared.

Baertram 03/12/24 03:34 AM

Quote:

Originally Posted by cyxui (Post 49468)
You dont even need multiple addons to break this. Just reload UI twice and boom. The whole thing is locked up.

As long as I use the Guild History UI to query the past 10 days it's smooth and fast there (all addons off!).
Only if I got LibHistoire enablede solo (which you need to set any guild's category to "Force" then so it updaes that data of the category), or any addon enabled that wants to query the data, then I experience the same as Sharlikran:

I wait forever and cannot get anything except that "Guild history requests can be done once every 2 seconds", even if clicking the normal UI buttons / "Get more" keybind then.
After like 1 hour the banked ites was updated then :) For 1 guild...

So I guess every API usage outside of the Guild History UI will lock it at the moment.

sirinsidiator 03/12/24 06:08 AM

Quote:

Originally Posted by nightstrike2 (Post 49465)
Now that the update has arrived, I can't see any obvious way to set this retention information. Where do you configure it for more than 30 days?

That's one of the things I had to cut from this version of the lib. You can manually adjust it in your UserSettings.txt in the meantime.

peniku8 03/12/24 09:21 AM

Quote:

Originally Posted by Sharlikran (Post 49463)
So I'm not convinced that mods should use the API directly and they should all be using LibHistorie so that if it wants data it can request it, then transmit that to all the mods that are waiting for the data.

Since my addons have been relying on my modded version of Shissu's history scanner, which now broke, I've been thinking about doing this to track member join dates and donations. Is it documented anywhere how I would go about implementing LibHistoire for this kind of purpose? I've talked to Sirinsidiator about this two years ago, when this seemed impossible to me, but I suppose a lot has changed in that time. Is there a function to request chached data for let's say all gank transactions for a guild over a specific time frame?
If this question is too OT I'd be happy to talk about this elsewhere, because I'd love to implement an efficient solution instead of just making another addon that constantly requests data from the server.

Sharlikran 03/12/24 11:34 AM

My mod already does that but it relies on the data existing. So say your user joined 2 years ago and the guild history has 180 days, that won't work. A new GM using the mod will not see the join date.

Deposits are also already available for those with permission. Now that users see their own activities then that can be used per user. Although I already show what they deposited using an event.

nightstrike2 03/12/24 12:57 PM

Quote:

Originally Posted by sirinsidiator (Post 49477)
That's one of the things I had to cut from this version of the lib. You can manually adjust it in your UserSettings.txt in the meantime.

Ah, thanks for the pointer. I found the settings in UserSettings.txt. Much appreciated.

Quote:

Originally Posted by sirinsidiator (Post 48688)
Data will be available on the server for 30 days for all categories except the roster and milestone category

It looks like at least for now, the server only has 11 days. Presumably they started with the same 10 as before and are now incrementally adding back up to 30, but they didn't collect 30 days before the U41 drop.

cyxui 03/12/24 03:03 PM

Quote:

Originally Posted by Baertram (Post 49476)
As long as I use the Guild History UI to query the past 10 days it's smooth and fast there (all addons off!).
Only if I got LibHistoire enablede solo (which you need to set any guild's category to "Force" then so it updaes that data of the category), or any addon enabled that wants to query the data, then I experience the same as Sharlikran:

I wait forever and cannot get anything except that "Guild history requests can be done once every 2 seconds", even if clicking the normal UI buttons / "Get more" keybind then.
After like 1 hour the banked ites was updated then :) For 1 guild...

So I guess every API usage outside of the Guild History UI will lock it at the moment.

Before the update i was getting like 100 per 2min and i thought its slow enough. Now its 30-60min for 500 if you are lucky that the request didnt get stuck.

ImpOfThePerverse 03/16/24 05:46 PM

Is there documentation somewhere on how to use the new system? I see function and event signatures, and constants, but nothing giving an overview of basic operations like creating a guild history request and iterating over the results.

I'm used to figuring out game APIs through trial and error, but with requests taking that long to process it doesn't seem like that will be very effective.

My best guess is:

Use CreateGuildHistoryRequest to get the server to start caching events.

Poll with GetGuildHistoryRequestFlags until GUILD_HISTORY_REQUEST_FLAG_COMPLETE is true (unless there's an event I can register for that will let me know?)

Iterate over the results via GetGuildHistoryEventBasicInfo (or one of the event category specific variants of it) in a for loop, using starting and ending event indices obtained via GetGuildHistoryEventIndicesForTimeRange.

Is this close? What happens when there are more than 500 events in the range specified?

Thank you, any help will be greatly appreciated.

Sharlikran 03/16/24 06:02 PM

At this point it's best to explain what you want to do or upgrade. The reason is if you use the low level API (meaning what you see in the docs) then it could cause your mod to appear to work properly. However, LibHistorie may not be able to function because you did something that is taboo, and it put everything on cooldown by accident. Like for example, don't try to fix mods like Shissu's.

If you want to make something from scratch, go ahead. Read the the LibHistorie description page.

ImpOfThePerverse 03/16/24 07:15 PM

Oh, sorry, I meant writing something to work directly with ESO's updated guild history API. I wasn't planning to make use of LibHistoire.

I would like to request guild event history within a specific category and iterate over the results.

Sharlikran 03/16/24 07:53 PM

If it's not for a mod, so you don't create incompatibility or lock up functioning mods, then sorry, never mind. It's a large binary file you don't request anything. I apologize for misunderstanding, sorry for interrupting.

ImpOfThePerverse 03/16/24 08:17 PM

The addon won't be in general circulation, or run alongside other addons that create guild history requests. To keep things simple I would prefer to just work directly with the ESO API.

sirinsidiator 03/17/24 08:17 AM

You should not use the low-level api directly. Instead it's advised to utilize the high-level api found here.

But you will run into a lot of issues if you do that right now, since the game currently has a couple bugs in regards to requesting data. If you don't want to spend countless hours navigating around them, you are better off relying on LibHistoire.

If you still insist on not using it after hearing that, be warned that you will inadvertently interfere with operation of the library and make it slower. So don't blame me when it ends up taking even longer to request data. ;)

ImpOfThePerverse 03/17/24 11:32 AM

Ah, gotcha, thanks. Dealing with a buggy API can be an enormous, onerous headache. Hopefully ZOS cleans it up in the not too distant future. Still, they should probably try to communicate their new system better - I'm sure right now there are a bunch of addon authors banging away at the API trying to figure it out, jamming the servers with messed up history requests.


All times are GMT -6. The time now is 03:34 PM.

vBulletin © 2024, Jelsoft Enterprises Ltd
© 2014 - 2022 MMOUI