Quantcast Why don't addons include dependant libs anymore? - ESOUI
Thread Tools Display Modes
05/31/19, 01:48 AM   #1
Arato
Join Date: May 2019
Posts: 2
Why don't addons include dependant libs anymore?

Why do we need to hunt down and download and install the libs manually? Minion won't even update them so if an addon breaks because of a dependency we have to do it manually. Defeats the whole point of an addon manager.
  Reply With Quote
05/31/19, 02:43 AM   #2
sirinsidiator
 
sirinsidiator's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 1,280
There are many reasons for it:
  • The game offers a new system for loading libraries that ensures that only the newest version of the library code is run. In the past an arbitrary number of old versions of the code embedded in each addon had to be executed before the newest one was found, increasing loading time and opening the way for bugs that are impossible to fix without having to replace all older versions of the libraries in each installed addon.
  • The new library system allows libraries to access all features regular addons can use, like saved variables, XML UI controls, dependencies, multiple source files, etc.
  • It takes less disk space and also slightly improves loading time, since the game doesn't have to check multiple copies of a library.
  • Less "hidden features". Users can see which libraries an addon is using
  • Abandoned addons will receive updates to the libraries they use, potentially making them live longer.
  • Minion cannot update libraries contained within other addons, so installing them separately will ensure that they stay up to date
  • Minion4 will be able to install dependencies automatically
  • Authors spend less time updating bundled libs and as a result have more time for implementing new features and fixing bugs

And not many reasons against it:
  • Users have to install dependencies "manually" a single time, until Minion4 is released
__________________
Like what I do? Support me on Patreon!
>siri.exe MyAddon
Does your addon work? [y/n] n
There is a typo in there.

Last edited by sirinsidiator : 05/31/19 at 02:46 AM.
  Reply With Quote
05/31/19, 03:24 AM   #3
Arato
Join Date: May 2019
Posts: 2
I installed them manually at first before deciding it was a big hassle and getting Minion. Minion does not update them.
  Reply With Quote
05/31/19, 05:37 AM   #4
Marazota
Join Date: Sep 2015
Posts: 141
Originally Posted by Arato View Post
I installed them manually at first before deciding it was a big hassle and getting Minion. Minion does not update them.
minion updating and installing libs just fine for me
problem on your side
  Reply With Quote
05/31/19, 06:41 AM   #5
Baertram
 
Baertram's Avatar
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 2,415
Originally Posted by Arato View Post
I installed them manually at first before deciding it was a big hassle and getting Minion. Minion does not update them.
Minion is updating them IF you just instal lthem via Minion at least 1 time.
If they are not installed as "standalone" library, (just like an addon is) it won't be updated via Minion.
  Reply With Quote
05/31/19, 08:51 PM   #6
Wulfrun
Join Date: Feb 2015
Posts: 5
We need a lib management addon

Is there a FAQ on libs and AddOns; how to MANAGE them?
Minion was nice but my addons are a mess and some complain about missing libs. If a lib is out of date will the dependent addon just yell that it doesn't have a lib rather than say the lib is out of date?
IMHO, it is not easier than two years ago to get and use an AddOn (or add-ons as some are now calling them???).

Thanks sirinsidator but I don't understand how to make this all work. I see and appreciate it makes programming easier for the addon creator. But if we can't use the addon is seems a bit less than optimal.
Happy Trails

Last edited by Wulfrun : 05/31/19 at 08:57 PM.
  Reply With Quote
06/01/19, 01:50 PM   #7
t31os
AddOn Author - Click to view addons
Join Date: Jun 2015
Posts: 26
I prefer the new method and it's going to be a reoccuring issue(create confusion for people getting into addons) until authors update their addons(or someone else does) and everyone installs the libraries independently, but it'll be a good change long term.
  Reply With Quote
06/12/19, 03:32 AM   #8
Aaxc
 
Aaxc's Avatar
AddOn Author - Click to view addons
Join Date: Feb 2019
Posts: 18
Also, have you went to Minion settings and set "Automatically update" option on?
  Reply With Quote
06/28/19, 02:05 PM   #9
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 679
Originally Posted by sirinsidiator View Post
There are many reasons for it:
  • The game offers a new system for loading libraries that ensures that only the newest version of the library code is run. In the past an arbitrary number of old versions of the code embedded in each addon had to be executed before the newest one was found, increasing loading time and opening the way for bugs that are impossible to fix without having to replace all older versions of the libraries in each installed addon.
  • The new library system allows libraries to access all features regular addons can use, like saved variables, XML UI controls, dependencies, multiple source files, etc.
  • It takes less disk space and also slightly improves loading time, since the game doesn't have to check multiple copies of a library.
  • Less "hidden features". Users can see which libraries an addon is using
  • Abandoned addons will receive updates to the libraries they use, potentially making them live longer.
  • Minion cannot update libraries contained within other addons, so installing them separately will ensure that they stay up to date
  • Minion4 will be able to install dependencies automatically
  • Authors spend less time updating bundled libs and as a result have more time for implementing new features and fixing bugs

And not many reasons against it:
  • Users have to install dependencies "manually" a single time, until Minion4 is released
Good points, however not addressing the question in the title. The option to not include libs in add-ons was there since 2014. Embedded libraries became the norm probably because the drawbacks of embedding were deemed not bad enough to require separate installation of libraries, which would be less convenient for users.

Originally Posted by Arato
"Why don't addons include dependant libs anymore?"
Because some authors decided to change their packaging method without utilizing the new AddOnVersion capability of ESO client.

AddOnVersion enables packaging libraries together with an add-on, without embedding them in the add-on manifest (which was clumsy, inefficient and error-prone). If another add-on packages a newer library version, or a newer library version is installed separately, older versions are not loaded. What baffles me is that this new feature prompted some add-ons to change their packaging while ignoring the feature. Instead of switching to the newly enabled packaging method (including libs with their own manifests), they switched from user-friendly method of 2014 (embedded libs) to user-unfriendly method of 2014 (separately installed libs), completely missing the point of AddOnVersion.
  Reply With Quote
06/28/19, 05:04 PM   #10
Baertram
 
Baertram's Avatar
AddOn Author - Click to view addons
Join Date: Mar 2014
Posts: 2,415
The problem with the packaged libraries within an addon, including the whole library folder and txt manifest files, is:

Putting the library folders in a subfolder of the addon but using their own txt files
-The txt files will only be read from the addon manager until a folder depth of 2 or 3 (not sure anymore) subfolders.
-You have a copy of the libraries AGAIN inside the addon subfolders and need to update this subfolder each time the library updates (same problem as with /libs/libraryname/libraryfile.lua reference in the addon manifest txt from 2014)
+ no possibilities to "only update the library" via Minion, and much effort to update it manually
-Not easy to find the loaded library in case of errors as every addon and subfolder could contain the library again. This was one of the reasons why I'm using the libraries in the main AddOns folder now e.g.!

Libraries packaged with the addon but installed to main AddOns folder
-If you do not have it inside the addon subfolder itsself but in the main folder of the addon (as own "library folder"), it should work but it might not update properly via Minion because of multiple found txt files and folders.
-every addon updated will overwrite the folder again. If only one got a "bad or old version" the folder might be broken and thus several addons will

Benefit of the manually / Minion standalone installations of libraries - IMO
-You have only 1 source where the error could be and thus it's easy to support and fix it
-Less disk space needed
-Update via Minion is easy after it has been installed once
-Manual update is easy as wella s you only need to update 1 folder and are able to overwrite all contents (the AddOnVersion tag in updated libraries assure that only the newest version will be read from this AddOns/libraryName folder then)
So I don't get why you say we ignore the AddOnVersion here @merlight?

I guess it's up to the authors how they solve this. But including the libs again into the addons got more disadvantages (afai understood the whole complex system) than keeping them seperated in their own folder.
  Reply With Quote
06/28/19, 06:56 PM   #11
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 679
Originally Posted by Baertram View Post
The problem with the packaged libraries within an addon, including the whole library folder and txt manifest files, is:

Putting the library folders in a subfolder of the addon but using their own txt files
-The txt files will only be read from the addon manager until a folder depth of 2 or 3 (not sure anymore) subfolders.
I don't think that's an issue at all, what use case would need manifest 4+ levels deep?

Originally Posted by Baertram View Post
-You have a copy of the libraries AGAIN inside the addon subfolders and need to update this subfolder each time the library updates (same problem as with /libs/libraryname/libraryfile.lua reference in the addon manifest txt from 2014)
As an author, you generally don't want to test your add-on against stale library versions (older than what your users will install separately, or get from other add-ons using the same library), so you should already have all libraries you use up-to-date.
You don't need to have copies of libs in every add-on. You can have your packaging script add libs from a shared location.
And it's hardly the same problem as with lib sources embedded in main add-on manifest, where you had to manually edit the manifest and copy the paths to sources, in case some were added/removed.

Originally Posted by Baertram View Post
+ no possibilities to "only update the library" via Minion, and much effort to update it manually
For libraries packaged together with add-on, users still have the option to install libraries separately (and if they're newer version, they'll be used). It's exactly as much effort for users as if the library is not packaged, except that it's optional (until there's an essential update to the library and none of the add-ons using that library gets the updated version).

Originally Posted by Baertram View Post
-Not easy to find the loaded library in case of errors as every addon and subfolder could contain the library again. This was one of the reasons why I'm using the libraries in the main AddOns folder now e.g.!
I'm not sure whether it's base UI or Votan's Addon List feature, but addon tooltips show path from which they loaded. Error messages contain path as well.

Originally Posted by Baertram View Post
Libraries packaged with the addon but installed to main AddOns folder
-If you do not have it inside the addon subfolder itsself but in the main folder of the addon (as own "library folder"), it should work but it might not update properly via Minion because of multiple found txt files and folders.
-every addon updated will overwrite the folder again. If only one got a "bad or old version" the folder might be broken and thus several addons will
This would obviously be broken, evil and also completely miss the point of AddOnVersion.

Originally Posted by Baertram View Post
Benefit of the manually / Minion standalone installations of libraries - IMO
-You have only 1 source where the error could be and thus it's easy to support and fix it
-Less disk space needed
-Update via Minion is easy after it has been installed once
-Manual update is easy as wella s you only need to update 1 folder and are able to overwrite all contents (the AddOnVersion tag in updated libraries assure that only the newest version will be read from this AddOns/libraryName folder then)
Originally Posted by Baertram View Post
So I don't get why you say we ignore the AddOnVersion here @merlight?
The purpose of AddOnVersion is to help game client pick the latest version of an add-on/library when it encounters multiple copies. If you're not shipping any copies other than the separately installed one, you're not utilizing AddOnVersion.

Originally Posted by Baertram View Post
I guess it's up to the authors how they solve this. But including the libs again into the addons got more disadvantages (afai understood the whole complex system) than keeping them seperated in their own folder.
Yes, of course it's everyone's choice, I'm not trying to take that away. I just find it sadly amusing that this 'rm -rf lib/' movement started after AddOnVersion became a thing. Imagine you decided to stop packaging libraries with your add-ons back in 2015, when ESO didn't recognize AddOnVersion. There was nothing stopping you from doing just that.
  Reply With Quote
06/29/19, 04:03 AM   #12
sirinsidiator
 
sirinsidiator's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 1,280
I do agree that the whole unbundling movement happened a bit too soon, but stopping it is no longer possible and reverting to bundled libs doesn't make any sense now that it has happened and most users already installed the necessary libs. But everyone is free to handle it the way how they want any ways. I personally would have liked to wait with it until Minion 4 is ready to avoid the confusion we have seen.

stale libraries) Next game update we'll get a tiny new feature, allowing us to specify a minimum AddOnVersion for dependencies (e.g. "LibAddonMenu>=29"). That way authors will utilize the AddOnVersion, even when they don't bundle the lib and won't have to worry about old unsupported versions of libs being loaded.

lib path) One of the reasons I added the loaded addon logging to LibDebugLogger is exactly that. If you utilize it and educate users to send you the log file when they have a problem, it won't be an obstacle as it tells you which folder the libs are loaded from.

addons including libs in the root) Unfortunately I noticed some addons doing exactly that while working on Minion 4. And it's really bad, because Minion 3 will simply replace the folder without a notice and won't even recognize that the version of the installed standalone library has changed on the disk and still show it as up to date. I plan to add something to Minion 4 to handle this case better, but I believe in the long run, ESOUI should recognize bundled dependencies on the root level at upload time and deny the upload.
__________________
Like what I do? Support me on Patreon!
>siri.exe MyAddon
Does your addon work? [y/n] n
There is a typo in there.

Last edited by sirinsidiator : 06/29/19 at 04:08 AM.
  Reply With Quote
06/29/19, 07:45 AM   #13
Marazota
Join Date: Sep 2015
Posts: 141
any ETA of minion 4?
  Reply With Quote
06/29/19, 12:43 PM   #14
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 679
Originally Posted by sirinsidiator View Post
stale libraries) Next game update we'll get a tiny new feature, allowing us to specify a minimum AddOnVersion for dependencies (e.g. "LibAddonMenu>=29"). That way authors will utilize the AddOnVersion, even when they don't bundle the lib and won't have to worry about old unsupported versions of libs being loaded.
Nice. Finally some plus points for separate libraries
  Reply With Quote
06/29/19, 12:48 PM   #15
sirinsidiator
 
sirinsidiator's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2014
Posts: 1,280
Originally Posted by Marazota View Post
any ETA of minion 4?
I'm aiming for next game update, but no promises.
__________________
Like what I do? Support me on Patreon!
>siri.exe MyAddon
Does your addon work? [y/n] n
There is a typo in there.
  Reply With Quote
06/29/19, 02:39 PM   #16
Drakanwulf
 
Drakanwulf's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2017
Posts: 50
Read the ESOUI Wiki!

There are two sections beneaththe ESOUI Wiki "Packaging:" topic that explain the recent add-on packaging and manifest file directive changes. I suggest everyone who has posted to this discussion take the time to read them; both sections have been updated and enhanced.

Packaging:
https://wiki.esoui.com/Addon_Structure
https://wiki.esoui.com/Addon_manifest_(.txt)_format

If you think what these Packaging wiki sections say is not correct, then ask about them; someone will take the time to help you and perhaps update the wiki text to correct the inaccuracies.

If you don't understand what these Packaging wiki sections say, then ask about them; someone will take the time to help you and perhaps update the wiki text to clarify your confusion.

If you don't like what these Packaging wiki sections say, then state the reasons for your dislike; someone will take the time to help you and perhaps update the wiki text to remove the reasons for your dislike.

If all you wish to do is to complain about the recent changes to how the game loads libraries and add-ons, then I suggest you start writing an add-on to force the game to revert back to how it used to load libraries and add-ons; that should keep you busy for the rest of your life!
__________________
Drakanwulf, the dire wolf that hunts dragons!
  Reply With Quote
06/29/19, 06:03 PM   #17
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 679
Originally Posted by Drakanwulf View Post
https://wiki.esoui.com/Addon_manifest_(.txt)_format
DependsOn

A space-separated list of subordinate add-on folder names ...
This is inaccurate. Dependencies listed in "DependsOn" can be loaded from anywhere else, e.g. from live/AddOns/libX or any other (unrelated) add-on's subdirectory. The description of "OptionalDependsOn" further on that page correctly omits the word "subordinate".

Originally Posted by Drakanwulf View Post
https://wiki.esoui.com/Addon_Structure
(3a) When you load any library add-on (e.g. LibStub) directly into your .../AddOns/... folder as a stand-alone, "new style" library, you should remove all references to its folder name in all DependsOn: or OptionalDependsOn: directives within any subordinate manifests ...
This is wrong. (Optional)DependsOn directives also affect execution order. If you don't list the dependencies there, and attempt to refer to a library from add-on before EVENT_ADD_ON_LOADED, it will bite you.

Originally Posted by Drakanwulf View Post
(3a) ... and you should remove all its folders from within any subordinate folders. For example, (3b) is wrong; the LibStub folder should not be there.
I'd replace "should" with "may". There's nothing wrong with keeping LibStub (3b) there.

Here my constructive feedback ends.


Originally Posted by Drakanwulf View Post
If all you wish to do is to complain about the recent changes to how the game loads libraries and add-ons, then I suggest you start writing an add-on to force the game to revert back to how it used to load libraries and add-ons; that should keep you busy for the rest of your life!
Something tells me I should refrain from responding, but I can't leave this nonsense unaddressed. Nobody complains about changes to how the game loads libraries and add-ons. The question was about changes to how add-ons are packaged. A user asked why add-ons stopped bundling libs. "Because the game has this new AddOnVersion feature" is not a proper answer; the introduction of AddOnVersion made a more efficient method of bundling libraries possible; it didn't make the previously available methods of bundling-embedding or not-bundling libraries any better or worse, so why would anyone stop bundling because of that? The decision to stop bundling libs should not be blamed on changes to how the game loads libraries, because the not-bundling method was always available and doesn't exploit the AddOnVersion (yet).

Last edited by merlight : 06/30/19 at 04:12 PM.
  Reply With Quote
06/29/19, 09:16 PM   #18
Phuein
 
Phuein's Avatar
AddOn Author - Click to view addons
Join Date: Mar 2018
Posts: 95
Post

In the broader spectrum of this conversation, I agree with those that dislike this change.

It's akin to how Microsoft requires installing separate libraries (Directx and .NET), or how Linux depends on a network of libraries. It's a pain for the end-user, and to this day is very unpopular. Most MS applications offer to install those libraries together with the app, or download it automatically; but that relies on the app doing this - not some manager. Linux distributions with their Package Managers are even less popular, by far, and for good reason.

I personally, as a programmer, hate dependencies. They are only justified in the most extreme and reliable cases. The drawbacks are immense and endless; from neglected code to badly documented APIs and unexpected bugs.

Hopefully, either the good work on M4 will make this moot, or future addon devs will figure out a friendly way of including the libs with less hassle.
__________________

Author of beloved and rejuvenated addons:

ShowMount             PvP FPS 2018

Notebook 2018         Junkee 2018

BestFriends 2018       New Addon
  Reply With Quote
06/30/19, 02:45 PM   #19
Drakanwulf
 
Drakanwulf's Avatar
AddOn Author - Click to view addons
Join Date: Apr 2017
Posts: 50
Thank you for your comments. They are appreciated.

Originally Posted by merlight View Post
This is inaccurate. Dependencies listed in "DependsOn" can be loaded from anywhere else, e.g. from live/AddOns/libX or any other (unrelated) add-on's subdirectory. The description of "OptionalDependsOn" further on that page correctly omits the word "subordinate".
The text has been corrected.

Originally Posted by merlight View Post
This is wrong. (Optional)DependsOn directives also affect execution order. If you don't list the dependencies there, and attempt to refer to a library from add-on before EVENT_ADD_ON_LOADED, it will bite you.
The text has been corrected.

Originally Posted by merlight View Post
I'd replace "should" with "may". There's nothing wrong with keeping LibStub (3b) there.
The text has been expanded to explain why leaving LibStub(3b) in there can lead to unforeseen and unwanted problems. Here is a GitHub link to the screenshot library I built to document the WrathStone (100026) add-on loader confusion problem:

https://github.com/Drakanwulf/Add-on-loading-issue

The expanded answer to LibStub(3b) may also provide some insight into why leaving old, embedded, library add-ons in their old, possibly incorrect, packages could be a bad decision. It became the rationale for my following Baertram's lead and converting to "standalone" library and support add-ons as quickly as I could.

You should know that I took the time to reload all of the add-ons that I use and their libraries with the OOD checkbox turned OFF to learn from their packaging mistakes and oversights and to gain knowledge about how the game's add-on loader functions. It was my Don Quixote "windmill" project. I was able to complete my "windmill" successfully.

I apologize for the harsh wording I used when I made reference to complainers. BTW, you may wish to update the manifest and packaging for the "mer Torch Bug" add-on. I use that add-on a lot for research and debugging.
__________________
Drakanwulf, the dire wolf that hunts dragons!

Last edited by Drakanwulf : 06/30/19 at 02:48 PM.
  Reply With Quote
06/30/19, 04:11 PM   #20
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 679
I've just noticed I swapped wiki links in previous post, will fix that.


Originally Posted by Drakanwulf View Post
The text has been expanded to explain why leaving LibStub(3b) in there can lead to unforeseen and unwanted problems.
I see, that explains a lot. I assumed the LibStub(3b) directory in the example stood for "some complete new-style library with proper manifest". Didn't even think about the possible pitfalls with dragging old libs.


Originally Posted by Drakanwulf View Post
I apologize for the harsh wording I used when I made reference to complainers. BTW, you may wish to update the manifest and packaging for the "mer Torch Bug" add-on. I use that add-on a lot for research and debugging.
No promises. I still use it as well, but I'm not spending nearly as much time with ESO as in the early days, much less on add-ons. I never did updates only to bump API version, I need some code changes to justify a release
  Reply With Quote

ESOUI » AddOns » AddOn Help/Support » Why don't addons include dependant libs anymore?

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off