I have been thinking about how libraries in ESO work and am unsatisfied with the current state.
Ideally each addon would have its own dependencies isolated into a sandbox, but that's not how ESO works so instead all addons have to share the same libraries in the same sandbox. That's why we use LibStub to keep only the newest version in memory and hope nothing breaks. In most cases we also just bundle them into the addon, which pushes a lot of responsibilities onto the author of the addon.
I came to the conclusion that they should not be bundled with addons at all. Instead it should be the responsibility of the user to install the necessary dependencies like in any other environment. Having a standalone-only libary also has the benefit that it can use saved variables and xml controls and most importantly, specify dependencies for the library. Right now if a library depends on another library, the library author has to add a check to make sure it is loaded and the addon author who wants to use the library is responsible to load them in the correct order. We already have a dependency system in ESO which could easily handle this for us if we stopped bundling the libraries.
Of course we cannot expect the average user to do this manually, so it needs to become a feature of ESOUI and Minion to install the necessary dependencies automatically. For this to work we should be able to somehow specify the addon id or name and minimum required version of a dependency when we upload the addon. Minion then just installs and updates them in addition to the selected addon and ESOUI could for example automatically supply a bundled zip alongside the normal one. This would also make it possible to create collections of addons without having to put all of them into one zip file manually (e.g. PhinixUI) and also allow it to show lists of who uses what on the addon pages.
In order to compare version numbers properly we should decide on a common format for libraries and addons. I am currently thinking of using the following number based format:
Major reflects changes that break compatibility with other addons, minor symbolizes any small feature update that does not break anything and build number is automatically increased whenever a new build is made. For authors who don't use a build system it's probably just some arbitrary number. Either way it should never become smaller than it was before.
AddonA specifies LibSomething 1.23.* as a minimum dependency
Minion/ESOUI would supply the user with any 1.*.* version after it (e.g. 1.23.456), but not with version 2.*.*.
Another addonB depends on LibSomething 2.0.*
Minion/ESOUI adds any version after that (e.g. 2.0.789) for the user.
The author of LibSomething has to make sure that v1 and v2 are able to run side by side and any future v1 or v2 stays backwards compatible.
The author of AddonA has to specify LibSomething and a minimum version as a dependency and does not care about updates or dependencies of the library
The user has to make sure that all dependencies are fulfilled
Minion/ESOUI makes this simple for users by automatically resolving them
For library authors this means, the library should still use LibStub which would get an update to support this version format and maybe also to separate libraries and addons into two lists in the addon manger.
For addon authors it means they no longer should bundle libraries and instead make sure that they reference them in the meta file and on ESOUI.
For Dolby it means a lot of work to implement the required features.
For users not much would change.
In Minion we could probably do it on the fly because we could just tell Minion to download the 2 zip files, etc. However to provide a download from the website we need to give the user just 1 zip and to generate a new zip on each download would require a ton of cpu and disk i/o so we would have to cache it. If we are going to cache it might as well just serve that cached file to Minion too otherwise there might be a point where Minion is giving a different version of the lib then if you just download from the site manually. Also the issue with alerting users via email, rss, etc when just the lib is updated and not the main addon.
So how about when the author releases a new version and uploads it here our site would then scan the zip for a file that holds dependency info. If the file is there and it has valid data we inject that dependency into the release zip. If the file is missing or has invalid data such as version numbers no longer available it just ignores it. Maybe add a button and api endpoint for the author to easily release an update if its only a dependency update?
side thought... if you are going with a travis ci release system you could bundle everything on your end and use the upload api I'll have added to esoui soon.
I'll keep thinking about this. sorry for the short response and typos, on my phone atm.
Injecting it in the uploaded zip only when an update is made won't work because if addonA uses some library v1.0 and some time later addonB starts to use library v1.1 with some new feature that didn't exist in 1.0, addonB would break depending on the order the user extracted the bundles in. Same goes for building it on my end via travis.
I also don't think the bundled zip should be generated when the user downloads the file. As you pointed out, that would cost way too much resources. If anything, it should be generated when any author uploads a new version of any of the parts of the bundle. This would make it necessary that you keep the original upload as you have to rebuild the bundle whenever a dependency gets updated. Then you could also offer a less prominent download button for the unbundled version for expert users.
As for update notifications, I don't think they are required for dependency updates. Just rebuild the bundle silently as a new versions of libraries should not change the functionality of the addon and if a library update contains a fix for the specific addon, the author of the addon can reupload a new zip file with an incremented build version to trigger the notification.
Minion should still download the new version like for other installed addons. I don't think downloading the bundle is the way to go here. It would maybe also need a feature to recognize when a dependency is no longer needed and clean it up.
The dependencies could be specified in an optional JSON file in the root of the uploaded zip which contains an array of ESOUI addon ids and the minimum required versions: