Thread Tools Display Modes
07/15/15, 07:04 AM   #1
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 671
LuaEcoSearch - fighting global namespace pollution

I wrote a script that looks for global symbols used in add-ons. It's far from perfect, yet its output might already prove useful, so here you are

http://ge.tt/3zjp9QK2

luaEcoSearch.sh is a Unix shell script, it requires perl, luac5.1 (Lua 5.1 compiler) and optionally 7-zip. You give it a bunch of directories and/or zip files and it prints globals it finds to stdout.
Code:
$ sh luaEcoSearch.sh SomeAddonFolder AnotherAddon.zip ...
Assuming not everyone will be able to run it, I also included gzipped output generated from the latest versions of most of the add-ons here on esoui. I can't promise keeping it up to date, though.

Sample output:
Code:
cache/AddonProfiles+1416618983.zip

 +? AddonProfilesComboBox_ObjectFromContainer   AddonProfiles/AddonProfilesComboBox.lua +1
 +? AddonProfiles_CreateProfile                 AddonProfiles/AddonProfiles.lua +1
 +? AddonProfiles_DeleteProfile                 AddonProfiles/AddonProfiles.lua +1
 +? AddonProfiles_SavedVariables2               AddonProfiles/AddonProfiles.lua +185
 +  AddonProfiles_SwitchToProfile               AddonProfiles/AddonProfiles.lua +1
  ? AddMenuItem
  ? AnchorMenu
  ? ClearMenu
  ? CreateControlFromVirtual
The first line contains the file given on the command line.

The remaining lines begin with FLAGS:
+ marks a symbol that the add-on SETS
? marks a symbol that the add-on GETS

The second column is the symbol name, and the third column (if present) is the location of the start of the function in which the symbol is SET (luac doesn't tell the line number of the actual assignment).

If there is a + but no ? in front of a symbol, it may mean that it is unused, but not always. AddonProfiles_SwitchToProfile in the above sample is actually used from XML, which the script doesn't check.

Like I said, it's far from perfect, and here is why - it only finds simple, unqualified accesses/assigments. For example, when run on this code:
Lua Code:
  1. foo = bar
  2. _G.chop = _G.wood
  3. _G["cook"] = _G["food"]

The output would be:
Code:
/tmp/foo/

 +  foo                                         foo/bar.lua +1
  ? _G
  ? bar
That is, only foo, bar and _G got caught. The script can't detect you're also reading globals wood and food, and setting globals chop and cook.
  Reply With Quote
07/16/15, 12:10 AM   #2
votan
 
votan's Avatar
AddOn Author - Click to view addons
Join Date: Oct 2014
Posts: 577
Ah. It's great.

Typos and missing declaration happens very easy without a compiler warning you.
What about passing the manifest file to get the list of Lua files?
This way wrong paths can be found quicker.
And an extra loop filling $(language) with all (unoffical) supported languages to find syntax errors in language files one normally does not use. (Already happend)

The list was very useful.

Thanks!
  Reply With Quote
07/16/15, 03:02 AM   #3
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 671
Originally Posted by votan View Post
What about passing the manifest file to get the list of Lua files?
This way wrong paths can be found quicker.
And an extra loop filling $(language) with all (unoffical) supported languages to find syntax errors in language files one normally does not use. (Already happend)
I didn't even think about parsing add-on manifest. The script just checks all .lua files it finds, there's no ESO specific logic (except stripping UTF-8 BOM). That's how I once found a broken string in french translation, which I don't understand

As for finding missing/misspelled files, here you go:

bash Code:
  1. find -name '*.txt' -exec perl -ne '
  2.    chomp;
  3.    next unless /^\w/;
  4.    for $lang (qw( en de fr ru es pt )) {
  5.        $fn = $_;
  6.        $rep = ($fn =~ s/\$\(language\)/$lang/);
  7.        print "missing $fn\n" unless -f $fn;
  8.        last unless $rep;
  9.    }
  10. ' '{}' +

Last edited by merlight : 07/16/15 at 09:28 AM.
  Reply With Quote
07/16/15, 12:03 PM   #4
Lodur
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 108
Would love to see you filter out the reads (?) of the LUA system functions and ZOS API Functions.
  Reply With Quote
07/16/15, 12:37 PM   #5
Ayantir
 
Ayantir's Avatar
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 1,019
To update the wiki I'm writing some kind of similar script (but in lua). I'll try to update it aswell next zos patches come.
  Reply With Quote
07/16/15, 07:45 PM   #6
Lodur
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 108
And thanks! You found me two bugs in my current addon.
  Reply With Quote
07/17/15, 02:24 AM   #7
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 671
Originally Posted by Lodur View Post
Would love to see you filter out the reads (?) of the LUA system functions and ZOS API Functions.
Yea that should be easy. The hardest part will be getting a clean and complete list
  Reply With Quote
07/18/15, 05:09 AM   #8
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 671
Added a filtered list. Shouldn't contain read-only globals available ingame without add-ons (functions, tables, controls, enums).
  Reply With Quote
07/20/15, 10:17 AM   #9
Lodur
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 108
Will there be a new luaEcoSearch.sh released that does the filter? The link is still to the old one.
  Reply With Quote
07/20/15, 10:31 AM   #10
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 671
This was kinda an afternoon fun demo, but if I return to it, I'll put it up on github with all the parts. I did the filtering with another script, which extracts a list of names from a dumper add-on (which will need to be included) and then essentially does grep -xf
  Reply With Quote
07/29/15, 11:36 PM   #11
Lodur
AddOn Author - Click to view addons
Join Date: Feb 2014
Posts: 108
Thanks again for another pre-reason bug found...
  Reply With Quote
09/03/15, 08:24 AM   #12
merlight
AddOn Author - Click to view addons
Join Date: Jul 2014
Posts: 671
Updated dump as of 3 Sep 2015 http://ge.tt/3zjp9QK2/v/7
  Reply With Quote

ESOUI » Developer Discussions » Dev Tools » LuaEcoSearch - fighting global namespace pollution

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off