LibConstantMapper

[zgrssd]

While trying to figure out wich unambigious Chat Category Constant name relates to wich category I ran into the issue that the Constant Names are the keys in the global table. They can be identified via Pattern Matching, but they are not a subtable that you can easily itterate over.

Doing excessive pattern matching is bad for performance. Also there are other similary organised groups of values (like the Chat_Channels or all the Event ID's) you might want to find an easy match for.
So I started development on this library. It consists of two files:
mappings.lua

ConstantMapperMappings = {
    { mapping = "ChatCategories", pattern = "^CHAT_CATEGORY_", exclude = "^CHAT_CATEGORY_HEADER" }
}

Just a table. It contains the mapping (wich Key the table will be saved under and will be requestable), the pattern to match (in this case for chat categories) and a pattern to not include (in this case I want all the entries starting with "CHAT_CATEGORY_", except the ones starting wiht "CHAT_CATEGORY_HEADER").
If I also want the chat channels using this library, I just have to modify it like this:

ConstantMapperMappings = {
    { mapping = "ChatCategories", pattern = "^CHAT_CATEGORY_", exclude = "^CHAT_CATEGORY_HEADER" },
    { mapping = "ChatChannels", pattern = "^CHAT_CHANNEL_"}
}

main.lua

local Mappings = ConstantMapperMappings
local data = { }
 
--Make certain data contains a table for each mapping
for i=1, #Mappings, 1 do
    local currentMap = Mappings[i]
    
    if(data[currentMap.mapping] == nil) then
        data[currentMap.mapping] = {}
    end
end
 
--For every entry in the Global table
for key, value in zo_insecurePairs(_G) do
    --Compare it to every mapping
    for i=1, #Mappings, 1 do
        local currentMap = Mappings[i]
        
        if (key):find(currentMap.pattern) and (currentMap.exclude == nil or not(key):find(currentMap.exclude)) then
            --found a Value for this mapping, so store it in the table
            local newentry = { key = key, value = value}
            
            table.insert(data[currentMap.mapping], newentry)
        end
    end
end

I go over all entries in the global table. Compare each entry to each mapping. And if it matches, I store the value as a subtable in Data.
That way I get a easy to itterate over table containing wich ConstantName matches wich Category.

Going over the whoel global table is a lot faster then I anticipated so maybe I switch to a "retreive on demand, but cache" approach instead.

Of course the whole thing will use libStub to avoid multiple execution in the final version. I just wanted to ask if there are any mistakes in this code I have to look out for.

[merlight]

Some suggestions.

a) either allow more exludes (make it a table) to make things like this work (because Lua's regular expressions are so weak they can't accomplish this in one go):

{ mapping = "Stats", pattern = "^STAT_", exclude = {"^STAT_BONUS_OPTION_", "^STAT_SOFT_CAP_OPTION", "^STAT_VALUE_COLOR"} }

b) or replace excludes with a non-mapping...

{ mapping = false, pattern = "^CHAT_CATEGORY_HEADER_" }, -- non-mapping
{ mapping = "ChatCategories", pattern = "^CHAT_CATEGORY_" },
{ mapping = "StatBonusOptions", pattern = "^STAT_BONUS_OPTION_" },
{ mapping = "StatSoftCapOptions", pattern = "^STAT_SOFT_CAP_OPTION_" },
{ mapping = false, pattern = "^STAT_VALUE_COLOR" }, -- non-mapping
{ mapping = "Stats", pattern = "^STAT_" },

...and stop after the first match

--For every entry in the Global table
for key, value in zo_insecurePairs(_G) do
    --Compare it to every mapping
    local foundMap = nil
    for _, currentMap in ipairs(Mappings) do
        if key:find(currentMap.pattern) then
            foundMap = currentMap
            break -- stop at the first match
        end
    end
    if foundMap and foundMap.mapping then
            --found a Value for this mapping, so store it in the table
            local newentry = { key = key, value = value}
            table.insert(data[foundMap.mapping], newentry)
    end
end

Also it would be nice to have direct [value] => "key" mapping in addition to the table of pairs (I'm aware there might be collisions and the library would have to deal somehow, that's why wrote in addition to )

[Garkin]

Some more suggestions:

local MAJOR, MINOR = "LibConstantMapper-1.0", 1
 
local lib, oldminor = LibStub:NewLibrary(MAJOR, MINOR)
if not lib then return end
 
local mappingsTable = {
   ["ACTION_TYPE"] = { exclude = "DEPRECATED" },
   ["ATTRIBUTE"] ={ exclude = "ATTRIBUTE_.+_" },
   ["CHAT_CATEGORY"] = { exclude = "^CHAT_CATEGORY_HEADER" },
   ["CURRENCY_CHANGE_REASON"] = {},
   ["DAMAGE_TYPE" = {},
   ["ITEM_QUALITY"] = {},
   ["ITEM_TRAIT_TYPE"] = {},
   ["ITEMSTYLE"] = { exclude = "DEPRECATED" },
   ["ITEMTYPE"] = {},
   ["LINK_TYPE"] = { pattern = "_LINK_TYPE$" }
   ["STAT"] = { exclude = {"^STAT_BONUS_OPTION_", "^STAT_SOFT_CAP_OPTION", "^STAT_VALUE_COLOR"} }
}
 
--do not overwrite existing data
lib.data = lib.data or {}
 
--internal functions
function lib:AddData(mapping, pattern, exclude)
   for name, value in zo_insecurePairs(_G) do
      if type(mapping) == "table" then
         for mappingName, mappingTable in pairs(mapping) do
            lib:SaveDataEntry(name, value, mappingName, mappingTable.pattern, mappingTable.exclude)
         end
      elseif type(mapping) == "string" then
         lib:SaveDataEntry(name, value, mapping, pattern, exclude)
      end
   end
end
 
function lib:SaveDataEntry(name, value, mapping, pattern, exclude)
   if (name):find(pattern or "^"..mapping.."_") == nil then return end
 
   local skip = false
   if exclude then 
      if type(exclude) == "table" then
         for i,v in ipairs(exclude) do 
            if (name):find(v) then
               skip = true
               break
            end
         end
      else
         skip = (name):find(exclude) ~= nil
      end
   end
   
   if not skip then 
      lib.data[mapping] = data[mapping] or {}
      lib.data[mapping][name] = value
   end
end
 
if oldminor then
   --library update, add only new/changed items?
   local newItems = {}
   for name, value in pairs(mappingsTable) do
      if value ~= lib.mappings[name] then
         newItems[name] = value
         lib.mappings = value
      end
   end
   lib:AddData(newItems) 
else
   lib.mappings = mappingsTable
   lib:AddData(mappingsTable)
end
 
 
--external functions
function lib:AddMapping(mapping, pattern, exclude)
   --add or replace enxisting entry
   if type(mapping) == "table" then
      for name, value in pairs(mapping) do
         lib.mappings[name] = value
      end
   else  
      lib.mappings[mapping] = { ["pattern"] = pattern, ["exclude"] = exclude }
   end
 
   lib:AddData(mapping, pattern, exclude)   
end
 
function lib:GetConstants(mapping)
   return lib.data[mapping]
end

1) Mapping can be part of the name and pattern is defined only in special cases
2) Do not create a new global table ConstantMapperMappings, just make it accessible using the library reference
3) As merlight said, it would be nice to have direct [value] => "key" mapping, so I changed it that way

[zgrssd]

Originally Posted by merlight Some suggestions. a) either allow more exludes (make it a table) to make things like this work (because Lua's regular expressions are so weak they can't accomplish this in one go): { mapping = "Stats", pattern = "^STAT_", exclude = {"^STAT_BONUS_OPTION_", "^STAT_SOFT_CAP_OPTION", "^STAT_VALUE_COLOR"} } b) or replace excludes with a non-mapping... [highlight="Lua"] [...] Also it would be nice to have direct [value] => "key" mapping in addition to the table of pairs (I'm aware there might be collisions and the library would have to deal somehow, that's why wrote in addition to )

I want the same Key to be able to pop up in multiple mappings (i.e. Events appareing ina big "all events" mapping as well as sublist for each class like guild), so non-mapping is out of the question.

The final structure for mappings might looks something like this:
1 Mapping (the mapping name)
-> with 1...N patterns to match
-> each pattern with 0...M exclusions

But that is pretty far in the future. First things first.

Originally Posted by Garkin 1) Mapping can be part of the name and pattern is defined only in special cases 2) Do not create a new global table ConstantMapperMappings, just make it accessible using the library reference 3) As merlight said, it would be nice to have direct [value] => "key" mapping, so I changed it that way

1) Due to the long term plans, a mapping might be able to have multiple patterns. Combinign them would just make the code mroe complex
2) The global table was only a temporary solution for my first trials. Currently I actually tend towards mappings being registered at runtime the same way you register media with LibMedia or libaries with LibStub. And only being processed on demand. if so the mappings.lua file will propably run after the actuall library and just call the register function in a loop.
3) My original idea was a [value] => "key" mapping. But that would cause issues if I ever end up with the same value more then once (the whole Chat Cat header issues pointed me at that).
The output tables are designed to be itterable via # operator, no use of "pairs" needed. But taht is only personal preference of int indexes over string indexes.
Not using the value as key also means I can accomodate non-key worthy values. For example, afaik "" is a valid value. But not a valid index. BugEaters inability to deal with nil and "" values pointed me towards that issue.
A simple int indexed array of key&value allows a lot more ways to process the data.

Edit (too 3):
The more I think about it, the better my idea sounds. The global table is full of values that would never make a valid index for a table: empty stings, functions, other tables - non of that could ever make a proper index. Plus sometimes the same function/table might be used as value multiple keys (so I had the same index used multiple times).
Clear index-worthy int and string values seem to be a odd thing in the global table, rather then a common one.

[merlight]

Originally Posted by zgrssd For example, afaik "" is a valid value. But not a valid index. BugEaters inability to deal with nil and "" values pointed me towards that issue.

Anything but nil is a valid index (you can even use another table as key). In SavedVars only int/string/boolean keys are allowed, but that is SavedVars limitation, not Lua limitation. Empty string is a valid key in both.

Skip down to *CONSTRUCTIVE* if you only want something constructive, this below is just rant

Originally Posted by zgrssd A simple int indexed array of key&value allows a lot more ways to process the data.

But not the one way I find most convenient - quickly answering the question "what's the symbolic name of this value?".

Originally Posted by zgrssd Edit (too 3): The more I think about it, the better my idea sounds. The global table is full of values that would never make a valid index for a table: empty stings, functions, other tables - non of that could ever make a proper index.

I don't think I would ask the name of anything but integer constants (except when I was writing a function to dump _G, I used some heuristics to give names to metatables, and that was when I used [{table}] => "string" mapping; but that's something completely out of scope of this library, I guess).

Originally Posted by zgrssd Plus sometimes the same function/table might be used as value multiple keys (so I had the same index used multiple times). Clear index-worthy int and string values seem to be a odd thing in the global table, rather then a common one.

Look at it this way. You give me an array of pairs. I want to know the name of a thing. So I'll write this function:

local function nameIt1(value, mapping)
    for i, m in ipairs(mapping) do
        if value == m.value then
            return m.key
        end
    end
end

Someone else might instead do it this way:

local function nameIt2(mapping, value)
    local name = nil
    for i, m in ipairs(mapping) do
        if value == m.value then
            name = m.key
        end
    end
    return name
end

nameIt1 returns the first key with matching value, nameIt2 returns the last. Not to mention that the order of values in mapping depends on the order in which you get them from zo_insecurePairs! Which may be different every time you /reloadui, so addons using any nameIt function might behave strangely sometimes. I know it's hard do deal with conflicts properly, but if it's done in a library, at least it can be done consistently.


*CONSTRUCTIVE*

You could:
1) Add type requirement to your mapping spec. For example, "only int values allowed", everything else thrown away.
2) Sort your data={{value=5,key="foo"}, {value=5,key="bar"}} pairs table on value and key (sorting elements with equal values on key is important if you want consistent search results). Or zo_binaryinsert right away, instead of table.insert's and table.sort afterwards.
3) Provide a lookup function which would zo_binarysearch for given value in the sorted data.

[zgrssd]

Originally Posted by merlight But not the one way I find most convenient - quickly answering the question "what's the symbolic name of this value?".

If you want it done differently, you can always do it differently.

My experience tells me that a int-indexed, properly formed array consisting of key/value tables offers the best overall usage while still having acceptable looping performance.
Looping over the array with # and comparing a string to "something" is going to have the same performance as asking the system for the value to key "something". Because all those tables are "under the hood" are one List without allowed duplicates and one with allowed duplicates.

Also I cannot guarantee that any value will be there only once. That is simply a matter of how the data looks and the patterns/exclusiong are set.
Better ending up with the same key twice in the array (easy to diagnose) then having sometimes the one key in use, sometimes the other (hard to diagnose).