Help with metatables

[tilen.curin]

Hi,

I’d like to ask for some explanation for metatables in lua.

I’ve read a bunch of tutorials on this already, but nothing really worked for me. Only got me more doubts and confusion.

What I already know is that any table in lua can have a metatable, which can contain anything, but they respond to certain keys.

My first question would be about this syntax:

t = setmetatable({}, {})

I see almost everyone use this syntax. This is a “shorter” way to make a table and give it a metatable. I put shorter in quotes cause it only saves you 2 lines of code and to me it only looks confusing.

And how does it even make sense?

It looks like we stored a function in a variable and gave it 2 empty tables for arguments, the first one being the base table and the second one is the metatable. And I guess we don’t have to name the metatable, since if we need it we can get it with getmetatable(). Correct me if I’m wrong, please.

Next, I’m having my mind blown by operator keys like __mul or __add.

This for example is some random code I found somewhere on the internet:

t = setmetatable({ 1, 2, 3 }, { \_\_mul = function(t, other) new = {} for i = 1, other do for \_, v in ipairs(t) do table.insert(new, v) end end return new end }) t = t \* 2 -- { 1, 2, 3, 1, 2, 3 }

In the above example- what’s the point of using the multiplication operator if we don’t really multiply tables. We just defined what happens to tables once they’re passed as arguments and then return a completely new table. Couldn’t we have done exactly the same using the addition operator? Or any other operator? I can’t see the point of having so many operators.

And I guess I understand most other metatable keys…kinda.

For now I just need some help with the above things.

[StarCrunch]

Hi.

The function is no more than a distraction, in the first example (basically, built-in functions without a prefix tend to be useful things that can’t be done, at least not efficiently, through raw Lua, but which don’t quite amount to standalone library material), but not naming the metatable would certainly be one of the reasons to use that construction, yes. I would find it unusual to see an empty metatable, though, versus one such as in your second example. That said, just as often doing something like this will be about not naming the first argument, with the understanding that “it’s not ready” until after a metatable has been assigned. The first argument and the return value are one and the same; the return value is a convenience, nothing more. Some people, myself included, simply prefer to make things more concise, once we’ve internalized the language.

Heh, that is a peculiar use of __mul.  :) Operators are actually a big source of contention, in many programming languages, in fact probably anywhere such facilities are available. Look up “operator overloading” on your C++ forum of choice, for instance, and you’ll find a sharp divide between those who consider them useful and expressive, and as many others saying they obscure the structure and intent of one’s code. (A similar dilemma, being waged in the minds of the core developers no less, is the reason we’ll probably never see macros in Lua.)

One thing to keep in mind is that Lua wasn’t written with Corona specifically in mind, and thus some of its features may be more broad (or more narrow!) than what you find yourself needing in Corona. Speaking of metatables, for instance, few Corona-centric things will ever require weak tables, since "finalize"  event listeners are known to be available and tend to be a better fit. Going in the other direction, Lua’s original purpose was scientific work, where the whole panoply of operator-intensive math might more conveniently be driven through operators: for vectors, matrices (and we’re talking the 500-by-500 sort, not just some wussy 4-by-4 perspective matrix!), tensors, quaternions (standard, dual), geometric algebra, and even simple colors and such. Lately, operators have also shown up completely repurposed in stuff like LPEG. (Roberto did hint, once upon a time on the Lua mailing list, that he wanted to support custom operators, but thus far this hasn’t gone anywhere. Alas, I cannot find any of the threads where this was discussed. :()

[tilen.curin]

So by using __mul etc, we customize operators for operands of user-defined types?

We can define what will happen to the table if we use multiplication operator on it.

If that’s the case, we could program it to do anything. Not necessary multiply.

[StarCrunch]

Yep, although if you’re expecting to have other users it would be impolite to hijack them for something COMPLETELY different.  :D Typically you’ll want __mul for multiplication-ish behaviors, or problem domains which have already co-opted the symbol (or something resembling it); likewise for the rest of the arithmetic and relational operators.

Also worth noting is that all the arithmetic operators expect to be used in assignments, arguments, or return values (i.e. they make up expressions, not statements), the upshot of this being that the corresponding metamethods must return a value. This value may be one of your original inputs, and your “what will happen to the table” applies. But you’re equally free to return a new result, as in the example you posted.

There’s quite a variety to what can be done with __index and __newindex. Some great examples are found in Programming in Lua. (For that matter, the rest of the chapter, and the book proper, are great reads as well.)

[tilen.curin]

Alright, if I understand correctly, operator keys can only be used to store functions (as their values). These functions will be called when we use the corresponding operator with the table. First argument these functions receive is the target table and the second one is the value on the right hand side.

But all the operator keys can basically do anything. The only diffirence between them is the name and what operator they respond to.

This just seemed kinda wierd to me and made me think I misunderstood something.

Oh, and about __index and __newindex…those haven’t caused me as much confusion, but something related did: rawget() and rawset().

I know that they can be used to get or set values in tables without invoking metamethods, but I haven’t seen any good examples of how/ when to use them.

[StarCrunch]

Pretty close, as far as the function goes. When a “multiplication” happens, the left operand gets checked. If it has a __mul available, we call that. Otherwise, we make the same query of the right operand. (If that fails too, KABOOM!) In either case, when we call the function, the left operand goes into the first parameter, and the right one into the second.

If both operands are of the same type (two colors, say), and you can rely on that, this might not matter much. Otherwise, you’ll need some up-front work to know which argument is which (e.g. check that one is a table and has some known ID). From there you can swap them or stuff them into local variables or just spice up the rest of the function with tests or whatever. Some operations may change according the parameter type, too. A vector type, for instance, might reasonably be multiplied by a number (scaling), matrix (matrix-vector multiply), or another vector (cross product, dot product, outer product, or even component-wise multiply).

rawget () and rawset () are sort of “you’ll know it when you see it”, in my experience.  :P But as an example, consider something like this: automagic tables You might not know if a key is in use, but your very act of checking it has the side effect of filling it in. With raw access, you avoid that.

As far as rawset , this is a little artificial, but if you have a proxy table (mentioned in that Programming in Lua chapter), you could disable some of its keys by stuffing some value into them. I wish I had a better example off hand. 

[tilen.curin]

Cool, that cleared up a pretty big mess in my head. It looks like I’ll need some more experience with metatables to fully understand them, but at least I have a decent idea of how they work now.

I’ve been reading everywhere, that using metatables can make your lua programming a lot better, but I couldn’t picture an example where it would be good to implement them. Guess I’ll have to get to a point when I will actually have to use them. Or try to implement them into one of my projects.

I understand most other metamethods like __tostring or __call, so I think that’s all I needed to ask for now.

Anyways, thanks a lot for the explanation. I really appreciate your help!

[thomas6]

In my opinion - since you say you couldn’t picture an example where it would be good to implement metatables - the absolute best reason to implement is to create a sort of object oriented programming, where you can create one “prototype” object, and create instances of this object with the same properties, methods and behaviour.

I don’t know if you need an explanation or not, but the simplest way to imagine this is that, by setting metatables, you can create instances super-easily.

Image creating a table called Rex to represent a dog. Then you could give that table Rex a couple of keys that could be either properties like Rex.furColor = “black” or functions like Rex.bark = function() doSomething end

Now the cool thing is that you can now create a second table called PuppyPaws or whatever you want. And you can say to Lua: set the Metatable of PuppyPaws to Rex’s (to put it simply). This boils down to saying: hey Lua, if I call a key of PuppyPaws, and PuppyPaws does not have this key (i.e. this property or function) then look in Rex’s keys to see if Rex has a key with this name.

In practice, this means that you have created an instance, or another object (or table) based on Rex, but with a new name.

As an extra note: you may have seen people using “self” inside functions. The reason for this is exactly what I outlined above. If you don’t use “self” to describe the base-object’s name, some things could go wrong quickly. For example, if you would say:

Rex.printColor = function() print(Rex.furColor) end

and then setting PuppyPaws’ metaTable to Rex’s then you would have a problem, because calling PuppyPaws.printColor() would definitely find a function, but it would not print out PuppyPaws’ fur color, because the function just says “print Rex’s furColor”!

If you use “self” then that function would do: “print my own furColor”, wether or not it is called for Rex or PuppyPaws - or a million other dog objects, for that matter.

Voila. That’s the biggest importance to me. Took me a while to understand, I might add, but I am SUPER-happy to have mastered metatables, because I would not be able to make the games I make thanks to that knowledge!

[tilen.curin]

Great to see more people joining the discussion!

In your Rex example, the metatable would have an __index key? If our table doesn’t have the key that we’re looking, lua checks the metatable for __index key. Now if that one contains a table, lua checks that table for the key that we originally wanted.

If it contains a function, the key that we were looking for is passed as an argument along with our table and we can do a lot of things from there.

But what about other keys like __tostring? Haven’t seen any good examples.

Take a look at this code (from some tutorial I was reading):

t = setmetatable({ 1, 2, 3 }, { \_\_tostring = function(t) sum = 0 for \_, v in pairs(t) do sum = sum + v end return "Sum: " .. sum end }) print(t) -- prints out "Sum: 6"

Did __tostring do anything? Sum gets concatenated with a string, so it would be converted to a string anyways.

What’s the point of having a metatable and messing around with __tostring key when we can get the exact same result with a simple for loop:

myTable = {1, 2, 3} sum = 0 for k, v in pairs(myTable) do sum = sum + v end print("Sum " .. sum) --we can concatenate a number with a string to convert it to a string

…And it’s a lot more straightforward.

Every example I’ve seen has been something useless and every time there was a way around.

[StarCrunch]

It’s really your call whether to go through __tostring or just write individual routines as the need arises. I think a motivation for both it and print () was to complement stuff like the command line interpreter (available with the Lua source code, but usually not bundled with stuff like Corona) or the live demo, which obviously aren’t going to know about your routines. But it’s only a library metamethod; nothing in the language proper would break if you removed  tostring () and print ().

If you had objects of several different types in an array or whatever, rather than figuring out what each one is and calling its special logic, you can just call print () on each one blindly and let it do its thing.

(In 5.2 and on, pairs() itself honors a library metamethod. But Corona uses 5.1, so this is merely a curiosity.)

A plain old loop and printing each element can fall short, at times. If the table isn’t shallow, for instance, which is to say some or all of its elements are themselves tables, with their own elements. You can recurse or use a stack to handle this, but then you need to account for cycles. At this point, you’re also likely to notice some structure to your data, so you might start to indent it and try to draw pretty ASCII art.  :D Also, if you have large data sets, your printouts might end up flooding the console (which will probably confuse rather than help), and so you’ll start adding features to stagger, abridge, or reroute the output. And so on. These kind of things call for a lot of work, which you won’t want to throw away.

Once that stuff is available, it might be convenient to wire it into some of your objects from the outset, and  __tostring is right there…

[thomas6]

@ tilen.curin: yes, you would be using the __index key, naturally. I used “shorthand” in my example because it is only about the principles, not the syntax.

Regarding your “Every example I’ve seen has been something useless and every time there was a way around.” I do believe that there is no other way to do OOP-like instancing that is this powerful or flexible without using metatables.

[tilen.curin]

I’ll try to connect everything I’ve read so far and try out a few things.

Really want to get comfortable using metatables. Guess I’ll have to learn through experience.

I pretty much get how they work, but I’ll need to actually start working with them.

[StarCrunch]

Hi.

The function is no more than a distraction, in the first example (basically, built-in functions without a prefix tend to be useful things that can’t be done, at least not efficiently, through raw Lua, but which don’t quite amount to standalone library material), but not naming the metatable would certainly be one of the reasons to use that construction, yes. I would find it unusual to see an empty metatable, though, versus one such as in your second example. That said, just as often doing something like this will be about not naming the first argument, with the understanding that “it’s not ready” until after a metatable has been assigned. The first argument and the return value are one and the same; the return value is a convenience, nothing more. Some people, myself included, simply prefer to make things more concise, once we’ve internalized the language.

Heh, that is a peculiar use of __mul.  :) Operators are actually a big source of contention, in many programming languages, in fact probably anywhere such facilities are available. Look up “operator overloading” on your C++ forum of choice, for instance, and you’ll find a sharp divide between those who consider them useful and expressive, and as many others saying they obscure the structure and intent of one’s code. (A similar dilemma, being waged in the minds of the core developers no less, is the reason we’ll probably never see macros in Lua.)

One thing to keep in mind is that Lua wasn’t written with Corona specifically in mind, and thus some of its features may be more broad (or more narrow!) than what you find yourself needing in Corona. Speaking of metatables, for instance, few Corona-centric things will ever require weak tables, since "finalize"  event listeners are known to be available and tend to be a better fit. Going in the other direction, Lua’s original purpose was scientific work, where the whole panoply of operator-intensive math might more conveniently be driven through operators: for vectors, matrices (and we’re talking the 500-by-500 sort, not just some wussy 4-by-4 perspective matrix!), tensors, quaternions (standard, dual), geometric algebra, and even simple colors and such. Lately, operators have also shown up completely repurposed in stuff like LPEG. (Roberto did hint, once upon a time on the Lua mailing list, that he wanted to support custom operators, but thus far this hasn’t gone anywhere. Alas, I cannot find any of the threads where this was discussed. :()

[tilen.curin]

So by using __mul etc, we customize operators for operands of user-defined types?

We can define what will happen to the table if we use multiplication operator on it.

If that’s the case, we could program it to do anything. Not necessary multiply.

[StarCrunch]

Yep, although if you’re expecting to have other users it would be impolite to hijack them for something COMPLETELY different.  :D Typically you’ll want __mul for multiplication-ish behaviors, or problem domains which have already co-opted the symbol (or something resembling it); likewise for the rest of the arithmetic and relational operators.

Also worth noting is that all the arithmetic operators expect to be used in assignments, arguments, or return values (i.e. they make up expressions, not statements), the upshot of this being that the corresponding metamethods must return a value. This value may be one of your original inputs, and your “what will happen to the table” applies. But you’re equally free to return a new result, as in the example you posted.

There’s quite a variety to what can be done with __index and __newindex. Some great examples are found in Programming in Lua. (For that matter, the rest of the chapter, and the book proper, are great reads as well.)

[tilen.curin]

Alright, if I understand correctly, operator keys can only be used to store functions (as their values). These functions will be called when we use the corresponding operator with the table. First argument these functions receive is the target table and the second one is the value on the right hand side.

But all the operator keys can basically do anything. The only diffirence between them is the name and what operator they respond to.

This just seemed kinda wierd to me and made me think I misunderstood something.

Oh, and about __index and __newindex…those haven’t caused me as much confusion, but something related did: rawget() and rawset().

I know that they can be used to get or set values in tables without invoking metamethods, but I haven’t seen any good examples of how/ when to use them.

[StarCrunch]

Pretty close, as far as the function goes. When a “multiplication” happens, the left operand gets checked. If it has a __mul available, we call that. Otherwise, we make the same query of the right operand. (If that fails too, KABOOM!) In either case, when we call the function, the left operand goes into the first parameter, and the right one into the second.

If both operands are of the same type (two colors, say), and you can rely on that, this might not matter much. Otherwise, you’ll need some up-front work to know which argument is which (e.g. check that one is a table and has some known ID). From there you can swap them or stuff them into local variables or just spice up the rest of the function with tests or whatever. Some operations may change according the parameter type, too. A vector type, for instance, might reasonably be multiplied by a number (scaling), matrix (matrix-vector multiply), or another vector (cross product, dot product, outer product, or even component-wise multiply).

rawget () and rawset () are sort of “you’ll know it when you see it”, in my experience.  :P But as an example, consider something like this: automagic tables You might not know if a key is in use, but your very act of checking it has the side effect of filling it in. With raw access, you avoid that.

As far as rawset , this is a little artificial, but if you have a proxy table (mentioned in that Programming in Lua chapter), you could disable some of its keys by stuffing some value into them. I wish I had a better example off hand. 

[tilen.curin]

Cool, that cleared up a pretty big mess in my head. It looks like I’ll need some more experience with metatables to fully understand them, but at least I have a decent idea of how they work now.

I’ve been reading everywhere, that using metatables can make your lua programming a lot better, but I couldn’t picture an example where it would be good to implement them. Guess I’ll have to get to a point when I will actually have to use them. Or try to implement them into one of my projects.

I understand most other metamethods like __tostring or __call, so I think that’s all I needed to ask for now.

Anyways, thanks a lot for the explanation. I really appreciate your help!

[thomas6]

In my opinion - since you say you couldn’t picture an example where it would be good to implement metatables - the absolute best reason to implement is to create a sort of object oriented programming, where you can create one “prototype” object, and create instances of this object with the same properties, methods and behaviour.

I don’t know if you need an explanation or not, but the simplest way to imagine this is that, by setting metatables, you can create instances super-easily.

Image creating a table called Rex to represent a dog. Then you could give that table Rex a couple of keys that could be either properties like Rex.furColor = “black” or functions like Rex.bark = function() doSomething end

Now the cool thing is that you can now create a second table called PuppyPaws or whatever you want. And you can say to Lua: set the Metatable of PuppyPaws to Rex’s (to put it simply). This boils down to saying: hey Lua, if I call a key of PuppyPaws, and PuppyPaws does not have this key (i.e. this property or function) then look in Rex’s keys to see if Rex has a key with this name.

In practice, this means that you have created an instance, or another object (or table) based on Rex, but with a new name.

As an extra note: you may have seen people using “self” inside functions. The reason for this is exactly what I outlined above. If you don’t use “self” to describe the base-object’s name, some things could go wrong quickly. For example, if you would say:

Rex.printColor = function() print(Rex.furColor) end

and then setting PuppyPaws’ metaTable to Rex’s then you would have a problem, because calling PuppyPaws.printColor() would definitely find a function, but it would not print out PuppyPaws’ fur color, because the function just says “print Rex’s furColor”!

If you use “self” then that function would do: “print my own furColor”, wether or not it is called for Rex or PuppyPaws - or a million other dog objects, for that matter.

Voila. That’s the biggest importance to me. Took me a while to understand, I might add, but I am SUPER-happy to have mastered metatables, because I would not be able to make the games I make thanks to that knowledge!

[tilen.curin]

Great to see more people joining the discussion!

In your Rex example, the metatable would have an __index key? If our table doesn’t have the key that we’re looking, lua checks the metatable for __index key. Now if that one contains a table, lua checks that table for the key that we originally wanted.

If it contains a function, the key that we were looking for is passed as an argument along with our table and we can do a lot of things from there.

But what about other keys like __tostring? Haven’t seen any good examples.

Take a look at this code (from some tutorial I was reading):

t = setmetatable({ 1, 2, 3 }, { \_\_tostring = function(t) sum = 0 for \_, v in pairs(t) do sum = sum + v end return "Sum: " .. sum end }) print(t) -- prints out "Sum: 6"

Did __tostring do anything? Sum gets concatenated with a string, so it would be converted to a string anyways.

What’s the point of having a metatable and messing around with __tostring key when we can get the exact same result with a simple for loop:

myTable = {1, 2, 3} sum = 0 for k, v in pairs(myTable) do sum = sum + v end print("Sum " .. sum) --we can concatenate a number with a string to convert it to a string

…And it’s a lot more straightforward.

Every example I’ve seen has been something useless and every time there was a way around.