Plugins: impack and Bytemap

Corona Marketplace Marketplace Announcements
#21

Going on the “Maybe this would be better served with some C++ techniques, making reclamation deterministic.” hunch, I spent the afternoon playing around with another sort of allocator, a hybrid between a small arena (for small allocations) and the heap. It was interesting to debug, to say the least. The APIs that underlie impack do indeed perform a ton of allocation, and in particular re allocation. I don’t think my previous design fared very well in light of the latter.

I’ll try to push this in the next day or two, after stressing it a bit.

#22

Ok, no haste on my part. I’ve moved on to an OpenGL-based solution that works ok (stable, but not perfect looking scaling). I also discovered some speed issues.

The scaling of impack worked well (when not crashing) on my Nexus 6P, The result looked nice and the speed wasn’t too bad (ca 1 second). This is native camera resolution -> 800x800 scaling.

But on older phones the scaling could take 5-6 seconds and that came as a small shock to me to be honest. I was hoping that the smaller native resolution on older phones would in some ways would make up for the weaker processor, but that didn’t quite play out the way I had hoped. Not sure if there is anything to be done with this though. Scaling an image is not the best for single threaded CPU processes. 

#23

This is awesome. Quick question (not having read more than the post above): Can SVG’s be loaded in at any size?

#24

Thanks.  :slight_smile:

A few months back I tried (though using stock nanosvg, sans r-lyeh’s updates) to load the SVG found here, which is reasonably large: luarocks dependency graph

It succeded, but was pretty slow. (No text support either, alas.) Do cases as large as that matter? As time goes on, I figure that will become more clear, and whether I should attempt optimizations and such.

The Spot stuff was a rather late addition and still needs some streamlining with the overall API, working into the sample, etc. I’m hoping to start making some improvements once the iOS8 policy changes have rolled out, since these will also bring libc++ support along with the Accelerate framework.

#25

“First and foremost, it lets you load and save images, in several more formats than just PNG and JPEG.”

This is slightly inaccurate as it’s not possible to SAVE to JPGs, or is it? I bought the plugin and to my surprise (I should have read more carefuly) it doesn’t seem to support JPG writes.

Is this somehting in the works or have I misunderstood and it already supports JPG writes?

#26

Hi.

I was a bit careless in writing that up. In my mind JPEG-saving was indeed located in the write submodule.  :slight_smile:

Assuming no bugs, you should be able to do it via a Spot image’s save method. Something like:

local image = impack.image.load\_image\_object("elephant.tga") local color = impack.image.new\_color\_rgba(55, 0, 0, 128) image:add\_mutate(color) image:save("tinted\_elephant.JPG")

I realize this is a bit unintuitive. The stuff in image and write  mostly consist of bits from stb, whereas the Spot parts come from the eponymous library. (To make matters more confusing, Spot itself uses many of those same components from stb.) In time I’d like to streamline some of this.

I need to update the samples with some of the Spot stuff eventually. Maybe sooner than later; with the new release, I can now assume everybody is adhering to the new iOS minimum versions and using libc++, which will let me unsnarl all sorts of nastiness (some of this is alluded to in the docs). Just have to get around to it.  :slight_smile:

#27

Hmm…ok, if a jpg write is possible, that’s surely good news for me. The one and only thing I need to do is take one jpg file, resize it (scale down (typically from the high res camera image to something manageable, like 400x300 pixels) and save it to another jpg file.

Is there a resize/scale down function in impack? The seems to be something called image.shrink() and image.scale() in eponymous library, but I’m not sure if they are supported by impack

I thought the impack.write.png() method had a built in scaling, but I never got it to work so I’m not sure:

local ok = impack.write.png(“resizedImage.png”, system.DocumentsDirectory, w, h, comp, data)

Can you show me how to do the following?

  1. Load the image “aquired,jpg” (from system.TemporaryDirectory)

  2. Scale it to 300x400

  3. Save it as “resized.jpg” (in system.DocumentsDirectory)

Thanks!

-Rune

#28

Hi.

The width and height (and component count) are just a means of the API to make sense of the image bytes. They’re a pretty straight port from the stb interface. There’s not enough context in them to allow for scaling.

There are resize operations, however, e.g. this one might be good for JPEG images. (Looks like that has a doc bug, as it’s known to only have the three channels.) You can get some sense of how they’re used in the plugin sample.

Unfortunately I do seem to have overlooked adding a method to populate Spot images straight from bytes. I’ll see about addressing that. At the moment you could probably save your scaled bytes to an in-memory PNG and then reload that as a Spot image, then save; or make a dummy color, convert it to RGBA mode, then iterate the bytes three at a time, set the color’s “hsl” properties, and assign it to the current pixel. These are pretty hacky solutions though.  :slight_smile: Nevertheless I’ll see if I can’t throw one or the other together after I wake up tomorrow (very late here now).

I could probably incorporate some of this under the hood in the image and write submodules to add the missing file types. There would still be some minor discrepancies, but probably better than nothing.

#29

I certainly hope you can do something to make it a bit more intuitive, because the above way of doing it seems to be only slightly easier than writing the scaling code myself  :blink: .

Eagerly awaiting!

-Rune

#30

Heh, yeah, I figured.  :D That said, those would probably be decent test cases on my end.

Anyhow, I’ve added some stuff, though I’m not able at the moment to sign in to the remote machine where I do my plugins’ non-Windows builds (there was a rather loud thunderstorm a couple hours ago…). I’ll try to do them all when I head over to the office tomorrow, perhaps with some further refinements, and push them soon after. I’ll let you know.

As for the changes, basically, after doing something like the following:

local data, w, h, bpp = impack.image.load("turtle.bmp") local w2, h2 = 400, 300 local result = impack.ops.resize\_custom(data, w, h, w2, h2, bpp)

you can then do either:

impack.write.jpg("resized.jpg", w2, h2, bpp, result)

or the more roundabout

local image = impack.image.new\_image\_object(w2, h2) local method = bpp == 3 and "set\_from\_bytes\_rgb" or "set\_from\_bytes\_rgba" -- should these be unified with a -- comp parameter? image[method](image, 1, 1, result) image:save(system.pathForFile("resized\_by\_spot.JPG", system.DocumentsDirectory)) -- todo: need to wire paths -- into the Spot methods

Obviously the first is preferable here. However, now that the latter is possible, it should be fairly easy to route any additional formats through Spot, with some minor overhead.

#31

The first method looks great!

I’ll check it out as soon as I get word that it’s ready!

Thanks!

-Rune

#32

If I haven’t botched anything, this should now be available. Let me know if anything’s awry.

#33

Ah… I interpreted the below as the new stuff not being ready:

“Anyhow, I’ve added some stuff, though I’m not able at the moment to sign in to the remote machine where I do my plugins’ non-Windows builds (there was a rather loud thunderstorm a couple hours ago…). I’ll try to do them all when I head over to the office tomorrow, perhaps with some further refinements, and push them soon after. I’ll let you know.”

But thruth to be said, I really didn’t understand what you meant…

#34

Do you use a .bmp as input in the example because it’s not going to work with .jpgs? I’m thinking specifically about the 6th parameter in the resize_custom function which is an output from the image.load() function when loading .bmp images.

Am I wrong in guessing that the forth return value from image.load() is bpp (bits per pixel) when loading .BMP files and comp (copression value) when loading .JPG files?

In that case should I just hard write 24 as the bpp value in resize_custom() when I have loaded a .JPG?

Edit: I see now that “comp” means “image components”, but not sure what that is for .JPGs.

#35

OK, I think I’ve get it to work properly now. It seems to work with JPGs in the same way aswith BMPs.

Thanks!

#36

Okay, great.

There wasn’t anything special about loading a BMP. I simply had the example on hand, ready to adapt.  :slight_smile:

For consistency I ought to use “comp” rather than “bpp”. I’ll keep that in mind when I get a chance to clean things up.

I’m not sure I’ve run across a 4-component JPEG in the wild, but you can force-load it that way. In theory, this might let you get some benefit by using  impack.ops.resize_rgba , but if performance is fine I wouldn’t worry.

The interface for impack.write.jpg is indeed largely the same, although it can also take an options table as the final parameter. Implicitly, this is the call that’s happening if you don’t provide your own:

impack.write.jpg("resized.jpg", w2, h2, comp, result, { quality = 90 })

where quality is the JPEG quality between 0 and 100.

#37

OK, have been testing this for some hours now and unfortunately the app seems to crash now and again (as far as I can tell somewhere inside the scaling). There is no logging so it’s difficult to tell. I’ll see if I can do some more logging on my side to isolate the crash better.

#38

I’m inclined to trust the code that underlies the resize API, though I notice it does do a memory allocation, as would the resize itself. This might be conflicting with some things I did in what would then turn out to be a failed attempt to gracefully handle out-of-memory errors… basically allocating through Lua, though this could pile up (with all those 400 x 300 x 3 images, which might actually be doubled and change) if you’re doing several in a row per frame and not aggressively calling collectgarbage(). Maybe this would be better served with some C++ techniques, making reclamation deterministic.

What platform(s) is it failing on? Do you get an error message, or is it a hard crash? Any different if you surround calls like the resize in assert()?

#39

The app has crashed again on an Android 7.1 device. I log before and after each call to impack functions and 

the last logging is right before the call to impack.image.load().

The code looks like this:

 utils.LogTime("Calling impack.image.load()", composer.state.startLogTime) local data, w, h, comp = impack.image.load("aquiredImage.jpg", system.TemporaryDirectory, 0) utils.LogTime("Called impack.image.load()", composer.state.startLogTime)

And the last bit of logging looks like this:

04-21 13:26:35.608 32081 980 I Corona : photoAquired() 04-21 13:26:35.608 32081 980 I Corona : startLogTime = 127685.759 04-21 13:26:35.608 32081 980 I Corona : Delta: 0.033999999999651 : Calling impack.image.load()

I do “stress the system” by repeatedly loading/rescaling/saving images, but certainly not several in a frame (if you mean screen redraw frame), not even several in a second as the loading of the image takes about 700ms and the scaling about 670 ms. The writing only takes about 100ms or so. Anyway, I also have to take the image with the camera or select it from the image library on the device. All in all it takes several seconds between each scaling.

#40

I forgot about this one. Oh, it’s certainly what you would call a hard crash. The only message I get is from Android that “the app has crashed”. No reasons or explanations. And the log just stops abruptly.

Not sure how to use assert() with impack.image.load(). The app crashes before I get the change to use any of its return values.