Tar Module - Extract tar files with Corona

[Danny]

Hey guys, I am aware that loads of you have been looking for a way to extract archives with Corona for a long time now.

I am planning on implementing more archive support for corona in C/objective c but more info on that later.

So for now, here is a solution done in pure lua.

From benchmarking it took 1.3 seconds to extract a 10mb tar archive.

I created the test tar archive using a mac app named “betterZip” however any archiving tool should be able to do this for you.

Notes :

1. It only supports uncompressed tar archives (.tar).
2. It doesn’t support folders inside the archive.

So it won’t save you space, but it will help you break past app size barrier limits. For instance the 50mb limit on the android market can be overcome by downloading your assets individually from a server. Obviously if you have thousands of images this is going to be tiresome, so this way you can pack them into one file and download that single file and extract it where it’s needed.

The sample includes the tar module and also a sample demonstrating how to use it (just one line of code )

Enjoy

Update :

Changes :

1. Minor optimizations, even faster extraction
2. Added a onComplete handler that you can pass a function to upon successful extraction of the archive.

http://www.infuseddreams.com/apps/Tar_Lib.zip
[import]uid: 84637 topic_id: 23242 reply_id: 323242[/import]

[bryan.vaccaro]

Danny, this is good news to hear after I just benchmarked another pure lua unzip module at 15 seconds for a 2 meg file!
One of my particular use cases involves extracting and reading one file from a tar/gzip that contains thousands of files without decompressing the entire archive… do you think that will be possible under the native C implementation? [import]uid: 120 topic_id: 23242 reply_id: 93040[/import]

[Danny]

Yeah of course, i have no eta on the c implementation but i will provide info as soon as i have it.
It might also be possible with what i created here, il look into it

[import]uid: 84637 topic_id: 23242 reply_id: 93043[/import]

[john_c]

doesn’t seem to work. i keep getting “Invalid header magic” error.

i created the tar archive using 7zip. [import]uid: 71024 topic_id: 23242 reply_id: 93106[/import]

[Danny]

Hey jon.

Just to confirm.
Your file is a uncompressed .tar file and it doesnt contain any folders?

If it is like i said above please post up your tar file.

Did you try it with the sample tar file i included? [import]uid: 84637 topic_id: 23242 reply_id: 93109[/import]

[Danny]

Can you guys try with winrar? They have a free trial.

As i said i dont have a windows machine and made my archives using betterzip which i presume is mac only.
It sounds like 7zip is writing the headers differently.

Try with winrar for me and if it still doesnt work il install windows on a virtualbox and work this out for you guys [import]uid: 84637 topic_id: 23242 reply_id: 93114[/import]

[Danny]

Can you guys try with winrar? They have a free trial.

As i said i dont have a windows machine and made my archives using betterzip which i presume is mac only.
It sounds like 7zip is writing the headers differently.

Try with winrar for me and if it still doesnt work il install windows on a virtualbox and work this out for you guys [import]uid: 84637 topic_id: 23242 reply_id: 93115[/import]

[Danny]

Ezraanderson : is that with my sample ran unmodified? [import]uid: 84637 topic_id: 23242 reply_id: 93116[/import]

[ezraanderson1979]

i think winrar can only uncompress tar, i don’t think it can write them.

i will try it on my mac.

@danny

your example works fine on a mac

7-zip compressed file doesn’t work on a mac, same error as above [import]uid: 89663 topic_id: 23242 reply_id: 93117[/import]

[ezraanderson1979]

yes unmodified just spacing between functions

–*******************************
–i do this between my functions [import]uid: 89663 topic_id: 23242 reply_id: 93118[/import]

[Danny]

Weird, so my example works on mac but not on windows. It definitely works on IOS also.

This must be a simulator thing or something else. I will install virtualbox and windows tomorrow and fix this for you windows users.

My apologies for this… [import]uid: 84637 topic_id: 23242 reply_id: 93119[/import]

[john_c]

yes, the tar file is uncompressed and does not contain any folders. where should i post the tar file? it is also rather large (~75 mb)…

also, untarring test.tar results in “attempt to perform arithmetic on a nil value” [import]uid: 71024 topic_id: 23242 reply_id: 93120[/import]

[Danny]

@john, there seems to be some issues with the windows version of the simulator and my module.

I am looking into fixing them asap, in the mean time if you have a mac, mac and ios are confirmed working.

Sorry for any inconvenience. [import]uid: 84637 topic_id: 23242 reply_id: 93121[/import]

[Danny]

In betterzip i used uncompressed(tar), [import]uid: 84637 topic_id: 23242 reply_id: 93124[/import]

[john_c]

Ok, thanks Danny. I tried it on a Mac and it worked but the tar archive created on a Windows 7 machine with 7zip still resulted in a header error. I had to create another one using BetterZip. [import]uid: 71024 topic_id: 23242 reply_id: 93141[/import]

[ezraanderson1979]

it took me a very long time to debug and figure it out, but the solution is very simple
function:
function untar(filename, fileDir, destdir, onComplete)

code:
local tar_handle = _ioOpen(filePath, “r”)

change to:
local tar_handle = _ioOpen(filePath, “rb”)
i guess it needs to opened as a binary in windows. [import]uid: 89663 topic_id: 23242 reply_id: 93142[/import]

[ezraanderson1979]

@John_c

betterzip for a mac seams to work. i have tried 7zip but i think it must use some compression, and for the tar.lua to work the file needs to be stored in an uncompressed format.

and for “attempt to perform arithmetic on a nil value” just open the file as a binary on the windows simulator for it to work.
I have been waiting for this for a long time. thx all for making it happen.

if someone can find a windows based app for compressing .tar files that works with this please post:) [import]uid: 89663 topic_id: 23242 reply_id: 93148[/import]

[ezraanderson1979]

@john

i am in the same boat i can’t get it to work(invalid header magic), using xp and 7-zip compressing to a .tar

i would like to work with you to sort this out:)

module:
tar.lua

error:
attempt to index local ‘header’ (a boolean value)

function:
function untar(filename, fileDir, destdir, onComplete)

line:
local file_data = tar_handle:read(_ceil(header.size / blocksize) * blocksize):sub(1,header.size)

**------------

@danny

i have no folder in the compressed file, same error as above.

using your example on windows simulator

example:
test.tar

module:
tar.lua

error:
attempting to perform arithmetic on a nil value

function:
local function checksum_header(block)

line:
sum = sum + block:byte(i)
[import]uid: 89663 topic_id: 23242 reply_id: 93112[/import]

[ezraanderson1979]

in betterZip which Tar format do you use, uncompressed(tar), tgz,tbz?

edit:

just reread your post

1. It only supports uncompressed tar archives (.tar). [import]uid: 89663 topic_id: 23242 reply_id: 93122[/import]

[sharp1959]

O.K. guys,

Make sure you change the line in the code like

ezraanderson1979 above did…(windows needs rb) for
binary…then
Then I read this, about the header (errors)
"ustar "… found these links

http://sourceforge.net/projects/sevenzip/forums/forum/45797/topic/3686403

http://www.izarc.org/download.html

Install the izarc program for windows and make a
uncompressed .tar file and wooot…life is good again!
Have fun!
Larry
Willow Road games

[import]uid: 107633 topic_id: 23242 reply_id: 93170[/import]