Jump to content

[TOPIC: topicViewTemplate]
[GLOBAL: userSmallPhoto]
Photo

Tar Module - Extract tar files with Corona
Started by Danny Mar 12 2012 11:19 PM

- - - - -
68 replies to this topic
[TOPIC CONTROLS]
Page 1 of 3 1 2 3
This topic has been archived. This means that you cannot reply to this topic.
[/TOPIC CONTROLS]
[modOptionsDropdown]
[/modOptionsDropdown]
[reputationFilter]
[TOPIC: post.html]
#1

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Hey guys, I am aware that loads of you have been looking for a way to extract archives with Corona for a long time now.

I am planning on implementing more archive support for corona in C/objective c but more info on that later.

So for now, here is a solution done in pure lua.

From benchmarking it took 1.3 seconds to extract a 10mb tar archive.

I created the test tar archive using a mac app named "betterZip" however any archiving tool should be able to do this for you.

Notes :

1) It only supports uncompressed tar archives (.tar).
2) It doesn't support folders inside the archive.

So it won't save you space, but it will help you break past app size barrier limits. For instance the 50mb limit on the android market can be overcome by downloading your assets individually from a server. Obviously if you have thousands of images this is going to be tiresome, so this way you can pack them into one file and download that single file and extract it where it's needed.

The sample includes the tar module and also a sample demonstrating how to use it (just one line of code :) )

Enjoy

Update :

Changes :

1) Minor optimizations, even faster extraction
2) Added a onComplete handler that you can pass a function to upon successful extraction of the archive.

http://www.infuseddreams.com/apps/Tar_Lib.zip
uid: 84637 topic_id: 23242 reply_id: 323242


[TOPIC: post.html]
#2

bryan.vaccaro

[GLOBAL: userInfoPane.html]
bryan.vaccaro
  • Enthusiast

  • 95 posts
  • Guests

Danny, this is good news to hear after I just benchmarked another pure lua unzip module at 15 seconds for a 2 meg file!
One of my particular use cases involves extracting and reading one file from a tar/gzip that contains thousands of files without decompressing the entire archive.. do you think that will be possible under the native C implementation?
uid: 120 topic_id: 23242 reply_id: 93040


[TOPIC: post.html]
#3

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Yeah of course, i have no eta on the c implementation but i will provide info as soon as i have it.
It might also be possible with what i created here, il look into it

uid: 84637 topic_id: 23242 reply_id: 93043


[TOPIC: post.html]
#4

john_c

[GLOBAL: userInfoPane.html]
john_c
  • Observer

  • 16 posts
  • Corona SDK

doesn't seem to work. i keep getting "Invalid header magic" error.

i created the tar archive using 7zip.
uid: 71024 topic_id: 23242 reply_id: 93106


[TOPIC: post.html]
#5

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Hey jon.

Just to confirm.
Your file is a uncompressed .tar file and it doesnt contain any folders?

If it is like i said above please post up your tar file.

Did you try it with the sample tar file i included?
uid: 84637 topic_id: 23242 reply_id: 93109


[TOPIC: post.html]
#6

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Can you guys try with winrar? They have a free trial.

As i said i dont have a windows machine and made my archives using betterzip which i presume is mac only.
It sounds like 7zip is writing the headers differently.

Try with winrar for me and if it still doesnt work il install windows on a virtualbox and work this out for you guys
uid: 84637 topic_id: 23242 reply_id: 93114


[TOPIC: post.html]
#7

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Can you guys try with winrar? They have a free trial.

As i said i dont have a windows machine and made my archives using betterzip which i presume is mac only.
It sounds like 7zip is writing the headers differently.

Try with winrar for me and if it still doesnt work il install windows on a virtualbox and work this out for you guys
uid: 84637 topic_id: 23242 reply_id: 93115


[TOPIC: post.html]
#8

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Ezraanderson : is that with my sample ran unmodified?
uid: 84637 topic_id: 23242 reply_id: 93116


[TOPIC: post.html]
#9

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK

i think winrar can only uncompress tar, i don't think it can write them.

i will try it on my mac.

@danny

your example works fine on a mac

7-zip compressed file doesn't work on a mac, same error as above
uid: 89663 topic_id: 23242 reply_id: 93117


[TOPIC: post.html]
#10

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK

yes unmodified just spacing between functions

--*******************************
--i do this between my functions
uid: 89663 topic_id: 23242 reply_id: 93118


[TOPIC: post.html]
#11

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Weird, so my example works on mac but not on windows. It definitely works on IOS also.

This must be a simulator thing or something else. I will install virtualbox and windows tomorrow and fix this for you windows users.

My apologies for this....
uid: 84637 topic_id: 23242 reply_id: 93119


[TOPIC: post.html]
#12

john_c

[GLOBAL: userInfoPane.html]
john_c
  • Observer

  • 16 posts
  • Corona SDK

yes, the tar file is uncompressed and does not contain any folders. where should i post the tar file? it is also rather large (~75 mb)...

also, untarring test.tar results in "attempt to perform arithmetic on a nil value"
uid: 71024 topic_id: 23242 reply_id: 93120


[TOPIC: post.html]
#13

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

@john, there seems to be some issues with the windows version of the simulator and my module.

I am looking into fixing them asap, in the mean time if you have a mac, mac and ios are confirmed working.

Sorry for any inconvenience.
uid: 84637 topic_id: 23242 reply_id: 93121


[TOPIC: post.html]
#14

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

In betterzip i used uncompressed(tar),
uid: 84637 topic_id: 23242 reply_id: 93124


[TOPIC: post.html]
#15

john_c

[GLOBAL: userInfoPane.html]
john_c
  • Observer

  • 16 posts
  • Corona SDK

Ok, thanks Danny. I tried it on a Mac and it worked but the tar archive created on a Windows 7 machine with 7zip still resulted in a header error. I had to create another one using BetterZip.
uid: 71024 topic_id: 23242 reply_id: 93141


[TOPIC: post.html]
#16

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK

it took me a very long time to debug and figure it out, but the solution is very simple
function:
function untar(filename, fileDir, destdir, onComplete)

code:
local tar_handle = _ioOpen(filePath, "r")

change to:
local tar_handle = _ioOpen(filePath, "rb")
i guess it needs to opened as a binary in windows.
uid: 89663 topic_id: 23242 reply_id: 93142


[TOPIC: post.html]
#17

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK

@John_c

betterzip for a mac seams to work. i have tried 7zip but i think it must use some compression, and for the tar.lua to work the file needs to be stored in an uncompressed format.

and for "attempt to perform arithmetic on a nil value" just open the file as a binary on the windows simulator for it to work.
I have been waiting for this for a long time. thx all for making it happen.

if someone can find a windows based app for compressing .tar files that works with this please post:)
uid: 89663 topic_id: 23242 reply_id: 93148


[TOPIC: post.html]
#18

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK

@john

i am in the same boat i can't get it to work(invalid header magic), using xp and 7-zip compressing to a .tar

i would like to work with you to sort this out:)

module:
tar.lua

error:
attempt to index local 'header' (a boolean value)

function:
function untar(filename, fileDir, destdir, onComplete)

line:
local file_data = tar_handle:read(_ceil(header.size / blocksize) * blocksize):sub(1,header.size)

**------------
--------------

@danny

i have no folder in the compressed file, same error as above.

using your example on windows simulator

example:
test.tar

module:
tar.lua

error:
attempting to perform arithmetic on a nil value

function:
local function checksum_header(block)

line:
sum = sum + block:byte(i)
uid: 89663 topic_id: 23242 reply_id: 93112


[TOPIC: post.html]
#19

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK



in betterZip which Tar format do you use, uncompressed(tar), tgz,tbz?

edit:

just reread your post
1) It only supports uncompressed tar archives (.tar).
uid: 89663 topic_id: 23242 reply_id: 93122


[TOPIC: post.html]
#20

sharp1959

[GLOBAL: userInfoPane.html]
sharp1959
  • Enthusiast

  • 81 posts
  • Corona SDK

O.K. guys,

Make sure you change the line in the code like

ezraanderson1979 above did..(windows needs rb) for
binary..then
Then I read this, about the header (errors)
"ustar ".. found these links

http://sourceforge.net/projects/sevenzip/forums/forum/45797/topic/3686403

http://www.izarc.org/download.html

Install the izarc program for windows and make a
uncompressed .tar file and wooot....life is good again!
Have fun!
Larry
Willow Road games


uid: 107633 topic_id: 23242 reply_id: 93170


[TOPIC: post.html]
#21

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK

i can confirm that this works very well

http://www.izarc.org/download.html

nice find
uid: 89663 topic_id: 23242 reply_id: 93209


[TOPIC: post.html]
#22

john_c

[GLOBAL: userInfoPane.html]
john_c
  • Observer

  • 16 posts
  • Corona SDK

Wow, great job all!
uid: 71024 topic_id: 23242 reply_id: 93214


[TOPIC: post.html]
#23

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Hey guys, yeah after talking with josh quick i realised i should be opening it in "rb" binary. I will fix this in a few minutes and update the files for you.

Delighted to hear its working for people now :)
uid: 84637 topic_id: 23242 reply_id: 93254


[TOPIC: post.html]
#24

Danny

[GLOBAL: userInfoPane.html]
Danny
  • Corona Geek

  • 2,597 posts
  • Corona Staff

Edit : Just updated the file

Same link as the first post or here again : http://www.infuseddreams.com/corona/Tar_Lib.zip
Changes :

1) Fixed the library not working on windows
2) Fixed issues with onComplete function
3) Added Notes text that defines the requirements for tar archives.

Enjoy :)
uid: 84637 topic_id: 23242 reply_id: 93267


[TOPIC: post.html]
#25

ezraanderson1979

[GLOBAL: userInfoPane.html]
ezraanderson1979
  • Contributor

  • 103 posts
  • Corona SDK

@sharp1959 nice Find probably easier than my solution haha :)
this was my solution:

goto and download tare07.zip

http://geoffair.net/unix/tar-01.htm#downloads

extract to c:\tar

make tar.bat file and copy to c:\tar

@echo off

chdir /D %~dp0/tarMe

setlocal enabledelayedexpansion
set myvar 

rem for  %%i In (*.*) DO set myvar=!myvar! %%i
for  %%i In (*.*) DO set myvar=!myvar! %%i

echo %myvar%
c:\tar\tar.exe  -cvf c:\tar\myTar.tar %myvar%

pause


makeDir c:\tar\tarMe

copy files you want tared into c:\tar\tarMe\ and run c:\tar\tar.bat file


uid: 89663 topic_id: 23242 reply_id: 93206



[topic_controls]
Page 1 of 3 1 2 3
 
[/topic_controls]