Jump to content

[TOPIC: topicViewTemplate]
[GLOBAL: userSmallPhoto]
Photo

Augmented Reality apps - possible in Corona?
Started by kilopop Jul 17 2016 04:17 PM

- - - - -
26 replies to this topic
vuforia augmented image targets
[TOPIC CONTROLS]
Page 1 of 2 1 2
This topic has been archived. This means that you cannot reply to this topic.
[/TOPIC CONTROLS]
[modOptionsDropdown]
[/modOptionsDropdown]
[reputationFilter]
[TOPIC: post.html]
#1

kilopop

[GLOBAL: userInfoPane.html]
kilopop
  • Contributor

  • 258 posts
  • Corona SDK

Broadly this is already possible to add an image overlay to the camera feed.

 

But would it be possible to create content that reacts to the environment? Obviously not 3d models but 2d images that position and skew to where the device camera is.  This is very easy to achieve in Vuforia for instance using image targets (real world image to provide something for the device camera to track an image as though it is pinned to the image target).

 

Realistically, is this something that could be developed using Corona? Either from the ground up or via a Vuforia plugin?

 

Interested in any and all feedback or examples.

 

 



[TOPIC: post.html]
#2

roaminggamer

[GLOBAL: userInfoPane.html]
roaminggamer
  • Corona Geek

  • 7,613 posts
  • Corona SDK

1. General Augmented Reality - Yes some apps that would be considered AR can be made w/ Corona.  I wrote a game for a client last year that cared where in the world and what physical features were nearby to make game decisions and present you with 'encounters'.  (Game not released unfortunately as he got sidetracked by a bigger opportunity which I'm also coding for him.)

 

Note: I can't give any more details that the above general statement till I get the OK from client or it is finally released.

 

 

2. AR the way you described it (a very specific application of the concept).  No. I don't think you could do that exact kind of app in Corona.  At least now without doing some Enterprise coding and using some sophisticated image analysis code, and likely other bits like tracking phone orientation and position.

 

Wasn't Vuforia specifically designed for AR?  If so, we're talking apples and oranges comparison.  Corona is a general SDK.  

 

Maybe you could integrate elements of Vuforia w/ Corona via enterprise?

 

 

Note: AR is much more than a specific usage like Pokemon Go which is what it sounds like you're describing.  Corona can be used for a number of games and apps that would be considered AR.

 

PS - Some will disagree w/ my last statement which is cool, as I've read definitions of AR, but I tend to think any app that cares where you are in the world and adds additional data on top of that is enough to be AR.  

 

However, with the current Pokemon Go craze and the strong interest in various visualization technologies, most people will think that an app/game is only AR if it involves the overlaying of virtual elements onto a real-time video or camera capture.  Fair enough.

 

TL;DR  No I don't think you can make a Pokemon Go style game in Corona Pro, but you might be able to w/ Enterprise.


Edited by roaminggamer, 17 July 2016 - 07:08 PM.


[TOPIC: post.html]
#3

kilopop

[GLOBAL: userInfoPane.html]
kilopop
  • Contributor

  • 258 posts
  • Corona SDK

Thanks roaminggamer.

 

Yes, Vuforia is specifically designed for AR. So instead of having to create something from the ground up in Corona, would a Vuforia plugin be possible in Corona? Like it is with Unity: http://www.marcofolio.net/other/introduction_into_augmented_reality_with_vuforia.html



[TOPIC: post.html]
#4

roaminggamer

[GLOBAL: userInfoPane.html]
roaminggamer
  • Corona Geek

  • 7,613 posts
  • Corona SDK

I don't write native plugins any more.  However, I imagine if I did and could get my hands on the vuforia libs and docs in a format compatible / build-able with Corona Enterprise, then the answer would be yes.

 

Sounds like you need to find a Enterprise developer.



[TOPIC: post.html]
#5

kilopop

[GLOBAL: userInfoPane.html]
kilopop
  • Contributor

  • 258 posts
  • Corona SDK

Great, that's encouraging. I will post something over in the Corona Jobs forum at some stage. For now, it would be interesting to hear if any Enterprise developers have achieved this to date or any other thoughts about Vuforia with Corona. It would be fantastic to avoid having to dive into Unity for this and maintain our Corona framework.



[TOPIC: post.html]
#6

gogigoranic8

[GLOBAL: userInfoPane.html]
gogigoranic8
  • Contributor

  • 149 posts
  • Corona SDK

Corona cannot render 2.5, or so called 3D models?

As far as I can understand, that is required for game like Pokemon Go.

Furthermore, how to handle 2.5 (3D) collisions?

[TOPIC: post.html]
#7

roaminggamer

[GLOBAL: userInfoPane.html]
roaminggamer
  • Corona Geek

  • 7,613 posts
  • Corona SDK

Rendering - Correct, Corona is 2D only.  You can however implement a variant of projection styles on your own using math, properly constructed artwork and a solid understanding of the projection style you're using.

 

Collisions - In most cases, you'll need to use math tricks, rules-of-thumb based in the projection style's layout, and a throrough understanding of Box 2D collision beyond the basic collision filter rules.  In some cases, you'll have to roll-your-own COLDET and Response system.

 

Probably not a satisfying answering, but... an answer. :|



[TOPIC: post.html]
#8

gogigoranic8

[GLOBAL: userInfoPane.html]
gogigoranic8
  • Contributor

  • 149 posts
  • Corona SDK

I am not so skilled programmer as you Ed, so if I decide to move over to 3D, it is easier to go with Unity  :D

 

Thanks.



[TOPIC: post.html]
#9

roaminggamer

[GLOBAL: userInfoPane.html]
roaminggamer
  • Corona Geek

  • 7,613 posts
  • Corona SDK

Hey, if you're going to actually make a 3D game I'd go with a 3D engine/SDK too.  However, I will say, 2.5D ain't 3D, so you have to decide which way to go:

 

One MTE usage:



[TOPIC: post.html]
#10

roaminggamer

[GLOBAL: userInfoPane.html]
roaminggamer
  • Corona Geek

  • 7,613 posts
  • Corona SDK

Note: I think we've strayed a bit here.. so my apologies to all, but I did want to finish answering the 'appended' questions about projections and Corona with samples (above).



[TOPIC: post.html]
#11

gogigoranic8

[GLOBAL: userInfoPane.html]
gogigoranic8
  • Contributor

  • 149 posts
  • Corona SDK

Thanks Ed for all clarifications.

 

2.5D is good enough (like Zig-Zag and Crossy Road).

I think 3D is too much for indie (you need a team and graphic designer I think).

 

 

Is there any 2.5D relatively successful/known game made in Corona so far?


Edited by gogigoranic8, 19 July 2016 - 10:56 PM.


[TOPIC: post.html]
#12

Peter Rich

[GLOBAL: userInfoPane.html]
Peter Rich
  • Enthusiast

  • 47 posts
  • Corona SDK

It seems that a lot of this conversation has focused on 3D imaging concerns.  What about simple, flat (2D) overlays on the camera at a given location?  Imagine I had people collecting "stickers" at specific geographic locations using their GPS and camera (with the overlay in the camera).  How difficult would that be in CoronaSDK?

 

Also, how precise would the GPS be overall?  And how does that translate to indoors?  Thanks for any clarification you might provide.



[TOPIC: post.html]
#13

Rob Miracle

[GLOBAL: userInfoPane.html]
Rob Miracle
  • Moderator

  • 26,534 posts
  • Enterprise

Corona SDK can capture input from the video camera and use it to fill a Corona display object (newRect for example) but on iOS only. We do not and have to no plans to support this on Android. It's simply too difficult. You could display stickers on top of the camera input with little difficulty.

 

Now the GPS accuracy: You're really at the mercy of the GPS system. They are as accurate as they are. It has to do with how many satellites are in view. The more sats your phone can see the more accurate it will be. You will be less accurate indoors since building block the relatively low power GPS signal and it's harder to pick up enough, or it will fall back to using location data from WiFi routers.

 

Sometimes the GPS will put you with in a meter or so of reality other times it could be tens of meters off. This is the nature of GPS.

 

Rob



[TOPIC: post.html]
#14

roaminggamer

[GLOBAL: userInfoPane.html]
roaminggamer
  • Corona Geek

  • 7,613 posts
  • Corona SDK

@gogigoranic8

 

Hi.  This is my last answer re: 2.5D here because this is not really related to the original question.  If you have further questions, please start a new thread and link back to this one to supplement the question/discussion.

 

 

 

Is there any 2.5D relatively successful/known game made in Corona so far?

 

  • Successful game?  I don't know.
  • Successful example?  See my MTE links above.

 

 

 

@staypuffinpc

 

Adding to Rob's last statement, I'd say that it is especially the nature of civilian GPS receivers to be inaccurate sometimes. Civilian GPS receivers are still not (but steadily improving) as accurate as military receivers.

 

This is an interesting read that talks about some of the reasons for this: http://www.gps.gov/systems/gps/performance/accuracy/


Edited by roaminggamer, 20 July 2016 - 08:03 AM.


[TOPIC: post.html]
#15

Peter Rich

[GLOBAL: userInfoPane.html]
Peter Rich
  • Enthusiast

  • 47 posts
  • Corona SDK

Thanks @roaminggamer.  I fear you're right about GPS accuracy in that regard.  We'll have to wait a few more years still, perhaps.



[TOPIC: post.html]
#16

kilopop

[GLOBAL: userInfoPane.html]
kilopop
  • Contributor

  • 258 posts
  • Corona SDK

AR doesn't need to use GPS. Visual image targets can be used instead which is an image in the real world that the device app uses to motion track content. Here is an example of a flat 2d object but a shape that does adjust to the tilts and zooms of the device to track onto an image target. Check last part of the video:

 

As far as I can tell, Corona out of the box cannot interpret image targets in its camera view. But Vuforia can. So I am thinking the solution is using Vuforia via Corona Enterprise, correct?

 

So if anyone has had experience with Vuforia in Enterprise, it would be great to hear about it. I did find Walter responding in Vuforia to some specific questions so someone in Corona land is doing it: https://developer.vuforia.com/forum/ios/using-vuforia-without-eaglview



[TOPIC: post.html]
#17

1limakoh4

[GLOBAL: userInfoPane.html]
1limakoh4
  • Observer

  • 8 posts
  • Corona SDK

Hi

 

I am fairly new to Coronasdk. I am intrigued about AR and want to do similar things with CoronaSDK for Android. I know that CoronaSDK can do it for IOS. I can't afford the enterprise version since I am just playing around with coding and most of the projects I do is free.

 

I read in Rob's post somewhere here that we can put a sticker on top of the camera. This would be so cool too. I can think of so many things that can be done with this AR feature (if only available on Andriod as well).

 

Anyone has done this sticker on top of camera?

 

Thanks

Rgds

TH



[TOPIC: post.html]
#18

sskenth

[GLOBAL: userInfoPane.html]
sskenth
  • Observer

  • 10 posts
  • Corona SDK

Hi All

So I'm not the best coder in the world, but I managed to make a little PokemonGo AR clone in Corona in less than 100 lines of code.
You can see the video here 

You can download and use the code(however you like) here - https://bitbucket.org/sskenth/coronaopensource.git

Hope that helps you on your journey with Augmented Reality and CoronaSDK.

Good luck, and if I can answer any specific question on it, let me know

 

-Sat

 

P.S. can anyone tell me how to embed a video into a forum post, as you can see above I clearly dont know how to do it ahaha [SOLVED - Thanks Rob]



[TOPIC: post.html]
#19

Rob Miracle

[GLOBAL: userInfoPane.html]
Rob Miracle
  • Moderator

  • 26,534 posts
  • Enterprise

Nice tutorial.

 

Have you tried just putting in the URL? It might be smart enough to detect youtube links.



[TOPIC: post.html]
#20

sskenth

[GLOBAL: userInfoPane.html]
sskenth
  • Observer

  • 10 posts
  • Corona SDK

It couldn't have been any easier and I still got it wrong, I feel so silly right now  :unsure: 

Thanks Rob



[TOPIC: post.html]
#21

Rob Miracle

[GLOBAL: userInfoPane.html]
Rob Miracle
  • Moderator

  • 26,534 posts
  • Enterprise

By the way, great tutorial!



[TOPIC: post.html]
#22

sskenth

[GLOBAL: userInfoPane.html]
sskenth
  • Observer

  • 10 posts
  • Corona SDK

Thank you Rob, its a great compliment especially coming from a legend like yourself :)
 



[TOPIC: post.html]
#23

AidanWolf

[GLOBAL: userInfoPane.html]
AidanWolf
  • Enthusiast

  • 92 posts
  • Corona SDK

AR is definitely possible in Corona, check out my Corona app Stickar. Though I eventually switched my AR code from lua to native for pixel perfect movement.

 

Here's some really helpful code I've written in my year long exploration into AR. Haven't tested in a while, changed some variable names around, but it should be a good starting point. Note: does not adjust to Landscape orientation.

 

Augmented Reality tracking in <100 lines of code

 

0: Quick setup

system.setAccelerometerInterval( 100 ) -- 100hz for best effect, higher battery drain

local ar = {degrees = 0,height = 0,startDegrees = 0,startHeight = 0}
-- degrees comes from the compass (for x AR movement)
-- height comes from the accelerometer (for y AR movement)
-- startDegrees is an offset for x AR movement (default is 0)
-- startHeight is an offset for y AR movement (default is 0)

-- AR camera
local background = display.newRect(display.contentWidth*.5,display.contentHeight*.5,display.contentWidth,display.contentHeight)
background.fill = {type = "camera"} -- gives you a fullscreen camera view
background.width = display.contentHeight*(3/4) -- get proper aspect ratio for camera

1: Augmented Reality tracking

local track_degrees = nil
local track_degrees_last = nil

local track_group = display.newGroup() -- group that moves with device orientation changes

local test = display.newRect(track_group,display.contentWidth*.5,display.contentHeight*.5,64,64)

track = function (event)
  if not track_degrees then
    track_degrees = ar.degrees
    track_degrees_last = ar.degrees
  end

  -- flip tracking for full 360 deg rotation
  if ar.degrees-track_degrees_last >= 180 then
    track_degrees_last = track_degrees_last + 360
  elseif ar.degrees-track_degrees_last <= -180 then
    track_degrees_last = track_degrees_last - 360
  end
  
  track_degrees = track_degrees + (ar.degrees-track_degrees_last)
  track_degrees_last = ar.degrees
  
  local easex = 0
  local easey = ar.height*display.contentHeight

  if ar.startDegrees then
    easex = -(vt.track_degrees-ar.startDegrees)*16-(display.contentWidth*.5)
  end

  if ar.startHeight then
    easey = (ar.height*display.contentHeight)-(ar.startHeight*display.contentHeight)-display.contentHeight*.5
  end

  local track_speed = 3 -- lower value = faster tracking, more jittery

  track_group:translate(((easex-track_group.x)/track_speed),((easey-track_group.y)/track_speed))

end

Runtime:addEventListener("enterFrame",track)

2: Get ar.degrees

local compass = function( event )
    if _G.device == "Android" or event.geographic == nil then
        ar.degrees = event.magnetic
    else
        ar.degrees = event.geographic
    end
    
     if ar.degrees < 0 then
        ar.degrees = ar.degrees + 360
    elseif ar.degrees >= 360 then
        ar.degrees = ar.degrees - 360
    end

end

Runtime:addEventListener( "heading", compass )

3: Get ar.height

accelerometer = function ( event )
  ar.height = event.zGravity
end

Runtime:addEventListener( "accelerometer", accelerometer )

4: insert AR object

local create_AR_object = function (obj)
     local xc,yc = obj:localToContent(0,0 )
     track_group:insert(obj)
    
     obj.x = -track_group.x+(xc-W*.5)
     obj.y = -track_group.y+(yc-H*.5)
end

local myObject = display.newCircle(0,0,64)
create_AR_object(myObject)

Here are some other helpful pieces of code if you're planning on making an AR/GPS game like Pokemon Go

 

1: Distance between two map coordinates, accurate whether you're in China or the North Pole

Important because latitude and longitude distances change as you move away from the equator

local map_distance = function (lat1,long1,lat2,long2)
    local R = 6371000; -- metres
    local x1 = math.rad(lat1);
    local x2 = math.rad(lat2);
    local deltax = math.rad(lat2-lat1);
    local deltalambda = math.rad(lon2-lon1);
    
    local a = math.sin(deltax/2) * math.sin(deltax/2) + math.cos(x1) * math.cos(x2) * math.sin(deltalambda/2) * math.sin(deltalambda/2);
    local c = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a));
    
    local d = R * c;
    
    return d
end

2: Plot a point on a map using distance and angle

Simple, but a big time saver

local plotter = function (lat,lon,ang,dist)
  if lat and lon then
    local x = lat + dist*math.deg(math.cos(math.rad(ang)))
    local y = lon + dist*math.deg(math.sin(math.rad(ang)))
    return x,y
end

I also have code for translating latitude and longitude to screen pixels (for AR camera) if anyone is interested.

 

Hope I helped and got someone started on a new Corona AR project!



[TOPIC: post.html]
#24

nmichaud

[GLOBAL: userInfoPane.html]
nmichaud
  • Contributor

  • 492 posts
  • Corona SDK

Hi @AidanWolf, great tutorial. You also mention that you have some code for translating latitude and longitude to screen screen pixels ? Would you be open to share some of it please ?

 

regards.

Nick



[TOPIC: post.html]
#25

kilopop

[GLOBAL: userInfoPane.html]
kilopop
  • Contributor

  • 258 posts
  • Corona SDK

Stickar looks interesting. Would like to try It but keep getting: "not compatible with this iPad" message on multiple iPads we've tried.


[topic_controls]
Page 1 of 2 1 2
 
[/topic_controls]