Jump to content

[TOPIC: topicViewTemplate]
[GLOBAL: userSmallPhoto]
Photo

Native/Lua communication guide needed
Started by whammy Jun 28 2014 10:46 AM

50 replies to this topic
[TOPIC CONTROLS]
Page 2 of 3 1 2 3
This topic has been archived. This means that you cannot reply to this topic.
[/TOPIC CONTROLS]
[modOptionsDropdown]
[/modOptionsDropdown]
[reputationFilter]
[TOPIC: post.html]
#26

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Ahh... screencaptures on WP8 Direct3D apps is a challenge.  Which is why we don't support it yet.

 

I don't think a WriteableBitmap can capture what is custom Direct3D rendered to a DrawingSurfaceBackgroundGrid.  I think the WriteableBitmap class can only capture what was rendered to a XAML control.  (ie: Rendering via a DrawingSurfaceBackgroundGrid does not actually render to XAML, but gives us access to the app's main Direct3D context which the native XAML framework uses to render its content.)

 

That said, capturing via a WriteableBitmap will work if you disable the CoronaPanel's rendering to the DrawingSurfaceBackgroundGrid, because then it renders to a XAML SurfaceControl's texture instead, which can be captured.  The only problem with this is that you can't render at 60 FPS when it is in this mode.  But luckily we've set up our CoronaPanel to support hot-swapping between DrawingSurfaceBackgroundGrid and DrawingSurface rendering.

 

The following code successfully captures on the screen by doing what I've mentioned above.  I consider it a hack though, but it's the quickest thing I could come up with.  The only problem is that you have to wait until the 2nd/3rd frame for the capture to occur.

private CoronaLabs.Corona.WinRT.CoronaLuaEventHandler fEnterFrameEventHandler = null;
private int fEnterFrameCount = 0;

private void OnCoronaRuntimeLoaded(object sender,CoronaRuntimeEventArgs e)
{
   e.CoronaRuntimeEnvironment.AddEventListener("requestingScreenshot", OnRequestingScreenshot);
}

private ICoronaBoxedData OnRequestingScreenshot(
   CoronaRuntimeEnvironment sender, CoronaLuaEventArgs e)
{
   // Switch from fast background rendering to rendering within the CoronaPanel.
   // We do this background rendered content is not capturable via WriteableBitmap.
   fCoronaPanel.BackgroundRenderingEnabled = false;

   // Schedule to capture the screen on the next rendered frame.
   if (fEnterFrameEventHandler == null)
   {
      fEnterFrameEventHandler = new CoronaLuaEventHandler(OnEnterFrame);
   }
   fEnterFrameCount = 0;
   sender.AddEventListener("enterFrame", fEnterFrameEventHandler);
   return null;
}

private ICoronaBoxedData OnEnterFrame(
   CoronaRuntimeEnvironment sender, CoronaLuaEventArgs e)
{
   // The first frame still belongs to the background. Wait for the 2nd frame.
   fEnterFrameCount++;
   if (fEnterFrameCount < 2)
   {
      return null;
   }

   // Remove the "enterFrame" event listener.
   // We don't want to capture the screen on every rendered frame. :)
   sender.RemoveEventListener("enterFrame", fEnterFrameEventHandler);

   // Capture the contents of the CoronaPanel.
   var writeableBitmap = new System.Windows.Media.Imaging.WriteableBitmap(
         (int)fCoronaPanel.ActualWidth, (int)fCoronaPanel.ActualHeight);
   writeableBitmap.Render(fCoronaPanel, null);
   writeableBitmap.Invalidate();

   // As a quick test, display the captured image within an image control onscreen.
   var imageControl = new System.Windows.Controls.Image();
   imageControl.Source = writeableBitmap;
   imageControl.Margin = new Thickness(50);
   fCoronaPanel.Children.Add(imageControl);

   // Re-enable fast 60 FPS background rendering.
   fCoronaPanel.BackgroundRenderingEnabled = true;
   return null;
}



[TOPIC: post.html]
#27

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Hmm... you know what, the above is not working reliably for me.  I can only get it to capture while running under the .NET debugger.  It won't work while running under the "Native" debugger... or when not debugging the app (ie: release mode).

 

I think we're running into the following WP8 bug...

   http://connect.microsoft.com/VisualStudio/feedback/details/785537/windows-phone-8-direct3d-xaml-application-writeablebitmap-render-bug

 

Where capturing a XAML control or screen's content using WriteableBitmap while using a DrawingSurfaceBackgroundGrid becomes unreliable.  I've ran into this issue before.  I think the only reliable way of doing this is in C++ via Direct3D.

 

I hate to say this, but you might want to skip this feature for now.  I don't think we can implement this on our end by your due date.



[TOPIC: post.html]
#28

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

Yes, I've just tried it and the app crashes: "First-chance exception at 0x770259A3 in TaskHost.exe: Microsoft C++ exception: _com_error at memory location 0x01CEF600."

 

OK, I'll share a text message then.

 

There's another thing: is there a way to make fCoronaPanel modal for a while, so when I display a native popup no clicks would go through?



[TOPIC: post.html]
#29

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Are you saying the taps on the popup that is on top of Corona are getting dispatched to Lua?

 

Edit:

I just put a XAML WebBrowser control on top of the CoronaPanel.  Corona is not getting any tap events when I tap on the WebBrowser.  So, I'm not sure what the issue is.  Or perhaps what you want is to completely block all taps/touches from reaching Corona, but still allow Corona to be visible and animating/rendering?



[TOPIC: post.html]
#30

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

Maybe it's because I'm using a Windows Phone Toolkit to generate a popup? I'm not sure.

 

But yes - my clicks are getting through so I need to block all taps from reaching Corona
while remaining it
 visible under the popup.

 

Is there a way to do this?



[TOPIC: post.html]
#31

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Hold on.  I just figured out how to reproduce the issue you discovered.  In fact, I've found a TODO comment in my touch handling code to resolve this very issue in the future.  *Slaps-Forehead*

 

Okay.  I'll take care of this now.

 

Side Note:  We've just updated/uploaded our WP8 class documentation now.  You can find our Native/Lua bridge related classes via the link below for you quick reference...

   http://docs.coronalabs.com/daily/native/wp8/html/html/N_CoronaLabs_Corona_WinRT.htm



[TOPIC: post.html]
#32

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

I have a work-around for you at the moment.  You can suspend the Corona runtime yourself before displaying the popup and then resume it yourself when the popup is cleared.  You can do so via the following methods...

 

fCoronaPanel.Runtime.Suspend();

fCoronaPanel.Runtime.Resume();

 

Just note that this pauses rendering and audio too.



[TOPIC: post.html]
#33

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

OK, I needed to have audio playing so I removed all touch listeners on lua side before dispatching an event to C#

 

BTW: I call audio.pause() and audio.play() in applicationSuspend and applicationResume listeners but I still get stream audio muted sometimes.



[TOPIC: post.html]
#34

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

I've just uploaded a live Beta to Windows Phone Store and tested In App Purchases - the bridge is working great! Thanks!

 

The only issue is that the audio mutes almost every time I make a purchase  :huh: I'll try to add an audio.play() call at the end of the event sent from C# after finishing the purchase. Will let you know if that helped when I upload the next Beta and test it.



[TOPIC: post.html]
#35

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Olaf,

 

Today, I've fixed the audio suspend/resume issue and the touch/tap event issue.

David and I won't be able to send you these changes until this Monday.  I hope you don't mind.  (I know the clock is ticking for you.)

 

I'm guessing you'll want me to add support for disabling the idle timer, right?



[TOPIC: post.html]
#36

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

Joshua,

Thanks so much. This Monday is the ultimate day I can send the app to reviewers, considering that the certification takes 4 working days for them. So iI'll wait for the new build to get rid of pausing audio bug. If you manage to add the idle timer support until then as well, it would be great!
However, this support would be working for me only if I could enable/disable it inside app several times, not only to disable it once at the beginning. So if you find out that's not possible for WP8 apps, then leave it.

Would it be possible for David to send out the new bits in the morning?

Thanks,

[TOPIC: post.html]
#37

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

Joshua, 

 

I forgot to mention that my game occasionally crashes giving this output:

The program '[4428] TaskHost.exe' has exited with code -2146233078 (0x8013150a).

There are no lua Runtime errors displayed during the crash. Do you have an idea what might be the cause?



[TOPIC: post.html]
#38

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Olaf,

I'm not aware of any crashes in Corona on WP8.  Or at least I haven't seen any during my testing.

 

If a crash does occur, then Visual Studio should be outputting a stack dump that would tell you where the error occurred... and exception message if the error originated from .NET or COM interface C++/CX code.  Did you not see anything like this in Visual Studio's Output Panel?  If not, then trying changing your debug mode from "Native" to "Managed" under your application project's "Properties".  You'll get different debugging information then.  Especially if the error originated from the .NET side of the app.

 

I'm actually curious if you're running into an OutOfMemory exception.  There was 1 Corona app that I've played with that always ran into that on startup, because, well, they were loading too much stuff into memory than the device had available.  :)

But in this case, Visual Studio makes it very obvious that this is the case in the Output Panel.



[TOPIC: post.html]
#39

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

OK, I'll try to hunt it down. It happens almost never though and I regularly remove textures from memory. I'll monitor memUsage to find it out. Thanks for the clue.

 

Now I'm trying to make the app crash in a Managed debugging mode but can't reproduce the crash again, huh :-)



[TOPIC: post.html]
#40

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

Joshua,

 

Thanks a ton for sharing the new build with David so fast! He's put it in my Dropbox folder already.

The music is resuming after application resumes and the clicks aren't going through anymore.

Thanks to you I'm can submit my game yet today :)

 

We'll see how the Windows Store certification and later appCampus review goes

but here's a sneak peek of the game I'm developing in CoronaCards: 

 

http://www.madeitapp.com/omnomster2/



[TOPIC: post.html]
#41

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Happy to help!  Since you said you needed to upload your app by this Monday, we wanted to make sure that you had a reasonable amount of time to test it on your end.  Better to get it done early just in case.

 

And thanks for sharing a sneak peak of your app with me.  Can't wait to see it in action for myself.  :)

Your app will be the first Corona SDK built app to make it onto the Windows Store.  So, this is pretty exciting from my perspective too.  :)

 

Oh and regarding the random crashes, if you find any more clues for me, then I'll be more than happy to track it down.  I'm curious as to what that the real issue is too.



[TOPIC: post.html]
#42

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

Sure, I'll play with it under Managed mode and try to reproduce the crash. Although it happened very rarely. I'm thinking it could happen because the app couldn't fetch IAPs when I tested the local build, maybe? I don't encounter the crash on the beta submitted to windows phone store.

 

I'll be happy to pass you a link when the game goes live. This will be after the App Campus QA Team let me unhide it from the store (assuming that Windows Store certification goes fine first).

 

I'll be here anyway. Thanks for all!



[TOPIC: post.html]
#43

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

I did a bit of research on your crash and that exit code you're seeing matches the error code for a "SecurityException", which can happen when accessing a file location or feature without the required permissions.  I assume that you have all of the required capabilities and requirements checked in your WMAppManifest file or else IAP wouldn't have worked at all.  So, I'm not sure what might be causing this yet.  I'll ask Microsoft about this later this week.

 

In any case, good luck with your submitted app.  I look forward to it showing up on the app store soon.  :)



[TOPIC: post.html]
#44

whammy

[GLOBAL: userInfoPane.html]
whammy
  • Contributor

  • 202 posts
  • Corona SDK

[TOPIC: post.html]
#45

Tapps - enterprise

[GLOBAL: userInfoPane.html]
Tapps - enterprise
  • Enthusiast

  • 75 posts
  • Corona SDK

Hello Joshua,

 

I have just started testing the Native/Lua bridge and it seems to be working fine so far. Sending events to the Native side from Lua is pretty easy (register a listener and send the event to the lua Runtime object). But I have a question about sending events to Lua from the Native side.

 

In all your examples you register an enterFrame event listener in C# and use the RuntimeEnviroment that you get from the handler to dispatch the event to Lua. Is this the recommended way of getting access to the RuntimeEnviroment object ?

 

In my tests, I saved the RuntimeEnviroment object that I get from the OnCoronaRuntimeLoaded event listener and then use this object to dispatch my future events. Will I run into problems doing this (if, for example, this is not the only RuntimeEnviroment object or if it is not the same throughout the app lifecycle) ?



[TOPIC: post.html]
#46

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

The "enterFrame" listener I've set up is just an example demonstrating that you can listen into standard Corona events that are normally delivered to Lua.  I don't recommend that you set up an "enterFrame" listener on the C# side because that'll cause needless overhead and may lower your framerate.  I recommend that you create your own event name on the Lua side instead.

 

Have a look at my "requestingSum" example here...

   http://forums.coronalabs.com/topic/49041-nativelua-communication-guide-needed/?p=257633

 

In that example, I dispatch a "requestingSum" event from Lua which is received from C#.  The C# code then sums the 2 parameters delivered via the event arguments and then returns the sum back to Lua as a return value.



[TOPIC: post.html]
#47

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

Here are some more details if it helps you understand how this works.  Calling CoronaRuntimeEnvironment.DispatchEvent(), CoronaRuntimeEnvironment.AddEventLister(), and CoronaRuntimeEnvironment.RemoveEventListener() will literally call Lua functions Runtime:dispatchEvent(), Runtime:addEventListener(), and Runtime:removeEventListener() under the hood.  We're merely bridging to these functions between .NET and Lua.
 
For example, the following dispatches and listens to a custom event purely on the Lua side.  This is what's typically done with Lua libraries such as widgets, composer, etc.
-- Set up a custom event listener.
local function onCustomEventReceived(event)
   print("Received custom event")
end
Runtime:addEventListener("customEvent", onCustomEventReceived)

-- Dispatch the custom event.
Runtime:dispatchEvent({name="customEvent"})
 
And here is the equivalent of the above completely written in C#. Note that this adds an event listener and dispatches an event completely on the C# side (nothing written on the Lua side). Not a practical example, but it demonstrates the concept of how our event dispatcher/listener system works.
void OnCoronaRuntimeLoaded(object sender, CoronaRuntimeEventArgs e)
{
   // Add a custom event listener.
   e.CoronaRuntimeEnvironment.AddEventListener("customEvent", OnCustomEventReceived);
   
   // Dispatch the custom event.
   var eventProperties = CoronaLuaEventProperties.CreateWithName("customEvent");
   var eventArgs = new CoronaLuaEventArgs(eventProperties);
   e.CoronaRuntimeEnvironment.DispatchEvent(eventArgs);
}

ICoronaBoxedData OnCustomEventReceived(CoronaRuntimeEnvironment sender, CoronaLuaEventArgs e)
{
   System.Diagnostics.Debug.WriteLine("Received custom event");
}
 
Does this help any?

[TOPIC: post.html]
#48

Tapps - enterprise

[GLOBAL: userInfoPane.html]
Tapps - enterprise
  • Enthusiast

  • 75 posts
  • Corona SDK

Okay, I got that part. But I still have one doubt. All functions to use events, for example CoronaRuntimeEnvironment.DispatchEvent(), are object methods, instead of static methods, right ? So you need access to a CoronaRuntimeEnvironment object, and in the example you just posted, this done using the e.CoronaRuntimeEnviroment property.

 

What I am wondering is if this object stays the same during the application lifetime. If it is, I could just save it and use it to send events later, something like this:

private CoronaRuntimeEnvironment  environment = null;
private void OnCoronaRuntimeLoaded(object sender, CoronaLabs.Corona.WinRT.CoronaRuntimeEventArgs e)
{
  environment = e.CoronaRuntimeEnvironment;
}

private void SendEventToLua()
{
  environment.DispatchEvent();
}


[TOPIC: post.html]
#49

Joshua Quick

[GLOBAL: userInfoPane.html]
Joshua Quick
  • Moderator

  • 3,397 posts
  • Corona Staff

With the way the CoronaCards project template is made, you'll only have 1 CoronaRuntimeEnvironment object for the lifetime of your WP8 app.  So, what you're doing in your code example above will work fine.  In fact, you'll find tat the Loaded event only gets raised once in your app.

 

Now, here's a quick summary of the lifetime of a CoronaRuntimeEnvironment instance.  When you call the fCoronaPanel.Runtime.Run() method (or when it is auto-launched), it'll terminate the last CoronaRuntimeEnvironment instance (if any) and then create/load a new instance.  That CoronaRuntimeEnvironment instance will continue to be active until it has been terminated via a call to the fCoronaPanel.Runtime.Terminate() method or when the fCoronaPanel has been removed/unloaded from the page.  The "Terminating" event will be raised in this case, which is your last chance to do anything in Corona/Lua before the internal Lua state gets destroyed.  Afterwards, the CoronaRuntimeEnvironment instance will be in a terminated state and calling its methods such as DispatchEvent() will do nothing (ie: no-op).

 

Note that the CoronaRuntimeEnvironment.RuntimeState property will indicate the current state of your project such as Running, Suspended, Resumed, Terminating, Terminated, etc.

   http://docs.coronalabs.com/daily/native/wp8/html/html/P_CoronaLabs_Corona_WinRT_CoronaRuntimeEnvironment_RuntimeState.htm

   http://docs.coronalabs.com/daily/native/wp8/html/html/T_CoronaLabs_Corona_WinRT_CoronaRuntimeState.htm

 

This might interest you too:

   http://docs.coronalabs.com/daily/native/wp8/html/html/E_CoronaLabs_Corona_WinRT_CoronaRuntime_Terminating.htm

 

It's okay to use a member variable to hold on to a CoronaRuntimeEnvironment object reference like you are doing.  If you do happen to terminate the runtime yourself and create a new one, then the Loaded event will end up replacing your last CoronaRuntimeEnvironment reference.



[TOPIC: post.html]
#50

Tapps - enterprise

[GLOBAL: userInfoPane.html]
Tapps - enterprise
  • Enthusiast

  • 75 posts
  • Corona SDK

Thanks for the info Joshua!

 

I suspected that it would be a single object, but wanted to make sure before I started to write code around it.




[topic_controls]
Page 2 of 3 1 2 3
 
[/topic_controls]