Jump to content

[TOPIC: topicViewTemplate]
[GLOBAL: userSmallPhoto]
Photo

Multi-Pass Shader on camera in real-time
Started by RJ5 Sep 22 2014 01:03 PM

13 replies to this topic
[TOPIC CONTROLS]
This topic has been archived. This means that you cannot reply to this topic.
[/TOPIC CONTROLS]
[modOptionsDropdown]
[/modOptionsDropdown]
[reputationFilter]
[TOPIC: post.html]
#1

RJ5

[GLOBAL: userInfoPane.html]
RJ5
  • Enthusiast

  • 92 posts
  • Corona SDK

Has anyone managed to get a multi-pass shader to work in real-time with a device camera?? Is it even possible??

 

So far I've got single filter effects working in real-time with my device camera, but now I need to combine duotone and grayscale. When I select this custom effect the camera just shows a black image (all the other UI is showing just a blank camera image)

 

Anyone know if this is even possible??



[TOPIC: post.html]
#2

bjsorrentino

[GLOBAL: userInfoPane.html]
bjsorrentino
  • Veteran

  • 8,506 posts
  • Corona SDK

Hi @RJ5,

Just to clarify, you're attempting to apply this multi-pass filter on a live camera "fill" on iOS?

 

Brent



[TOPIC: post.html]
#3

RJ5

[GLOBAL: userInfoPane.html]
RJ5
  • Enthusiast

  • 92 posts
  • Corona SDK

Yes Brent that's correct, I'm trying to apply the multi-pass filter to a fill on a live camera

[TOPIC: post.html]
#4

bjsorrentino

[GLOBAL: userInfoPane.html]
bjsorrentino
  • Veteran

  • 8,506 posts
  • Corona SDK

Hi @RJ5,

I haven't experimented before using filters on the live camera feed. Have you confirmed that your multi-pass filter first works as expected on static images?

 

Thanks,

Brent



[TOPIC: post.html]
#5

RJ5

[GLOBAL: userInfoPane.html]
RJ5
  • Enthusiast

  • 92 posts
  • Corona SDK

Hi Brent

 

Yes I can confirm, if I use a static image my multi-pass filter works correctly

 

I can do a single filter on a live camera feed no problem, just can't get multiple filters to work

Is there any other way that I can stack 2 filters on top of each other (on a live camera feed)... other than with a multi-pass filter??



[TOPIC: post.html]
#6

bjsorrentino

[GLOBAL: userInfoPane.html]
bjsorrentino
  • Veteran

  • 8,506 posts
  • Corona SDK

Hi @RJ5,

I would need to confirm with engineering, but most likely multi-pass isn't supported on live camera feeds simply for performance reasons. Multi-pass filters typically wouldn't tax performance if done once (on a static image), but applying them "live" at 30 or 60 FPS could potentially impact performance on the device. Is there any way you can work with the camera feed using one filter, then post-snap, you capture that image and apply the next filter to it?

 

Brent



[TOPIC: post.html]
#7

RJ5

[GLOBAL: userInfoPane.html]
RJ5
  • Enthusiast

  • 92 posts
  • Corona SDK

Hi Brent

 

Thanks for the help, I did think it was asking a bit too much trying to do a multi-pass filter on a live feed

 

Unfortunately I can't do the method which you suggested. I need to be able to show the user a live black and white feed, but also they need to be able to edit the rgba on the fly. So combining monotone and greyscale filters would have been the perfect solution. If the greyscale filter had colour channels to which could have been edited it would have solved the problem.

 

I dont suppose theres another way to convert a live camera feed to black and white is there??



[TOPIC: post.html]
#8

bjsorrentino

[GLOBAL: userInfoPane.html]
bjsorrentino
  • Veteran

  • 8,506 posts
  • Corona SDK

Hi @RJ5,

For a "monotone over greyscale" effect, have you experimented with either the "colorMatrix" or "colorPolynomial" filters? One of these might give you a reasonably close effect, but I'm not overly familiar with how the coefficient table will affect your live feed.

 

Brent



[TOPIC: post.html]
#9

RJ5

[GLOBAL: userInfoPane.html]
RJ5
  • Enthusiast

  • 92 posts
  • Corona SDK

Hi Brent

 

Unfortunately these 2 effects will create colour images (which I'm not wanting)

 

I'm looking to create a similar effect to that of using coloured filters in black and white photography

 

In black and white photography you use different coloured filters to ensure that objects of different colours are separated and clearly defined. So for e.g. you would use a red filter to make all the blue objects in a scene increase in contrast (blue skies turn black). The final image is still in b&w, but the contrast of the blacks and the whites has shifted.

 

Hence my reason for wanting to combine monotone and greyscale. (works fine on static images just not on the live feed)



[TOPIC: post.html]
#10

bjsorrentino

[GLOBAL: userInfoPane.html]
bjsorrentino
  • Veteran

  • 8,506 posts
  • Corona SDK

Hi @RJ5,

Can you give me a link to (or post) some examples of what the image should look like? As in, what they'd look like for a B&W photo with a red filter applied?

 

Brent



[TOPIC: post.html]
#11

RJ5

[GLOBAL: userInfoPane.html]
RJ5
  • Enthusiast

  • 92 posts
  • Corona SDK

Hi Brent Here's a link to a article on using coloured filters in black and white photography http://www.photographymad.com/pages/view/using-coloured-filters-in-black-and-white-photography

[TOPIC: post.html]
#12

bjsorrentino

[GLOBAL: userInfoPane.html]
bjsorrentino
  • Veteran

  • 8,506 posts
  • Corona SDK

Hi @RJ5,

Thanks for the link, I've done quite a bit of color photography in my day (and used some color filters in some cases) but I've not done any B&W. I can ask the engineers if multi-pass can be added to the live camera feed, or why it might not be supported in specific.

 

I wonder if you might try putting the contents of the camera feed into a snapshot, then applying a multi-pass to that?

 

Brent

 

P.S. - Out of curiosity, have you tested single-pass filters on the live feed on an actual iOS device? Some filters can hit the GPU harder than others, especially if applied on an object which must update every frame (i.e. the live camera feed).



[TOPIC: post.html]
#13

RJ5

[GLOBAL: userInfoPane.html]
RJ5
  • Enthusiast

  • 92 posts
  • Corona SDK

Hi Brent

I grew up on B&W photography, it was easy to develop yourself and was loads cheaper than colour. 

 

 

When you say single-pass filter do you mean this method??

display.setDrawMode( "forceRender" )

local x = display.contentCenterX
local y = display.contentCenterY

local image = display.newRect( x, y, display.contentWidth, display.contentHeight )
image.fill = { type = "camera" }
image.fill.effect = "filter.grayscale"

if so, then it works no problem at all.

 

Multi-pass also works no problem at all on a static image, it just won't work on a live feed.

 

Unfortunately I can't take a snapshot of the camera feed and multi-pass that as I need the user to be able to live edit the feed and view in real-time the changes that they are making. The only way I can see to do it with the snapshot method is to have it set up in an enterFrame event (taking 30/60 snapshots of the feed every second) , but surely there would be performance issues with that??



[TOPIC: post.html]
#14

bjsorrentino

[GLOBAL: userInfoPane.html]
bjsorrentino
  • Veteran

  • 8,506 posts
  • Corona SDK

You might want to test out the snapshot method, with a Runtime event essentially "invalidating" (refreshing) the snapshot every so often. But first, make sure that your multi-pass filter works as expected on a snapshot, before trying to make that snapshot update with the camera feed. :)




[topic_controls]
[/topic_controls]